CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

File uploading, file storage and CakePHPs MediaView class

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this:

echo $form->create('Media', array('action' => 'upload', 'type' => 'file'));
echo $form->file('file');
echo $form->submit(__('Upload', true));

 

The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form.

When you submit the form now, you'll get data like this in $this->data of your controller:

Array
(
	[Media] => Array
	(
		[file] => Array
		(
			[name] => cake.jpg
			[type] => image/jpeg
			[tmp_name] => /tmp/hp1083.tmp
			[error] => 0
			[size] => 24530
		)
	)
)

Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour.

If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model.

We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:

  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system.

Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems.

Advantages of storage in the file system:

  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB

The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system.

Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it.

I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space.

/**
 * Builds a semi random path based on the id to avoid having thousands of files
 * or directories in one directory. This would result in a slowdown on most file systems.
 *
 * Works up to 5 level deep
 *
 * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
 * @param mixed $string
 * @param integer $level
 * @return mixed
 * @access protected
 */
	protected function _randomPath($string, $level = 3) {
		if (!$string) {
			throw new Exception(__('First argument is not a string!', true));
		}

		$string = crc32($string);
		$decrement = 0;
		$path = null;
		
		for ($i = 0; $i < $level; $i++) {
			$decrement = $decrement -2;
			$path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2));
		}

		return $path;
	}

You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class.

I'll now explain you how to use this class to send files to the client.

/**
 * Sends a file to the client
 *
 * @param string $id UUID
 * @access public
 */
	public function download($id = null) {
		$this->Media->recursive = -1;
		$media = $this->Media->read(null, $id);

		if (empty($media)) {
		$this->redirect('/', 404, true);
		}
		
		$this->set('cache', '3 days');
		$this->set('download', true);
		$this->set('name', $media['Media']['slug']);
		$this->set('id', $media['Media']['filename']);
		$this->set('path', APP . 'media' . DS . $media['Media']['path']);
		$this->set('modified', $media['Media']['modified']);
		$this->set('mimeType', $media['Media']['mime_type']);
		$this->set('extension', $media['Media']['extension']);

		$this->view = 'Media';
		$this->autoLayout = false;
		if ($this->render() !== false) {
			$this->Media->updateAll(
				array('Media.downloads' => 'Media.downloads + 1'),
				array('Media.id' => $id));
		}
	}

You simply have to set autoLayout to false and the view class to media.

$this->view = 'Media';
$this->autoLayout = false;

There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false;

	$this->set('download', true);

You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching.

	$this->set('cache', '3 days');

The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server.

	$this->set('name', $media['Media']['slug']);
$this->set('id', $media['Media']['filename']);
$this->set('path', APP . 'media' . DS . $media['Media']['path']);

If you want to send a mime type that does not already in the MediaView class you can set it.

	$this->set('mimeType', $media['Media']['mime_type']);

If you don't set it, the class will try to determine the mime type by the extension.

	$this->set('extension', $media['Media']['extension']);

Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up.

When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download.

	if ($this->render() !== false) {
	$this->Media->updateAll(
	array('Media.downloads' => 'Media.downloads + 1'),
	array('Media.id' => $id));
}

 

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Latest articles

One CakePHP Project Per Day

The whole team here at CakeDC are big supporters and contributors of the CakePHP community. For this month, I decided to do “one CakePHP project per day” to share with the community.  Here are some of my projects so far:

Project 01 - Notes App

A one page note application using CakePHP 4 and Bootstrap 5. This project is  a good starting point to learn the framework. Link: https://github.com/rochamarcelo/one-project-a-day-challenge-01-notes  

Project 02 - Contact List

An application to manage contacts - you are able to list, add, edit and delete contacts, upload contact avatar images or use avatar images from gravatar.com . It was built using CakePHP 4, plugin friendsofcake/search, plugin josegonzalez/cakephp-upload, Gravatar, and Bootstrap 5.  Link: https://github.com/rochamarcelo/one-project-a-day-challenge-02-contact-list  

Project 03 - Recipe Box

An application to manage recipes, using CakePHP 4,  CouchDB and Bootstrap 5. This one is a good starting point to learn to use CouchDB with CakePHP, including how to list, add and edit recipes (documents). Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-03-recipe-box  

Project 04 - Service Plan with Exchange rate

An application to list services and apply exchange rate using the api https://exchangeratesapi.io/documentation/ and CakePHP 4. In this one you see the custom namespace WebService to handle logic related to api as client. Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-04-service-plans-ex-rate  

Project 05 - Polls

A fun poll app, using the awesome Bulma CSS Framework and CakePHP 4. A good example of model association and the CounterCache Behavior. Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-05-polls-emmy  

Project 06 - Movie Theater Schedule

An application to see which movies are in the theaters and which hours by screen each day of the week. A good example of complex queries, model associations and seed data. Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-06-movie-theater-schedule  

Project 07 - Podcast Finder

An application to help easily find podcasts and download episodes. In the source code you’ll find how to use the itunes api,  a structure to handle Model actions (that I think is a good option to make your models cleaner), and a way to parse podcasts feed (XML); example usage of dependency injection. The application was built with CakePHP 4 and Bulma CSS Framework. Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-07-podcast-finder  

Project 08 - Url Shortener

An application to create short urls - a good example of how to create custom routes and use custom primary key types for a model. The application was built with CakePHP 4. Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-08-url-shortener  

Project 09 - Quiz

Users can list quizzes, create quizzes and answer at any time. A good example of how to use MongoDB with CakePHP 4 with a base structure for Collection classes.  Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-09-quiz  

Project 10 - File Transfer

An application to easily send files to anyone, create an account, upload the file and inform the person email to send to. Built with CakePHP 4, plugin CakeDC/Users,  plugin Josegonzalez/Upload,  plugin friendsofcake/bootstrap-ui, SMTP and Bootstrap. A good example to see the usage of these plugins. Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-10-file-transfer  

Project 11 - Tasks

A one page application for  users to manage their tasks. The user can create and remove decks, create and complete tasks, and list tasks grouped by decks. Built with CakePHP 4, plugin CakeDC/Users and Bootstrap 5 Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-11-tasks  

Project 12 - Blog

A blog website with blog posts and tags management, WYSIWYG editor, blog search, tags filtering. Built with CakePHP 4, CakeDC/Users plugin, friendsofcake/bootstrap-ui, Muffin/Slug, friendsofcake/search and Bootstrap 4 . A good example of usage of custom routes, route prefix, finders and multiple plugins. Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-12-blog  

Project 13 - Olympic Medal Count

Perfect time for this project, right?! An application to display olympic medal count by country and sports. The source code uses CouterCache behavior and aggregated query. Built with CakePHP 4 and Bootstrap 5. Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-13-olympic-medal-count
 

Project 14 - Smart Home Dashboard

An awesome dashboard to manage smart devices using MQTT Messaging, CakePHP 4, CakeDC/Users plugin, php-mqtt/client (testing with Mosquitto Broker) and Bootstrap 5. The application is able to publish messages to change device status and subscribe for status changes. Link: https://github.com/rochamarcelo/one-cakephp-project-a-day-challenge-14-smart-home-dashboard-mqtt    I hope that this initiative will somehow inspire others to put their Cake skills to work, and share their projects with the community. If you’d like to see my future projects and posts, you can follow me on Twitter, and I will share them all there! https://twitter.com/mrcodex

Logging CakePHP Applications To Team Communication

The log of applications is gold. It's an important part of the software, they represent the health of the application. By default, CakePHP will use the FileLog adapter which will write to /logs/ folder. It's hard to track the live issues, and by hard I mean you will need to connect to the server, open the file on /logs/ and look at the issue which you want to investigate.   What do you think if your application sends the error directly to your team communication (Slack, Teams, RocketChat) application? Will be easier to know about a new error after some deployment? This error is sneaky, and can be in command applications. Often, we only look at the errors when the users report it.   For this sample I will use Slack, but this approach can be implemented for any application.  All we need is to create a Log adapter and configure it. So…let’s bake that:     Now we may get errors like this:   That’s all bakers! I hope this article can be useful and you can improve your logs.  

A CakePHP Docker Development Environment

We sponsor a monthly CakePHP training session (register here https://training.cakephp.org ) where we cover different topics about the framework. One of our sessions, the "Getting Started with CakePHP 4" is aimed to help developers starting a new project quickly and following the best practices.   Our previous "recommended" quick setting for a CakePHP development environment was using a vagrant box. See details here:  https://www.cakedc.com/jorge_gonzalez/2018/01/17/using-a-vagrant-box-as-quick-environment-for-the-getting-started-with-cakephp-training-session. However, we've switched internally to use docker as our primary development environment and also we started using docker in our training sessions.   Here's a quick overview of a simple docker based development environment for CakePHP.  

1. Create a new CakePHP project skeleton using 

composer create-project cakephp/app myproject   A new folder "myproject" will be created with a CakePHP project skeleton inside. Go to this new directory and proceed.  

2. Create a new "docker-compose.yaml" file with the following contents

version: '3' services:   mysql8:     image: mysql:8     restart: always     container_name: mysql     environment:         MYSQL_ROOT_PASSWORD: root         MYSQL_DATABASE: my_app         MYSQL_USER: my_app         MYSQL_PASSWORD: secret     volumes:       - ./:/application     ports:       - '9306:3306'     cakephp:     image: webdevops/php-apache:8.0     container_name: cakephp     working_dir: /application/webroot     volumes:       - ./:/application     environment:       - WEB_DOCUMENT_ROOT=/application/webroot       - DATABASE_URL=mysql://my_app:secret@mysql/my_app     ports:       - "8099:80"
 

3. Run "docker-compose up"

You'll create 2 containers named mysql and cakephp -  check the docker-compose configuration to see default database and users created in the mysql container, and the same environment params passed to the cakephp container via DATABASE_URL to allow the cakephp container to connect with the mysql database.   NOTE: the ports exposed are 9306 for mysql and 8099 for cakephp webserver. You can list them using docker-compose ps.  

4. Access your database and cakephp shell

  • To access the database you can use the command:
mysql --port 9306 -umy_app -psecret my_app   To restore a database dump for example, you can use the command: curl -L https://raw.githubusercontent.com/CakeDC/cakephp4-getting-started-session/master/my_app.sql |mysql --port 9306 -umy_app -psecret my_app   You can also configure any database tool to access the database in: localhost:9306  
  • To access the cakephp environment and shell you can use the command:
docker exec -it --user application cakephp bash   You'll go to the webroot folder, so in order to run the cake shell you'll need to: cd .. bin/cake 
  Now you have a working environment to play with the training session contents.   In this previous article, we covered another approach to setting up a local docker environment: https://www.cakedc.com/rochamarcelo/2020/07/20/a-quick-cakephp-local-environment-with-docker    We hope to see you in our next training session! https://training.cakephp.org   

We Bake with CakePHP