CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

File uploading, file storage and CakeP...

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this: echo $form->create('Media', array('action' => 'upload', 'type' => 'file')); echo $form->file('file'); echo $form->submit(__('Upload', true));   The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form. When you submit the form now, you'll get data like this in $this->data of your controller: Array
(
[Media] => Array
(
[file] => Array
(
[name] => cake.jpg
[type] => image/jpeg
[tmp_name] => /tmp/hp1083.tmp
[error] => 0
[size] => 24530
)
)
)
Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour. If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model. We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:
  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system. Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems.

Advantages of storage in the file system:

  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB
The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system. Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it. I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space. /** * Builds a semi random path based on the id to avoid having thousands of files * or directories in one directory. This would result in a slowdown on most file systems. * * Works up to 5 level deep * * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits * @param mixed $string * @param integer $level * @return mixed * @access protected */ protected function _randomPath($string, $level = 3) { if (!$string) { throw new Exception(__('First argument is not a string!', true)); } $string = crc32($string); $decrement = 0; $path = null; for ($i = 0; $i < $level; $i++) { $decrement = $decrement -2; $path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2)); } return $path; } You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class. http://api.cakephp.org/class/media-view I'll now explain you how to use this class to send files to the client. /** * Sends a file to the client * * @param string $id UUID * @access public */ public function download($id = null) { $this->Media->recursive = -1; $media = $this->Media->read(null, $id); if (empty($media)) { $this->redirect('/', 404, true); } $this->set('cache', '3 days'); $this->set('download', true); $this->set('name', $media['Media']['slug']); $this->set('id', $media['Media']['filename']); $this->set('path', APP . 'media' . DS . $media['Media']['path']); $this->set('modified', $media['Media']['modified']); $this->set('mimeType', $media['Media']['mime_type']); $this->set('extension', $media['Media']['extension']); $this->view = 'Media'; $this->autoLayout = false; if ($this->render() !== false) { $this->Media->updateAll( array('Media.downloads' => 'Media.downloads + 1'), array('Media.id' => $id)); } } You simply have to set autoLayout to false and the view class to media. $this->view = 'Media'; $this->autoLayout = false; There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false; $this->set('download', true); You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching. $this->set('cache', '3 days'); The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server. $this->set('name', $media['Media']['slug']); $this->set('id', $media['Media']['filename']); $this->set('path', APP . 'media' . DS . $media['Media']['path']); If you want to send a mime type that does not already in the MediaView class you can set it. $this->set('mimeType', $media['Media']['mime_type']); If you don't set it, the class will try to determine the mime type by the extension. $this->set('extension', $media['Media']['extension']); Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up. When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download. if ($this->render() !== false) { $this->Media->updateAll( array('Media.downloads' => 'Media.downloads + 1'), array('Media.id' => $id)); }

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Felix Geisendörfer - Javascript and Git

Felix gave a demonstration of the production level javascript separation and management that the team at Debuggable use in order to minimise the amount of Javasript that needs to be sent to the client for any specific page view, and to ensure the logic is separated into the pages that it is used for. This creates a better management system for Javascript than using a single file. In addition to this separation, Felix gave an overview of common practices and operations for using Git for version control in a day to day environment. This included: merges, conflict resolution, fast forwarding branches, and managing multiple repositories. Largely this presentation was an interactive one, and to gain the most out of it, you really needed to be there.

Marius Wilms - The CakePHP Media Plugin

If Marius had more than an hour to talk about the Media Plugin, he most certainly would have taken it. To go over the features and functionality of the entire plugin would have been many hours as there is a lot there. A brief touch on the features provided by the plugin was discussed, with some examples. Requirements are in the high end, but considering the state of PHP and the upcoming version of CakePHP, developers should be moving forward in terms of their PHP version and library support anyway. The Media plugin requires CakePHP 1.2.x.x and PHP 5.2.0+. It enables the transfer, manipulation and embedding of files in many varied ways. You can find the media plugin at: http://github.com/davidpersson/media Marius' focus was on doing media manipulation and embedding "properly", and identified that while there are lots of user contributions floating around the net, none of them were meeting his needs and were flexible enough. One of the main points he made here was that if done incorrectly, potential security risks arise due to command line interaction and file saving. Validation was one particular section of the code that made this a tricky plugin to develop, but allowed tests to be implemented to ensure security. Some common points that we hear all the time came through, and they make sense for CakePHP as well as any web application for security reasons:  

  1. Don't trust users supplied filenames
  2. Don't store files in an accessible webroot, rather have them accessible to scripts.
  3. Make the upload location (and local filenames) unguessable (like referencing files by UUIDs)
The media plugin contains about 8 new rules for file validation purposes to ensure that submitted data meets the application needs. Beyond validation, it handles all kinds of uploads, HTTP Post, Remote HTTP and local file inclusion.
A console is included to initialize the default directory structure, and as such, could be included as part of a deployment script with the CakePHP console.examples.
To ensure flexibility of use, a behavior is included to allow attachment to any number of models, and generioc storage and linking provided to ease integration into existing apps.
Marius concluded his talk with a plea for feedback. There are plenty of people using the plugin, but more feedback is required to ensure its the best it can be, and that all bugs  (if any) are squashed. Checkout the code at: http://github.com/davidpersson/media

Robert Scherer - Multi-Tenancy in CakePHP

Robert's talk was unscheduled, but ended up being a great case study for an insurance sales white-labelling solution that his company had undertaken and completed. Robert talked about multi-tenancy, and what this means for a web application, and how it relates to SaaS architecture. Challenges to be solved included:

  1. Differences in functionality
  2. Workflow differences
  3. Separation and security of data
  4. Branding and visual differences
Auth and Acl Components were used to solve a lot of the problems described, but in addition, Robert discussed the development of Modules as a new addon / plugin structure that allowed the addition, removal or configuration of application items at any level (Model, View or Controller).
Configuration of the modules was broken up into system default, mandators, and dealers configuration, allowing for inheritance of options along the way. To solve the view specific differences, built in themes were used to provide the differences required. This is a CakePHP builtin mechanism that serviced their needs well. Much of Robert's talk went through visuals of the site itself, and should we be able to get our hands on these, will post them up to see the various differences in presentation, and the module structure in terms of MVC.

Neil Crookes - Bake Master Class

After an introduction to bake, and what this shell means within CakePHP, Neil went on to explain and show examples of the code generation templates and capabilities provided by default. The bake shell is broken down into tasks and a main shell. These tasks separate out the logic required for various main task subsets including controller, model and view baking, amongst others. The main bake shell is found in the CakePHP directory cake/console/libs/bake.php. Tasks used by this shell are defined in the $tasks variable. Bake extends the CakeShell class and executes calls based on whether the users want interactive or non-interactive tasks through the __interactive() and bake() methods respectively. Neil made the suggestion that a persistent MySQL connection might be a good idea to stop database connection timeouts. Following this introduction, a great walkthrough of customisation of the bake process and templates was demonstrated. This included the addition of a new Shell that allows for multiple bakes to be done automatically of the same type. Neil has been kind enough to host the code, and you can find this over at the CakeFest downloads page.

We Bake with CakePHP