CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

File uploading, file storage and CakePHPs MediaView class

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this:

echo $form->create('Media', array('action' => 'upload', 'type' => 'file'));
echo $form->file('file');
echo $form->submit(__('Upload', true));

 

The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form.

When you submit the form now, you'll get data like this in $this->data of your controller:

Array
(
	[Media] => Array
	(
		[file] => Array
		(
			[name] => cake.jpg
			[type] => image/jpeg
			[tmp_name] => /tmp/hp1083.tmp
			[error] => 0
			[size] => 24530
		)
	)
)

Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour.

If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model.

We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:

  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system.

Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems.

Advantages of storage in the file system:

  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB

The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system.

Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it.

I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space.

/**
 * Builds a semi random path based on the id to avoid having thousands of files
 * or directories in one directory. This would result in a slowdown on most file systems.
 *
 * Works up to 5 level deep
 *
 * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
 * @param mixed $string
 * @param integer $level
 * @return mixed
 * @access protected
 */
	protected function _randomPath($string, $level = 3) {
		if (!$string) {
			throw new Exception(__('First argument is not a string!', true));
		}

		$string = crc32($string);
		$decrement = 0;
		$path = null;
		
		for ($i = 0; $i < $level; $i++) {
			$decrement = $decrement -2;
			$path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2));
		}

		return $path;
	}

You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class.

I'll now explain you how to use this class to send files to the client.

/**
 * Sends a file to the client
 *
 * @param string $id UUID
 * @access public
 */
	public function download($id = null) {
		$this->Media->recursive = -1;
		$media = $this->Media->read(null, $id);

		if (empty($media)) {
		$this->redirect('/', 404, true);
		}
		
		$this->set('cache', '3 days');
		$this->set('download', true);
		$this->set('name', $media['Media']['slug']);
		$this->set('id', $media['Media']['filename']);
		$this->set('path', APP . 'media' . DS . $media['Media']['path']);
		$this->set('modified', $media['Media']['modified']);
		$this->set('mimeType', $media['Media']['mime_type']);
		$this->set('extension', $media['Media']['extension']);

		$this->view = 'Media';
		$this->autoLayout = false;
		if ($this->render() !== false) {
			$this->Media->updateAll(
				array('Media.downloads' => 'Media.downloads + 1'),
				array('Media.id' => $id));
		}
	}

You simply have to set autoLayout to false and the view class to media.

$this->view = 'Media';
$this->autoLayout = false;

There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false;

	$this->set('download', true);

You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching.

	$this->set('cache', '3 days');

The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server.

	$this->set('name', $media['Media']['slug']);
$this->set('id', $media['Media']['filename']);
$this->set('path', APP . 'media' . DS . $media['Media']['path']);

If you want to send a mime type that does not already in the MediaView class you can set it.

	$this->set('mimeType', $media['Media']['mime_type']);

If you don't set it, the class will try to determine the mime type by the extension.

	$this->set('extension', $media['Media']['extension']);

Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up.

When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download.

	if ($this->render() !== false) {
	$this->Media->updateAll(
	array('Media.downloads' => 'Media.downloads + 1'),
	array('Media.id' => $id));
}

 

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Latest articles

CakePHP 4 - First Look

Last december, the CakePHP team announced the immediate availability of 4.0.0. This release begins a new chapter for CakePHP, as 4.0 is now API stable. With this release, Cake 3.x moves into maintenance mode, while 2.x moves into security release mode. The promise of the version is: cleaner, faster and still tasty as usual. I had the opportunity to bake a new application from scratch and I will give my feedback about my process.  

Skeleton Design

The new version refreshes the skeleton design of the application. Now we have 2 new folders on root:
  • Templates

The templates folder has presentational files placed here: elements, error pages, layouts, and view template files. Pay attention for subfolders: 
  • Core templates are lowercase: cell, element, email, layout
  • App templates still uppercase: Error, Pages
  • Resources

The resources folder has subfolders for various types of resource files.  The locales* sub folder stores string files for internationalization.   If you are familiar with i18n, you will see the difference:
  • src/Locale/pt_BR/default.po (3.x)
  • resources/locales/pt_BR/default.po (4.x)
  Another important change was the .ctp files. They are moved for .php. CakePHP template files have a default extension of .php now. We have a new config/app_local.php file, which contains the configuration data that varies between environments and should be managed by configuration management, or your deployment tooling.  

PHP Strict Type Mode

In PHP the declare (strict_types = 1); directive enables strict mode. In strict mode, only a variable of exact type of the “type declaration” will be accepted, or a TypeError will be thrown. The only exception to this rule is that an integer may be given to a function expecting a float. This is a feature from PHP 7 - which we strongly recommended. All codebase from the skeleton and files generated by bake will include the function.  

Entities

The preferred way of getting new entities is using the newEmptyEntity() method: $product = $this->Products->newEmptyEntity();  

Authentication

After 10 years baking, that's a really big change for me. I’m not usually use plugins for authentication, I really like the Auth Component. I think many bakers would agree, as I remember on the first international meetup, the co-host shared the same opinion.   The Auth Component is deprecated, so it's better move on and save the good memories. The new way for implementing Authentication is more verbose. It requires a few steps, I don’t will detail that,  because you can easily check on book:
  • Install Authentication Plugin
  • Load the Plugin
  • Apply the Middleware
  • Load the Component
  My first look is like I said,  too verbose, for me anyway. We need to write a lot of code. Also it is not included on the skeleton of CakePHP applications, you need include by your own. https://book.cakephp.org/authentication/2/en/index.html  

HTTPS Enforcer Middleware

Contrary to the Authentication, I was really surprised how easy it was to force my Application to use HTTPS. If you are familiar with CakePHP, you will use the Security Component for that: class AppController extends Controller {      public function initialize()    {        parent::initialize();        $this->loadComponent('Security', [            'blackHoleCallback' => 'forceSSL',        ]);    }      public function beforeFilter(Event $event)    {        if (!Configure::read('debug')) {            $this->Security->requireSecure();        }    }      public function forceSSL()    {        return $this->redirect(            'https://' .            env('SERVER_NAME') .            Router::url($this->request->getRequestTarget())        );    }   }
  The implementation on version 4 is less verbose and easy, kudos for the new version:    public function middleware(MiddlewareQueue $middlewareQueue)    {        $middlewareQueue            ->add(new HttpsEnforcerMiddleware([                'redirect' => true,                'statusCode' => 302,                'disableOnDebug' => true,            ]));          return $middlewareQueue;    }   What I know is a drop, what I don’t know is an ocean. The new version is here to stay, and this article it's a just one overview of basic usage of the new version. * Version 4.1.0 is released already with more improvements and features.  

Links 

[1] Book https://book.cakephp.org/4/en/contents.html [2] Migration Guide https://book.cakephp.org/4/en/appendices/migration-guides.html  

CakeDC API plugin - Authentication and Authorization

This article covers new changes for CakePHP 4 version of plugin. So it covers versions starting from 8.x (8.0) and later.  

Permissions system. RBAC

By default, the plugin uses CakeDC Users and CakeDC Auth plugins for authentication. For RBAC it uses the same style as defined in the Auth plugin RBAC system with minor changes required for the API plugin. First, let's consider the case when we want public api without any authorization. In this case the most simple way would be is to define in config/api_permissions.php next rule   return [     'CakeDC/Auth.api_permissions' => [         [             'role' => '*',             'service' => '*',             'action' => '*',             'method' => '*',             'bypassAuth' => true,         ],      ], ];   Now, consider the case we want to use users plugin authentication. Since Api is supposed to be used from another domain, we should allow all requests with OPTIONS type. To do this we should add this rule as first on in config/api_permissions.php       [         'role' => '*',         'service' => '*',         'action' => '*',         'method' => 'OPTIONS',         'bypassAuth' => true,     ],    Here, method define OPTIONS and bypassAuth means that such actions should work for any users, including not authenticated. Now we should allow Auth service methods       [         'role' => '*',         'service' => '*',         'action' => ['login', 'jwt_login', 'register', 'jwt_refresh',],         'method' => ['POST'],         'bypassAuth' => true,     ],    All other services/actions should be declared in api_permissions file to define what user roles are allowed to access them. Imagine we want to allow the admin role to access the add/edit/delete posts and make index and view public. We can do it based on method or based on action names.       [         'role' => 'admin',         'service' => 'posts',         'action' => '*',         'method' => ['POST', 'PUT', 'DELETE'],     ],      [         'role' => 'admin',         'service' => 'posts',         'action' => ['index', 'view'],         'method' => '*',         'bypassAuth' => true,     ],   

 Routers and Middlewares

Starting from the 8.x version, API Plugin uses router middlewares. This gives great abilities to configure the plugin. So now it is possible to have separate authentication and authorization configuration for website and for api. Also, It is possible to have more then one api prefix, and as result provide more then single api for website with different configuration. Let’s take a look on the default configuration for middlewares   'Middleware' => [     'authentication' => [         'class' => AuthenticationMiddleware::class,         'request' => ApiInitializer::class,         'method' => 'getAuthenticationService',     ],     'bodyParser' => [         'class' => BodyParserMiddleware::class,     ],     'apiParser' => [         'class' => ParseApiRequestMiddleware::class,     ],     'apiAuthorize' => [         'class' => AuthorizationMiddleware::class,         'request' => ApiInitializer::class,         'params' => [             'unauthorizedHandler' => 'CakeDC/Api.ApiException',         ],     ],     'apiAuthorizeRequest' => [         'class' => RequestAuthorizationMiddleware::class,     ],     'apiProcessor' => [         'class' => ProcessApiRequestMiddleware::class,     ], ],   First we see the order of middlewares that proceed api request. It passes through AuthenticationMiddleware, AuthorizationMiddleware, and RequestAuthorizationMiddleware to perform generic auth tasks. It passes through BodyParserMiddleware to unpack the json request. And finally ParseApiRequestMiddleware does initial service analysis and ProcessApiRequestMiddleware performs the request. Also we can note CakeDC\Api\ApiInitializer class used to define Authentication and Authorization configuration. It can be redefined in the application layer to provide needed Identifiers and  Authenticators.  

 Jwt authentication - Refreshing tokens

New plugin feature is embedded jwt_login action which allows the user to get access_token and refresh_token included into the login response. Tokens should be passed in the Authorization header with bearer prefix. Access token is supposed to be used as default token and refresh token needed to get a new access token when it's expired. So for refreshing provided additional jwt_refresh action which should be used in this case.  

 Configuration

Configuration should be defined on application level in config/api.php. Need to note that it is important to enable this file to load by the Api plugin. It could be done in config/bootstrap_app.php using global configuration: Configure::write('Api.config', ['api']);       'Api' => [          ...                  'Jwt' => [             'enabled' => true,             'AccessToken' => [                 'lifetime' => 600,                 'secret' => 'accesssecret',             ],             'RefreshToken' => [                 'lifetime' => 2 * WEEK,                 'secret' => 'refreshsecret',             ],         ],    Hopefully, this was helpful. Our team is always working on adding new features and plugins. You can check out more available plugins HERE.

CakePHP Meetup: Unit Test Fixtures, Queue Plugin, PPM Bridge

Developers are used to living in a virtual world, so adjusting has been easier than expected. Recently, we’ve been holding virtual meetups, and we are so happy with the feedback. Digital training sessions allow bakers from all over the world to come together and enjoy. Our plan is to host one each month, and coordinate time zones so that everyone gets a chance to attend. Our latest one was based around a good time for our Japanese community.  If you missed the meetup, no problem. We always post the recording for playback, and I’ll even give you a quick rundown of the topics covered. Let’s jump in:

CakePHP Fixture Factory Plugin

by Juan Pablo Ramirez CakePHP Fixture Factory Plugin https://github.com/pakacuda/cakephp-fixture-factories  helps to improve the way fixtures are generated, when having a big database writing fixtures can get so complicated. This plugin provides Fixture Factories in replacement of the fixtures found out of the box in CakePHP.
Generating fixtures can be done in a few code lines reducing the effort of writing and maintaining tests. There are some other plugins to manage fixtures: 

CakePHP Queue Plugin

By Mark Scherer @dereuromark CakePHP Queue Plugin https://github.com/dereuromark/cakephp-queue is a simple Queue solution, it can be used for small applications and it’s a good one to get started with Job Queues, having something easy to maintain at the beginning is a good starting point.
Queues are a good option for functionalities like: image processing, email sending, PDF generation; to improve the response-time for heavy-processing tasks. For more robust solutions can be used:
  • CakePHP Queuesadilla  https://github.com/josegonzalez/cakephp-queuesadilla This plugin is a simple wrapper around the Queuesadilla queuing library, providing tighter integration with the CakePHP framework. We have used this plugin in CakeDC in several projects, we also had to build  a Mongo Engine for a specific client.

CakePHP PHP PM Bridge

By Jorge Gonzalez @steinkel CakePHP Bridge https://github.com/CakeDC/cakephp-phppm  to use with PHP-PM project.
PPM is a process manager, supercharger and load balancer for modern PHP applications. PHP PM It's based on ReactPHP, the approach of this is to kill the expensive bootstrap of PHP (declaring symbols, loading/parsing files) and the bootstrap of feature-rich frameworks.
It’s a good option If you want to significantly improve the responsiveness of an application that could have spikes. PM works as PHP FPM, it’s a replacement for it.  Below some benchmark:  50 Concurrent threads in 10 seconds
  • FPM 83 transactions per second, Failed 0,  Concurrency 6.58.
  • PPM 90.30 transactions per second, Failed 0, Concurrency 3.86.
200 Concurrent threads in 10 seconds
  • FPM 116,49 transactions per second, Failed 142,  Concurrency 116.64.
  • PPM 207.35 transactions per second, Failed 0, Concurrency 85.59.
1000 Concurrent threads in 10 seconds
  • FPM 109,88 transactions per second, Failed 1759, Concurrency 187.49.
  • PPM 214.91 transactions per second, Failed 0,  Concurrency 302.39.
PPM is able to handle a lot of concurrency connections coming in spike to the server  in a better way than PHP FPM.
For watching the Meetup visit the following link https://www.youtube.com/watch?v=POI0IwyqULo Stay up to date on all virtual meetups here  https://cakephp.org/pages/meetups      

We Bake with CakePHP