CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

File uploading, file storage and CakePHPs MediaView class

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this:

echo $form->create('Media', array('action' => 'upload', 'type' => 'file'));
echo $form->file('file');
echo $form->submit(__('Upload', true));

 

The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form.

When you submit the form now, you'll get data like this in $this->data of your controller:

Array
(
	[Media] => Array
	(
		[file] => Array
		(
			[name] => cake.jpg
			[type] => image/jpeg
			[tmp_name] => /tmp/hp1083.tmp
			[error] => 0
			[size] => 24530
		)
	)
)

Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour.

If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model.

We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:

  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system.

Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems.

Advantages of storage in the file system:

  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB

The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system.

Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it.

I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space.

/**
 * Builds a semi random path based on the id to avoid having thousands of files
 * or directories in one directory. This would result in a slowdown on most file systems.
 *
 * Works up to 5 level deep
 *
 * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
 * @param mixed $string
 * @param integer $level
 * @return mixed
 * @access protected
 */
	protected function _randomPath($string, $level = 3) {
		if (!$string) {
			throw new Exception(__('First argument is not a string!', true));
		}

		$string = crc32($string);
		$decrement = 0;
		$path = null;
		
		for ($i = 0; $i < $level; $i++) {
			$decrement = $decrement -2;
			$path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2));
		}

		return $path;
	}

You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class.

I'll now explain you how to use this class to send files to the client.

/**
 * Sends a file to the client
 *
 * @param string $id UUID
 * @access public
 */
	public function download($id = null) {
		$this->Media->recursive = -1;
		$media = $this->Media->read(null, $id);

		if (empty($media)) {
		$this->redirect('/', 404, true);
		}
		
		$this->set('cache', '3 days');
		$this->set('download', true);
		$this->set('name', $media['Media']['slug']);
		$this->set('id', $media['Media']['filename']);
		$this->set('path', APP . 'media' . DS . $media['Media']['path']);
		$this->set('modified', $media['Media']['modified']);
		$this->set('mimeType', $media['Media']['mime_type']);
		$this->set('extension', $media['Media']['extension']);

		$this->view = 'Media';
		$this->autoLayout = false;
		if ($this->render() !== false) {
			$this->Media->updateAll(
				array('Media.downloads' => 'Media.downloads + 1'),
				array('Media.id' => $id));
		}
	}

You simply have to set autoLayout to false and the view class to media.

$this->view = 'Media';
$this->autoLayout = false;

There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false;

	$this->set('download', true);

You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching.

	$this->set('cache', '3 days');

The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server.

	$this->set('name', $media['Media']['slug']);
$this->set('id', $media['Media']['filename']);
$this->set('path', APP . 'media' . DS . $media['Media']['path']);

If you want to send a mime type that does not already in the MediaView class you can set it.

	$this->set('mimeType', $media['Media']['mime_type']);

If you don't set it, the class will try to determine the mime type by the extension.

	$this->set('extension', $media['Media']['extension']);

Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up.

When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download.

	if ($this->render() !== false) {
	$this->Media->updateAll(
	array('Media.downloads' => 'Media.downloads + 1'),
	array('Media.id' => $id));
}

 

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Latest articles

Integrating Users and ACL plugins in CakePHP

In previous posts, we saw how CakeDC Users plugin can help you to build an application that manages everything related to users: registration, social login, permissions, etc. Recently it has been noted by the team that there are some use cases where a deeper control of permissions is needed - more than is offered in RBAC. Today we’ll go into this using the ACL approach. ACL or Access Control List, refers to the application using a detailed list of objects to decide who can access what. It can be as detailed as particular users and rows through to specifying which action can be performed (i.e user XX has permissions to edit articles but does not have permissions to delete articles). One of the big features of ACL is that both the accessed objects; and objects who ask for access, can be organized in trees. There’s a good explanation of how ACL works in the CakePHP 2.x version of the Book. ACL does not form part of CakePHP core V 3.0 and can be accessed through the use of the cakephp/acl plugin. Let’s just refresh the key concepts of ACL:

  • ACL: Access Control List (the whole paradigm)
  • ACO: Access Control Object (a thing that is wanted), e.g. an action in a controller: creating an article
  • ARO: Access Request Object (a thing that wants to use stuff), e.g. a user or a group of users
  • Permission: relation between an ACO and an ARO
For the purpose of this article - we shall use this use case: You are using CakeDC/users plugin and now want to implement ACL in your application.

Installation

Starting with a brand new CakePHP app: composer selfupdate && composer create-project --prefer-dist cakephp/app acl_app_demo && cd acl_app_demo We are going to use CakeDC/users and cakephp/acl plugins. In a single step we can install them with composer: composer require cakedc/users cakephp/acl Create a DB and set its name and credentials in the config/app.php file of the just created app (in the Datasources/default section). This command can help you out if you are using MySQL: mysql -u root -p -e "create user acl_demo; create database acl_demo; grant all privileges on acl_demo.* to acl_demo;" Plugins will be loaded always with the app. Let’s set them on the bootstrap file: bin/cake plugin load -br CakeDC/Users
bin/cake plugin load -b Acl Now let’s insert a line in bootstrap.php before Users plugin loading, so cakedc/users will read the configuration from the config/users.php file of our app. Configure::write('Users.config', ['users']); This file does not exist yet. The plugin provides a default file which is very good to start with. Just copy it to your app running: cp -i vendor/cakedc/users/config/users.php config/ Also, let’s copy the permissions file the same way to avoid warnings in our log files: cp -i vendor/cakedc/users/config/permissions.php config/ We need to change cakedc/users config: remove RBAC, add ACL. In cakephp/acl there’s ActionsAuthorize & CrudAuthorize. We’ll start just using ActionsAuthorize. We will tell ActionsAuthorize that actions will be under the 'controllers/' node and that the users entity will be MyUsers (an override of the Users entity from the plugin). Edit the Auth/authorize section of config/users.php so that it sets: 'authorize' => [ 'CakeDC/Auth.Superuser', 'Acl.Actions' => [ 'actionPath' => 'controllers/', 'userModel' => 'MyUsers', ], ], Add calls to load components both from Acl & Users plugin in the initialize() method in AppController: class AppController extends Controller { public function initialize() { parent::initialize(); // (...) $this->loadComponent('Acl', [ 'className' => 'Acl.Acl' ]); $this->loadComponent('CakeDC/Users.UsersAuth'); // (...) } // (...) }

Database tables

Some tables are required in the database to let the plugins work. Those are created automatically just by running their own migrations: bin/cake migrations migrate -p CakeDC/Users
bin/cake migrations migrate -p Acl One table from the Acl plugin needs to be fixed because Users migration creates users.id as UUID (CHAR(36)) and Acl migrations creates AROs foreing keys as int(11). Types must match. Let’s fix it adapting the aros table field: ALTER TABLE aros CHANGE foreign_key foreign_key CHAR(36) NULL DEFAULT NULL; Now, it’s time to set our own tables as needed for our app. Let’s suppose we are developing a CMS app as specified in the CMS Tutorial from the CakePHP book. Based on the tutorial, we can create a simplified articles table: CREATE TABLE articles ( id INT AUTO_INCREMENT PRIMARY KEY, user_id CHAR(36) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL, title VARCHAR(255) NOT NULL, body TEXT, published BOOLEAN DEFAULT FALSE, created DATETIME, modified DATETIME, FOREIGN KEY user_key (user_id) REFERENCES users(id) ); Note: Specify CHARACTER SET and COLLATE for user_id only if the table CHARACTER SET and COLLATE of the table differ from users.id (than may happen running migrations). They must match. Roles will be dynamic: admin will be allowed to manage them. That means that they has to be stored in a table. CREATE TABLE roles ( id CHAR(36) NOT NULL PRIMARY KEY, name VARCHAR(100) NOT NULL, created DATETIME, modified DATETIME ); Association between users and roles bill be belongsTo, so we’ll need a foreign key in the users table instead of a role varchar field: ALTER TABLE users ADD role_id CHAR(36) CHARACTER SET latin1 COLLATE latin1_swedish_ci NULL DEFAULT NULL AFTER role, ADD INDEX role_id (role_id), ADD FOREIGN KEY (role_id) REFERENCES roles(id); ALTER TABLE users DROP role;

Baking

Time to think about what will be ACOs and AROs. In most cases, Users will be the only AROs. To do that, we need to link the Users entity and table to the ACL plugin. In this case that we are using CakeDC/users plugin, we first need to extend the plugin as it is explained in the docs. We will also add the behavior and parentNode() as shown in the cakephp/acl readme file, so at the end we’ll need to create those files: src/Model/Entity/MyUser.php: <?php namespace App\Model\Entity; use CakeDC\Users\Model\Entity\User; /** * Application specific User Entity with non plugin conform field(s) */ class MyUser extends User { public function parentNode() { return ['Roles' => ['id' => $this->role_id]]; } } src/Model/Table/MyUsersTable.php: <?php namespace App\Model\Table; use CakeDC\Users\Model\Table\UsersTable; class MyUsersTable extends UsersTable { public function initialize(array $config) { parent::initialize($config); $this->addBehavior('Acl.Acl', ['requester']); $this->belongsTo('Roles'); $this->hasMany('Articles'); } } Run bin/cake bake controller MyUsers (beware of case) Then, edit the top of src/Controller/MyUsersController.php as: <?php namespace App\Controller; use App\Controller\AppController; use CakeDC\Users\Controller\Traits\LinkSocialTrait; use CakeDC\Users\Controller\Traits\LoginTrait; use CakeDC\Users\Controller\Traits\ProfileTrait; use CakeDC\Users\Controller\Traits\ReCaptchaTrait; use CakeDC\Users\Controller\Traits\RegisterTrait; use CakeDC\Users\Controller\Traits\SimpleCrudTrait; use CakeDC\Users\Controller\Traits\SocialTrait; class MyUsersController extends AppController { use LinkSocialTrait; use LoginTrait; use ProfileTrait; use ReCaptchaTrait; use RegisterTrait; use SimpleCrudTrait; use SocialTrait; // CRUD methods ... To generate the template files for MyUsers we can run: bin/cake bake template MyUsers Next, just let Cake bake all objects for articles and roles: bin/cake bake all Articles
bin/cake bake all Roles Add behavior to their tables. ArticlesTable will act as controlled because it will represent ACOs: class ArticlesTable extends Table { public function initialize(array $config) { parent::initialize($config); // (...) $this->addBehavior('Acl.Acl', ['controlled']); // (...) The case of RolesTable will be similar but it will act as requester, as it will represent AROs: class RolesTable extends Table { public function initialize(array $config) { parent::initialize($config); // (...) $this->addBehavior('Acl.Acl', ['requester']); // (...) Create the parentNode() method in both entities: Article and Role. public function parentNode() { return null; }

Testing

Ok, time to test the whole system! At this point, the app should be ready to use. At least, for an administrator. Let’s quickly create one: it is as easy as running bin/cake users add_superuser. New credentials will appear on screen. When accessing our app in the URL that we installed it, a login form will appear. Log as the just created admin. First, let’s create some roles. Go to /roles in your app’s URL. Then, click on "New Role". Create the roles:
  • Author
  • Editor
  • Reader
Then, we can create two users an author and a reader. Head to /my-users and add them. Remember to select the Active checkbox and the proper role in the dropdown menu. Because MyUsers has the AclBehavior, AROs has been automatically created while creating users, along with the created roles. Check it out with bin/cake acl view aro Aro tree: --------------------------------------------------------------- [1] Roles.24c5646d-133d-496d-846b-af951ddc60f3 [4] MyUsers.7c1ba036-f04b-4f7b-bc91-b468aa0b7c55 [2] Roles.5b221256-0ca8-4021-b262-c6d279f192ad [3] Roles.25908824-15e7-4693-b340-238973f77b59 [5] MyUsers.f512fcbe-af31-49ab-a5f6-94d25189dc78 --------------------------------------------------------------- Imagine that we decided that authors will be able to write new articles and readers will be able to view them. First, let’s create the root node for all controllers: bin/cake acl create aco root controllers Then, let’s inform ACL that there are such things as articles: bin/cake acl create aco controllers Articles Now, we will tell that there are 5 actions related to Articles: bin/cake acl create aco Articles index bin/cake acl create aco Articles view bin/cake acl create aco Articles add bin/cake acl create aco Articles edit bin/cake acl create aco Articles delete We can see the first branch of the ACOs tree here: bin/cake acl view aco Aco tree: --------------------------------------------------------------- [1] controllers [2] Articles [3] index [4] view [5] add [6] edit [7] delete --------------------------------------------------------------- ACL knows that articles can be added, so let’s tell who can do that. We can check which aro.id belongs to role Author with: mysql> select id from roles where name like 'Author'; +--------------------------------------+ | id | +--------------------------------------+ | 24c5646d-133d-496d-846b-af951ddc60f3 | +--------------------------------------+ 1 row in set (0.00 sec) And the same with the Reader role:: mysql> select id from roles where name like 'Reader'; +--------------------------------------+ | id | +--------------------------------------+ | 25908824-15e7-4693-b340-238973f77b59 | +--------------------------------------+ 1 row in set (0.00 sec) So, if we look up this id in the bin/cake acl view aro output, it turns out that aro.id 1 is Author and that aro.id 3 is Reader. If we want to let authors (ARO 1) add articles (ACO 5), we must grant permission to Articles/add to editors by running: bin/cake acl grant 1 5 And we'll grant readers (ARO 3) view articles (ACO 4) with: bin/cake acl grant 3 4 Don't forget to grant access to Articles/index for all roles, or nobody would access /articles: bin/cake acl grant 1 3 bin/cake acl grant 2 3 bin/cake acl grant 3 3 Note: Obviously, it would be easier to set a "super role" which includes the 3 roles and grant access to index to it, but we don't want to add too many steps in this tutorial. You can try it for yourself. Then, aros_acos table becomes: mysql> select * from aros_acos; +----+--------+--------+---------+-------+---------+---------+ | id | aro_id | aco_id | _create | _read | _update | _delete | +----+--------+--------+---------+-------+---------+---------+ | 1 | 1 | 5 | 1 | 1 | 1 | 1 | | 2 | 3 | 4 | 1 | 1 | 1 | 1 | | 3 | 1 | 3 | 1 | 1 | 1 | 1 | | 4 | 2 | 3 | 1 | 1 | 1 | 1 | | 5 | 3 | 3 | 1 | 1 | 1 | 1 | +----+--------+--------+---------+-------+---------+---------+ 5 rows in set (0.00 sec) Let’s create a new article as the first user. To do that:
  • Log out (we are still logged in as superadmin) going to /logout
  • Log in as the first created user
  • Go to /articles
  • Create an article
Right now, author can add an article but not view it, since we only set the add permission. Check it out clicking in View next to the article. Log in as a reader to check how the reader can really view the article. Obviously, more than a couple of permissions have to be grant in a big app. This tutorial served just as an example to start.

Last words

That's all for now related to the use of ACL in a webapp made with CakePHP. A lot more can be done with ACL. Next step would be to use CrudAuthorize to specify which CRUD permissions are granted for any ARO to any ACO. Keep visiting the blog for new articles! This tutorial has been tested with:
  • CakePHP 3.5.10
  • CakeDC/users 6.0.0
  • cakephp/acl 0.2.6
An example app with the steps followed in this tutorial is available in this GitHub repo. Please let us know if you use it, we are always improving on them - And happy to get issues and pull requests for our open source plugins. As part of our open source work in CakeDC, we maintain many open source plugins as well as contribute to the CakePHP Community. Reference

Using a vagrant box as quick environment for the Getting Started with...

We've decided to create a simple vagrant box with all the required packages to improve the environment setup step in our free Getting Started with CakePHP training session. We used other tools in the past, but we hope vagrant will help users to install a common environment before the session to get the most of it.

Requirements

Setup

  • Create a new folder where the code will be located
  • Create a new file called Vagrantfile with the following contents
# -*- mode: ruby -*- # vi: set ft=ruby : Vagrant.configure("2") do |config| config.vm.box = "cakedc/cakephp-training" config.vm.network :forwarded_port, guest: 8765, host: 8765 config.vm.provider "virtualbox" do |vb| vb.memory = "1024" vb.customize ['modifyvm', :id, '--cableconnected1', 'on'] end end
  • Run vagrant up
  • Wait (download could take several minutes depending on your internet connection)
  • Run vagrant ssh
Now you have ssh access to a training ubuntu (16.04) based virtual machine, with all the requirements to run your training CakePHP application.
  • Setup a new CakePHP project
cd /vagrant composer create-project cakephp/app
  • Start the local server
cd /vagrant/app php bin/cake.php server --host 0.0.0.0
  • From your host machine, open a browser and navigate to http://localhost:8765
  • You should be able to see the CakePHP welcome page
  We think this VM will enable faster environment setups, and an easier entry point to the training session. Please let us know if you find issues with this process.

Boosting your API with CakePHP API and PHP-PM (ReactPHP)

A couple days ago AlexMax commented in CakePHP's IRC channel about the https://github.com/php-pm/php-pm project and it rang a bell for us. We did a couple tests internally and found this could be a great companion to our API plugin, so we wrote a new Bridge for CakePHP and ran some benchmarks.

The Cast

We put all together and created a sample application (1 posts table with 30 records) to do some benchmarks.

Benchmark configuration

We are not aiming to provide detailed or production figures, just a reference of the results obtained for your comparison. Results are generated from a development box, using PHP 7.1.12-3+ubuntu16.04.1+deb.sury.org+1 with xdebug enabled on ubuntu xenial, 8x Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz We baked the application using the latest CakePHP 3.5.10, and set application debug to false, and log output to syslog. As we are interested in boosting API response times the most, we tested the following scenarios
  • A) CakePHP json output, served from nginx+phpfpm
  • B) CakePHP + API Plugin Middleware integration json output, served from nginx+phpfpm
  • C) CakePHP + API Plugin Middleware integration json output, served from php-pm
Benchmark figures were obtained using ab -n 5000 -c 100 URL

Results

Scenario requests/second avg time
A) CakePHP json output, served from nginx+phpfpm using php7.1 372.97 [#/sec] (mean) 268.120 [ms] (mean)
B) CakePHP + API Plugin Middleware integration json output, served from nginx+phpfpm using php7.1 399.79 [#/sec] (mean) 250.133 [ms] (mean)
C) CakePHP + API Plugin Middleware integration json output, served from php-pm using php7.1 911.95 [#/sec] (mean) 109.656 [ms] (mean)
D) CakePHP + API Plugin Middleware integration json output, served from php-pm using php7.2 1811.66 [#/sec] (mean) 55.198 [ms] (mean)
  These results for a NOT OPTIMIZED CakePHP application are promising, and the improvement using PHP-PM is huge in this case. There are some important considerations though:
  • PHP-FPM is mature and stable, PHP-PM is still in early development, although there is a 1.0 version released already.
  • Processes need monitoring, specially regarding memory leaks, we would need to manage a restart policy and be able to hot-restart individual workers
  • System integration, init scripts are not provided, even if this is something easy to manage nowadays via systemd or monit, would be good to have for production
  • Application bootstrapping should not be affected by the request. If your application bootstrapping depends on the request params, or logged in user, you'll need to refactor your code
  • Session handling was not tested, issues are reported for PHP-PM for other frameworks. We were aiming to stateless API's so we don't know if this would be an issue for a regular application
Performance is always a concern for the API developer, applying proven paradigms like the event driven development (https://reactphp.org/) to your existing code would be the way to go and ensure backend frameworks like CakePHP will perform as required when dealing with the peaks we all love and hate. Edit: We've added a php7.2 based benchmark, with a huge performance improvement.

Giving back to the community

This Plugin's development has been sponsored by the Cake Development Corporation. Contact us if you are interested in:      

We Bake with CakePHP