CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

File uploading, file storage and CakePHPs MediaView class

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this:

echo $form->create('Media', array('action' => 'upload', 'type' => 'file'));
echo $form->file('file');
echo $form->submit(__('Upload', true));

 

The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form.

When you submit the form now, you'll get data like this in $this->data of your controller:

Array
(
	[Media] => Array
	(
		[file] => Array
		(
			[name] => cake.jpg
			[type] => image/jpeg
			[tmp_name] => /tmp/hp1083.tmp
			[error] => 0
			[size] => 24530
		)
	)
)

Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour.

If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model.

We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:

  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system.

Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems.

Advantages of storage in the file system:

  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB

The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system.

Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it.

I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space.

/**
 * Builds a semi random path based on the id to avoid having thousands of files
 * or directories in one directory. This would result in a slowdown on most file systems.
 *
 * Works up to 5 level deep
 *
 * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
 * @param mixed $string
 * @param integer $level
 * @return mixed
 * @access protected
 */
	protected function _randomPath($string, $level = 3) {
		if (!$string) {
			throw new Exception(__('First argument is not a string!', true));
		}

		$string = crc32($string);
		$decrement = 0;
		$path = null;
		
		for ($i = 0; $i < $level; $i++) {
			$decrement = $decrement -2;
			$path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2));
		}

		return $path;
	}

You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class.

I'll now explain you how to use this class to send files to the client.

/**
 * Sends a file to the client
 *
 * @param string $id UUID
 * @access public
 */
	public function download($id = null) {
		$this->Media->recursive = -1;
		$media = $this->Media->read(null, $id);

		if (empty($media)) {
		$this->redirect('/', 404, true);
		}
		
		$this->set('cache', '3 days');
		$this->set('download', true);
		$this->set('name', $media['Media']['slug']);
		$this->set('id', $media['Media']['filename']);
		$this->set('path', APP . 'media' . DS . $media['Media']['path']);
		$this->set('modified', $media['Media']['modified']);
		$this->set('mimeType', $media['Media']['mime_type']);
		$this->set('extension', $media['Media']['extension']);

		$this->view = 'Media';
		$this->autoLayout = false;
		if ($this->render() !== false) {
			$this->Media->updateAll(
				array('Media.downloads' => 'Media.downloads + 1'),
				array('Media.id' => $id));
		}
	}

You simply have to set autoLayout to false and the view class to media.

$this->view = 'Media';
$this->autoLayout = false;

There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false;

	$this->set('download', true);

You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching.

	$this->set('cache', '3 days');

The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server.

	$this->set('name', $media['Media']['slug']);
$this->set('id', $media['Media']['filename']);
$this->set('path', APP . 'media' . DS . $media['Media']['path']);

If you want to send a mime type that does not already in the MediaView class you can set it.

	$this->set('mimeType', $media['Media']['mime_type']);

If you don't set it, the class will try to determine the mime type by the extension.

	$this->set('extension', $media['Media']['extension']);

Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up.

When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download.

	if ($this->render() !== false) {
	$this->Media->updateAll(
	array('Media.downloads' => 'Media.downloads + 1'),
	array('Media.id' => $id));
}

 

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Latest articles

How to take a rest in five minutes

There are a lot of toys these days. Let me show you one very simple and powerful brick for your next revolutionary invention. Let's assume that you're either:

  • A curious person, and you like to play with things you don't know, just for the case
  • A frontend developer with no PHP coding experience, and your javascript components could use some dynamic backend json resource. Maybe you want to do this without playing with all mocking options or Node.js modules out there...and you don't want to set up MySQL database, user, grant permissions and all that stuff
  • building dynamic user interface, and your code needs to be aware of continuously changing structure of databases using unpredictable table names
  • A creator of a wireless network of IoT devices or swarm of flying educational robots. Or, looking for some stable and performant data backend running on your central Raspberry Pi Zero W with tmpfs mounts and ridiculously huge memory card, while sniffing around for bluetooth signatures
  • aware of services like ngrok and wanting to consume payloads of some IFTTT webhooks locally. Or, allowing a friend with Postman to access and/or feed up your database created (or temporarily exposed) just for that reason.
  • Having a rich html application and flirting with the idea of full blown CakePHP backend running in kiosk mode of the DraftCode application right on your iOS device
  • able to pack a PHP application as a single file Phar archive. For example: to simplify all sorts of domain logic/responsibility distribution across bazillions of machines spawned on demand and utilizing jq in your provisioning scripts
  • praying for headless access to any database that your CakePHP application is able connect to, bypassing implemented validation, callbacks, events, and even sneaking through your authentication/authorization backdoor if you want to
  • going to try following example right away, or at least very soon
If your machine meets the CakePHP Requirements, and you have composer in your $PATH then we are good to go for a CakePHP application using SQLite database accessible through REST. This would have obvious api goodies like CRUD, CORS, HATEOAS, versioning, list, describe, relations, sorting and pagination included by default.

Clock is ticking, time to rest with some book(s)

I don't know how much time of the rest limit we lost on the preface (quick learners are quick readers), but don't worry, unless you are on the slow internet connection and packagist is taking a break - we are almost done. We have a clear picture of what we're going to create, so the rest is a piece of cake.
  • create a CakePHP 4.x project named rest with CakeDC/Api plugin unlocked and loaded:
composer create-project \ --prefer-dist --no-interaction \ cakephp/app:~4.0 rest cd $_ composer config \ minimum-stability dev composer require \ cakedc/cakephp-api:~8.0
  • Unlock and load! Auth bypass, "allow all" switch (don't try this at home)
cp vendor/cakedc/cakephp-api/config/api_permissions.php.default \ ./config/api_permissions.php bin/cake plugin load CakeDC/Api
  • manually (meh, we should be resting now) configure Database.default connection in the file config/app_local.php
'database' => TMP . 'rest.sqlite', 'driver' => 'Cake\Database\Driver\Sqlite',
  • create example table books in the database and start local server
bin/cake bake migration create_books \ title:string:unique \ description:text \ price:integer:index \ available:boolean:index \ created \ modified bin/cake migrations migrate bin/cake server Open http://localhost:8765/api/books in your browser to see the (empty) list of your books, then pick some RESTful Route and take a rest from the back end, even without backend coding, scratching who knows where, etc. You maybe have some SQLite relation databases laying around, and that would be also worth a try. I forgot to remind you to stop that 300s timer, sorry. You will get lost in time as well, once you'll realize what all is possible with the CakeDC/Api plugin using little to no configuration, and what dimensions are opening to you. If you combine its powerful multilayer services approach with solid implementation of your custom ORM classes embraced by CakePHP core and conventions, the results are amazing. Oh, and a quick note for CakePHP 3.x developers - we like you, but please take your time to upgrade your toolset. Feel free to ask us for help, or use following warranty-limited commands: composer create-project \ --prefer-dist --no-interaction \ cakephp/app:^3.9 rest composer require \ cakedc/cakephp-api:~7.0 bin/cake plugin load -b -r CakeDC/Api

What's next

CakeFest 2020 of course. I hope you all are as excited as we are, together in these hard times. Be safe and take care, folks! And rest too, even if it's a quick one :-)

CakeFest Insider

In case we haven’t reminded you enough lately, CakeFest 2020 is less than a month away. While we have recently hosted a few virtual meetups, a fully virtual conference is uncharted territory for our team. I like to look on the bright side of things, so I will say how excited I am that bakers from all over the world will be able to join in and participate this year. Obviously, with international travel, this hasn’t always been the case for individuals. So my last CakeFest blog, I went into details of what to expect, and how the conference will (hopefully) run - smoothly. However, we’ve had a lot of emails and interest in hearing about what topics will be covered. Our lineup is excellent this year - so we wanted to share some things to look forward to:  

SPONSORS

First, we have to give a shout out to this year’s sponsors Cake Development Corporation  Passbolt Mark’s Software  RingCentral Companies that support Open Source are essential for our communities to move forward and grow!   

SCHEDULE

Day 1 will feature our popular workshops. These sessions are different than normal conference talks because they are basically training sessions to help grow your recipe book. You can see examples of previous years HERE. There will be basic workshops, as well as advanced during the full day session. We have condensed this year’s duration, so what is normally 2 days of workshops will be done in 2 hour sessions by each baker.  Mark Story, Jorge Gonzalez, and Jose Lorenzo are back. This year, we’ve also recruited another core member, Mark Scherer… you may be familiar with Mark from hosting many of our virtual meetups.  Topics that our core members MIGHT include are (these are being discussed): CakePHP 4.x, as well as project examples -  a repost builder, a headless micro cms, a media server to upload/download cache files from S3. The speaker’s will build through a project with you, answering questions as they progress.  Have an idea you’d like the workshops to include? Email us! Cakefest@cakephp.org.
  Now, day 2. This year, we decided to go ahead and try a few new things. Of course being 100% digital, condensing times, and some topics and speakers that may be unexpected. I personally invited some speakers that I thought could bring something fresh to the tech conference table.  One of these being OSMI (Open Source Mental Illness). I wanted to bring mental health importance to light, and this group is doing amazing things for the development world. Mental health doesn’t have to be a taboo subject. Dr. Jennifer Akullian will be talking about mental health, stress, and burnout in the tech industry. In addition to an overview of the research, distinctive considerations of the industry we work in will be discussed, as well as how to improve the management of common stressors in technology during an incomparable time in the world. We will also hear how some people got their start, like Michael Hoffman, and learn how to build solid architecture with CakePHP Plugins thanks to Ed Barnard. Ever wondered how to release your own plugin? Jose Gonzalez will shed some light. We are also delighted to have one of our involved Japanese community bakers, Junichi Okuyama, joining us as a speaker this year talking about helpful tips that he has learned for baking with CakePHP. Our keynote will be given by our diamond sponsor representative from Passbolt, Remy Bertot.  Other talks will include: well known podcast host Cal Evans talking about all of the cool things that can be done with PHP's built in DateTime math, Mariano Iglesias,  core members Mark Story, Chris Nizzardini and Mark Scherer. Popular contributors Juan Pablo Ramirez and Nicolas Masson will join us and share more details about CakePHP fixture factories.   You can see the full schedule HERE, and plan your attendance accordingly. 
 

SLACK CHATS 

We will also have slack chat rooms for attendees. This will give everyone the opportunity to interact with other watchers as well as speakers. Questions are welcomed before, during, and after the event. We will have team members monitoring the chat and trying to get all questions answered as soon as possible.   Slack channel tag: #cakefest  So basically, have a coffee and a chat with other attendees and presenters.    Have you purchased your ticket yet? WHAT ARE YOU WAITING FOR?    

Upgrading to CakePHP 4

As you know, CakePHP announced the version 4.x last December.I recommend that you consider upgrading your applications to the next version, to keep up to date and get all the benefits. Now, let's see how to bake!  

Step 1: Upgrade PHP

First things first, if you are not running on PHP 7.2 or higher, you will need to upgrade PHP before updating CakePHP. CakePHP 4.0 requires a minimum of PHP 7.2.  

Step 2: Upgrade Templates and Resources

There is an upgrade CLI tool for rename and moving the templates and resources:   Templates and Resources must have been moved and renamed, check the result below: * This project doesn't have Resources files   Now, let's create a new constant for Resources on /config/paths.php: Finally, update the paths on config/app.php:  

Step 3: Upgrade CakePHP

The next step is optional (and the Migration Guide included this) - run the rector command to automatically fix many deprecated method calls: The rector applied on codebase some return type declarations: https://github.com/rafaelqueiroz/cakephp-upgrade-sample/commit/d7e5c2ecc5dc28045700a270721f07098a8e189c?branch=d7e5c2ecc5dc28045700a270721f07098a8e189c&diff=split Pay attention: It is important to apply rector before you upgrade your dependencies.   Upgrade CakePHP and PHPUnit: PHPUnit can be upgraded easily. Most of the time, the --update-with-dependencies doesn’t work with me for CakePHP: The root of the issue is the packages using Caret Version Range, so let’s update debug_kit, migrations and bake using editor:   Here we go:   Now, let see how the project looks: Here, we have few deprecations and warnings. Do you remember I mentioned the rector is optional? So, the question is the rector and it's not always able to handle these issues.   I will use the PHPStan to fix this - we will install with composer: Now, we can run the phpstan analyse and fix the issues:   It's up to you how much effort you will put in with PHPStan issues. I recommend fixing everything. For this post, I did fix only what was needed to run the project after the update, you can check the fixes on this commit.   After the last fixes, the project is running well:  That’s all? No. But we upgraded CakePHP? Yes. Real applications probably use many plugins, and if these plugins don't have a version for CakePHP 4, you will need to update. Depending on the size and level of complexity of the project, the upgrade could be hard, but never impossible.    If you do not feel confident or your company would like to outsource support for this, don't hesitate to contact us at Cake Development Corporation. Our team is offering a full upgrade from CakePHP 2/3 to CakePHP 4. This will be a migration of your current application code to make it compatible with CakePHP 4 features, plugins, security settings, etc. We will be doing these migration services for a special rate - something we have never done before! Learn more about our Upgrade Services You can check the codebase of the examples on this repository. The branch upgrade has all steps by commit.  With every release CakePHP gets better, and version 4.x is no exception. There are many benefits that come with upgrading, and it makes baking a lot easier.

We Bake with CakePHP