CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

File uploading, file storage and CakePHPs MediaView class

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this:

echo $form->create('Media', array('action' => 'upload', 'type' => 'file'));
echo $form->file('file');
echo $form->submit(__('Upload', true));

 

The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form.

When you submit the form now, you'll get data like this in $this->data of your controller:

Array
(
	[Media] => Array
	(
		[file] => Array
		(
			[name] => cake.jpg
			[type] => image/jpeg
			[tmp_name] => /tmp/hp1083.tmp
			[error] => 0
			[size] => 24530
		)
	)
)

Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour.

If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model.

We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:

  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system.

Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems.

Advantages of storage in the file system:

  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB

The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system.

Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it.

I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space.

/**
 * Builds a semi random path based on the id to avoid having thousands of files
 * or directories in one directory. This would result in a slowdown on most file systems.
 *
 * Works up to 5 level deep
 *
 * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
 * @param mixed $string
 * @param integer $level
 * @return mixed
 * @access protected
 */
	protected function _randomPath($string, $level = 3) {
		if (!$string) {
			throw new Exception(__('First argument is not a string!', true));
		}

		$string = crc32($string);
		$decrement = 0;
		$path = null;
		
		for ($i = 0; $i < $level; $i++) {
			$decrement = $decrement -2;
			$path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2));
		}

		return $path;
	}

You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class.

I'll now explain you how to use this class to send files to the client.

/**
 * Sends a file to the client
 *
 * @param string $id UUID
 * @access public
 */
	public function download($id = null) {
		$this->Media->recursive = -1;
		$media = $this->Media->read(null, $id);

		if (empty($media)) {
		$this->redirect('/', 404, true);
		}
		
		$this->set('cache', '3 days');
		$this->set('download', true);
		$this->set('name', $media['Media']['slug']);
		$this->set('id', $media['Media']['filename']);
		$this->set('path', APP . 'media' . DS . $media['Media']['path']);
		$this->set('modified', $media['Media']['modified']);
		$this->set('mimeType', $media['Media']['mime_type']);
		$this->set('extension', $media['Media']['extension']);

		$this->view = 'Media';
		$this->autoLayout = false;
		if ($this->render() !== false) {
			$this->Media->updateAll(
				array('Media.downloads' => 'Media.downloads + 1'),
				array('Media.id' => $id));
		}
	}

You simply have to set autoLayout to false and the view class to media.

$this->view = 'Media';
$this->autoLayout = false;

There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false;

	$this->set('download', true);

You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching.

	$this->set('cache', '3 days');

The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server.

	$this->set('name', $media['Media']['slug']);
$this->set('id', $media['Media']['filename']);
$this->set('path', APP . 'media' . DS . $media['Media']['path']);

If you want to send a mime type that does not already in the MediaView class you can set it.

	$this->set('mimeType', $media['Media']['mime_type']);

If you don't set it, the class will try to determine the mime type by the extension.

	$this->set('extension', $media['Media']['extension']);

Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up.

When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download.

	if ($this->render() !== false) {
	$this->Media->updateAll(
	array('Media.downloads' => 'Media.downloads + 1'),
	array('Media.id' => $id));
}

 

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Latest articles

Why Choosing The Right Development Team Is Important

 Listen, we get it… it is the age of do-it-yourself everything, but one thing that I will remain firm on is the fact that a good web presence (and web application/website) is absolutely essential for any business. The fast track to a good application starts with choosing the right development team. Whether you decide as a business owner to have an internal development team, or outsource the work, it is a very important action for success.   Having a user friendly site will do just that, get you friends. Or in this case, “traffic” to your webpage. What can a development company or team do that those “quick and easy build a site” tools can’t? A lot. But firstly, they can build custom features and integrations. If someone visits your site, it is likely they will stick around if there are things to do/see/read, thus decreasing your “bounce rate” - which is a nice way to say clicking the little red x or back button.    Another benefit to choosing the right development team is having someone on hand to work on debugging, error fixing, and training for maintainers you may have on your team. For example, with CakeDC, we offer consulting, project management and training. This means your project manager can be with you for the long term (always available to fix those ERROR codes or upgrade), or train your tech team to manage the application themselves. Development is a long term commitment.  Our team keeps clients accompanied during all stages of development, from the initial call and vision building, to implementation and delivery… and even offer after build management in case there is a future need. It is important to make sure your site/application is always running smoothly and dedicate time to enhancing it. Nothing is more frustrating to a potential client or consumer than trying to run an action and getting broken links or useless site features. A good developer will ensure security is enabled as well (number one priority). The more features you have, and the easier they are to use, means the more conversions and leads you can get and the easier they are to obtain.   Hiring a team does not always mean things will be perfect, there are so many options available: commercial teams, freelancers, internal tech departments… so it is important to do your research before making the plunge. When speaking to a lot of our potential clients, a common complaint I get is that developers are not accessible (or at least not quickly). Many times, I hear the same story… a company hires a freelance developer working on their own time, the freelancer builds the initial project, then disappears and the company is left with a site full of errors. Our team focuses a lot on communication with clients, and we make ourselves reachable at any time. We even have developers in different timezones to suit the needs of clients, and we keep them in the loop every step of the way. CakeDC has a project management system that is accessible to every client so that they are able to track work and time being used. Not to mention, we chat via email, skype, slack, phone… you name it.    Plus, our team works together, that way the work promised gets delivered.     Basically, what I am trying to say is to choose a development team, and choose wisely. The team should start with asking your needs and wants, and your vision for the project. An experienced development team can bring that project to life. The decision you make now, can save time, money, and many meltdowns in the future. Don’t say I didn’t warn you (just kidding…kinda).   

Users plugin 9.x for CakePHP4

CakePHP 4 is out for some time (4.0.2 at the time of writing this post) and some of you already asked "Is there a cakedc/users version for cake4?" a couple times... The answer is YES!. We had a meeting after we realized AuthComponent was going to be deprecated, the authentication/authorization layer was leaning towards
Middlewares and getting a new Plugin home ourside of the core.
We decided to embrace the new middlewares and provide native compatibility from version 9.x of the plugin. Before we explain how is it going to work, and proceed to setup a project from scratch a couple answers to potential questions: * Are you going to maintain cakedc/users 8.x for CakePHP 3.x?
  * Yes, as long as you use it, we'll keep it maintained. Version 8.x will be a LTS version. We'll consider backporting some features from 9.x but the
  main objective of 8.x is to provide stability over time to existing CakePHP 3 projects.
* I'm planning to migrate my project from CakePHP 3 to CakePHP 4, what's the migration path for cakedc/users?
  * Keep the version 8.x of the plugin for some time. Complete your upgrade and then decide if you want to migrate to cakedc/users 9.x.
  We wrote a migration guide here when you decide it's time to move forward.
* AuthComponent is going to explode in CakePHP 4.x ?
  * No, it's deprecated, but deprecated also means it's completely functional. Just keep it until you decide to upgrade it.
 
This blog post is the first of a series of articles about cakedc/users 9.x where we'll explain how to use the plugin from scratch, it's benefits and
also the way the new authentication & authorization layers work for a CakePHP 4.x application. We've kept our main objectives for this plugin: provide a quick, easy, flexible users management plugin to help you build awesome products on top of CakePHP 4. So let's create a new project * Ensure you have a working develop environment, download a development vagrant machine or pick some other's environment. composer create-project cakephp/app:^4 users9 Now we have a new CakePHP 4 project skeleton created under folder users9.
Test it using bin/cake server then go to http://localhost:8765 and check it's all green.   all green checks for cakephp 4   * Install the plugin using
composer require cakedc/users:^9 * Now configure your application to use the plugin in your src/Application.php bootstrap() method, add
$this->addPlugin(\CakeDC\Users\Plugin::class); * Create the required tables in your configured Datasource using Migrations:
bin/cake migrations migrate -p CakeDC/Users This will create an empty `users` table and `social_accounts` to hold your authenticatio data. * Now create a superadmin user
bin/cake users addSuperuser It will create a superadmin user and a random password, copy the user and password to a safe place. Now start the standalone server using bin/cake server again, and go to the home page http://localhost:8765... you'll see the login form.   login form cakephp 4 and cakedc/users
If you use the superadmin credentials previously generated, you'll be able to login and continue to the home page. Total setup process, possibly less than 5 minutes if your network allows it... How is it actually working? * Once the plugin is added to you Application class through the Plugin class, we implement the  `AuthenticationService` and `AuthorizationService` provider interfaces, and use
the `middleware()` plugin callback to inject the configured middlewares into the Application middleware queue, see the `MiddlewareQueueLoader`.
We do it to keep the loading and configuration of the middlewares in one place, and decide the middlewares needed based on your configuration, for example if you are using
social login with Facebook, we configure which middlewares you need loaded and the correct order.
Check LINK for customization options of the plugin. We'll deal with customization in other articles of this series. * CakePHP core Authentication and Authorization middlewares are also loaded, with the provided configuration. Check file vendor/cakedc/users/config/users.php for the
default configuratio used. You'll see there we're using by default Session, Form, Token, Cookie and Social. Depending on your configuration we'll check if you're authenticated in the following order:
  * Your identity already in the session
  * You've posted login form and your credentials are in the request data
  * There is a token present we can retrieve, usually for API stateless token based auth
  * There is a remember me cookie present
  * Your identity is available after social login, and we can use it to login into the app
 
If all these methods fail, you're redirected to the login page, `/login` by default (configurable, of course).   * But that's the first step, once we can identify who are you, the next step (Authorization) is to determine if you're allowed to access the page you're trying to open. The plugin default configuration has 2
Authorization methods, superuser and rbac.
  * If you're user is a superuser, you are granted
  * If there is a rule in the rbac configuration to match your role and the current page, you are granted
 
If none of the above, you are not authorized, and redirected to the home page. One of the important concepts about the new authentication layer in CakePHP is: "Authentication happens before you hit AppController".
So when you get to your Controller, CakePHP alreay knows you're an existing user and you have permission to access the page. All the abstraction and complexity
of maintaining the authentication is now extracted and managed OUTSIDE of your controllers, reducing their complexity. Give it a try in your next project! Let us know how it goes and share you experiences with us, we'll be happy to help in the community channels.  

CakeFest 2019 Recap

Here we are, a couple weeks out of CakeFest 2019. What an experience in Japan!  We didn’t really know what to expect when we decided to finally make the trip to host in Tokyo. We were given great welcoming from the PHP community, and each person we met left us with wonderful memories. Another welcoming surprise was the low price of food, and delicious ramen. Our team definitely utilized all of the free time we had.            Let’s talk about the event, and the workshops. Jorge Gonzalez, Jose Rodriquez, and Mark Story definitely delivered in the knowledge department. The participation was fantastic, although the class size was smaller than in other years. We’ve had a lot of requests for their slides, so those are included in the link below ⬇️ One thing that was different this year, is that we had different venues for the workshops and conference. This makes it difficult for the team, with transferring our equipment in such a busy city (shout out to the quick responding taxi services).  We did try the metro when we had less baggage, and got up close and personal with the locals. Speaking of venues, we cannot thank DMM.com and SmartNews enough. We are still dreaming of an office like DMM’s with live plants growing up the walls and a complete installed watering system.  These venues were overly accommodating, making this one of the best conferences we’ve had.          If you are a PHPer or specifically working with CakePHP, the speakers topics were overflowing with useful information. Like Yuki Kanazawa’s tips for a smooth upgrade to CakePHP 3, or Tadahisa Motooka’s ideas about database replication. Kazuki Higashiguchi helped talk us through painful testing of code, and Sho Ito walked us through an initial OSS with CakePHP. We had such a great lineup this year, and we cannot wait to have some speakers return. Other great talks included David Yell, Daniel Voyce, Jose Gonzalez, and Wim Godden, and superstar core members Mark Story and Jose Rodriguez.  We even had to be confronted with details about life after CakePHP (GASP!) from Andrej Griniuk.    Unfortunately, no event can be executed without some roadblocks, and we aren’t exempt. We had a couple late cancellations (understandable) from speakers, but definitely made up the time with chats and lightning talks. There was so much information exerted during the short 2 days, that we all probably needed and extra day to take notes. Luckily, we did that for you. All of the slides included during CakeFest are available at the link below as well.    So, would we come back and host in Japan again? YES! We hope to do so sooner rather than later. Are there some things we will change on our end? Yes, again.  We hoped for higher numbers for workshops, as the information given is invaluable. We hope that in the future, all conference attendees will take advantage of those sessions as well. You can stay up to date with all things CakeFest at CakeFest.org - we are actually working on adding a history feature to reference past events.      We could not have done all of this without the amazing sponsors we had this year:    Cake Development Corporation  Shizen Energy  BASE  Lancers  DMM JetBrains Connehito  Marks Software SmartNews  ESM   Follow our speakers on Twitter:   Yuki Kanazawa - @yakitori009  Mark Story - @mark_story Jose Rodriguez - @jose_zap Jorge Gonzalez - @steinkelz Tadahisa Motooka - @t_motooka Kazuki Higashiguchi -  @hgsgtk Sho Ito - @itosho David Yell - @Yelldavid Daniel Voyce - @voycey_web Jose Gonzalez - @savant Wim Godden - @wimgtr Andrej Griniuk - @andrej_gr   CLICK HERE to view the CakeFest 2019 workshop and speaker slides.   Now, we want to hear from you! If you attended, what did you think about CakeFest Japan? What did you enjoy the most/least? If you did not attend: what has held you back from joining us? Let us know - email: community@cakephp.org.

We Bake with CakePHP