CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

File uploading, file storage and CakePHPs MediaView class

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this:

echo $form->create('Media', array('action' => 'upload', 'type' => 'file'));
echo $form->file('file');
echo $form->submit(__('Upload', true));

 

The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form.

When you submit the form now, you'll get data like this in $this->data of your controller:

Array
(
	[Media] => Array
	(
		[file] => Array
		(
			[name] => cake.jpg
			[type] => image/jpeg
			[tmp_name] => /tmp/hp1083.tmp
			[error] => 0
			[size] => 24530
		)
	)
)

Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour.

If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model.

We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:

  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system.

Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems.

Advantages of storage in the file system:

  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB

The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system.

Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it.

I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space.

/**
 * Builds a semi random path based on the id to avoid having thousands of files
 * or directories in one directory. This would result in a slowdown on most file systems.
 *
 * Works up to 5 level deep
 *
 * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
 * @param mixed $string
 * @param integer $level
 * @return mixed
 * @access protected
 */
	protected function _randomPath($string, $level = 3) {
		if (!$string) {
			throw new Exception(__('First argument is not a string!', true));
		}

		$string = crc32($string);
		$decrement = 0;
		$path = null;
		
		for ($i = 0; $i < $level; $i++) {
			$decrement = $decrement -2;
			$path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2));
		}

		return $path;
	}

You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class.

I'll now explain you how to use this class to send files to the client.

/**
 * Sends a file to the client
 *
 * @param string $id UUID
 * @access public
 */
	public function download($id = null) {
		$this->Media->recursive = -1;
		$media = $this->Media->read(null, $id);

		if (empty($media)) {
		$this->redirect('/', 404, true);
		}
		
		$this->set('cache', '3 days');
		$this->set('download', true);
		$this->set('name', $media['Media']['slug']);
		$this->set('id', $media['Media']['filename']);
		$this->set('path', APP . 'media' . DS . $media['Media']['path']);
		$this->set('modified', $media['Media']['modified']);
		$this->set('mimeType', $media['Media']['mime_type']);
		$this->set('extension', $media['Media']['extension']);

		$this->view = 'Media';
		$this->autoLayout = false;
		if ($this->render() !== false) {
			$this->Media->updateAll(
				array('Media.downloads' => 'Media.downloads + 1'),
				array('Media.id' => $id));
		}
	}

You simply have to set autoLayout to false and the view class to media.

$this->view = 'Media';
$this->autoLayout = false;

There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false;

	$this->set('download', true);

You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching.

	$this->set('cache', '3 days');

The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server.

	$this->set('name', $media['Media']['slug']);
$this->set('id', $media['Media']['filename']);
$this->set('path', APP . 'media' . DS . $media['Media']['path']);

If you want to send a mime type that does not already in the MediaView class you can set it.

	$this->set('mimeType', $media['Media']['mime_type']);

If you don't set it, the class will try to determine the mime type by the extension.

	$this->set('extension', $media['Media']['extension']);

Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up.

When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download.

	if ($this->render() !== false) {
	$this->Media->updateAll(
	array('Media.downloads' => 'Media.downloads + 1'),
	array('Media.id' => $id));
}

 

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Latest articles

Remote Work, Actually Works!

As a fully remote company, the Cake Development Corporation team is used to working from home. We communicate with our team daily, keep on top of tasks, hold each other accountable and support one another. Heck, a lot of us even do it with kids in the household, too! I consider us extremely lucky to be able to work while juggling an at home life at the same time.  It has worked for CakeDC over the past decade, and in my opinion, can work for most companies.   As of last month, an estimated 4.7 million people were working remotely, which grew 44% over the last 5 years. This is just in the United States. Remote work is becoming the norm.  Obviously for the next few weeks, this number will be drastically increased, but perhaps this will educate companies on the advantages of a WFH culture. Advantages to employers, besides the operations cost (other than payroll, of course), which can decrease by close to 90%, includes increased productivity. Decreased overhead results in higher salaries, which results in more quality candidates and employees.  I understand the concern of the ability to micro-manage (UGH) being unavailable, but according to statistics, 85% of businesses that work remotely confirmed that productivity increased in their companies. When there is more flexibility, there will be higher employee morale.  With the current situation arising from COVID-19, a lot of businesses are forced to transition employees to WFH in order to stay afloat. This not only keeps employees and clients safe, but family members too.  I have put together some stats and resources that may help CEO’s and employees transition a little bit easier.  

Communication:

It is absolutely essential to keep open communication among a team when everyone is working remotely. Our team uses RocketChat* ( I will include some links in the resource section at the end of this blog), and it has proved to be effective. A chat allows for quicker response time, as well as allowing individuals to set their status (like busy, away, at lunch, sick, etc.). This is a good way to get quick answers, as users can be alerted when they have been messaged or tagged in a company chat. Most of our team work in different timezones, so this is a good way to “stay in the know” about everything happening day to day. We separate chats according to their department. For example: marketing, development, general, etc. We also have the option to private message with co-workers when needed.  Other ideas, if not daily chat interaction, include scheduled meetings. For most of our team meetings, we use Zoom. This tool allows for audio only, as well as video chats.  

Accountability & Time Management:

It is important that tasks are managed and followed through. We use programs like Redmine* to track hours and work, in addition to weekly, or monthly conference calls for each department.  If you or your team are new to remote work, it may be in your best interest to assign a project manager, someone who will assign work, track hours, and ensure that work needed is being completed in a timely manner. Without each person being held accountable, the ship will sink, fast. For personal accountability, there are many free apps and tools available. One example is Trello*. This is a scheduling board so that tasks are not forgotten and you can plan your work week and stay organized. Once tasks placed on your “schedule board” are completed, you can make note of it and stay focused on each one according to their priority. You can also keep track of documents and reports. The boards look like this:    

Resources:

Documents & Recording - We <3 Google Docs - we are able to share and edit internally, we couldn’t function without it.  Docusign is a good tool for contracts / documents needing signatures Invision Freehand - this is a tool where you can create presentations, and allows comments and feedback between designers. Good for freelance designers!    Organization/Tasks -  Trello - for individual time management scheduling.  Redmine - for project assigning, time recording, HR management,    Communication -  RocketChat - allows for multiple internal chats all rolled into one link (allows for individual logins) Zoom - good for meetings. Allows audio and video chats for teams or reps and clients.  Slack - also a great option for expanded chats. Each person has a “screen name” and can be personally messaged, or public groups can be created (we use this as well). Slack also allows video calls with their paid subscription.  Google Hangouts WhatsApp - if your team is diverse, like ours, WhatsApp is a must. We are able to text each other, regardless of location - no fees, no service problems (if you have wifi of course).  World Time Buddy - this is a tool that I am not familiar with, but being the designated “scheduler of meetings”, I think I would find it useful. If your team works within different timezones, this allows you to add the location of your teammates, compare times, and find ideal times for meetings.    Community - In the development world, community support sites are absolutely one of the most important tools. This allows for individuals - inside or outside of your company - to communicate and help each other out. Most developers are aware and utilize these, but if not, may I suggest: Discourse - chat support  GitHub - our favorite team collaboration tool. GitHub allows for hosting, editing and managing products. We use it for building software and allow for community interaction. It also integrates with a lot of other tools, which is a plus!  

Take Away:

These resources are just a drop in the bucket compared to what is available to remote workers. I think this is a reflection of how WFH is becoming more accepted and more normal in the corporate world. I’d love to hear some of your favorites: amanda.goff@cakedc.com.  Let’s take away some positivity to the current quarantined times, and encourage more companies to follow suit. In today’s world, flexibility goes a long way and this type of transition can be mutually beneficial for employers and employees. I mean look at us, we are PRETTY normal… right?  Speaking of being in quarantine - stay healthy, stay inside, and wash your hands!  

ddsds

Two Factor Authentication & CakeDC Users Plugin

Why 2FA?

Nowadays we have noticed that many of the websites or applications that we access offer the option to activate an extra layer of security called Two Factor Authentication, better known as 2FA. Most of our lives happen on our mobile devices and laptops, so it’s not a secret that cyber-thieves would like to gain access to our personal and financial data. This is why adding an extra layer for protecting logins is worth it.  2FA  is an extra layer of security to make sure that someone that is trying to gain access to an account is who they say they are. The first layer is generally a combination of a username and password, and the second layer could ask for a code that is sent to your phone, a fingerprint scan or the name of your best friend. Currently 2FA has become a security standard in the digital world.

How does it work?

First the user will enter his username and password, then instead of getting in immediately into the system, he will be required to provide  additional information. Which could be one of the following options or factors:
  • Something you know : This could be a password, a personal identification number (PIN), answers to a secret question or a specific keystroke pattern.
  • Something you have: This is something the user owns, a physical device, like a mobile phone, an id card, an usb stick, a token, etc.
  • Something you are: This could be face or voice recognition, retina scan,  fingerprint, DNA, handwriting.

CakeDC Users Plugin and 2FA

There are various ways to implement Time-based One-Time Password (TOTP), Short Message Service (SMS), Electronic Mail (Email),  Universal Second Factor (U2F). CakeDC Users Plugin provides the ability to enable in your site TOTP or U2F. 
 

TOTP Google Authenticator

Enabling 2FA Google Authenticator in CakeDC Users Plugin is quite easy, it just takes a few minutes. In case you have not installed CakeDC Users Plugin in your application, follow the installation steps described here. Once you have installed the plugin and your basic login is working, you just need to do the following:
  1. Run the next command: composer require robthree/twofactorauth
  2. In Application::pluginBootstrap() add the following: Configure::write('OneTimePasswordAuthenticator.login', true);
  Once you have 2FA enabled in your site, when you try to login will happen next 
  1. Type your username and password.   
  2. You proceed to the next step where you are asked for the authentication code
    • First time you will be shown a QR code that you need to scan from your authenticator application.   
    • Next time you will only get the input to type your authentication code  
  3. You open the authenticator application to get a secondary code called a one-time password (OTP)—usually six characters in length. There are many options in the market for the authenticator application, some of the most used are: Google Authenticator, Duo Mobile, FreeOTP etc.
  4. You type the 6-digit code into the website, and you’re in!
 

FIDO U2F

If you want something more solid and reliable, then you could use U2F (Universal 2nd Factor) standard created by the FIDO Alliance. With this kind of authentication you use a physical security key, and insert that into your PC, touch the key’s button, and you’re “automatically” logged in.  U2F standard was implemented in CakeDC Users Plugin by using  the YubiKey, the most famous and common example of U2F. To enable 2FA via Yubico follow the next steps:
  1. Run the next command: composer require yubico/u2flib-server:^1.0
  2. In Application::pluginBootstrap() add the following: Configure::write(‘U2f.enabled’, true);
     
Yubico is a hardware based 2FA, it’s a small device with one end that slots into a standard Type-A USB port. You just need to Insert your YubiKey and touch it! You won’t need to manually enter the code. Take into account that you will need to use https to be able to use 2FA features in your applicatins.

So, what to choose for two-factor authentication? There is no universal answer, it will depend on the level of security you are expecting, but start protecting your account by enabling 2FA! In this article you could noticed how easy is to enable 2FA in any CakePHP application by using CakeDC Users Plugin.
  References: https://github.com/CakeDC/users https://en.wikipedia.org/wiki/Multi-factor_authentication https://en.wikipedia.org/wiki/One-time_password https://en.wikipedia.org/wiki/FIDO_Alliance https://en.wikipedia.org/wiki/Universal_2nd_Factor  

We Bake with CakePHP