CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

File uploading, file storage and CakePHPs MediaView class

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this:

echo $form->create('Media', array('action' => 'upload', 'type' => 'file'));
echo $form->file('file');
echo $form->submit(__('Upload', true));

 

The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form.

When you submit the form now, you'll get data like this in $this->data of your controller:

Array
(
[Media] => Array
(
[file] => Array
(
[name] => cake.jpg
[type] => image/jpeg
[tmp_name] => /tmp/hp1083.tmp
[error] => 0
[size] => 24530
)
)
)

Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour.

If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model.

We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:

  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system.

Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems.

Advantages of storage in the file system:

  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB

The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system.

Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it.

I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space.

/**
 * Builds a semi random path based on the id to avoid having thousands of files
 * or directories in one directory. This would result in a slowdown on most file systems.
 *
 * Works up to 5 level deep
 *
 * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
 * @param mixed $string
 * @param integer $level
 * @return mixed
 * @access protected
 */
	protected function _randomPath($string, $level = 3) {
		if (!$string) {
			throw new Exception(__('First argument is not a string!', true));
		}

		$string = crc32($string);
		$decrement = 0;
		$path = null;
		
		for ($i = 0; $i < $level; $i++) {
			$decrement = $decrement -2;
			$path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2));
		}

		return $path;
	}

You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class.

http://api.cakephp.org/class/media-view

I'll now explain you how to use this class to send files to the client.

/**
 * Sends a file to the client
 *
 * @param string $id UUID
 * @access public
 */
	public function download($id = null) {
		$this->Media->recursive = -1;
		$media = $this->Media->read(null, $id);

		if (empty($media)) {
		$this->redirect('/', 404, true);
		}
		
		$this->set('cache', '3 days');
		$this->set('download', true);
		$this->set('name', $media['Media']['slug']);
		$this->set('id', $media['Media']['filename']);
		$this->set('path', APP . 'media' . DS . $media['Media']['path']);
		$this->set('modified', $media['Media']['modified']);
		$this->set('mimeType', $media['Media']['mime_type']);
		$this->set('extension', $media['Media']['extension']);

		$this->view = 'Media';
		$this->autoLayout = false;
		if ($this->render() !== false) {
			$this->Media->updateAll(
				array('Media.downloads' => 'Media.downloads + 1'),
				array('Media.id' => $id));
		}
	}

You simply have to set autoLayout to false and the view class to media.

$this->view = 'Media';
$this->autoLayout = false;

There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false;

	$this->set('download', true);

You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching.

	$this->set('cache', '3 days');

The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server.

	$this->set('name', $media['Media']['slug']);
$this->set('id', $media['Media']['filename']);
$this->set('path', APP . 'media' . DS . $media['Media']['path']);

If you want to send a mime type that does not already in the MediaView class you can set it.

	$this->set('mimeType', $media['Media']['mime_type']);

If you don't set it, the class will try to determine the mime type by the extension.

	$this->set('extension', $media['Media']['extension']);

Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up.

When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download.

	if ($this->render() !== false) {
	$this->Media->updateAll(
	array('Media.downloads' => 'Media.downloads + 1'),
	array('Media.id' => $id));
}

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Latest articles

A quick guide to agile development - why it has become such an important...

Agile has been around for a while, but recently it has come back into focus. But what is agile development and why has it become such an important concept? It has become such a buzz word but do you really know what it means or why it could be a great addition to your development process? Agile is a project management term that uses short development cycles, or iterations, to focus on continuous improvement in the development of a product or service. Agile was originally developed to improve the development process - allowing it to rapidly identify and adjust for issues and defects. A major benefit is that it allows development to keep ahead of customer expectations, competition etc.  There are 12 key principles that guide an agile project

  1. Customer satisfaction is always the highest priority; achieved through rapid and continuous delivery.
  2. Changing environments are embraced at any stage of the process to provide the customer with a competitive advantage.
  3. A product or service is delivered with higher frequency.
  4. Stakeholders and developers closely collaborate on a daily basis.
  5. All stakeholders and team members remain motivated for optimal project outcomes, while teams are provided with all the necessary tools and support, and trusted to accomplish project goals.
  6. Face-to-face meetings are deemed the most efficient and effective format for project success.
  7. A final working product is the ultimate measure of success.
  8. Sustainable development is accomplished through agile processes whereby development teams and stakeholders are able to maintain a constant and ongoing pace.
  9. Agility is enhanced through a continuous focus on technical excellence and proper design.
  10. Simplicity is an essential element.
  11. Self-organizing teams are most likely to develop the best architectures, designs and meet requirements.
  12. Regular intervals are used by teams to improve efficiency through fining tuning behaviors.
 Many industries actually make use of an agile development process and follow these key principles. It is highly collaborative and is seen to be more efficient. Some of the more popular agile methods used are
  • Scrum
  • Kanban
  • Lean (LN)
  • Dynamic System Development Model, (DSDM)
  • Extreme Programming (XP)
  • Crystal
  • Adaptive software development (ASD)
  • Agile Unified Process (AUP)
  • Crystal Clear methods
  • Disciplined agile delivery
  • Feature-driven development (FDD)
  • Scrumban
  • RAD(Rapid Application Development)
While agile development has many advantages, it also has a few disadvantages - it certainly is not for every project or project team. It also favors developers, project teams and customer goals, not necessarily the end user’s experience.

Quick glossary: DevOps

Has your team gotten you down with the use of so many terms that seem so unfamiliar? Don’t despair! The ability to rapidly develop, deploy and integrate new software is essential to success - but you should be aware of the terms that the dev ops team will be using! First starting off with devops - which is a mash-up of two terms: "software development" and "information technology operations. But there are more A/B testing A technique for testing new software or new features whereby two or more versions are deployed to users for testing. The metrics from each variant are then compared and assessed based on the testing criteria. Acceptance testing The testing performed near the end of the development cycle that determines whether software is ready for deployment. Agile development Agile development refers to a methodology that emphasizes short iterative planning and development cycles. The idea is that iterative development affords more control and establishes predictability.   Behaviour driven development A development methodology that asserts software should be specified in terms of the desired behavior of the application, and with syntax that is readable for business managers. Build Automation Tools or frameworks that allow source code to be automatically compiled into releasable binaries. Usually includes code-level unit testing to ensure individual pieces of code behave as expected. CA Release Automation CA Release Automation is an enterprise-class, continuous delivery solution that automates complex, multi-tier release deployments through orchestration and promotion of applications from development through production. Continuous delivery Continuous Delivery is a set of processes and practices that radically removes waste from your software production process, enables faster delivery of high-quality functionality and sets up a rapid and effective feedback loop between your business and your users. Deployment Manager Cloud Deployment Manager allows developers to easily design, deploy, and reuse complex Cloud Platform solutions using simple and flexible declarative templates. From simple web servers to complex highly available clusters, Deployment Manager allows teams to spend less time managing, and more time building. Delivery pipeline A sequence of orchestrated, automated tasks implementing the software delivery process for a new application version. Each step in the pipeline is intended to increase the level of confidence in the new version to the point where a go/ no-go decision can be made. A delivery pipeline can be considered the result of optimizing an organization’s release process. Functional testing Testing of the end-to-end system to validate (new) functionality. With executable specifications, Functional Testing is carried out by running the specifications against the application. Gitlab GitLab is a web-based Git repository manager with wiki and issue tracking features. GitLab is similar to GitHub, but GitLab has an open source version, unlike GitHub. Github GitHub is a web-based Git repository hosting service, which offers all of the distributed revision control and source code management (SCM) functionality of Git as well as adding its own features. Unlike Git, which is strictly a command-line tool, GitHub provides a web-based graphical interface and desktop as well as mobile integration. Lean “Lean manufacturing” or “lean production” is an approach or methodology that aims to reduce waste in a production process by focussing on preserving value. Largely derived from practices developed by Toyota in car manufacturing, lean concepts have been applied to software development as part of agile methodologies. The Value Stream Map (VSM), which attempts to visually identify valuable and wasteful process steps, is a key lean tool. Micro services Microservices is a software architecture design pattern, in which complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs. These services are small, highly decoupled and focus on doing a small task. NoOps A type of organization in which the management of systems on which applications run is either handled completely by an external party (such as a PaaS vendor) or fully automated. A NoOps organization aims to maintain little or no in-house operations capability or staff. Non-Functional•Requirements (NFRs) The specification of system qualities such as ease-of-use, clarity of design, latency, speed, ability to handle large numbers of users etc. that describe how easily or effectively a piece of functionality can be used, rather than simply whether it exists. These characteristics can also be addressed and improved using the Continuous Delivery feedback loop. Orchestration pipeline Tools or products that enable the various automated tasks that make up a Continuous Delivery pipeline to be invoked at the right time. They generally also record the state and output of each of those tasks and visualize the flow of features through the pipeline. Whitebox testing A testing or quality assurance practice which is based on verifying the correct functioning of the internals of a system by examining its (internal) behavior and state as it runs.  

Ed Finkler - Founder, Open Sourcing Mental Illness

Do you know who Ed Finkler is or what OSMI does? If you are in the developer community, then it definitely is a name you should get to know. Open Sourcing Mental Illness is a non-profit organization  dedicated to raising awareness, educating, and providing resources to support mental wellness in the tech and open source communities. CakeDC and CakePHP has long supported and stood behind OSMI - Ed Finkler has been instrumental in making mental health a topic of discussion, and opening up lines of support for mental wellness in tech. Mental health and wellness are close to our hearts and we want to share with you OSMI and why you should support it. Ed has been active in bringing forward a previously rarely discussed topic - mental health. Being an advocate of mental health awareness and using his own experiences as a developer, he has recently announced that he is now able to go full time into OSMI. This is really fantastic news and CakeDC stands 100% behind him. We caught us with him to find out more. We love that you are now putting all your time into OSMI - but what was the Catalyst for your decision to focus full time into OSMI?
What we found is that we simply had to much to do, and not enough time to do it. Everyone at OSMI are volunteers, and it was becoming increasingly challenging to find the bandwidth for anyone to complete major tasks. We are ambitious, and our ambition far exceeded the time available. I couldn’t ask it of anyone else, but I could make a decision myself -- that I would step away from my CTO role at a tech startup and dedicate myself to OSMI full-time.
What is your favorite thing to do out of ‘office’ hours (Hobbies/activities etc)?
Generally I find myself watching movies or good TV shows, or playing video games (I’m deep in Mass Effect: Andromeda right now). I also write electronic music, which you can hear at deadagent.net.
Do you think that companies are becoming more receptive to your message and becoming more open about speaking about mental health?
Yes, I think so. Companies in general are gradually becoming more aware of the need to discuss mental health openly, the same way we discuss other serious public health issues, like cancer and heart disease. But there’s a long, long way to go, and we are just taking our first steps as an industry to deal with this in a healthy way.
Have you seen a marked difference in people opening up about their personal experiences?
I definitely have observed, over and over, that when someone takes that first step forward, others follow. Fear is the thing that keeps mental illness hidden, and fear is why so many suffer in silence. Seeing someone speak without fear about their own issues empowers the listener. They may not need to stand up on stage like I do, but I’ve had numerous people tell me that hearing someone speak openly was what allowed them to seek help and/or start speaking openly about the subject.
What would you say is the biggest misconception that you have encountered when speaking about and sharing your personal experiences?
I think the biggest misconception I encounter is companies believing that by simply offering some level of mental health care in medical coverage, they’ve done all they can. That would be fine if we treated mental disorders like we do cancer or heart disease or diabetes, but we don’t -- we are afraid to discuss it, and as a consequence, we don’t know what to look for, why it matters, and how to seek help. In the absence of consistent, positive affirmation that it’s a safe topic, our default is to be afraid to discuss it. That keeps people from seeking the help they need.
Biggest piece of advice that you would give someone battling with mental health issues
You are not alone. Lots of people are like you. There is no shame in what you deal with. You are stronger than you know.
You recently spoke about mental health breaks on the OSMI blog, how would someone know they are in need of one and how would you suggest for employees to bring this topic up with their employers?
I am leery of giving specific health advice, but in general I’d say this: listen to your mind and your body, and remember that your own health is far, far more important than any job. Plus, if you’re healthy, you’ll be able to do your job much better.
In the last 5 years, you have achieved incredible breakthroughs and achievements in bringing this to the fore - where do you see OSMI and mental illness awareness in the next 5 years?
Ultimately, those two things are intertwined. OSMI will continue to grow because so many of us suffer from this, and more and more of us are realizing that we aren’t alone. That we aren’t broken. That we aren’t without hope. OSMI is about giving hope to those that felt they had none. Giving compassion to those who are hardest on themselves.
It’s my sincere hope that OSMI will drive the awareness of mental health in the tech workplace and change what we choose to value in employers and employees. However we get there, I believe we will succeed.

As someone suffering and wanting to find out more or be involved, how do we reach out, what should we expect and where should we go?
There are lots of ways to help OSMI, and all you really need is a willingness to spend some of your time working with us. You should visit https://osmihelp.org and learn more about our work, and then email info@osmihelp.org to talk to us about volunteering.
As a business with employees in the tech industry, what should we do to make mental health more accessible
For each employer there’s a different answer, but there are some general things to keep in mind. The biggest one is that the well-being of your employees must be a top priority. It’s an easy thing to say, but if you truly value it, you’ll avoid doing what so many organizations do: rewarding overwork and unhealthy “loyalty.” Ping pong tables and bean bag chairs don’t make people healthier, and neither do free snacks and beer at the office. They’re short-term tricks to get people to come to you and maybe stay in the office longer, but they don’t encourage a healthy work/life balance. Too many developers think their work IS their life. That’s a mistake.
Long term, what works are reasonable work hours, easy access to mental and physical health care, and promoting healthy preventative habits. Employees who feel that their well-being is demonstrably valued will be more productive and stay with your organization longer.
I also strongly encourage everyone in a leadership position to take Mental Health First Aid <https://www.mentalhealthfirstaid.org>, a program that teaches the skills to respond to the signs of mental illness and substance use.
Quote to live by or key advice to follow every day
One time I was encouraged to do a six-word memoir, and this is what I came up with:
“By helping others, I save myself.”
Thanks to Ed! We absolutely loved catching up with him about OSMI, we hope that you take a moment to check out the links and find out more to get involved and continue this important conversation! For more information, be sure to check out https://osmihelp.org/about/about-osmi  

We Bake with CakePHP