CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

Utils Plugin release v1.1

The Utils plugin is our mixed bag of "awesome". If you've not yet checked it out, definitely hop over to github to check it out. It aggregates a lot of useful code and miscellaneous ideas into a single plugin thats portable and dead easy to use in your applications. Since its release in September, we've made a few changes and updates, and we've bundled a new version for release. Here's a summary of the commits:

  • Commit [7bdf401]: Update license and readme.
  • Commit [e7630bd]: Added tests for data retrieval and false return from model delete.
  • Commit [8510fe4]: Updated documentation for Soft Delete tests.
  • Commit [f7d9983]: Removed empty test file.
  • Commit [c5db61b]: Changed the behavior saving the position manipulation without running model callbacks and validation by default. This is now also configureable by setting 'callbacks' and 'validate' in the behavior settings to true/false.
  • Commit [ca98003]: updating readme
  • Commit [edc6576]: updating readme
  • Commit [da6ec86]: Add a russian translation
  • Commit [a2319ca]: Adding spanish translation
  • Commit [752f1d7]: Added a Portuguese translation
The release is available now on the master branch of the repository, or you can download a release archive here. Don't forget if you have any issues, suggestions or fixes for the utils plugin, you can lodge a ticket on Github. Enjoy!

CakeDC Plugins updates, October 2010

Its been a little while since we launched our plugins at CakeFest 2010 to the community, and a few things have been changed and updated in that time, so its time to throw out a new release for the community. We have received a huge response after opening our code to the community, and we're absolutely thrilled to know that you're taking advantage of the experience and effort that CakeDC has put into making these plugins. Getting feedback and hearing stories about usage makes it all worthwhile. The team has been monitoring tickets, and cleaning up where we can in-between "real work" :) Thanks to everyone that lodged tickets, submitted patches, we're overwhelmed with the generosity that people have shown by contributing to help benefit the community and to further the work we began. This blog marks the beginning of a run of updates we're doing with the plugins that have been released. We'll process tickets, package and release new versions every couple of weeks to ensure we're on top of tickets, and getting any updates published for people to use on a regular basis. We hope you enjoy the upcoming releases, and thanks again for the support! From all the team at CakeDC.

i18n routes with CakePHP 1.3

Internationalizing a CakePHP application can be tricky when it comes to deal with i18n urls. We will see in this article how the Custom route classes introduced by CakePHP 1.3 could be used to add the current language to your urls in a few lines of code. EDIT: This proof of concept has now been improved and a better version of the code below can be found in CakeDC's I18n plugin on Github

Requirements

This article will not go too deep in internationalizing an application as many resources already exist about it. We suppose the following:
  • Your application defines the current language on given the language code passed in the url
  • The available languages are configured via Configure::write('Config.languages', array('eng', 'fre', 'deu'));
  • You use the CakePHP array syntax for defining urls:
    • $this->Html->link('link', array('controller' => 'posts', 'action' => 'view', $post['Post']['id']));
    • $this->redirect(array('controller' => 'posts', 'action' => 'index'));
    • Router::url(array('controller' => 'posts', 'action' => 'index'), true);
Custom routes were already introduced by Mark Story on his blog, so we will not do it again here... before continuing be sure you have read "Using custom Route classes in CakePHP"

Show me some code!

I18nRoute

As I said (or not), routes are probably the best place for customizing your urls and add information in them... much more better at least than overriding the Helper::url() method in an AppHelper class! Custom routes introduced a way to customize how routes are processed in a very easy and powerful way (i.e ~20 lines of code). It is a bit like wrapping the Router class in CakePHP 1.2, a good example of this was the CroogoRouter. First, we are going to create an I18nRoute class extending CakeRoute in the "/libs/routes/i18n_route.php" file. Here is its code: <?php class I18nRoute extends CakeRoute { /** * Constructor for a Route * Add a regex condition on the lang param to be sure it matches the available langs * * @param string $template Template string with parameter placeholders * @param array $defaults Array of defaults for the route. * @param string $params Array of parameters and additional options for the Route * @return void * @access public */ public function __construct($template, $defaults = array(), $options = array()) { $options = array_merge((array)$options, array( 'lang' => join('|', Configure::read('Config.languages')) )); parent::__construct($template, $defaults, $options); } /** * Attempt to match a url array. If the url matches the route parameters + settings, then * return a generated string url. If the url doesn't match the route parameters false will be returned. * This method handles the reverse routing or conversion of url arrays into string urls. * * @param array $url An array of parameters to check matching with. * @return mixed Either a string url for the parameters if they match or false. * @access public */ public function match($url) { if (empty($url['lang'])) { $url['lang'] = Configure::read('Config.language'); } return parent::match($url); } } The most important part of the code is in the "match()" method. We just add the current language to the url "lang" named param if it was not set. The constructor was also overriden to add a regex pattern for the "lang" param. Thus, only lang prefixes defined in your list of available languages will be parsed by the route.

Define your routes

It is now time to use this custom route in your application. Here is how the default route for pages could be defined in "/config/routes.php": App::import('Lib', 'routes/I18nRoute'); Router::connect('/:lang/pages/*', array('controller' => 'pages', 'action' => 'display'), array('routeClass' => 'I18nRoute'));
  1. import the library file containing the custom route
  2. add a ":lang" param in where you want the language code appear in the url
  3. tell the Router you want to use this custom class (third param)

Link from everywhere!

Now you won't have to worry about the language code transmitted in your urls... every generated link will contain the current language code. If you want to switch the language (for instance switching to the French version of your application), you will just have to add the "lang" param to the url array. Here are some examples of urls which would be generated on the "/eng/posts/index" page: $this->Html->link(__('French', true), array_merge($this->passedArgs, array('lang' => 'fre'))); // /fre/posts/index $this->Html->link('link', array('controller' => 'posts', 'action' => 'view', $post['Post']['id'])); // /eng/posts/view/2

Disclaimer

This code is experimental and the article shows you how to use CustomRoutes to implement this basic feature. Many improvements could be added to fit your needs (no language code for the default application lang, short languages code...) Even if the tests we made were successful, we have not used this code in production yet so there may be "real word" use cases that are not handled correctly with this solution... if you find one, please tell us in the comments!

Feature rich, customizable comments pl...

Freshly baked by the friendly team here at CakeDC is the Comments plugin. The comments plugin allows you to enable comments on any controller for any existing model in you application. Built in a manner to allow complete separation from your application, enabling and including the comments functionality is almost too easy. A good use case is the addition of comments to blog posts. In this case you can facilitate user feedback on information posted on your web site to further enhance the facilities of your existing application. The documentation takes you through a practical example of how you can include this into an existing application with only a couple of code lines.  

Quick start with Migrations plugin

In a previous post I gave an overview of the CakePHP Migrations plugin, what it does and why you should use it in your applications. This article will explain how to use it in a practical way. We are going to bake a simple blog application recipe application and see how migrations are integrated in the development process. Since we recently moved all our open source projects on http://cakedc.github.com/, this sample application source code is also available there: Sample Migrations Application - Github (it is a CakePHP 1.3 application). Ready?

Bake a new application and add the migrations plugin

First of all, we need to bake a new CakePHP application. Easy enough to do using cake bake, then configure your database (an empty database is sufficient for now) and check that the home page is all green! If you have not set up your environment to use the CakePHP command line yet, take some time to do so... it worth it! Adding the migrations plugin might also be a straightforward task. You can either download the archive containing the plugin code and unzip it in the "/plugins/migrations" folder of your application, or  add it as a git submodule with the following command: git submodule add git://github.com/CakeDC/Migrations.git plugins/migrations Then check that it is correctly installed by executing the following command from your application root: cake migration help If you see a list of available commands you can move on next step.

Create initial tables and bake the MVC

We now need something to migrate! Let's create some tables in the database. The application will have Users who can publish Recipes, each one having several Ingredients (of course Ingredients can be used in many Recipes). Here is a SQL dump of this simple database schema: CREATE TABLE `ingredients` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` varchar(100) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; CREATE TABLE `ingredients_recipes` ( `id` int(11) NOT NULL AUTO_INCREMENT, `ingredient_id` int(11) NOT NULL, `recipe_id` int(11) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; CREATE TABLE `recipes` ( `id` int(11) NOT NULL AUTO_INCREMENT, `user_id` int(11) NOT NULL, `name` varchar(100) NOT NULL, `content` text NOT NULL, `created` datetime NOT NULL, `modified` datetime NOT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; CREATE TABLE `users` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` varchar(100) NOT NULL, `password` varchar(255) NOT NULL, `created` datetime NOT NULL, `modified` datetime NOT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; As our goal here is not to focus on the application code itself, baked MVC from these tables might be sufficient... just run the command cake bake all for User, Recipe and Ingredient to bake'em all! At this point we must have an application with an initial architecture ready to share. To start from here, one will just have to checkout the related commit... but don't you see a problem with this? How will he create the initial database? Maybe we could send him the SQL dump by email, or better commit it with the application! It is where the Migrations plugin comes in.

Generate the initial migration

"Be kind with your coworkers and include the database schema with your code... along with some sample data." Let's use the migrations shell to generate an agnostic database schema containing our 4 tables, and an initial admin user account. To do so we just need to run the following command: cake migration generate After entering a name for the migration and selected the database dump option, we might have a new "/config/migrations" directory containing two files:
  • map.php representing the different migrations order,
  • name_of_the_migration.php a migration file containing all the necessary information to create your actual database. In the sample application it is named: "001_added_users_recipes_and_ingredients_tables.php". You might have noticed that we added a 001 prefix to the migration name to make it easier to see migrations order, it is a good practice.
We can now open the generated migration file (/config/migrations/001_added_users_recipes_and_ingredients_tables.php) and take a look at it. If you need more information and understand all available migration directives, you can read the plugin documentation. For now we are just going to focus on the empty "after()" callback. This callback is triggered once the migration has been executed, and allow you to do whatever you want, given the direction of the migration: applied (up) or reverted (down). We are going to use this callback to create an initial admin User. Here is the code of the callback (as you are a CakePHP developer you might understand it quite easily): function after($direction) { if ($direction === 'up') { if (!class_exists('Security')) { App::import('Core', 'Security'); } $User = $this->generateModel('User'); $user = array( 'User' => array( 'name' => 'admin', 'password' => Security::hash('unsecurepassword', null, true))); $User->save($user); } return true; } Notice the use of the generateModel() method provided by the Migrations plugin. It is a shorthand allowing you to cleanly load a model in the callback to insert new data or update the existing. We could explain the reason of it more deeply but it is not the goal of this article, so just keep in mind that it is the best way to load a Model from callbacks! Here we are! We can now share the application with anyone. After checked out the application, one will just have to run cake migration all to turn an empty database to a database containing all the needed tables, and an initial admin user to start using the application.

Categorize the recipes!

As the application evolves, we need to sort recipes by categories. This change involves two changes in the current database schema: a new categories table must be created, and a category_id field added to the recipes table. Note: If you later want to use the migrations diff feature to generate a migration containing a diff between your previous database schema and the current one, you have to generate a Cake Schema of your database at this point. Simply run cake schema generate. We can now update the recipes table and create a new categories table. Here is a simple SQL script: CREATE TABLE `categories` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` varchar(100) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1; ALTER TABLE `recipes` ADD `category_id` INT NOT NULL Bake the MVC for categories and update recipes view pages to display the category so the application reflect these database changes. Before sharing these code changes, we need to generate a second migration describing the above SQL snippet in an agnostic way... and creating initial categories! Nothing different than what we did previously: run cake migration generate, give a name to the migration, and choose between generating a diff from the schema.php file (if one was generated), generating a dump of the database (we will remove unnecessary instructions later) or generating an empty migration file. Once generated, it is always important to check the generated directives for the migration and fix them if needed. The migration must look like this: var $migration = array( 'up' => array( 'create_table' => array( 'categories' => array( 'id' => array('type' => 'integer', 'null' => false, 'default' => NULL, 'key' => 'primary'), 'name' => array('type' => 'string', 'null' => false, 'default' => NULL, 'length' => 100), 'indexes' => array( 'PRIMARY' => array('column' => 'id', 'unique' => 1), ), 'tableParameters' => array('charset' => 'latin1', 'collate' => 'latin1_swedish_ci', 'engine' => 'MyISAM'), ), ), 'create_field' => array( 'recipes' => array( 'category_id' => array('type' => 'integer', 'null' => false, 'default' => NULL) ), ), ), 'down' => array( 'drop_table' => array( 'categories' ), 'drop_field' => array( 'recipes' => array( 'category_id' ), ), ), ); If you understood what we did in the first migration callback to add an initial user you might be able to implement this one. We would like to add initial categories: Starters, Main Dish and Desserts. For lazy people, the code is here: function after($direction) { if ($direction === 'up') { $Category = $this->generateModel('Category'); $categories = array( array('name' => 'Starters'), array('name' => 'Main Dish'), array('name' => 'Desserts')); $Category->saveAll($categories); } return true; } Here we are again! The changes are ready to commit, and the commit will contains both code and database changes. One could update the database after checking out this commit by running: cake migration all.

The end

I hope this very simple use case and the code we built will help you to start using Migrations. As you could see it is very simple to use and will make your life much more easier: you would not have to worry anymore about the state of your database schema. The source code of this tutorial is available on Github. If you found any bug or have any suggestion about the Migrations plugin, please create a ticket on Github. Comment this article if you have any question, and do not hesitate to share it if you found it useful!

CakePHP Migrations plugin: easily vers...

This article is a quick introduction to the Migrations plugin, open sourced a few weeks ago by our company. You will see how simple it is to use the plugin and what you could do with it. I hope this article will show you the benefits of using migrations in your CakePHP applications and make you give it a try right after the reading! Here is a one-sentence description of the plugin: the Migrations plugin allows developers to easily version and automate the creation / update process of any database schema and application data from the command line. For information, CakeDC uses this plugin on its project since several years to make team collaboration and deployment easier. The plugin has been entirely rewritten a few months ago and fully tested (code coverage >95% as always at CakeDC) before being open sourced under the MIT license. It is now available to the community along with its documentation... and it is free!

Why is it useful?

It has been a while since companies integrated Source Code Management in their development process and CVS, SVN, Mercurial or Git are now common tools. Inspired from the open source movement it is also a good practice for single developers to version application source code. As you might know, an application almost always depends of the database schema it is aimed to use... however it is not easy to version both the source code and database schema with a SCM. Let's take the example of a CakePHP application: until now the only way to do was to version a single file, either a sql dump or a CakePHP schema.php file generated with the cake schema shell. These two approaches are not very convenient to use on a daily basis, the first one forcing the developer to drop and recreate the whole database every time! Moreover, a web application development is never really finished (there are always new features to add, software updates or bug fixing to do...) and deploying these change on a test or production server is always a delicate task. Here comes the Migrations plugin! It provides a simple and easy way to version a database... and to perform many other different tasks thanks to its callback system. Here are some features:
  • keep a local database schema up-to-date: you just have to run all non applied migrations to update the local database schema to the latest version
  • make team work easier: when several developers work on the same application it is important that all of them work with the same database schema during all the development cycle. With migrations every commit is tied to the database schema at this precise instant, which makes easy switching branches and resetting a branch to a specific commit.
  • make installation and updates easier: ready to push the new version of your application live? You will only have to push the sources on the server and run all non applied migrations!
  • migrate more than database schema: the callback system allows you to do everything you want before (or after) applying (or reverting) each migration. Here are some examples: creating an initial admin account, add initial or test data to the application (lorem ipsums, categories, content...), update values from the database, send an email if debug > 0... The only limit may be your imagination ;)

Where can I find the code?

Announced a few weeks ago, a packaged version of the plugin can be downloaded from the "Plugins" section of CakeDC.com. This page contains a link to download the 1.0 version, the plugin documentation and the Github project for tickets and direct Git access to the repository. To make people aware of the need to show their support to the Cake Software Fundation by donating a few bucks (this is unfortunately not done enough), the plugin was first available to donors only. The "Download without donation" button was added later, when the repository was made public! However, if you find this plugin useful please consider making a donation to the CSF... that is the best thing you could do for thanking us. Click here to lend your support to: cakephp1x and make a donation at www.pledgie.com ! Even better! A sample application was also released for those who want to see how migrations could be used and integrated in an application. To play with it, Download the code or git clone the project using: git clone git://codaset.com/cakedc/sample-migrations-application.git sample_migrations You will only need to create a database.php configuration file and update CakePHP's core location to make the application work. Git users, run git submodule init git submodule update to automatically add the migrations plugin as a submodule!

What do I need to use it in my application?

Note: the packaged plugin is for the CakePHP 1.3 version only. You can either download the 1.3-beta package of the framework, or use the 1.2 branch available in the Git repository. Adding the plugin to an existing application is very simple. If you downloaded the archive containing the plugin code, unzip it in the "/plugins/migrations" folder of your application. Git users can add it as a submodule with the following command: git submodule add git://codaset.com/cakedc/migrations.git plugins/migrations To check that it is installed correctly, execute the following command from your application root (it will display the available command to use the plugin): cake migration help If you encounter any problem here, please read the official documentation about CakePHP's console usage.

How does it work?

This post is not aimed at providing a comprehensive tutorial on how to use the plugin, thus I will just introduce the most useful commands along with some use cases. For a complete documentation, please read the official documentation provided on the plugin page. For a simple (but useful for understanding purpose) use case you can take a look at the sample application introduced above. Going through the commit history will allow you to understand how migrations could be used in a development process.

Create a migration

To generate a new migration, type the following command cake migration generate The tool will ask you to give a name to the migration and suggest to do a dump of the current database schema. If a "schema.php" file is found in the application, it will ask you if you want to generate a diff between this schema and your current database one. Generated migration files will be added to the "/config/migrations" application directory.

Apply / Revert migrations

When you pull an application containing migrations, several commands are available to apply or revert migrations. The simplest one is: cake migration It will display all the found migrations along with their status (applied or not applied) and id number. Just enter a migration number to update your database to the correct version. Some convenience commands are also available. You can use: cake migration up, down, all or reset These commands will respectively:
  • apply the next migration
  • revert the latest applied migration
  • apply all non applied migrations (and thus update the schema to the most recent version)
  • revert all applied migrations (and empty the database)

Migrations for plugins

Adding plugins to an existing application often implies adding new tables to the database or altering existing ones. The Migrations plugin brings a quick and efficient way to automate this installation. On the one hand developers can easily add necessary migrations to their plugin (making upgrades easier), on the other hand users can apply them as easily. The only difference compared with commands introduced above is the parameter "-plugin pluginname" that needs to be added. Here is how the user will install the database for the newly added / updated plugin "test": cake migration run all -plugin test I would like to highlight the fact that callbacks allow the developer to do everything they want before / after each migration. It is convenient for adding initial data, and one can even implement a callback method opening the bootstrap.php file to append plugin's configuration entries there (it is just an example ;)).

... going further

Of course, feel free to add any remark or example of migrations use in the comments. As this post is not aimed at providing support for the plugin, I recommend you to use the official tools available:
  • If you found a bug or want to suggest enhancements: open a ticket!
  • An installation problem or a question about the plugin usage? Ask your question to the community!
  • You would like a custom version of this plugin, or professional related services... contact us, it is our job ;)
I hope you enjoyed this post, it is now time for you to start playing with the Migrations plugin...

File uploading, file storage and CakeP...

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this: echo $form->create('Media', array('action' => 'upload', 'type' => 'file')); echo $form->file('file'); echo $form->submit(__('Upload', true));   The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form. When you submit the form now, you'll get data like this in $this->data of your controller: Array ( [Media] => Array ( [file] => Array ( [name] => cake.jpg [type] => image/jpeg [tmp_name] => /tmp/hp1083.tmp [error] => 0 [size] => 24530 ) ) ) Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour. If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model. We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:
  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system. Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems. Advantages of storage in the file system:
  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB
The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system. Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it. I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space. /** * Builds a semi random path based on the id to avoid having thousands of files * or directories in one directory. This would result in a slowdown on most file systems. * * Works up to 5 level deep * * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits * @param mixed $string * @param integer $level * @return mixed * @access protected */ protected function _randomPath($string, $level = 3) { if (!$string) { throw new Exception(__('First argument is not a string!', true)); } $string = crc32($string); $decrement = 0; $path = null; for ($i = 0; $i < $level; $i++) { $decrement = $decrement -2; $path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2)); } return $path; } You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class. I'll now explain you how to use this class to send files to the client. /** * Sends a file to the client * * @param string $id UUID * @access public */ public function download($id = null) { $this->Media->recursive = -1; $media = $this->Media->read(null, $id); if (empty($media)) { $this->redirect('/', 404, true); } $this->set('cache', '3 days'); $this->set('download', true); $this->set('name', $media['Media']['slug']); $this->set('id', $media['Media']['filename']); $this->set('path', APP . 'media' . DS . $media['Media']['path']); $this->set('modified', $media['Media']['modified']); $this->set('mimeType', $media['Media']['mime_type']); $this->set('extension', $media['Media']['extension']); $this->view = 'Media'; $this->autoLayout = false; if ($this->render() !== false) { $this->Media->updateAll( array('Media.downloads' => 'Media.downloads + 1'), array('Media.id' => $id)); } } You simply have to set autoLayout to false and the view class to media. $this->view = 'Media'; $this->autoLayout = false; There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false; $this->set('download', true); You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching. $this->set('cache', '3 days'); The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server. $this->set('name', $media['Media']['slug']); $this->set('id', $media['Media']['filename']); $this->set('path', APP . 'media' . DS . $media['Media']['path']); If you want to send a mime type that does not already in the MediaView class you can set it. $this->set('mimeType', $media['Media']['mime_type']); If you don't set it, the class will try to determine the mime type by the extension. $this->set('extension', $media['Media']['extension']); Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up. When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download. if ($this->render() !== false) { $this->Media->updateAll( array('Media.downloads' => 'Media.downloads + 1'), array('Media.id' => $id)); }

 

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Felix Geisendörfer - Javascript and Git

Felix gave a demonstration of the production level javascript separation and management that the team at Debuggable use in order to minimise the amount of Javasript that needs to be sent to the client for any specific page view, and to ensure the logic is separated into the pages that it is used for. This creates a better management system for Javascript than using a single file. In addition to this separation, Felix gave an overview of common practices and operations for using Git for version control in a day to day environment. This included: merges, conflict resolution, fast forwarding branches, and managing multiple repositories. Largely this presentation was an interactive one, and to gain the most out of it, you really needed to be there.

Marius Wilms - The CakePHP Media Plugin

If Marius had more than an hour to talk about the Media Plugin, he most certainly would have taken it. To go over the features and functionality of the entire plugin would have been many hours as there is a lot there. A brief touch on the features provided by the plugin was discussed, with some examples. Requirements are in the high end, but considering the state of PHP and the upcoming version of CakePHP, developers should be moving forward in terms of their PHP version and library support anyway. The Media plugin requires CakePHP 1.2.x.x and PHP 5.2.0+. It enables the transfer, manipulation and embedding of files in many varied ways. You can find the media plugin at: http://github.com/davidpersson/media Marius' focus was on doing media manipulation and embedding "properly", and identified that while there are lots of user contributions floating around the net, none of them were meeting his needs and were flexible enough. One of the main points he made here was that if done incorrectly, potential security risks arise due to command line interaction and file saving. Validation was one particular section of the code that made this a tricky plugin to develop, but allowed tests to be implemented to ensure security. Some common points that we hear all the time came through, and they make sense for CakePHP as well as any web application for security reasons:  

  1. Don't trust users supplied filenames
  2. Don't store files in an accessible webroot, rather have them accessible to scripts.
  3. Make the upload location (and local filenames) unguessable (like referencing files by UUIDs)
The media plugin contains about 8 new rules for file validation purposes to ensure that submitted data meets the application needs. Beyond validation, it handles all kinds of uploads, HTTP Post, Remote HTTP and local file inclusion.
A console is included to initialize the default directory structure, and as such, could be included as part of a deployment script with the CakePHP console.examples.
To ensure flexibility of use, a behavior is included to allow attachment to any number of models, and generioc storage and linking provided to ease integration into existing apps.
Marius concluded his talk with a plea for feedback. There are plenty of people using the plugin, but more feedback is required to ensure its the best it can be, and that all bugs  (if any) are squashed. Checkout the code at: http://github.com/davidpersson/media

Robert Scherer - Multi-Tenancy in CakePHP

Robert's talk was unscheduled, but ended up being a great case study for an insurance sales white-labelling solution that his company had undertaken and completed. Robert talked about multi-tenancy, and what this means for a web application, and how it relates to SaaS architecture. Challenges to be solved included:

  1. Differences in functionality
  2. Workflow differences
  3. Separation and security of data
  4. Branding and visual differences
Auth and Acl Components were used to solve a lot of the problems described, but in addition, Robert discussed the development of Modules as a new addon / plugin structure that allowed the addition, removal or configuration of application items at any level (Model, View or Controller).
Configuration of the modules was broken up into system default, mandators, and dealers configuration, allowing for inheritance of options along the way. To solve the view specific differences, built in themes were used to provide the differences required. This is a CakePHP builtin mechanism that serviced their needs well. Much of Robert's talk went through visuals of the site itself, and should we be able to get our hands on these, will post them up to see the various differences in presentation, and the module structure in terms of MVC.

Neil Crookes - Bake Master Class

After an introduction to bake, and what this shell means within CakePHP, Neil went on to explain and show examples of the code generation templates and capabilities provided by default. The bake shell is broken down into tasks and a main shell. These tasks separate out the logic required for various main task subsets including controller, model and view baking, amongst others. The main bake shell is found in the CakePHP directory cake/console/libs/bake.php. Tasks used by this shell are defined in the $tasks variable. Bake extends the CakeShell class and executes calls based on whether the users want interactive or non-interactive tasks through the __interactive() and bake() methods respectively. Neil made the suggestion that a persistent MySQL connection might be a good idea to stop database connection timeouts. Following this introduction, a great walkthrough of customisation of the bake process and templates was demonstrated. This included the addition of a new Shell that allows for multiple bakes to be done automatically of the same type. Neil has been kind enough to host the code, and you can find this over at the CakeFest downloads page.

Felix Geisendörfer - Recipies for succ...

Felix liked to Get Things Done™. And through experience and what became an interactive idea and experience sharing productivity session, he explained mechanisms and methods that he has used to achieve the best results for projects in the shortest time possible. Communication. While this means your team should be in contact, and that those contact points should be quality communication, it doesn't mean that instant communication is a requirement. Communication mechanisms would ideally be: decentralised and work in an offline capacity (at least for partial functionality). Return on investment is an interesting statistic to consider when responding to or creating a new item of communication. Provide a concise message. Enough to ensure the intention of the communication is clear, and ambiguity is reduced if not eliminated. Email is a great tool, especially for the following: Timezone differences, announcements, spawning debates that require discussion, emergency notifications / reports, mailing lists, shared email accounts and automated reporting / information. Using Email over an instant messaging mechanism for spawning debates allows contributors to formulate a constructed response. This can assist the better understanding of some ones input to the discussions, as instant messaging can be difficult for items that require discussion. Instant messaging has good and bad traits:

  1. Good Stuff
    1. Instant
    2. Group Chats / conference calls
    3. Varying methods of communication
    4. Various formats (text, voice, video)
  2. Bad Stuff
    1. Distracting (interrupts workflow)
    2. History tracking / compatibility
    3. Citations / logs
Task management helps keep projects on track. However the truth is that there is no overall solution. We do the best we can to manage all the information we need to be successful, through a variety of tools.
Problems that exist are:
  1. Getting tasks into the system (May be the laziness of users)
  2. Tracking tasks that manage to make it into the system
  3. Getting those tasks done
Tools available:
  1. Pen and Paper (plain text files)
  2. OmniFocus (Mac Only)
  3. Lighthouse / TheChaw
One of the CakeFest attendees suggested post-it notes on a wall, so that the tam involved in pursuing the tasks can have some physical interaction with them, making the experience more productive and fun.
So with this in mind, Felix quickly went over what has worked for him and his company:
  1. Check emails twice a day only.
  2. Turn off instant messaging tools in the morning
  3. Set clear distinct goals for the day, and achieve those goals
How to fail at unit testing. Felix described some common myths about unit testing, how he feels failing is possible and how to improve your approach.
Failed unit testing can come from factors such as: Attempting to reach 100% code coverage all the time, misunderstanding test driven development and expecting that all developers / users can write unit tests.
Success can be achieved by approaching projects with a top-down approach, incorporating performance tests and re-factoring code. Continuous integration was presented in a basic format, going through the setup that Felix has found useful, making use of git post-receive hooks, and parsing of results to send notifications when necessary. In terms of increasing productivity and performance overall, Felix has begun testing Pair programming, where there is one computer for two people, and the development process is discussed ongoing, meaning that a lot of bugs an individual would miss are caught by the secondary developer on the first code pass. Virtualization though open source and free products like VirtualBox and VMWare Server are worthwhile investigating for cheap scaling testing during development.
Version control was introduced, referencing the change of CakePHP to git from subversion, pointing out the benefits including: decentralization, can work offline, can work semi-online though the use of adhoc networks, its very fast to operate and its more intelligent with the storage of information. Following this, a detailed example of three separate working repositories was demonstrated.

Joël Perras - Demystifying Webservices...

Joël's presentation on Web Services and CakePHP identifies important and interesting points that really demystify both implementation of datasources, and what web services mean for developers trying to take advantages of their offerings. A Web Service is a defined interface. The interface is made known and public, however the implementation may not be known (and its not really important). The developer should be interested in the data supply and the data returned from the web service. Various mechanisms are available for communicating with a web service. Such as: RPC, SOA, REST and more. Much of this presentation covered best practices, better practices, and why people tend to make decisions like implementing components when they really want datasources, as well as implementing datasources, and going about the implementation the wrong way. In the case of web services datasources implementation, curl is presented as a good example of something that works, but a better solution is available through the use of HttpSocket. HttpSocket being one of the CakePHP core libraries provided, allowing a complete implementation of Http communication, extending the CakeSocket class. Authentication and Authorization options were presented, with specific reference to OpenID and OAuth. Authentication and Authorzation are part of the application flow graph. This means implementation should be at the controller level, and in terms of implementing easily managed pluggable sections of code in cakephp converntions, this means a component. Data Sources are the closest layer to the actual data. Correct implementation of a data source will allow models to connect and communicate in a transparent fashion, meaning easy access to data in a standard way. The basics of a datasource should implement the following: __construct, listSources, describe, create, read, update, delete as well as defining $_schema. Some great datasource examples can be seen in the core. When implementing a datasource, to ensure maximum use and compatibility, try to make use of CakePHP libraries such as HttpSocket in the place of curl. Google Charts was presented as a good example of what should not be implemented as a datasource. The data in this instance is handed by some other data source, and the formatted chart request is sent with an image response supplied. This is more appropriate for a helper than a datasource. Joël mentioned that he has a partial google charts helper that he would be willing to share if someone asked.

Garret Woodworth - CakePHP then, now a...

Beginning with an overview of the CakePHP project, changes and evolution of direction and development team members, Garret provided a great overview of where the project stands, and how it has grown to be as successful as it has today. Garret gave a great description of the types of participation that are seen in open source teams, and these are relevant to CakePHP. He also described the attributes that make a good team member in such projects. Contribution Levels:  

  1. No effort (tickets are subimitted with little explanation)
  2. Some effort (well explain the ticket, and have attempted to reproduce the issue to confirm it)
  3. Attempted effort ("Some effort" with patch)
  4. Good Effort ("Some effort" with test case)
  5. Ultimate effort ("Some effort" with test case and patch)
Good team member attributes:
  1. Communicate often.
    1. To keep people motivated and interested on working for / with a project, its important to talk about what they want to work on, and what they feel they can assign some of their time to. Developing for open source shouldn't feel like "work".
  2. Show diffs of code, and get feedback to ensure the quality of work overall for the project is as high as it can be.
  3. Think longer about the problems faced, and as a result, write code faster.
  4. Details, Details, Details.
  5. Give back to the project more than you take from it.
  6. Think outside the box, and be creative.
CakePHP is growing, and the stats presented spoke for themselves, with America, Japan, India, France and Germany being the top countries at the moment in terms of hits on the CakePHP websites at the moment. This is resulting in 24% unique new visitors per month. A statistics that is truly extraordinary.   With the feature development and more developers available to the CakePHP Core Development Team, git has been implemented widely and is the future of version control for source code for the CakePHP project. This should ease feature development, and remove some of the pain associated with merging with Subversion. Announcements! Garret announced new versions of CakePHP, currently being actively developed by the CakePHP core development team. Version 1.3 is a Step up with several enhancements over 1.2. Most notably Bake, Session, Javascript changes, Inflector and some library renames. Deprecated methods were also removed. There is even a wiki page describing migration steps from 1.2, to help ease the transition. CakePHP 2.0 was also announced. This is a huge move, stepping forward to drop PHP4 support, and move towards PHP5 Strict compliance, and much better Object Orientation and performance throughout. This new version is in active development,  but does not yet have a stable release for download. code.cakephp.org was launched at the time of the Keynote, and is designed to consolidate systems. it's running on thechaw.com code, and uses git for the main projects. Its available now for everyone to use. Closing things up, Garrett urged the community to "get involved". CakePHP isn't where it is today without the extensive help and support of the community. There are a number of ways that you can contribute, and he mentioned the following in particular:  
  1. Interact with the community and the core developers.
  2. Get interested in Bakery 2.0 which is currently under development
  3. Plugins and Plugin Server
  4. Forks
  5. Join #cakephp-bakery on the IRC server
 

Benchmarking requestAction

Now there has been a lot of discussion in the past few months about requestAction() and how it can very easily create a negative impact on your application. In fact I even wrote such an article myself. However, its high time that someone did the number crunching to really see if requestAction() is actually as slow as we all seem to think it is. So onto the testing method and the results.

Testing method

To test this theory I used a small CakePHP application and the SVN head (revision 8064) of CakePHP. I used a simple sample application with 2 controllers and 2 models. My model method directly returned the results without touching the database, so that database retrieval time and model processing would not be a factor in these tests. As I was only interested in the performance implications inherent in requestAction() itself, I wanted to remove the variance created by connecting to a database. I set debug = 0, and used basic file caching. After warming up the cake core caches, I tested 4 different controller actions.
  • Using Relations / ClassRegistry::init() - The method I originally proposed, and often touted as the 'best' solution to requestAction()
  • Using RequestAction with a string URL
  • Using RequestAction with and Array URL
  • Using a cached RequestAction - This more accurately simulates how we use requestAction at CakeDC.
Benchmarks were generated with Siege I used 10 concurrent users with 110 reps each. My local development web-server is running Apache 2.2/PHP 5.2.6 o n a 2.6GHz Core 2 Duo iMac with 2GB of ram. I ran each test 3 times and took the best result of each. Using model relations / ClassRegistry::init() First up was my originally proposed solution of using model relations to access the correct information. I used the following command and got the following results. siege -b http://localhost/benchmark/posts/using_relations Transactions: 1100 hits Availability: 100.00 % Elapsed time: 63.21 secs Data transferred: 1.50 MB Response time: 0.55 secs Transaction rate: 17.40 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.60 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.76 Shortest transaction: 0.10 Using RequestAction with a string URL Up next was using request action with a string url. String URL's are often the slower way to perform a requestAction as parsing the URL string is one of the more expensive operations in request dispatching. I used the following command and the best results were. siege -b http://localhost/benchmark/posts/using_requestaction Transactions: 1100 hits Availability: 100.00 % Elapsed time: 64.60 secs Data transferred: 1.51 MB Response time: 0.57 secs Transaction rate: 17.03 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.72 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.76 Shortest transaction: 0.11 RequestAction with an Array URL Up next is requestAction() witn an array url. Using an array URL is supposed to expedite the dispatching process as it bypasses much of the parameter parsing done by Router. This theory turned out to be true, as Array URL's clocked in marginally faster than their string counterparts. siege -b http://localhost/benchmark/posts/using_requestaction_array Transactions: 1100 hits Availability: 100.00 % Elapsed time: 64.08 secs Data transferred: 1.53 MB Response time: 0.57 secs Transaction rate: 17.17 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.78 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.66 Shortest transaction: 0.11 RequestAction using Array URL's and Caching In my mind this was going to be the most performant requestAction option, due to the cached nature. The results were as expected with this method clocking to be only slightly behind the relation call. It is important to note as well, that this test does not reflect the time savings earned from not having to make an additional query/ round of result parsing. In a real world situation, the savings of using a cached element would be magnified by the cost of the query. siege -b http://localhost/benchmark/posts/using_cached_requestaction Transactions: 1100 hits Availability: 100.00 % Elapsed time: 63.60 secs Data transferred: 1.52 MB Response time: 0.56 secs Transaction rate: 17.30 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.62 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.77 Shortest transaction: 0.09 Results Summary In case you quickly scanned through the full results here is a summary of what happened.
Method Requests per second (mean) Total time taken (seconds)
Using relations/ClassRegistry::init() 17.40 63.21
Using requestAction and string urls 17.03 64.60
Using requestAction and array urls 17.17 64.08
Using cached requestaction 17.30 63.60
In closing requestAction() can be slower than a direct method call. There are some benefits to using requestAction though.
  • You have the opportunity to reduce the number of repeated lines of code by putting the requestAction inside the element. In doing so, you create an encapsulated element, that can be included anywhere without having to worry about having the correct method calls in your controller.
  • You can more easily cache the element. By using requestAction in conjunction with element caching you have an easy to use, simple to implement caching. Getting the same results with model method calls in your controller requires additional caching logic in your models.
  • The potential for increased performance. As we saw in the benchmarks above, a cached element performed almost as fast as the direct method call. This margin will grow when a database query is added into the mix.
Now am I retracting my previous stance on requestAction? No, I still feel that there are many situations where requestAction is the incorrect solution and signals poor application design. However, when the need arises it is good to know that requestAction can be as fast or faster than other approaches when implemented properly.  

RSS Feeds, Fast and Easy

For my first entry, I am going to talk about how to create an RSS Feed on your website. RSS (Really Simple Syndication) is a format used to publish frequently updated works such as blogs or featured products. RSS defines a set of XML elements that are used to describe a channel or feed of information. An RSS feed is comprised of two parts, first is the metadata describing the channel and second is the records that make up the elements of the feed. RSS feeds allow your sites visitors to access the information on your site using software that reads these feeds. This will allow your site's visitors to stay up-to-date on the information on your site. CakePHP allows for easy integration of RSS feeds into existing controller actions through the automatic router extension parsing. This allows us to specify what type of response we want from a URL through adding the proper extension to the URL such as http://www.yoursite.com/entries.rss. This alerts the router that your are asking for RSS formatted data in return. In addition, CakePHP has an RssHelper class that can be used to output parts of the metadata and elements in the feed through an easy to use helper.

Preparation

Before we begin making the feed we must alert the router that we want to allow for extensions to be parsed in the URL and that we want it to accept .rss as a valid extension. In your sites router file we add the following: Router::parseExtensions('rss'); Also for CakePHP to work it magic we must also have the RequestHandler in our controller's $components array. Now the router knows that we would like to parse urls that end in .rss as requesting RSS formatted responses. The next step of preparation is to add a default layout for rss feeds on your site. When you request a different format response the layout that is rendered will be selected from a sub-folder with the same name as the format. So in this case we would need a folder called /rss in the layouts folder in our CakePHP install. The view class will search for a file that has the same name as the layout that would be rendered if you were just rendering the html. In most cases this is the default.ctp layout file in the main layouts directory, but because we are requesting the response in RSS format we must add a default.ctp layout in the /layouts/rss/ sub-directory. This layout is our default RSS Feed layout. echo $rss->header(); if (!isset($channel)) { $channel = array(); } if (!isset($channel['title'])) { $channel['title'] = $title_for_layout; } echo $rss->document($rss->channel(array(), $channel, $content_for_layout)); Here in the layout our RssHelper shines through. We use the method RssHelper::channel() which generates the element and associated metadata elements. The $content_for_layout variable contains the output from the view. These then get passed to the RssHelper::document() method, which wraps the RSS document in the respective elements.

Controller

The controller needs no modification in the case of a simple RSS feed. This is because we are only adding a second view that is xml/rss to the action. The same data is used in both views and because CakePHP automatically sets the correct response type we don't need to tell it to render the correct view and layout for RSS. Here is the action method in the EntriesController for a basic view sorted by a published_date field and showing only if it is published. public function index() { $this->paginate['Entry'] = array( 'conditions' => array('Entry.published' => 1), 'order' => 'Entry.published_date DESC'); $this->set('entries', $this->paginate()); } If you do have code that is specific for only the RSS view you can use the RequestHandler::isRss() to see if the action was called with the request for xml/rss formatting on response. This method returns a boolean value based on if the .rss extension was parsed in the URL. if ($this->RequestHandler->isRss()) { // RSS feed specific code goes here }

Note About Channel Metadata

It may feel right to put your metadata information in the index method in the controller, using Controller::set() to send the information to the views. This is inappropriate and is one of the most common snags that we have seen in the CakePHP community with creating RSS feeds. That information which is passed in the layout file to the RssHelper::channel() method should be set in the view using View::set() which will set the $channel variable for the layout in the view.

Views

As we had to put the layout in a subdirectory of the layouts folder we also need to create a view for the index action for the blogs controller. This is done by creating a directory /views/entries/rss/ which will hold our view file that will generate the RSS to render. You will need to add your RssHelper to the list of helpers in your controller so that it is automatically loaded in the view and the layout. Our view begins by setting the $channel variable for the layout, this contains all the metadata for our RSS feed. $homeUrl = $html->url('/', true); $this->set('channel', array( 'title' => __("Daniel's Recent Articles", true), 'link' => $homeUrl, 'description' => __("Most recent articles from Daniel.", true), 'language' => 'en-us', 'image' => array( 'title' => 'Recent Articles from Daniel', 'url' => FULL_BASE_URL . $this->webroot('/img/rss_feed_image', true), 'link' => $homeUrl)); First we get the URL link for the website home that we will use for the links. Also we set the title, description and image to use for the RSS feed icon. By setting the channel variable using View::set() we are providing the layout the information to render the RSS feed's metadata elements. The second part of the view generates the elements for the actual records of the feed. This is accomplished by looping through the data that has been passed to the view and using the RssHelper::item() method. The other method you can use, RssHelper::items() which takes a callback and an array of items for the feed. (The method I have seen used for the callback has always been called transformRss(). There is one downfall to this method, which is that you cannot use any of the other helper classes to prepare your data inside the callback method because the scope inside the method does not include anything that is not passed inside, thus not giving access to the TimeHelper or any other helper that you may need. The RssHelper::item() transforms the associative array into an element for each key value pair. foreach ($entries as $entry) { $postTime = strtotime($entry['Entry']['created']); $entryLink = array( 'controller' => 'entries', 'action' => 'view', 'year' => date('Y', $postTime), 'month' => date('m', $postTime), 'day' => date('d', $postTime), $entry['Entry']['slug']); // This is the part where we clean the body text for output as the description // of the rss item, this needs to have only text to make sure the feed validates $bodyText = preg_replace('=\(.*?)\=is', '', $entry['Entry']['body']); $bodyText = $text->stripLinks($bodyText); $bodyText = Sanitize::stripAll($bodyText); $bodyText = $text->truncate($bodyText, 400, '...', true, true); echo $rss->item(array(), array( 'title' => $entry['Entry']['title'], 'link' => $entryLink, 'guid' => array('url' => $entryLink, 'isPermaLink' => 'true'), 'description' => $bodyText, 'dc:creator' => $entry['Entry']['author'], 'pubDate' => $entry['Entry']['created'])); } You can see above that we can use the loop to prepare the data to be transformed into XML elements. It is important to filter out any non-plain text charictars out of the description, especially if you are using a rich text editor for the body of your blog. In the code above we use the TextHelper::stripLinks() method and a few methods from the Sanitize class, but we recommend writing a comprehensive text cleaning helper to really scrub the text clean. Once we have set up the data for the feed, we can then use the RssHelper::item() method to create the XML in RSS format. Once you have all this setup, you can test your RSS feed by going to your site /entries/index.rss and you will see your new feed. It is always important that you validate your RSS feed before making it live. This can be done by visiting sites that validate the XML such as Feed Validator or the w3c site at http://validator.w3.org/feed/.

Lighty Story

I will tell you a story. Once upon a time... Seriously though, it was not too long ago in the past - but it happened and it is possible you can benefit from it.

What?

This tutorial will show how to make lighttpd 1.4.20 serve virtual hosts with CakePHP applications. Our scenario is quite simple:
  1. For admin purposes, lighttpd will listen on localhost, it will serve several CakePHP applications on several external ip addresses, without SSL.
  2. Virtual hosts will be organized in groups and every group will use one CakePHP core checkout for its virtual hosts.
  3. Every virtual host will have it own access log (this server will not run hundreds of virtual hosts, so we can afford to waste one file descriptor for each) and its own directory for caching of compressed static files.
  4. Management of virtual hosts, their default and custom settings should be as easy as possible, so we can delegate the management of some ip addresses or just groups of virthosts to someone else and sleep well, because nobody will have to touch our precious configuration files.
However, our scenario has some special requirements which we need to solve. By the way, I will be showing you how to do things the hard way from the start. In hopes to spare you a lot of headaches in future. Lighttpd is sweet piece of software, and is under active development. Unfortunately, there are things that are not easy to set up. For example - when using any of provided virtual host modules, it is impossible to set up different access logs and cache directories for compressed content etc. dynamically in a pure lighty config file without external scripts. Everything (except for per virtual host errorlog) is possible by writing necessary configuration by hand. But we willing to work more now, so we can be lazy later! There are several approaches for bash, Ruby etc. However, nothing usable in PHP as far as I know. I will show you how easy it could be. Take this as a working example, I am sharing ideas here, not bullet-proof all-mighty solutions. Lets go for it - and utilize PHP and the include_shell command in our lighttpd configuration file. The motto of this article is: it is easier read generated configuration, then write it by hand.

How? Lighty!

Don't think this is not a good answer. Lets set up a decent lighttpd installation. We'll assume you have it compiled and installed. Lets also assume that you have PHP prepared for lighttpd's ModFastCGI and are just waiting for configuration and the first test run. Also, for shell commands which need to be executed under root account, I'll use sudo in following examples. sudo mkdir /usr/local/etc/lighttpd First of all, we need a directory for our custom configuration. When in doubt, a fast look into its contents will tell you everything one should know about virtual hosts configuration. sudo mkdir -p /usr/local/www/data/default/webroot echo "<html><head><title>It works<body>It works" > /usr/local/www/data/default/webroot/index.html Next we created a directory for our default webroot. It will be used on localhost only, with index.html. sudo touch /var/log/lighttpd.error.log /var/log/lighttpd.access.log sudo chown www:www /var/log/lighttpd.error.log /var/log/lighttpd.access.log Now we need to create error and access log files. The first one will be common for whole server, the second will be used for localhost only. sudo mkdir -p /var/cache/lighttpd/compress/default sudo chown -R www:www /var/cache/lighttpd The last thing we had to prepare was the default directory for caching of compressed static files. In /usr/local/etc/lighttpd.conf we will setup a simple config file containing the common configuration we will utilize later: server.modules = ( "mod_simple_vhost", "mod_magnet", "mod_redirect", "mod_access", "mod_auth", "mod_expire", "mod_compress", "mod_fastcgi", "mod_accesslog" ) server.document-root = "/usr/local/www/data/default/webroot/" server.errorlog = "/var/log/lighttpd.error.log" accesslog.filename = "/var/log/lighttpd.access.log" server.port = 80 server.bind = "127.0.0.1" server.username = "www" server.groupname = "www" server.pid-file = "/var/run/lighttpd.pid" index-file.names = ( "index.php", "index.html", "index.htm", "default.htm" ) # shortened !!! mimetype.assign = ( ... ) url.access-deny = ( "~", ".inc" ) static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" ) dir-listing.activate = "disable" etag.use-mtime = "enable" static-file.etags = "enable" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ( "" => "access 7 days" ) } compress.cache-dir = "/var/cache/lighttpd/compress/default/" compress.filetype = ( "text/plain", "text/html", "text/xml", "text/javascript", "text/css" ) fastcgi.server = ( ".php" => (( "bin-path" => "/usr/local/bin/php-cgi -c /usr/local/etc/php.ini", "socket" => "/tmp/lighttpd_php5.socket", "min-procs" => 1, "max-procs" => 1, "bin-environment" => ( "FCGI_WEB_SERVER_ADDRS" => "127.0.0.1", "PHP_FCGI_CHILDREN" => "4", "PHP_FCGI_MAX_REQUESTS" => "1000" ), "bin-copy-environment" => ( "PATH", "SHELL", "USER"), "broken-scriptfilename" => "enable" )) ) simple-vhost.server-root = "/usr/local/www/data/" simple-vhost.document-root = "webroot" simple-vhost.default-host = "default" $HTTP["host"] =~ "^www\.(.*)" { url.redirect = ( "^/(.*)" => "http://%1/$1" ) } How far along are we? So far we have a configured webserver with few preloaded modules and simple common configuration. Our sever is currently:
  1. Listening on localhost:80.
  2. Refusing directory listing or sending some filetypes as plain text.
  3. Using etags and sending expiration headers for a set of static resources to 7 days by default. This allows us to schedule an upgrade of any virtual host just a week before it will happen.
  4. Using compression and caching of compressed static files for several mimetypes.
  5. Starting PHP as FastCGI, with only one parent process (we are going to use opcode cache). We are allowing only few child processes for this example tutorial and killing fcgi child processes after every 1000 requests
  6. Using mod_simple_vhost for name-based virtual hosting (preconfigured for fallback to default webroot).
  7. Redirecting all domains using www subdomain to the shorter version.
You will probably want to tweak some other settings. I am not going to describe all the server.max* configuration options, or talk about other pretty obvious things like mod_evasive, mod_status, mod_rrdtool etc, don't worry. Two things you should consider if some of your visitors will use one of the major browsers. $HTTP["url"] =~ "\.pdf$" { server.range-requests = "disable" } You do not want to cut off IE users from your pdf documents, right? compress.filetype = ( "text/plain", "text/html", "text/xml" ) $HTTP["useragent"] =~ "Firefox" { compress.filetype += ("text/javascript", "text/css" ) } If your visitors are using an old (and/or above mentioned undesirable) internet browser, you can control compression settings per useragent in this way. Instead of the above example, compressing all 5 crucial mimetypes. Ready to go? Ok, start lighttpd and make sure you see what you expect at http://localhost/ echo "<?php phpinfo(); ?>" > /usr/local/www/data/default/webroot/phpinfo.php Just to be sure that fcgi works as expected, try to see info about your current PHP setup at http://localhost/phpinfo.php and watch /var/log/lighttpd.error.log.

Url rewriting

It is possible to use lighttpd's mod_rewrite and create pattern for our static files if we are sure they exist. This approach has downsides though. We want to setup this part of webserver up and forget it exists. This is not possible with mod_rewrite, because for example, we are not going to force our developers to forget about /js/something.js as url for some of application controllers. Instead, we will use mod_magnet and custom Lua script. Visit this thread at CakePHP Google Group. Save the provided script to /usr/local/etc/lighttpd/cleanurl-v6.lua and add the following line to bottom of /usr/local/etc/lighttpd.conf: magnet.attract-physical-path-to = ( "/usr/local/etc/lighttpd/cleanurl-v6.lua" ) After restarting lighttpd, we are ready to remove all the .htaccess files from our filesystem and forget they exist. All requests for non-existing static files will be rewritten to /index.php?url=xxx like CakePHP requires.

Virtual hosts

Now we want to set up a directory structure and custom configuration for our virtual hosts and their groups. We will design a directory structure that can be used for dynamic configuration later, with no need to repeat anything obvious in configuration files. In this case, only logs folder matters (make sure it is writable by webserver). We will symlink everything else. Lets use the following directory structure with CakePHP core and our applications checkouts like our standard: # example.com (with redirect from www.example.com) /home/company/ logs/ www/ cake/ mainsite/ ... webroot/ vendors/ # dev-main.example.com and dev-product.example.com /home/development/ logs/ www/ cake/ mainsite/ ... webroot/ product/ ... webroot/ vendors/ # stage-main.example.com and stage-product.example.com /home/staging/ logs/ www/ cake/ mainsite/ ... webroot/ product/ ... webroot/ vendors/ # api.example.com, book.example.com, product.com ( with redirect from www.product.com) /home/product/ logs/ www/ api/ ... index.html book/ ... webroot/ cake/ product/ ... webroot/ vendors/ If you think the above directory tree is overcomplicated, or it seems too long for simple tutorial example, stop reading please, and feel free to come back any time later. It was nice to meet you :-) Things are only getting worse from here on in. For those brave enough to read on, you should have an idea of which domains will use which applications, and which applications will share one CakePHP core and folder for logs (not necessarily, read more). Now we are getting somewhere - we need tell our webserver on which external ip addresses it has to listen for incoming connections, and which virtual hosts map to each ip address. Our www subdomains (redirected) should listen on a different ip address then their short versions. This allows us to use different SSL certificates for them later, if there is a need for secure connections. To show what is possible with our config parser, api.example.com will not use a /webroot/ folder, it contains just static html files. To make things even more tricky, api.example.com and book.example.com will not listen on same ip like their neighbour application product.com. cd /usr/local/etc/lighttpd From now on, we will continue our work in this directory. Lets say that we want to use ip 1.2.3.4 for domains example.com, api.example.com and book.example.com. sudo mkdir -p ./1.2.3.4:80/company sudo ln -s /home/company/www/cake ./1.2.3.4:80/company/cake sudo ln -s /home/company/www/vendors ./1.2.3.4:80/company/vendors sudo ln -s /home/company/www/mainsite ./1.2.3.4:80/company/example.com sudo mkdir ./1.2.3.4:80/product sudo ln -s /home/product/www/cake ./1.2.3.4:80/product/cake sudo ln -s /home/product/www/vendors ./1.2.3.4:80/product/vendors sudo ln -s /home/product/www/api ./1.2.3.4:80/product/api.example.com sudo ln -s /home/product/www/book ./1.2.3.4:80/product/book.example.com What exactly did we just do? We created a folder named 1.2.3.4:80, containing 2 subfolders company and product. These will be used as groups of virtual hosts - their names should be the same as the name of their home directory (by default, path for logs can be adjusted). We will use them for setting paths to log files later. Both company and product have a symlinked cake and vendors folders and symlinks named as real domains and pointing to our app folders. Lets continue - ip 2.3.4:5:80 will be used for rest of the group product. sudo mkdir -p ./2.3.4.5:80/product sudo ln -s /home/product/www/cake ./2.3.4.5:80/product/cake sudo ln -s /home/product/www/vendors ./2.3.4.5:80/product/vendors sudo ln -s /home/product/www/product ./2.3.4.5:80/product/product.com That means only one virtual host for now. Ok, ip 3.4.5.6 is going to be used for the www subdomains. No symlinks to existing applications are necessary here, because lighttpd will redirect requests coming to www.example.com to example.com automatically. sudo mkdir -p ./3.4.5.6:80/company/www.example.com ./3.4.5.6:80/product/www.product.com We just had to create ip:port directory for the socket, group(s) of www virtualhosts and some domain-based directories just to have something to point default virtual host of this group at. Staging and development checkouts will all share one ip 4.5.6.7. sudo mkdir -p ./4.5.6.7:80/development sudo ln -s /home/development/www/cake ./4.5.6.7:80/development/cake sudo ln -s /home/development/www/vendors ./4.5.6.7:80/development/vendors sudo ln -s /home/development/www/mainsite ./4.5.6.7:80/development/dev-main.example.com sudo ln -s /home/development/www/product ./4.5.6.7:80/development/dev-product.example.com sudo mkdir ./4.5.6.7:80/staging sudo ln -s /home/staging/www/cake ./4.5.6.7:80/staging/cake sudo ln -s /home/staging/www/vendors ./4.5.6.7:80/staging/vendors sudo ln -s /home/staging/www/mainsite ./4.5.6.7:80/staging/stage-main.example.com sudo ln -s /home/staging/www/product ./4.5.6.7:80/staging/stage-product.example.com Four virtual hosts on one ip from different home folders (therefore placed in different groups). The hard part is complete. Lets go through the bothering part of this custom setup. Did I said already that everything is a file? Don't be scared from amount of necessary steps, it will all be worth it in the future. Lets look what we have done in directory /usr/local/etc/lighttpd/: 1.2.3.4:80/ company/ cake/ <-- /home/company/www/cake example.com/ <-- /home/company/www/mainsite vendors/ <-- /home/company/www/vendors product/ api.example.com/ <-- /home/product/www/api book.example.com/ <-- /home/product/www/book cake/ <-- /home/product/www/cake vendors/ <-- /home/product/www/vendors 2.3.4.5:80/ product/ cake/ <-- /home/product/www/cake product.com/ <-- /home/product/www/product vendors/ <-- /home/product/www/vendors 3.4.5.6:80/ company/www.example.com/ <-- empty directory (redirected), necessary for default virtual host product/www.product.com/ <-- empty directory (redirected), necessary for default virtual host 4.5.6:7:80/ development/ cake/ <-- /home/development/www/cake dev-main.example.com/ <-- /home/development/www/mainsite dev-product.example.com/ <-- /home/development/www/product vendors/ <-- /home/development/www/vendors staging/ cake/ <-- /home/staging/www/cake stage-main.example.com/ <-- /home/staging/www/mainsite stage-product.example.com/ <-- /home/staging/www/product vendors/ <-- /home/staging/www/vendors Some new folders with symlinks. Are you still with me? For those who know mod_simple_vhost, you should be already be pretty clear where we are going. Besides the accesslog path and compress folder path, we will also switch simple-vhost.server-root and simple-vhost.default-host in dependency of used socket and some hostname condition for virthost group. Actually, there is a bit more as well that I will show you. The above directory structure shows that we have 7 groups of virtual hosts in 4 sockets, so lets create 7 simple configuration files for our groups of virtual hosts. Configuration file for group is not required in very special case - no regex pattern for this group, only one virtual host inside and - either only group in socket, or (alphabetically) last one. <?php # /usr/local/etc/lighttpd/1.2.3.4:80/company/config.php $config['group'] = array( 'host' => '^example\.com', 'default' => 'example.com' ); ?> <?php # /usr/local/etc/lighttpd/1.2.3.4:80/product/config.php $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => 'book.example.com' ); ?> <?php # /usr/local/etc/lighttpd/2.3.4.5:80/product/config.php $config['group'] = array( 'host' => '^product\.com', 'default' => 'product.com' ); ?> <?php # /usr/local/etc/lighttpd/3.4.5.6:80/company/config.php $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => 'www.example.com' ); ?> <?php # /usr/local/etc/lighttpd/3.4.5.6:80/product/config.php $config['group'] = array( 'host' => '^(.*)\.product\.com', 'default' => 'www.product.com' ); ?> <?php # /usr/local/etc/lighttpd/4.5.6:7:80/development/config.php $config['group'] = array( 'host' => '^dev-(.*)\.example\.com', 'default' => 'dev-main.example.com' ); ?> <?php # /usr/local/etc/lighttpd/4.5.6:7:80/staging/config.php $config['group'] = array( 'host' => '^stage-(.*)\.example\.com', 'default' => 'stage-main.example.com' ); ?> And that's it. Every group (subfolder of ip.ad.dr.es:80 socket folder) has the required minimal configuration, and everything is properly set up. So lets see what we can take off from it.

Dynamic configuration

Extract this file in folder /usr/local/etc/lighttpd. sudo chmod a+x ./simple_config.php Make simple_config.php executable for everyone. Now run it as a non-privileged user. ./simple_config.php | more You should see a basic generated configuration for your sockets, virthosts and virthosts groups. Now we are already looking at a snippet of the generated configuration. # # Simple configuration parser output # # ERROR logfile /home/company/logs/example-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/example.com/ can not be created, SKIPPING # ERROR logfile /home/product/logs/api-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/api.example.com/ can not be created, SKIPPING # ERROR logfile /home/product/logs/book-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/book.example.com/ can not be created, SKIPPING # ERROR logfile /home/product/logs/product-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/product.com/ can not be created, SKIPPING # ERROR logfile /home/company/logs/www-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/www.example.com/ can not be created, SKIPPING # ERROR logfile /home/product/logs/www-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/www.product.com/ can not be created, SKIPPING # ERROR logfile /home/development/logs/dev-main-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/dev-main.example.com/ can not be created, SKIPPING # ERROR logfile /home/development/logs/dev-product-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/dev-product.example.com/ can not be created, SKIPPING # ERROR logfile /home/staging/logs/stage-main-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/stage-main.example.com/ can not be created, SKIPPING # ERROR logfile /home/staging/logs/stage-product-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/stage-product.example.com/ can not be created, SKIPPING # $SERVER["socket"] == "1.2.3.4:80" { $HTTP["host"] =~ "^example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/company/" simple-vhost.default-host = "example.com" $HTTP["host"] == "example.com" { .... You can see which files this script is trying to create. It will create all of them when you will run it as root once. But there are two things we would like to fix first: access logs /home/company/logs/www-access_log and /home/product/logs/www-access_log are generated for our redirected domains. Lets redirect these logs to those used by domains example.com and product.com: <?php # /usr/local/etc/lighttpd/3.4.5.6:80/company/config.php $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => 'www.example.com' ); $config['virthosts'] = array( 'www.example.com' => array( 'log' => 'example' ) ); ?> <?php # /usr/local/etc/lighttpd/3.4.5.6:80/product/config.php $config['group'] = array( 'host' => '^(.*)\.product\.com', 'default' => 'www.product.com' ); $config['virthosts'] = array( 'www.product.com' => array( 'log' => 'product' ) ); ?> Running ./simple_config.php as unprivileged user again shows this script is no longer trying to create any www-access_log files. We will not care about directories for compressed content, they can be used later, but we will never serve different content on example.com and www.example.com, so it is logical that they share one log file. Every decent logfile parser can handle several domains in one log file. Now, you can run this script as root: sudo ./simple_config.php and result will look much better now: # # Simple configuration parser output # # NOTICE created logfile /home/company/logs/example-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/example.com/ # NOTICE created logfile /home/product/logs/api-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/api.example.com/ # NOTICE created logfile /home/product/logs/book-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/book.example.com/ # NOTICE created logfile /home/product/logs/product-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/product.com/ # NOTICE created compress cache /var/cache/lighttpd/compress/www.example.com/ # NOTICE created compress cache /var/cache/lighttpd/compress/www.product.com/ # NOTICE created logfile /home/development/logs/dev-main-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/dev-main.example.com/ # NOTICE created logfile /home/development/logs/dev-product-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/dev-product.example.com/ # NOTICE created logfile /home/staging/logs/stage-main-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/stage-main.example.com/ # NOTICE created logfile /home/staging/logs/stage-product-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/stage-product.example.com/ # $SERVER["socket"] == "1.2.3.4:80" { $HTTP["host"] =~ "^example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/company/" simple-vhost.default-host = "example.com" $HTTP["host"] == "example.com" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/example.com/" } } else $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/product/" simple-vhost.default-host = "book.example.com" $HTTP["host"] == "api.example.com" { accesslog.filename = "/home/product/logs/api-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/api.example.com/" } else $HTTP["host"] == "book.example.com" { accesslog.filename = "/home/product/logs/book-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/book.example.com/" } } } $SERVER["socket"] == "2.3.4.5:80" { $HTTP["host"] =~ "^product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/2.3.4.5:80/product/" simple-vhost.default-host = "product.com" $HTTP["host"] == "product.com" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/product.com/" } } } $SERVER["socket"] == "3.4.5.6:80" { $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/company/" simple-vhost.default-host = "www.example.com" $HTTP["host"] == "www.example.com" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/www.example.com/" } } else $HTTP["host"] =~ "^(.*)\.product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/product/" simple-vhost.default-host = "www.product.com" $HTTP["host"] == "www.product.com" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/www.product.com/" } } } $SERVER["socket"] == "4.5.6.7:80" { $HTTP["host"] =~ "^dev-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/development/" simple-vhost.default-host = "dev-main.example.com" $HTTP["host"] == "dev-main.example.com" { accesslog.filename = "/home/development/logs/dev-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/dev-main.example.com/" } else $HTTP["host"] == "dev-product.example.com" { accesslog.filename = "/home/development/logs/dev-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/dev-product.example.com/" } } else $HTTP["host"] =~ "^stage-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/staging/" simple-vhost.default-host = "stage-main.example.com" $HTTP["host"] == "stage-main.example.com" { accesslog.filename = "/home/staging/logs/stage-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/stage-main.example.com/" } else $HTTP["host"] == "stage-product.example.com" { accesslog.filename = "/home/staging/logs/stage-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/stage-product.example.com/" } } } Getting close to what we need from this setup. I will process several steps now, and then I will paste here final output of config parser for you to compare with above one. We have another domain manual.example.com (with no virthost set) and we want to redirect it to api.example.com with configuration only, it will be using its own manual-access_log. Furthermore, we want book.example.com condition happen sooner then the condition on api.example.com, because book is gaining more traffic, and attach domain aliases bibliotheca.example.com and bookstore.example.com to book.example.com. Also, expire headers for book should be set for 2 years and as previously mentioned api.example.com is not using /webroot/ folder. <?php # /usr/local/etc/lighttpd/1.2.3.4:80/product/config.php $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => 'book.example.com' ); $config['virthosts'] = array( 'book.example.com' => array( 'expire' => array( '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 2 years' ), 'aliases' => array( 'bibliotheca.example.com', 'bookstore.example.com' ) ), 'api.example.com' => array( 'webroot' => '/' ), 'manual.example.com' => array( 'redirect' => 'http://api.example.org/' ) ); ?> All of it is fixed now. We even do not need folder/symlink for manual.example.com in this case. Important note: we do not have to create folders for domains bibliotheca.example.com and bookstore.example.com, because they are aliases for book.example.com and it is used as default virtual host for this group! If you will set alias for non-default virtual host, you have to symlink aliased application several times to group folder - every time with a different domain name. We want all staging sites to store logs in /home/development/logs. Also all staging and development sites should use expire headers for 5 minutes only and have to use http auth (one common file for now). <?php # /usr/local/etc/lighttpd/4.5.6:7:80/development/config.php $config['group'] = array( 'host' => '^dev-(.*)\.example\.com', 'default' => 'dev-main.example.com', 'expire' => array( '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 5 minutes' ), 'auth' => array( 'backend' => 'htpasswd', 'file' => '/var/projects/company/.trac.htpasswd', 'protect' => array( '/' => array( 'realm' => 'Development Access', 'require' => 'valid-user' ) ) ) ); ?> <?php # /usr/local/etc/lighttpd/4.5.6:7:80/staging/config.php $config['group'] = array( 'host' => '^stage-(.*)\.example\.com', 'default' => 'stage-main.example.com', 'expire' => array( '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 5 minutes' ), 'logs' => '/home/development/logs', 'auth' => array( 'backend' => 'htpasswd', 'file' => '/var/projects/company/.trac.htpasswd', 'protect' => array( '/' => array( 'realm' => 'Staging Access', 'require' => 'valid-user' ) ) ) ); ?> This has all been fixed now. Now our simple_config.php returns this: # # Simple configuration parser output # $SERVER["socket"] == "1.2.3.4:80" { $HTTP["host"] =~ "^example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/company/" simple-vhost.default-host = "example.com" $HTTP["host"] == "example.com" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/example.com/" } } else $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/product/" simple-vhost.default-host = "book.example.com" $HTTP["host"] =~ "^(book\.example\.com|bibliotheca\.example\.com|bookstore\.example\.com)" { accesslog.filename = "/home/product/logs/book-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/book.example.com/" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ("" => "access 2 years") } } else $HTTP["host"] == "api.example.com" { accesslog.filename = "/home/product/logs/api-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/api.example.com/" simple-vhost.document-root = "/" } else $HTTP["host"] == "manual.example.com" { accesslog.filename = "/home/product/logs/manual-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/manual.example.com/" url.redirect = ( ".*" => "http://api.example.org/" ) } } } $SERVER["socket"] == "2.3.4.5:80" { $HTTP["host"] =~ "^product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/2.3.4.5:80/product/" simple-vhost.default-host = "product.com" $HTTP["host"] == "product.com" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/product.com/" } } } $SERVER["socket"] == "3.4.5.6:80" { $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/company/" simple-vhost.default-host = "www.example.com" $HTTP["host"] == "www.example.com" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/www.example.com/" } } else $HTTP["host"] =~ "^(.*)\.product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/product/" simple-vhost.default-host = "www.product.com" $HTTP["host"] == "www.product.com" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/www.product.com/" } } } $SERVER["socket"] == "4.5.6.7:80" { $HTTP["host"] =~ "^dev-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/development/" simple-vhost.default-host = "dev-main.example.com" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ("" => "access 5 minutes") } auth.backend = "htpasswd" auth.backend.htpasswd.userfile = "/var/projects/company/.trac.htpasswd" auth.require = ( "/" => ( "method" => "basic", "realm" => "Development Access", "require" => "valid-user" ) ) $HTTP["host"] == "dev-main.example.com" { accesslog.filename = "/home/development/logs/dev-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/dev-main.example.com/" } else $HTTP["host"] == "dev-product.example.com" { accesslog.filename = "/home/development/logs/dev-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/dev-product.example.com/" } } else $HTTP["host"] =~ "^stage-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/staging/" simple-vhost.default-host = "stage-main.example.com" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ("" => "access 5 minutes") } auth.backend = "htpasswd" auth.backend.htpasswd.userfile = "/var/projects/company/.trac.htpasswd" auth.require = ( "/" => ( "method" => "basic", "realm" => "Staging Access", "require" => "valid-user" ) ) $HTTP["host"] == "stage-main.example.com" { accesslog.filename = "/home/development/logs/stage-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/stage-main.example.com/" } else $HTTP["host"] == "stage-product.example.com" { accesslog.filename = "/home/development/logs/stage-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/stage-product.example.com/" } } } Now it looks like we are set with everything we needed. One last line for /usr/local/etc/lighttpd.conf is: include_shell "/usr/local/etc/lighttpd/simple_config.php" And that's all. Before you will start or restart lighttpd, try and see if it can parse the new configuration (with our include) without errors, or inspect how it sees configuration after parsing: lighttpd -t -f /usr/local/etc/lighttpd.conf lighttpd -p -f /usr/local/etc/lighttpd.conf It is better to run the above commands as root, off course.

Now what?

Think twice about patterns for groups - don't be surprised if you get 'It works' page or default virthost of another group, if you are too lazy to read the generated configuration! Groups are processed in alphabetical order - just so you know which patterns are going to be checked first. Well, it is possible to change order of groups - change name of some company group folder to xxx_company and: $config['group'] = array( 'name' => 'company', Now you should be fine - this group in folder named xxx_company instead of company, and everything will still work. Everything that is necessary should be up and running now. Lighttpd should serve all virtual hosts from groups in sockets from now on. Read how to clear cache for mod_compress too. Smart brain should ask now, why we are using mod_simple_vhost, if our parser generates configuration for every virtual host it founds in our configuration files and directory structure. We don't do it, but you can - read code. Note for these who do not want or can not follow our default logs location, home directories, cache directories, user account lighttpd will use, or want to store directory structure with sockets/groups/virthosts somewhere else - read code too ;-) Reason why we set mod_simple_vhost for this example as default is simple - to get some domain serving some application, we need only one simple thing: symlink to app directory with domain name, placed in some virtual group in proper socket. This virtual host will be accessible immediately - although, restart of webserver is still necessary to have configuration for access logfile and compress directory for this virtual host (otherwise default accesslog and compress dir will be used), but not required. A few questions remain, what and how needs to be done in obvious use cases - adding new ip addresses, groups, virthosts, or moving whole groups over sockets, moving virthosts over sockets, etc... I assume this part will be sweet piece of cake for you. Definitely - feel free to call simple_config.php as often as you want to. It is highly reccommended to save functional configuration to a backup file by redirecting the output. Sure, one can use include "/some/path/generated_output.conf" exclusively, instead of include_shell - it is up to you. Backup, backup, backup. This is nothing more then a functional example, but the entire code lives in one class, so feel free to change or extend it for your needs. It is released under MIT license and is provided as it is, so you can do anything you want with it (except for removing license and copyright note). Keep in mind it was not tested in all possible situations and some of things I did not mention in this tutorial (but they are implemented in code) were not intensively tested yet. If you feel that some of the subdomains used in this tutorial sound familiar to you, you are probably right. I didn't said it was going to be a fairy tale. I said, I will tell you a story. To be continued...

Basic CakePHP markup for designers

This article assumes you practice semantic markup and have a fairly good understanding of CSS. CakePHP does a number of things automatically that designers might not expect, which can cause confusion for designers and difficulty for developers in implementing markup. I hope to provide some general purpose information and guidelines to designing for a CakePHP site. I would like to make a couple quick notes first however. CakePHP is a very flexible platform where markup can be re-used very easily. The more you as a designer try to make reusable markup, the easier your developers life will be. One of the most important things that you can do is to avoid id's other than for top level wrappers such as columns. This allows the developer to change content to address shifting specifications without having to rewrite chunks of CSS in order to avoid Id conflicts in the generated markup. CakePHP has a flexible output system that lets developers easily specify how output is generated. Unlike some applications such as older CMS systems which have a specified header and footer that are called before and after the content, Cake applications render their (x)html into a layout. Two main types of output are rendered into a layout, Views and Elements. Views are the page specific content, such as a blog post, or an entry form. Elements are blocks of markup that can be used across multiple pages, and can be used in the layout, in the view, or even from inside another element. Form related tags can appear in any of the output region types, but it is helpful if you as a designer try to avoid having forms span multiple elements or different sections. Doing so makes it much easier to ensure that the form will work correctly every time it appears. By default most form elements are wrapped in a div with descriptor classes such as “input text” for a text field. Check boxes, and radio buttons are wrapped in a fieldset instead of a div. Also by default the form helper automatically generates id's for the form input widget, and a label that matches it. This is another situation where it is easier to avoid an id attribute and use a class name as a designer. If you specify and ID, your developer has to check that it has not been used in CSS to specify a style, or choose to override it with your specified value and loose flexibility in the future. Examples of form widgets: Select box:   Blog blog. Text field:   Title Check Boxes:   Status Published Allow Comments Moderate   The other CakePHP core utility that generates a fair amount of HTML is the paginator helper. While the helper itself has been covered on the bakery before and is worthy of several blog posts itself, I'd at least like to touch on the default markup generated by the helper.   You can easily specify a string that will appear in between the spans, and each of the other elements are called separately, so they can be in other spots on the page, or have markup between. Unfortunately it is not easily possible to remove the span from the markup, but it would be fairly easy to change the pagination output to something similar to this:  

Meet the CakeDC

The articles section of our site will include best practices, business advice, and technical advice, updated weekly or more often by our talented developers. Our articles will touch upon a wide range of topics from our experiences and expertise. Check back frequently for fresh thoughts from our seasoned talent, including Mark Story who was at CakeFest in Argentina providing updates daily on the talks. As a founder of and lead developer at the Cake Development Corporation, I am proud to provide the inaugural article for the all new CakeDC.com. It has been a long and wonderful last 12 months, and CakeDC has helped bring reality to the limitless potential of our clients' projects. Like any great recipe, the ingredients of Cake Development Corporation have been hand–selected and carefully measured to create the very best blend of talent, imagination, and sophistication of any team in the world. It is my great pleasure to introduce you to our outstanding development team, as well as provide a little background about where our company has come from and where we are headed. After spending nearly 2 years working exclusively on CakePHP and seeing its popularity growing daily, Garrett Woodworth and I realized the need for a team of experts devoted to building the best applications for clients. Therefore, in 2007, Garrett and I formed Cake Development Corporation – a company dedicated to bringing the potential of CakePHP to life while supporting its further development and aiding clients in the rapid development of web applications. In this venture, Garrett and I recognized that we had a unique and rarely realized opportunity to do what we love as our full–time jobs – and, unlike during those initial two years of development, actually get paid to do it. As a business partner, I could not have asked for anyone better than Garrett. A key player in the development of CakePHP, Garrett continues to develop the code and drive the machines that keep users coming back for more. He is extremely passionate about the work he does and takes pride in delivering quality products that are the best of their kind, a true leader the team members can look up to. In fact, I have come to think of him as a younger brother – someone whom I respect greatly, but with whom I also spar from time to time (in a healthy way, of course). After a short time, it became necessary to bring on some new talent. Garrett and I welcomed a third developer Jitka Koukalová, to our small family in 2007. Jitka, a developer who was active in the CakePHP community and was vital contributor to its code, was an amazing addition to our team from the beginning, and exhibited great skill in, and equally great appreciation for, our open source platform. With great attention to detail, she became instrumental in server security and maintenance. Her ability to find an issue and fix it fast along with her classy and professional demeanor would make her a mentor to future members of the Cake development team. Florian Krämer, the next to join CakeDC, soon became our second great recruit. Florian has continually proven himself to be extremely knowledgeable when it comes to developing applications. Florian's meticulous attention to detail, outstanding work ethic, and dogged pursuit of excellence continue to be hugely important assets of our company. But more than a colleague, I consider Florian a good friend. Apart from his great skill, his fun and personable demeanor make him a great team member and someone I genuinely enjoy working with. He has a great sense of humor – or, at least, he appreciates mine! Having seen a great deal of growth in 2007, we sought to expand our development team in early 2008 by welcoming aboard a whole new crop of distinguished developers with a great working knowledge of CakePHP and a passion for its vast capabilities. The first of this group was Yevgeny Tomenko. Yevgeny knows an astounding amount about the inner workings of application development, and I am continually amazed at his dependability and sheer speed. Since joining our team, he has become a great "go–to" person who uses his years of experience to help guide the others on the team. I really love how passionate he is about our product, and it’s great to see such a talented and focused guy enjoying his work at CakeDC. Next, after spending some time watching Niles Rowland help others in the CakePHP IRC channel, we realized he would be a true asset to the CakeDC team. We recruited him in early 2008, and since then he has been a very knowledgeable and dependable part of the development team with a vast understanding of CakePHP and programming in general. Soon to follow Niles were Erin McCargar and Daniel Feinberg, both of whom came aboard in May of 2008. Erin has a long history working with CakePHP and is looked up to as an advisor by those on her team. Daniel, a key person when it comes to machine learning, is very knowledgeable with CakePHP. With outstanding attention to detail and a great skill in their fields of expertise, both are willing and able to go the extra mile to make a project shine. Finally, our most recent addition to the family is Mark Story who joined the team in November of 2008. Mark brings a lot to CakeDC; he is a core developer of CakePHP with design experience that is second to none. Mark is all about getting things done right the first time, and is great at working with others. The rest of the team truly looks up to him for both his skill and amicable personality. This eclectic group of talent is the heart and soul of CakeDC, the family unit that makes our company stand a head and shoulders above the rest. After seeing how far we have come in the last two years, I am very excited to see what this team can accomplish next! We are looking forward to all the challenges that lay ahead and can’t wait to help our clients create a brighter, more innovative future.

We Bake with CakePHP