CakeDC Blog


Joël Perras - Demystifying Webservices...

Joël's presentation on Web Services and CakePHP identifies important and interesting points that really demystify both implementation of datasources, and what web services mean for developers trying to take advantages of their offerings. A Web Service is a defined interface. The interface is made known and public, however the implementation may not be known (and its not really important). The developer should be interested in the data supply and the data returned from the web service. Various mechanisms are available for communicating with a web service. Such as: RPC, SOA, REST and more. Much of this presentation covered best practices, better practices, and why people tend to make decisions like implementing components when they really want datasources, as well as implementing datasources, and going about the implementation the wrong way. In the case of web services datasources implementation, curl is presented as a good example of something that works, but a better solution is available through the use of HttpSocket. HttpSocket being one of the CakePHP core libraries provided, allowing a complete implementation of Http communication, extending the CakeSocket class. Authentication and Authorization options were presented, with specific reference to OpenID and OAuth. Authentication and Authorzation are part of the application flow graph. This means implementation should be at the controller level, and in terms of implementing easily managed pluggable sections of code in cakephp converntions, this means a component. Data Sources are the closest layer to the actual data. Correct implementation of a data source will allow models to connect and communicate in a transparent fashion, meaning easy access to data in a standard way. The basics of a datasource should implement the following: __construct, listSources, describe, create, read, update, delete as well as defining $_schema. Some great datasource examples can be seen in the core. When implementing a datasource, to ensure maximum use and compatibility, try to make use of CakePHP libraries such as HttpSocket in the place of curl. Google Charts was presented as a good example of what should not be implemented as a datasource. The data in this instance is handed by some other data source, and the formatted chart request is sent with an image response supplied. This is more appropriate for a helper than a datasource. Joël mentioned that he has a partial google charts helper that he would be willing to share if someone asked.

Garret Woodworth - CakePHP then, now a...

Beginning with an overview of the CakePHP project, changes and evolution of direction and development team members, Garret provided a great overview of where the project stands, and how it has grown to be as successful as it has today. Garret gave a great description of the types of participation that are seen in open source teams, and these are relevant to CakePHP. He also described the attributes that make a good team member in such projects. Contribution Levels:  

  1. No effort (tickets are subimitted with little explanation)
  2. Some effort (well explain the ticket, and have attempted to reproduce the issue to confirm it)
  3. Attempted effort ("Some effort" with patch)
  4. Good Effort ("Some effort" with test case)
  5. Ultimate effort ("Some effort" with test case and patch)
Good team member attributes:
  1. Communicate often.
    1. To keep people motivated and interested on working for / with a project, its important to talk about what they want to work on, and what they feel they can assign some of their time to. Developing for open source shouldn't feel like "work".
  2. Show diffs of code, and get feedback to ensure the quality of work overall for the project is as high as it can be.
  3. Think longer about the problems faced, and as a result, write code faster.
  4. Details, Details, Details.
  5. Give back to the project more than you take from it.
  6. Think outside the box, and be creative.
CakePHP is growing, and the stats presented spoke for themselves, with America, Japan, India, France and Germany being the top countries at the moment in terms of hits on the CakePHP websites at the moment. This is resulting in 24% unique new visitors per month. A statistics that is truly extraordinary.
  With the feature development and more developers available to the CakePHP Core Development Team, git has been implemented widely and is the future of version control for source code for the CakePHP project. This should ease feature development, and remove some of the pain associated with merging with Subversion. Announcements! Garret announced new versions of CakePHP, currently being actively developed by the CakePHP core development team. Version 1.3 is available on and is a Step up with several enhancements over 1.2. Most notably Bake, Session, Javascript changes, Inflector and some library renames. Deprecated methods were also removed. There is even a wiki page describing migration steps from 1.2, to help ease the transition. CakePHP 2.0 was also announced. This is a huge move, stepping forward to drop PHP4 support, and move towards PHP5 Strict compliance, and much better Object Orientation and performance throughout. This new version is in active development, and code is also available on but does not yet have a stable release for download. was launched at the time of the Keynote, and is designed to consolidate systems. it's running on code, and uses git for the main projects. Its available now for everyone to use. Closing things up, Garrett urged the community to "get involved". CakePHP isn't where it is today without the extensive help and support of the community. There are a number of ways that you can contribute, and he mentioned the following in particular:  
  1. Interact with the community and the core developers.
  2. Get interested in Bakery 2.0 which is currently under development
  3. Plugins and Plugin Server
  4. Forks
  5. Join #cakephp-bakery on the IRC server

Benchmarking requestAction

Now there has been a lot of discussion in the past few months about requestAction() and how it can very easily create a negative impact on your application. In fact I even wrote such an article myself. However, its high time that someone did the number crunching to really see if requestAction() is actually as slow as we all seem to think it is. So onto the testing method and the results.

Testing method

To test this theory I used a small CakePHP application and the SVN head (revision 8064) of CakePHP. I used a simple sample application with 2 controllers and 2 models. My model method directly returned the results without touching the database, so that database retrieval time and model processing would not be a factor in these tests. As I was only interested in the performance implications inherent in requestAction() itself, I wanted to remove the variance created by connecting to a database. I set debug = 0, and used basic file caching. After warming up the cake core caches, I tested 4 different controller actions.
  • Using Relations / ClassRegistry::init() - The method I originally proposed, and often touted as the 'best' solution to requestAction()
  • Using RequestAction with a string URL
  • Using RequestAction with and Array URL
  • Using a cached RequestAction - This more accurately simulates how we use requestAction at CakeDC.
Benchmarks were generated with Siege I used 10 concurrent users with 110 reps each. My local development web-server is running Apache 2.2/PHP 5.2.6 o n a 2.6GHz Core 2 Duo iMac with 2GB of ram. I ran each test 3 times and took the best result of each.

Using model relations / ClassRegistry::init()

First up was my originally proposed solution of using model relations to access the correct information. I used the following command and got the following results. siege -b http://localhost/benchmark/posts/using_relations Transactions: 1100 hits Availability: 100.00 % Elapsed time: 63.21 secs Data transferred: 1.50 MB Response time: 0.55 secs Transaction rate: 17.40 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.60 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.76 Shortest transaction: 0.10

Using RequestAction with a string URL

Up next was using request action with a string url. String URL's are often the slower way to perform a requestAction as parsing the URL string is one of the more expensive operations in request dispatching. I used the following command and the best results were. siege -b http://localhost/benchmark/posts/using_requestaction Transactions: 1100 hits Availability: 100.00 % Elapsed time: 64.60 secs Data transferred: 1.51 MB Response time: 0.57 secs Transaction rate: 17.03 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.72 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.76 Shortest transaction: 0.11

RequestAction with an Array URL

Up next is requestAction() witn an array url. Using an array URL is supposed to expedite the dispatching process as it bypasses much of the parameter parsing done by Router. This theory turned out to be true, as Array URL's clocked in marginally faster than their string counterparts. siege -b http://localhost/benchmark/posts/using_requestaction_array Transactions: 1100 hits Availability: 100.00 % Elapsed time: 64.08 secs Data transferred: 1.53 MB Response time: 0.57 secs Transaction rate: 17.17 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.78 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.66 Shortest transaction: 0.11

RequestAction using Array URL's and Caching

In my mind this was going to be the most performant requestAction option, due to the cached nature. The results were as expected with this method clocking to be only slightly behind the relation call. It is important to note as well, that this test does not reflect the time savings earned from not having to make an additional query/ round of result parsing. In a real world situation, the savings of using a cached element would be magnified by the cost of the query. siege -b http://localhost/benchmark/posts/using_cached_requestaction Transactions: 1100 hits Availability: 100.00 % Elapsed time: 63.60 secs Data transferred: 1.52 MB Response time: 0.56 secs Transaction rate: 17.30 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.62 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.77 Shortest transaction: 0.09

Results Summary

In case you quickly scanned through the full results here is a summary of what happened.
Method Requests per second (mean) Total time taken (seconds)
Using relations/ClassRegistry::init() 17.40 63.21
Using requestAction and string urls 17.03 64.60
Using requestAction and array urls 17.17 64.08
Using cached requestaction 17.30 63.60
In closing requestAction() can be slower than a direct method call. There are some benefits to using requestAction though.
  • You have the opportunity to reduce the number of repeated lines of code by putting the requestAction inside the element. In doing so, you create an encapsulated element, that can be included anywhere without having to worry about having the correct method calls in your controller.
  • You can more easily cache the element. By using requestAction in conjunction with element caching you have an easy to use, simple to implement caching. Getting the same results with model method calls in your controller requires additional caching logic in your models.
  • The potential for increased performance. As we saw in the benchmarks above, a cached element performed almost as fast as the direct method call. This margin will grow when a database query is added into the mix.
Now am I retracting my previous stance on requestAction? No, I still feel that there are many situations where requestAction is the incorrect solution and signals poor application design. However, when the need arises it is good to know that requestAction can be as fast or faster than other approaches when implemented properly. You can download all the files I used for this process and run your own tests as well.


RSS Feeds, Fast and Easy

For my first entry, I am going to talk about how to create an RSS Feed on your website. RSS (Really Simple Syndication) is a format used to publish frequently updated works such as blogs or featured products. RSS defines a set of XML elements that are used to describe a channel or feed of information. An RSS feed is comprised of two parts, first is the metadata describing the channel and second is the records that make up the elements of the feed. RSS feeds allow your sites visitors to access the information on your site using software that reads these feeds. This will allow your site's visitors to stay up-to-date on the information on your site. CakePHP allows for easy integration of RSS feeds into existing controller actions through the automatic router extension parsing. This allows us to specify what type of response we want from a URL through adding the proper extension to the URL such as This alerts the router that your are asking for RSS formatted data in return. In addition, CakePHP has an RssHelper class that can be used to output parts of the metadata and elements in the feed through an easy to use helper.


Before we begin making the feed we must alert the router that we want to allow for extensions to be parsed in the URL and that we want it to accept .rss as a valid extension. In your sites router file we add the following: Router::parseExtensions('rss'); Also for CakePHP to work it magic we must also have the RequestHandler in our controller's $components array. Now the router knows that we would like to parse urls that end in .rss as requesting RSS formatted responses. The next step of preparation is to add a default layout for rss feeds on your site. When you request a different format response the layout that is rendered will be selected from a sub-folder with the same name as the format. So in this case we would need a folder called /rss in the layouts folder in our CakePHP install. The view class will search for a file that has the same name as the layout that would be rendered if you were just rendering the html. In most cases this is the default.ctp layout file in the main layouts directory, but because we are requesting the response in RSS format we must add a default.ctp layout in the /layouts/rss/ sub-directory. This layout is our default RSS Feed layout. echo $rss->header(); if (!isset($channel)) { $channel = array(); } if (!isset($channel['title'])) { $channel['title'] = $title_for_layout; } echo $rss->document($rss->channel(array(), $channel, $content_for_layout)); Here in the layout our RssHelper shines through. We use the method RssHelper::channel() which generates the element and associated metadata elements. The $content_for_layout variable contains the output from the view. These then get passed to the RssHelper::document() method, which wraps the RSS document in the respective elements.


The controller needs no modification in the case of a simple RSS feed. This is because we are only adding a second view that is xml/rss to the action. The same data is used in both views and because CakePHP automatically sets the correct response type we don't need to tell it to render the correct view and layout for RSS. Here is the action method in the EntriesController for a basic view sorted by a published_date field and showing only if it is published. public function index() { $this->paginate['Entry'] = array( 'conditions' => array('Entry.published' => 1), 'order' => 'Entry.published_date DESC'); $this->set('entries', $this->paginate()); } If you do have code that is specific for only the RSS view you can use the RequestHandler::isRss() to see if the action was called with the request for xml/rss formatting on response. This method returns a boolean value based on if the .rss extension was parsed in the URL. if ($this->RequestHandler->isRss()) { // RSS feed specific code goes here }

Note About Channel Metadata

It may feel right to put your metadata information in the index method in the controller, using Controller::set() to send the information to the views. This is inappropriate and is one of the most common snags that we have seen in the CakePHP community with creating RSS feeds. That information which is passed in the layout file to the RssHelper::channel() method should be set in the view using View::set() which will set the $channel variable for the layout in the view.


As we had to put the layout in a subdirectory of the layouts folder we also need to create a view for the index action for the blogs controller. This is done by creating a directory /views/entries/rss/ which will hold our view file that will generate the RSS to render. You will need to add your RssHelper to the list of helpers in your controller so that it is automatically loaded in the view and the layout. Our view begins by setting the $channel variable for the layout, this contains all the metadata for our RSS feed. $homeUrl = $html->url('/', true); $this->set('channel', array( 'title' => __("Daniel's Recent Articles", true), 'link' => $homeUrl, 'description' => __("Most recent articles from Daniel.", true), 'language' => 'en-us', 'image' => array( 'title' => 'Recent Articles from Daniel', 'url' => FULL_BASE_URL . $this->webroot('/img/rss_feed_image', true), 'link' => $homeUrl)); First we get the URL link for the website home that we will use for the links. Also we set the title, description and image to use for the RSS feed icon. By setting the channel variable using View::set() we are providing the layout the information to render the RSS feed's metadata elements. The second part of the view generates the elements for the actual records of the feed. This is accomplished by looping through the data that has been passed to the view and using the RssHelper::item() method. The other method you can use, RssHelper::items() which takes a callback and an array of items for the feed. (The method I have seen used for the callback has always been called transformRss(). There is one downfall to this method, which is that you cannot use any of the other helper classes to prepare your data inside the callback method because the scope inside the method does not include anything that is not passed inside, thus not giving access to the TimeHelper or any other helper that you may need. The RssHelper::item() transforms the associative array into an element for each key value pair. foreach ($entries as $entry) { $postTime = strtotime($entry['Entry']['created']); $entryLink = array( 'controller' => 'entries', 'action' => 'view', 'year' => date('Y', $postTime), 'month' => date('m', $postTime), 'day' => date('d', $postTime), $entry['Entry']['slug']); // This is the part where we clean the body text for output as the description // of the rss item, this needs to have only text to make sure the feed validates $bodyText = preg_replace('=\(.*?)\=is', '', $entry['Entry']['body']); $bodyText = $text->stripLinks($bodyText); $bodyText = Sanitize::stripAll($bodyText); $bodyText = $text->truncate($bodyText, 400, '...', true, true); echo $rss->item(array(), array( 'title' => $entry['Entry']['title'], 'link' => $entryLink, 'guid' => array('url' => $entryLink, 'isPermaLink' => 'true'), 'description' => $bodyText, 'dc:creator' => $entry['Entry']['author'], 'pubDate' => $entry['Entry']['created'])); } You can see above that we can use the loop to prepare the data to be transformed into XML elements. It is important to filter out any non-plain text charictars out of the description, especially if you are using a rich text editor for the body of your blog. In the code above we use the TextHelper::stripLinks() method and a few methods from the Sanitize class, but we recommend writing a comprehensive text cleaning helper to really scrub the text clean. Once we have set up the data for the feed, we can then use the RssHelper::item() method to create the XML in RSS format. Once you have all this setup, you can test your RSS feed by going to your site /entries/index.rss and you will see your new feed. It is always important that you validate your RSS feed before making it live. This can be done by visiting sites that validate the XML such as Feed Validator or the w3c site at

Lighty Story

I will tell you a story. Once upon a time... Seriously though, it was not too long ago in the past - but it happened and it is possible you can benefit from it.


This tutorial will show how to make lighttpd 1.4.20 serve virtual hosts with CakePHP applications. Our scenario is quite simple:
  1. For admin purposes, lighttpd will listen on localhost, it will serve several CakePHP applications on several external ip addresses, without SSL.
  2. Virtual hosts will be organized in groups and every group will use one CakePHP core checkout for its virtual hosts.
  3. Every virtual host will have it own access log (this server will not run hundreds of virtual hosts, so we can afford to waste one file descriptor for each) and its own directory for caching of compressed static files.
  4. Management of virtual hosts, their default and custom settings should be as easy as possible, so we can delegate the management of some ip addresses or just groups of virthosts to someone else and sleep well, because nobody will have to touch our precious configuration files.
However, our scenario has some special requirements which we need to solve. By the way, I will be showing you how to do things the hard way from the start. In hopes to spare you a lot of headaches in future. Lighttpd is sweet piece of software, and is under active development. Unfortunately, there are things that are not easy to set up. For example - when using any of provided virtual host modules, it is impossible to set up different access logs and cache directories for compressed content etc. dynamically in a pure lighty config file without external scripts. Everything (except for per virtual host errorlog) is possible by writing necessary configuration by hand. But we willing to work more now, so we can be lazy later! There are several approaches for bash, Ruby etc. However, nothing usable in PHP as far as I know. I will show you how easy it could be. Take this as a working example, I am sharing ideas here, not bullet-proof all-mighty solutions. Lets go for it - and utilize PHP and the include_shell command in our lighttpd configuration file. The motto of this article is: it is easier read generated configuration, then write it by hand.

How? Lighty!

Don't think this is not a good answer. Lets set up a decent lighttpd installation. We'll assume you have it compiled and installed. Lets also assume that you have PHP prepared for lighttpd's ModFastCGI and are just waiting for configuration and the first test run. Also, for shell commands which need to be executed under root account, I'll use sudo in following examples. sudo mkdir /usr/local/etc/lighttpd First of all, we need a directory for our custom configuration. When in doubt, a fast look into its contents will tell you everything one should know about virtual hosts configuration. sudo mkdir -p /usr/local/www/data/default/webroot echo "<html><head><title>It works<body>It works" > /usr/local/www/data/default/webroot/index.html Next we created a directory for our default webroot. It will be used on localhost only, with index.html. sudo touch /var/log/lighttpd.error.log /var/log/lighttpd.access.log sudo chown www:www /var/log/lighttpd.error.log /var/log/lighttpd.access.log Now we need to create error and access log files. The first one will be common for whole server, the second will be used for localhost only. sudo mkdir -p /var/cache/lighttpd/compress/default sudo chown -R www:www /var/cache/lighttpd The last thing we had to prepare was the default directory for caching of compressed static files. In /usr/local/etc/lighttpd.conf we will setup a simple config file containing the common configuration we will utilize later: server.modules = ( "mod_simple_vhost", "mod_magnet", "mod_redirect", "mod_access", "mod_auth", "mod_expire", "mod_compress", "mod_fastcgi", "mod_accesslog" ) server.document-root = "/usr/local/www/data/default/webroot/" server.errorlog = "/var/log/lighttpd.error.log" accesslog.filename = "/var/log/lighttpd.access.log" server.port = 80 server.bind = "" server.username = "www" server.groupname = "www" = "/var/run/" index-file.names = ( "index.php", "index.html", "index.htm", "default.htm" ) # shortened !!! mimetype.assign = ( ... ) url.access-deny = ( "~", ".inc" ) static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" ) dir-listing.activate = "disable" etag.use-mtime = "enable" static-file.etags = "enable" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ( "" => "access 7 days" ) } compress.cache-dir = "/var/cache/lighttpd/compress/default/" compress.filetype = ( "text/plain", "text/html", "text/xml", "text/javascript", "text/css" ) fastcgi.server = ( ".php" => (( "bin-path" => "/usr/local/bin/php-cgi -c /usr/local/etc/php.ini", "socket" => "/tmp/lighttpd_php5.socket", "min-procs" => 1, "max-procs" => 1, "bin-environment" => ( "FCGI_WEB_SERVER_ADDRS" => "", "PHP_FCGI_CHILDREN" => "4", "PHP_FCGI_MAX_REQUESTS" => "1000" ), "bin-copy-environment" => ( "PATH", "SHELL", "USER"), "broken-scriptfilename" => "enable" )) ) simple-vhost.server-root = "/usr/local/www/data/" simple-vhost.document-root = "webroot" simple-vhost.default-host = "default" $HTTP["host"] =~ "^www\.(.*)" { url.redirect = ( "^/(.*)" => "http://%1/$1" ) } How far along are we? So far we have a configured webserver with few preloaded modules and simple common configuration. Our sever is currently:
  1. Listening on localhost:80.
  2. Refusing directory listing or sending some filetypes as plain text.
  3. Using etags and sending expiration headers for a set of static resources to 7 days by default. This allows us to schedule an upgrade of any virtual host just a week before it will happen.
  4. Using compression and caching of compressed static files for several mimetypes.
  5. Starting PHP as FastCGI, with only one parent process (we are going to use opcode cache). We are allowing only few child processes for this example tutorial and killing fcgi child processes after every 1000 requests
  6. Using mod_simple_vhost for name-based virtual hosting (preconfigured for fallback to default webroot).
  7. Redirecting all domains using www subdomain to the shorter version.
You will probably want to tweak some other settings. I am not going to describe all the server.max* configuration options, or talk about other pretty obvious things like mod_evasive, mod_status, mod_rrdtool etc, don't worry. Two things you should consider if some of your visitors will use one of the major browsers. $HTTP["url"] =~ "\.pdf$" { server.range-requests = "disable" } You do not want to cut off IE users from your pdf documents, right? compress.filetype = ( "text/plain", "text/html", "text/xml" ) $HTTP["useragent"] =~ "Firefox" { compress.filetype += ("text/javascript", "text/css" ) } If your visitors are using an old (and/or above mentioned undesirable) internet browser, you can control compression settings per useragent in this way. Instead of the above example, compressing all 5 crucial mimetypes. Ready to go? Ok, start lighttpd and make sure you see what you expect at http://localhost/ echo "<?php phpinfo(); ?>" > /usr/local/www/data/default/webroot/phpinfo.php Just to be sure that fcgi works as expected, try to see info about your current PHP setup at http://localhost/phpinfo.php and watch /var/log/lighttpd.error.log.

Url rewriting

It is possible to use lighttpd's mod_rewrite and create pattern for our static files if we are sure they exist. This approach has downsides though. We want to setup this part of webserver up and forget it exists. This is not possible with mod_rewrite, because for example, we are not going to force our developers to forget about /js/something.js as url for some of application controllers. Instead, we will use mod_magnet and custom Lua script. Visit this thread at CakePHP Google Group. Save the provided script to /usr/local/etc/lighttpd/cleanurl-v6.lua and add the following line to bottom of /usr/local/etc/lighttpd.conf: magnet.attract-physical-path-to = ( "/usr/local/etc/lighttpd/cleanurl-v6.lua" ) After restarting lighttpd, we are ready to remove all the .htaccess files from our filesystem and forget they exist. All requests for non-existing static files will be rewritten to /index.php?url=xxx like CakePHP requires.

Virtual hosts

Now we want to set up a directory structure and custom configuration for our virtual hosts and their groups. We will design a directory structure that can be used for dynamic configuration later, with no need to repeat anything obvious in configuration files. In this case, only logs folder matters (make sure it is writable by webserver). We will symlink everything else. Lets use the following directory structure with CakePHP core and our applications checkouts like our standard: # (with redirect from /home/company/ logs/ www/ cake/ mainsite/ ... webroot/ vendors/ # and /home/development/ logs/ www/ cake/ mainsite/ ... webroot/ product/ ... webroot/ vendors/ # and /home/staging/ logs/ www/ cake/ mainsite/ ... webroot/ product/ ... webroot/ vendors/ #,, ( with redirect from /home/product/ logs/ www/ api/ ... index.html book/ ... webroot/ cake/ product/ ... webroot/ vendors/ If you think the above directory tree is overcomplicated, or it seems too long for simple tutorial example, stop reading please, and feel free to come back any time later. It was nice to meet you :-) Things are only getting worse from here on in. For those brave enough to read on, you should have an idea of which domains will use which applications, and which applications will share one CakePHP core and folder for logs (not necessarily, read more). Now we are getting somewhere - we need tell our webserver on which external ip addresses it has to listen for incoming connections, and which virtual hosts map to each ip address. Our www subdomains (redirected) should listen on a different ip address then their short versions. This allows us to use different SSL certificates for them later, if there is a need for secure connections. To show what is possible with our config parser, will not use a /webroot/ folder, it contains just static html files. To make things even more tricky, and will not listen on same ip like their neighbour application cd /usr/local/etc/lighttpd From now on, we will continue our work in this directory. Lets say that we want to use ip for domains, and sudo mkdir -p ./ sudo ln -s /home/company/www/cake ./ sudo ln -s /home/company/www/vendors ./ sudo ln -s /home/company/www/mainsite ./ sudo mkdir ./ sudo ln -s /home/product/www/cake ./ sudo ln -s /home/product/www/vendors ./ sudo ln -s /home/product/www/api ./ sudo ln -s /home/product/www/book ./ What exactly did we just do? We created a folder named, containing 2 subfolders company and product. These will be used as groups of virtual hosts - their names should be the same as the name of their home directory (by default, path for logs can be adjusted). We will use them for setting paths to log files later. Both company and product have a symlinked cake and vendors folders and symlinks named as real domains and pointing to our app folders. Lets continue - ip 2.3.4:5:80 will be used for rest of the group product. sudo mkdir -p ./ sudo ln -s /home/product/www/cake ./ sudo ln -s /home/product/www/vendors ./ sudo ln -s /home/product/www/product ./ That means only one virtual host for now. Ok, ip is going to be used for the www subdomains. No symlinks to existing applications are necessary here, because lighttpd will redirect requests coming to to automatically. sudo mkdir -p ./ ./ We just had to create ip:port directory for the socket, group(s) of www virtualhosts and some domain-based directories just to have something to point default virtual host of this group at. Staging and development checkouts will all share one ip sudo mkdir -p ./ sudo ln -s /home/development/www/cake ./ sudo ln -s /home/development/www/vendors ./ sudo ln -s /home/development/www/mainsite ./ sudo ln -s /home/development/www/product ./ sudo mkdir ./ sudo ln -s /home/staging/www/cake ./ sudo ln -s /home/staging/www/vendors ./ sudo ln -s /home/staging/www/mainsite ./ sudo ln -s /home/staging/www/product ./ Four virtual hosts on one ip from different home folders (therefore placed in different groups). The hard part is complete. Lets go through the bothering part of this custom setup. Did I said already that everything is a file? Don't be scared from amount of necessary steps, it will all be worth it in the future. Lets look what we have done in directory /usr/local/etc/lighttpd/: company/ cake/ <-- /home/company/www/cake <-- /home/company/www/mainsite vendors/ <-- /home/company/www/vendors product/ <-- /home/product/www/api <-- /home/product/www/book cake/ <-- /home/product/www/cake vendors/ <-- /home/product/www/vendors product/ cake/ <-- /home/product/www/cake <-- /home/product/www/product vendors/ <-- /home/product/www/vendors company/ <-- empty directory (redirected), necessary for default virtual host product/ <-- empty directory (redirected), necessary for default virtual host 4.5.6:7:80/ development/ cake/ <-- /home/development/www/cake <-- /home/development/www/mainsite <-- /home/development/www/product vendors/ <-- /home/development/www/vendors staging/ cake/ <-- /home/staging/www/cake <-- /home/staging/www/mainsite <-- /home/staging/www/product vendors/ <-- /home/staging/www/vendors Some new folders with symlinks. Are you still with me? For those who know mod_simple_vhost, you should be already be pretty clear where we are going. Besides the accesslog path and compress folder path, we will also switch simple-vhost.server-root and simple-vhost.default-host in dependency of used socket and some hostname condition for virthost group. Actually, there is a bit more as well that I will show you. The above directory structure shows that we have 7 groups of virtual hosts in 4 sockets, so lets create 7 simple configuration files for our groups of virtual hosts. Configuration file for group is not required in very special case - no regex pattern for this group, only one virtual host inside and - either only group in socket, or (alphabetically) last one. <?php # /usr/local/etc/lighttpd/ $config['group'] = array( 'host' => '^example\.com', 'default' => '' ); ?> <?php # /usr/local/etc/lighttpd/ $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => '' ); ?> <?php # /usr/local/etc/lighttpd/ $config['group'] = array( 'host' => '^product\.com', 'default' => '' ); ?> <?php # /usr/local/etc/lighttpd/ $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => '' ); ?> <?php # /usr/local/etc/lighttpd/ $config['group'] = array( 'host' => '^(.*)\.product\.com', 'default' => '' ); ?> <?php # /usr/local/etc/lighttpd/4.5.6:7:80/development/config.php $config['group'] = array( 'host' => '^dev-(.*)\.example\.com', 'default' => '' ); ?> <?php # /usr/local/etc/lighttpd/4.5.6:7:80/staging/config.php $config['group'] = array( 'host' => '^stage-(.*)\.example\.com', 'default' => '' ); ?> And that's it. Every group (subfolder of socket folder) has the required minimal configuration, and everything is properly set up. So lets see what we can take off from it.

Dynamic configuration

Extract this file in folder /usr/local/etc/lighttpd. sudo chmod a+x ./simple_config.php Make simple_config.php executable for everyone. Now run it as a non-privileged user. ./simple_config.php | more You should see a basic generated configuration for your sockets, virthosts and virthosts groups. Now we are already looking at a snippet of the generated configuration. # # Simple configuration parser output # # ERROR logfile /home/company/logs/example-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # ERROR logfile /home/product/logs/api-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # ERROR logfile /home/product/logs/book-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # ERROR logfile /home/product/logs/product-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # ERROR logfile /home/company/logs/www-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # ERROR logfile /home/product/logs/www-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # ERROR logfile /home/development/logs/dev-main-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # ERROR logfile /home/development/logs/dev-product-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # ERROR logfile /home/staging/logs/stage-main-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # ERROR logfile /home/staging/logs/stage-product-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/ can not be created, SKIPPING # $SERVER["socket"] == "" { $HTTP["host"] =~ "^example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { .... You can see which files this script is trying to create. It will create all of them when you will run it as root once. But there are two things we would like to fix first: access logs /home/company/logs/www-access_log and /home/product/logs/www-access_log are generated for our redirected domains. Lets redirect these logs to those used by domains and <?php # /usr/local/etc/lighttpd/ $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => '' ); $config['virthosts'] = array( '' => array( 'log' => 'example' ) ); ?> <?php # /usr/local/etc/lighttpd/ $config['group'] = array( 'host' => '^(.*)\.product\.com', 'default' => '' ); $config['virthosts'] = array( '' => array( 'log' => 'product' ) ); ?> Running ./simple_config.php as unprivileged user again shows this script is no longer trying to create any www-access_log files. We will not care about directories for compressed content, they can be used later, but we will never serve different content on and, so it is logical that they share one log file. Every decent logfile parser can handle several domains in one log file. Now, you can run this script as root: sudo ./simple_config.php and result will look much better now: # # Simple configuration parser output # # NOTICE created logfile /home/company/logs/example-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/ # NOTICE created logfile /home/product/logs/api-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/ # NOTICE created logfile /home/product/logs/book-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/ # NOTICE created logfile /home/product/logs/product-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/ # NOTICE created compress cache /var/cache/lighttpd/compress/ # NOTICE created compress cache /var/cache/lighttpd/compress/ # NOTICE created logfile /home/development/logs/dev-main-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/ # NOTICE created logfile /home/development/logs/dev-product-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/ # NOTICE created logfile /home/staging/logs/stage-main-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/ # NOTICE created logfile /home/staging/logs/stage-product-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/ # $SERVER["socket"] == "" { $HTTP["host"] =~ "^example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } else $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/product/logs/api-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } else $HTTP["host"] == "" { accesslog.filename = "/home/product/logs/book-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } } $SERVER["socket"] == "" { $HTTP["host"] =~ "^product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } } $SERVER["socket"] == "" { $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } else $HTTP["host"] =~ "^(.*)\.product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } } $SERVER["socket"] == "" { $HTTP["host"] =~ "^dev-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/development/logs/dev-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } else $HTTP["host"] == "" { accesslog.filename = "/home/development/logs/dev-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } else $HTTP["host"] =~ "^stage-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/staging/logs/stage-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } else $HTTP["host"] == "" { accesslog.filename = "/home/staging/logs/stage-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } } Getting close to what we need from this setup. I will process several steps now, and then I will paste here final output of config parser for you to compare with above one. We have another domain (with no virthost set) and we want to redirect it to with configuration only, it will be using its own manual-access_log. Furthermore, we want condition happen sooner then the condition on, because book is gaining more traffic, and attach domain aliases and to Also, expire headers for book should be set for 2 years and as previously mentioned is not using /webroot/ folder. <?php # /usr/local/etc/lighttpd/ $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => '' ); $config['virthosts'] = array( '' => array( 'expire' => array( '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 2 years' ), 'aliases' => array( '', '' ) ), '' => array( 'webroot' => '/' ), '' => array( 'redirect' => '' ) ); ?> All of it is fixed now. We even do not need folder/symlink for in this case. Important note: we do not have to create folders for domains and, because they are aliases for and it is used as default virtual host for this group! If you will set alias for non-default virtual host, you have to symlink aliased application several times to group folder - every time with a different domain name. We want all staging sites to store logs in /home/development/logs. Also all staging and development sites should use expire headers for 5 minutes only and have to use http auth (one common file for now). <?php # /usr/local/etc/lighttpd/4.5.6:7:80/development/config.php $config['group'] = array( 'host' => '^dev-(.*)\.example\.com', 'default' => '', 'expire' => array( '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 5 minutes' ), 'auth' => array( 'backend' => 'htpasswd', 'file' => '/var/projects/company/.trac.htpasswd', 'protect' => array( '/' => array( 'realm' => 'Development Access', 'require' => 'valid-user' ) ) ) ); ?> <?php # /usr/local/etc/lighttpd/4.5.6:7:80/staging/config.php $config['group'] = array( 'host' => '^stage-(.*)\.example\.com', 'default' => '', 'expire' => array( '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 5 minutes' ), 'logs' => '/home/development/logs', 'auth' => array( 'backend' => 'htpasswd', 'file' => '/var/projects/company/.trac.htpasswd', 'protect' => array( '/' => array( 'realm' => 'Staging Access', 'require' => 'valid-user' ) ) ) ); ?> This has all been fixed now. Now our simple_config.php returns this: # # Simple configuration parser output # $SERVER["socket"] == "" { $HTTP["host"] =~ "^example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } else $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] =~ "^(book\.example\.com|bibliotheca\.example\.com|bookstore\.example\.com)" { accesslog.filename = "/home/product/logs/book-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ("" => "access 2 years") } } else $HTTP["host"] == "" { accesslog.filename = "/home/product/logs/api-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" simple-vhost.document-root = "/" } else $HTTP["host"] == "" { accesslog.filename = "/home/product/logs/manual-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" url.redirect = ( ".*" => "" ) } } } $SERVER["socket"] == "" { $HTTP["host"] =~ "^product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } } $SERVER["socket"] == "" { $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } else $HTTP["host"] =~ "^(.*)\.product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["host"] == "" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } } $SERVER["socket"] == "" { $HTTP["host"] =~ "^dev-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ("" => "access 5 minutes") } auth.backend = "htpasswd" auth.backend.htpasswd.userfile = "/var/projects/company/.trac.htpasswd" auth.require = ( "/" => ( "method" => "basic", "realm" => "Development Access", "require" => "valid-user" ) ) $HTTP["host"] == "" { accesslog.filename = "/home/development/logs/dev-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } else $HTTP["host"] == "" { accesslog.filename = "/home/development/logs/dev-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } else $HTTP["host"] =~ "^stage-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/" simple-vhost.default-host = "" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ("" => "access 5 minutes") } auth.backend = "htpasswd" auth.backend.htpasswd.userfile = "/var/projects/company/.trac.htpasswd" auth.require = ( "/" => ( "method" => "basic", "realm" => "Staging Access", "require" => "valid-user" ) ) $HTTP["host"] == "" { accesslog.filename = "/home/development/logs/stage-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } else $HTTP["host"] == "" { accesslog.filename = "/home/development/logs/stage-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/" } } } Now it looks like we are set with everything we needed. One last line for /usr/local/etc/lighttpd.conf is: include_shell "/usr/local/etc/lighttpd/simple_config.php" And that's all. Before you will start or restart lighttpd, try and see if it can parse the new configuration (with our include) without errors, or inspect how it sees configuration after parsing: lighttpd -t -f /usr/local/etc/lighttpd.conf lighttpd -p -f /usr/local/etc/lighttpd.conf It is better to run the above commands as root, off course.

Now what?

Think twice about patterns for groups - don't be surprised if you get 'It works' page or default virthost of another group, if you are too lazy to read the generated configuration! Groups are processed in alphabetical order - just so you know which patterns are going to be checked first. Well, it is possible to change order of groups - change name of some company group folder to xxx_company and: $config['group'] = array( 'name' => 'company', Now you should be fine - this group in folder named xxx_company instead of company, and everything will still work. Everything that is necessary should be up and running now. Lighttpd should serve all virtual hosts from groups in sockets from now on. Read how to clear cache for mod_compress too. Smart brain should ask now, why we are using mod_simple_vhost, if our parser generates configuration for every virtual host it founds in our configuration files and directory structure. We don't do it, but you can - read code. Note for these who do not want or can not follow our default logs location, home directories, cache directories, user account lighttpd will use, or want to store directory structure with sockets/groups/virthosts somewhere else - read code too ;-) Reason why we set mod_simple_vhost for this example as default is simple - to get some domain serving some application, we need only one simple thing: symlink to app directory with domain name, placed in some virtual group in proper socket. This virtual host will be accessible immediately - although, restart of webserver is still necessary to have configuration for access logfile and compress directory for this virtual host (otherwise default accesslog and compress dir will be used), but not required. A few questions remain, what and how needs to be done in obvious use cases - adding new ip addresses, groups, virthosts, or moving whole groups over sockets, moving virthosts over sockets, etc... I assume this part will be sweet piece of cake for you. Definitely - feel free to call simple_config.php as often as you want to. It is highly reccommended to save functional configuration to a backup file by redirecting the output. Sure, one can use include "/some/path/generated_output.conf" exclusively, instead of include_shell - it is up to you. Backup, backup, backup. This is nothing more then a functional example, but the entire code lives in one class, so feel free to change or extend it for your needs. It is released under MIT license and is provided as it is, so you can do anything you want with it (except for removing license and copyright note). Keep in mind it was not tested in all possible situations and some of things I did not mention in this tutorial (but they are implemented in code) were not intensively tested yet. If you feel that some of the subdomains used in this tutorial sound familiar to you, you are probably right. I didn't said it was going to be a fairy tale. I said, I will tell you a story. To be continued...

We Bake with CakePHP