CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

Joël Perras - Demystifying Webservices...

Joël's presentation on Web Services and CakePHP identifies important and interesting points that really demystify both implementation of datasources, and what web services mean for developers trying to take advantages of their offerings. A Web Service is a defined interface. The interface is made known and public, however the implementation may not be known (and its not really important). The developer should be interested in the data supply and the data returned from the web service. Various mechanisms are available for communicating with a web service. Such as: RPC, SOA, REST and more. Much of this presentation covered best practices, better practices, and why people tend to make decisions like implementing components when they really want datasources, as well as implementing datasources, and going about the implementation the wrong way. In the case of web services datasources implementation, curl is presented as a good example of something that works, but a better solution is available through the use of HttpSocket. HttpSocket being one of the CakePHP core libraries provided, allowing a complete implementation of Http communication, extending the CakeSocket class. Authentication and Authorization options were presented, with specific reference to OpenID and OAuth. Authentication and Authorzation are part of the application flow graph. This means implementation should be at the controller level, and in terms of implementing easily managed pluggable sections of code in cakephp converntions, this means a component. Data Sources are the closest layer to the actual data. Correct implementation of a data source will allow models to connect and communicate in a transparent fashion, meaning easy access to data in a standard way. The basics of a datasource should implement the following: __construct, listSources, describe, create, read, update, delete as well as defining $_schema. Some great datasource examples can be seen in the core. When implementing a datasource, to ensure maximum use and compatibility, try to make use of CakePHP libraries such as HttpSocket in the place of curl. Google Charts was presented as a good example of what should not be implemented as a datasource. The data in this instance is handed by some other data source, and the formatted chart request is sent with an image response supplied. This is more appropriate for a helper than a datasource. Joël mentioned that he has a partial google charts helper that he would be willing to share if someone asked.

Garret Woodworth - CakePHP then, now a...

Beginning with an overview of the CakePHP project, changes and evolution of direction and development team members, Garret provided a great overview of where the project stands, and how it has grown to be as successful as it has today. Garret gave a great description of the types of participation that are seen in open source teams, and these are relevant to CakePHP. He also described the attributes that make a good team member in such projects. Contribution Levels:  

  1. No effort (tickets are subimitted with little explanation)
  2. Some effort (well explain the ticket, and have attempted to reproduce the issue to confirm it)
  3. Attempted effort ("Some effort" with patch)
  4. Good Effort ("Some effort" with test case)
  5. Ultimate effort ("Some effort" with test case and patch)
Good team member attributes:
  1. Communicate often.
    1. To keep people motivated and interested on working for / with a project, its important to talk about what they want to work on, and what they feel they can assign some of their time to. Developing for open source shouldn't feel like "work".
  2. Show diffs of code, and get feedback to ensure the quality of work overall for the project is as high as it can be.
  3. Think longer about the problems faced, and as a result, write code faster.
  4. Details, Details, Details.
  5. Give back to the project more than you take from it.
  6. Think outside the box, and be creative.
CakePHP is growing, and the stats presented spoke for themselves, with America, Japan, India, France and Germany being the top countries at the moment in terms of hits on the CakePHP websites at the moment. This is resulting in 24% unique new visitors per month. A statistics that is truly extraordinary.   With the feature development and more developers available to the CakePHP Core Development Team, git has been implemented widely and is the future of version control for source code for the CakePHP project. This should ease feature development, and remove some of the pain associated with merging with Subversion. Announcements! Garret announced new versions of CakePHP, currently being actively developed by the CakePHP core development team. Version 1.3 is a Step up with several enhancements over 1.2. Most notably Bake, Session, Javascript changes, Inflector and some library renames. Deprecated methods were also removed. There is even a wiki page describing migration steps from 1.2, to help ease the transition. CakePHP 2.0 was also announced. This is a huge move, stepping forward to drop PHP4 support, and move towards PHP5 Strict compliance, and much better Object Orientation and performance throughout. This new version is in active development,  but does not yet have a stable release for download. code.cakephp.org was launched at the time of the Keynote, and is designed to consolidate systems. it's running on thechaw.com code, and uses git for the main projects. Its available now for everyone to use. Closing things up, Garrett urged the community to "get involved". CakePHP isn't where it is today without the extensive help and support of the community. There are a number of ways that you can contribute, and he mentioned the following in particular:  
  1. Interact with the community and the core developers.
  2. Get interested in Bakery 2.0 which is currently under development
  3. Plugins and Plugin Server
  4. Forks
  5. Join #cakephp-bakery on the IRC server
 

Benchmarking requestAction

Now there has been a lot of discussion in the past few months about requestAction() and how it can very easily create a negative impact on your application. In fact I even wrote such an article myself. However, its high time that someone did the number crunching to really see if requestAction() is actually as slow as we all seem to think it is. So onto the testing method and the results.

Testing method

To test this theory I used a small CakePHP application and the SVN head (revision 8064) of CakePHP. I used a simple sample application with 2 controllers and 2 models. My model method directly returned the results without touching the database, so that database retrieval time and model processing would not be a factor in these tests. As I was only interested in the performance implications inherent in requestAction() itself, I wanted to remove the variance created by connecting to a database. I set debug = 0, and used basic file caching. After warming up the cake core caches, I tested 4 different controller actions.
  • Using Relations / ClassRegistry::init() - The method I originally proposed, and often touted as the 'best' solution to requestAction()
  • Using RequestAction with a string URL
  • Using RequestAction with and Array URL
  • Using a cached RequestAction - This more accurately simulates how we use requestAction at CakeDC.
Benchmarks were generated with Siege I used 10 concurrent users with 110 reps each. My local development web-server is running Apache 2.2/PHP 5.2.6 o n a 2.6GHz Core 2 Duo iMac with 2GB of ram. I ran each test 3 times and took the best result of each. Using model relations / ClassRegistry::init() First up was my originally proposed solution of using model relations to access the correct information. I used the following command and got the following results. siege -b http://localhost/benchmark/posts/using_relations Transactions: 1100 hits Availability: 100.00 % Elapsed time: 63.21 secs Data transferred: 1.50 MB Response time: 0.55 secs Transaction rate: 17.40 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.60 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.76 Shortest transaction: 0.10 Using RequestAction with a string URL Up next was using request action with a string url. String URL's are often the slower way to perform a requestAction as parsing the URL string is one of the more expensive operations in request dispatching. I used the following command and the best results were. siege -b http://localhost/benchmark/posts/using_requestaction Transactions: 1100 hits Availability: 100.00 % Elapsed time: 64.60 secs Data transferred: 1.51 MB Response time: 0.57 secs Transaction rate: 17.03 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.72 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.76 Shortest transaction: 0.11 RequestAction with an Array URL Up next is requestAction() witn an array url. Using an array URL is supposed to expedite the dispatching process as it bypasses much of the parameter parsing done by Router. This theory turned out to be true, as Array URL's clocked in marginally faster than their string counterparts. siege -b http://localhost/benchmark/posts/using_requestaction_array Transactions: 1100 hits Availability: 100.00 % Elapsed time: 64.08 secs Data transferred: 1.53 MB Response time: 0.57 secs Transaction rate: 17.17 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.78 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.66 Shortest transaction: 0.11 RequestAction using Array URL's and Caching In my mind this was going to be the most performant requestAction option, due to the cached nature. The results were as expected with this method clocking to be only slightly behind the relation call. It is important to note as well, that this test does not reflect the time savings earned from not having to make an additional query/ round of result parsing. In a real world situation, the savings of using a cached element would be magnified by the cost of the query. siege -b http://localhost/benchmark/posts/using_cached_requestaction Transactions: 1100 hits Availability: 100.00 % Elapsed time: 63.60 secs Data transferred: 1.52 MB Response time: 0.56 secs Transaction rate: 17.30 trans/sec Throughput: 0.02 MB/sec Concurrency: 9.62 Successful transactions: 1100 Failed transactions: 0 Longest transaction: 1.77 Shortest transaction: 0.09 Results Summary In case you quickly scanned through the full results here is a summary of what happened.
Method Requests per second (mean) Total time taken (seconds)
Using relations/ClassRegistry::init() 17.40 63.21
Using requestAction and string urls 17.03 64.60
Using requestAction and array urls 17.17 64.08
Using cached requestaction 17.30 63.60
In closing requestAction() can be slower than a direct method call. There are some benefits to using requestAction though.
  • You have the opportunity to reduce the number of repeated lines of code by putting the requestAction inside the element. In doing so, you create an encapsulated element, that can be included anywhere without having to worry about having the correct method calls in your controller.
  • You can more easily cache the element. By using requestAction in conjunction with element caching you have an easy to use, simple to implement caching. Getting the same results with model method calls in your controller requires additional caching logic in your models.
  • The potential for increased performance. As we saw in the benchmarks above, a cached element performed almost as fast as the direct method call. This margin will grow when a database query is added into the mix.
Now am I retracting my previous stance on requestAction? No, I still feel that there are many situations where requestAction is the incorrect solution and signals poor application design. However, when the need arises it is good to know that requestAction can be as fast or faster than other approaches when implemented properly.  

RSS Feeds, Fast and Easy

For my first entry, I am going to talk about how to create an RSS Feed on your website. RSS (Really Simple Syndication) is a format used to publish frequently updated works such as blogs or featured products. RSS defines a set of XML elements that are used to describe a channel or feed of information. An RSS feed is comprised of two parts, first is the metadata describing the channel and second is the records that make up the elements of the feed. RSS feeds allow your sites visitors to access the information on your site using software that reads these feeds. This will allow your site's visitors to stay up-to-date on the information on your site. CakePHP allows for easy integration of RSS feeds into existing controller actions through the automatic router extension parsing. This allows us to specify what type of response we want from a URL through adding the proper extension to the URL such as http://www.yoursite.com/entries.rss. This alerts the router that your are asking for RSS formatted data in return. In addition, CakePHP has an RssHelper class that can be used to output parts of the metadata and elements in the feed through an easy to use helper.

Preparation

Before we begin making the feed we must alert the router that we want to allow for extensions to be parsed in the URL and that we want it to accept .rss as a valid extension. In your sites router file we add the following: Router::parseExtensions('rss'); Also for CakePHP to work it magic we must also have the RequestHandler in our controller's $components array. Now the router knows that we would like to parse urls that end in .rss as requesting RSS formatted responses. The next step of preparation is to add a default layout for rss feeds on your site. When you request a different format response the layout that is rendered will be selected from a sub-folder with the same name as the format. So in this case we would need a folder called /rss in the layouts folder in our CakePHP install. The view class will search for a file that has the same name as the layout that would be rendered if you were just rendering the html. In most cases this is the default.ctp layout file in the main layouts directory, but because we are requesting the response in RSS format we must add a default.ctp layout in the /layouts/rss/ sub-directory. This layout is our default RSS Feed layout. echo $rss->header(); if (!isset($channel)) { $channel = array(); } if (!isset($channel['title'])) { $channel['title'] = $title_for_layout; } echo $rss->document($rss->channel(array(), $channel, $content_for_layout)); Here in the layout our RssHelper shines through. We use the method RssHelper::channel() which generates the element and associated metadata elements. The $content_for_layout variable contains the output from the view. These then get passed to the RssHelper::document() method, which wraps the RSS document in the respective elements.

Controller

The controller needs no modification in the case of a simple RSS feed. This is because we are only adding a second view that is xml/rss to the action. The same data is used in both views and because CakePHP automatically sets the correct response type we don't need to tell it to render the correct view and layout for RSS. Here is the action method in the EntriesController for a basic view sorted by a published_date field and showing only if it is published. public function index() { $this->paginate['Entry'] = array( 'conditions' => array('Entry.published' => 1), 'order' => 'Entry.published_date DESC'); $this->set('entries', $this->paginate()); } If you do have code that is specific for only the RSS view you can use the RequestHandler::isRss() to see if the action was called with the request for xml/rss formatting on response. This method returns a boolean value based on if the .rss extension was parsed in the URL. if ($this->RequestHandler->isRss()) { // RSS feed specific code goes here }

Note About Channel Metadata

It may feel right to put your metadata information in the index method in the controller, using Controller::set() to send the information to the views. This is inappropriate and is one of the most common snags that we have seen in the CakePHP community with creating RSS feeds. That information which is passed in the layout file to the RssHelper::channel() method should be set in the view using View::set() which will set the $channel variable for the layout in the view.

Views

As we had to put the layout in a subdirectory of the layouts folder we also need to create a view for the index action for the blogs controller. This is done by creating a directory /views/entries/rss/ which will hold our view file that will generate the RSS to render. You will need to add your RssHelper to the list of helpers in your controller so that it is automatically loaded in the view and the layout. Our view begins by setting the $channel variable for the layout, this contains all the metadata for our RSS feed. $homeUrl = $html->url('/', true); $this->set('channel', array( 'title' => __("Daniel's Recent Articles", true), 'link' => $homeUrl, 'description' => __("Most recent articles from Daniel.", true), 'language' => 'en-us', 'image' => array( 'title' => 'Recent Articles from Daniel', 'url' => FULL_BASE_URL . $this->webroot('/img/rss_feed_image', true), 'link' => $homeUrl)); First we get the URL link for the website home that we will use for the links. Also we set the title, description and image to use for the RSS feed icon. By setting the channel variable using View::set() we are providing the layout the information to render the RSS feed's metadata elements. The second part of the view generates the elements for the actual records of the feed. This is accomplished by looping through the data that has been passed to the view and using the RssHelper::item() method. The other method you can use, RssHelper::items() which takes a callback and an array of items for the feed. (The method I have seen used for the callback has always been called transformRss(). There is one downfall to this method, which is that you cannot use any of the other helper classes to prepare your data inside the callback method because the scope inside the method does not include anything that is not passed inside, thus not giving access to the TimeHelper or any other helper that you may need. The RssHelper::item() transforms the associative array into an element for each key value pair. foreach ($entries as $entry) { $postTime = strtotime($entry['Entry']['created']); $entryLink = array( 'controller' => 'entries', 'action' => 'view', 'year' => date('Y', $postTime), 'month' => date('m', $postTime), 'day' => date('d', $postTime), $entry['Entry']['slug']); // This is the part where we clean the body text for output as the description // of the rss item, this needs to have only text to make sure the feed validates $bodyText = preg_replace('=\(.*?)\=is', '', $entry['Entry']['body']); $bodyText = $text->stripLinks($bodyText); $bodyText = Sanitize::stripAll($bodyText); $bodyText = $text->truncate($bodyText, 400, '...', true, true); echo $rss->item(array(), array( 'title' => $entry['Entry']['title'], 'link' => $entryLink, 'guid' => array('url' => $entryLink, 'isPermaLink' => 'true'), 'description' => $bodyText, 'dc:creator' => $entry['Entry']['author'], 'pubDate' => $entry['Entry']['created'])); } You can see above that we can use the loop to prepare the data to be transformed into XML elements. It is important to filter out any non-plain text charictars out of the description, especially if you are using a rich text editor for the body of your blog. In the code above we use the TextHelper::stripLinks() method and a few methods from the Sanitize class, but we recommend writing a comprehensive text cleaning helper to really scrub the text clean. Once we have set up the data for the feed, we can then use the RssHelper::item() method to create the XML in RSS format. Once you have all this setup, you can test your RSS feed by going to your site /entries/index.rss and you will see your new feed. It is always important that you validate your RSS feed before making it live. This can be done by visiting sites that validate the XML such as Feed Validator or the w3c site at http://validator.w3.org/feed/.

Lighty Story

I will tell you a story. Once upon a time... Seriously though, it was not too long ago in the past - but it happened and it is possible you can benefit from it.

What?

This tutorial will show how to make lighttpd 1.4.20 serve virtual hosts with CakePHP applications. Our scenario is quite simple:
  1. For admin purposes, lighttpd will listen on localhost, it will serve several CakePHP applications on several external ip addresses, without SSL.
  2. Virtual hosts will be organized in groups and every group will use one CakePHP core checkout for its virtual hosts.
  3. Every virtual host will have it own access log (this server will not run hundreds of virtual hosts, so we can afford to waste one file descriptor for each) and its own directory for caching of compressed static files.
  4. Management of virtual hosts, their default and custom settings should be as easy as possible, so we can delegate the management of some ip addresses or just groups of virthosts to someone else and sleep well, because nobody will have to touch our precious configuration files.
However, our scenario has some special requirements which we need to solve. By the way, I will be showing you how to do things the hard way from the start. In hopes to spare you a lot of headaches in future. Lighttpd is sweet piece of software, and is under active development. Unfortunately, there are things that are not easy to set up. For example - when using any of provided virtual host modules, it is impossible to set up different access logs and cache directories for compressed content etc. dynamically in a pure lighty config file without external scripts. Everything (except for per virtual host errorlog) is possible by writing necessary configuration by hand. But we willing to work more now, so we can be lazy later! There are several approaches for bash, Ruby etc. However, nothing usable in PHP as far as I know. I will show you how easy it could be. Take this as a working example, I am sharing ideas here, not bullet-proof all-mighty solutions. Lets go for it - and utilize PHP and the include_shell command in our lighttpd configuration file. The motto of this article is: it is easier read generated configuration, then write it by hand.

How? Lighty!

Don't think this is not a good answer. Lets set up a decent lighttpd installation. We'll assume you have it compiled and installed. Lets also assume that you have PHP prepared for lighttpd's ModFastCGI and are just waiting for configuration and the first test run. Also, for shell commands which need to be executed under root account, I'll use sudo in following examples. sudo mkdir /usr/local/etc/lighttpd First of all, we need a directory for our custom configuration. When in doubt, a fast look into its contents will tell you everything one should know about virtual hosts configuration. sudo mkdir -p /usr/local/www/data/default/webroot echo "<html><head><title>It works<body>It works" > /usr/local/www/data/default/webroot/index.html Next we created a directory for our default webroot. It will be used on localhost only, with index.html. sudo touch /var/log/lighttpd.error.log /var/log/lighttpd.access.log sudo chown www:www /var/log/lighttpd.error.log /var/log/lighttpd.access.log Now we need to create error and access log files. The first one will be common for whole server, the second will be used for localhost only. sudo mkdir -p /var/cache/lighttpd/compress/default sudo chown -R www:www /var/cache/lighttpd The last thing we had to prepare was the default directory for caching of compressed static files. In /usr/local/etc/lighttpd.conf we will setup a simple config file containing the common configuration we will utilize later: server.modules = ( "mod_simple_vhost", "mod_magnet", "mod_redirect", "mod_access", "mod_auth", "mod_expire", "mod_compress", "mod_fastcgi", "mod_accesslog" ) server.document-root = "/usr/local/www/data/default/webroot/" server.errorlog = "/var/log/lighttpd.error.log" accesslog.filename = "/var/log/lighttpd.access.log" server.port = 80 server.bind = "127.0.0.1" server.username = "www" server.groupname = "www" server.pid-file = "/var/run/lighttpd.pid" index-file.names = ( "index.php", "index.html", "index.htm", "default.htm" ) # shortened !!! mimetype.assign = ( ... ) url.access-deny = ( "~", ".inc" ) static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" ) dir-listing.activate = "disable" etag.use-mtime = "enable" static-file.etags = "enable" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ( "" => "access 7 days" ) } compress.cache-dir = "/var/cache/lighttpd/compress/default/" compress.filetype = ( "text/plain", "text/html", "text/xml", "text/javascript", "text/css" ) fastcgi.server = ( ".php" => (( "bin-path" => "/usr/local/bin/php-cgi -c /usr/local/etc/php.ini", "socket" => "/tmp/lighttpd_php5.socket", "min-procs" => 1, "max-procs" => 1, "bin-environment" => ( "FCGI_WEB_SERVER_ADDRS" => "127.0.0.1", "PHP_FCGI_CHILDREN" => "4", "PHP_FCGI_MAX_REQUESTS" => "1000" ), "bin-copy-environment" => ( "PATH", "SHELL", "USER"), "broken-scriptfilename" => "enable" )) ) simple-vhost.server-root = "/usr/local/www/data/" simple-vhost.document-root = "webroot" simple-vhost.default-host = "default" $HTTP["host"] =~ "^www\.(.*)" { url.redirect = ( "^/(.*)" => "http://%1/$1" ) } How far along are we? So far we have a configured webserver with few preloaded modules and simple common configuration. Our sever is currently:
  1. Listening on localhost:80.
  2. Refusing directory listing or sending some filetypes as plain text.
  3. Using etags and sending expiration headers for a set of static resources to 7 days by default. This allows us to schedule an upgrade of any virtual host just a week before it will happen.
  4. Using compression and caching of compressed static files for several mimetypes.
  5. Starting PHP as FastCGI, with only one parent process (we are going to use opcode cache). We are allowing only few child processes for this example tutorial and killing fcgi child processes after every 1000 requests
  6. Using mod_simple_vhost for name-based virtual hosting (preconfigured for fallback to default webroot).
  7. Redirecting all domains using www subdomain to the shorter version.
You will probably want to tweak some other settings. I am not going to describe all the server.max* configuration options, or talk about other pretty obvious things like mod_evasive, mod_status, mod_rrdtool etc, don't worry. Two things you should consider if some of your visitors will use one of the major browsers. $HTTP["url"] =~ "\.pdf$" { server.range-requests = "disable" } You do not want to cut off IE users from your pdf documents, right? compress.filetype = ( "text/plain", "text/html", "text/xml" ) $HTTP["useragent"] =~ "Firefox" { compress.filetype += ("text/javascript", "text/css" ) } If your visitors are using an old (and/or above mentioned undesirable) internet browser, you can control compression settings per useragent in this way. Instead of the above example, compressing all 5 crucial mimetypes. Ready to go? Ok, start lighttpd and make sure you see what you expect at http://localhost/ echo "<?php phpinfo(); ?>" > /usr/local/www/data/default/webroot/phpinfo.php Just to be sure that fcgi works as expected, try to see info about your current PHP setup at http://localhost/phpinfo.php and watch /var/log/lighttpd.error.log.

Url rewriting

It is possible to use lighttpd's mod_rewrite and create pattern for our static files if we are sure they exist. This approach has downsides though. We want to setup this part of webserver up and forget it exists. This is not possible with mod_rewrite, because for example, we are not going to force our developers to forget about /js/something.js as url for some of application controllers. Instead, we will use mod_magnet and custom Lua script. Visit this thread at CakePHP Google Group. Save the provided script to /usr/local/etc/lighttpd/cleanurl-v6.lua and add the following line to bottom of /usr/local/etc/lighttpd.conf: magnet.attract-physical-path-to = ( "/usr/local/etc/lighttpd/cleanurl-v6.lua" ) After restarting lighttpd, we are ready to remove all the .htaccess files from our filesystem and forget they exist. All requests for non-existing static files will be rewritten to /index.php?url=xxx like CakePHP requires.

Virtual hosts

Now we want to set up a directory structure and custom configuration for our virtual hosts and their groups. We will design a directory structure that can be used for dynamic configuration later, with no need to repeat anything obvious in configuration files. In this case, only logs folder matters (make sure it is writable by webserver). We will symlink everything else. Lets use the following directory structure with CakePHP core and our applications checkouts like our standard: # example.com (with redirect from www.example.com) /home/company/ logs/ www/ cake/ mainsite/ ... webroot/ vendors/ # dev-main.example.com and dev-product.example.com /home/development/ logs/ www/ cake/ mainsite/ ... webroot/ product/ ... webroot/ vendors/ # stage-main.example.com and stage-product.example.com /home/staging/ logs/ www/ cake/ mainsite/ ... webroot/ product/ ... webroot/ vendors/ # api.example.com, book.example.com, product.com ( with redirect from www.product.com) /home/product/ logs/ www/ api/ ... index.html book/ ... webroot/ cake/ product/ ... webroot/ vendors/ If you think the above directory tree is overcomplicated, or it seems too long for simple tutorial example, stop reading please, and feel free to come back any time later. It was nice to meet you :-) Things are only getting worse from here on in. For those brave enough to read on, you should have an idea of which domains will use which applications, and which applications will share one CakePHP core and folder for logs (not necessarily, read more). Now we are getting somewhere - we need tell our webserver on which external ip addresses it has to listen for incoming connections, and which virtual hosts map to each ip address. Our www subdomains (redirected) should listen on a different ip address then their short versions. This allows us to use different SSL certificates for them later, if there is a need for secure connections. To show what is possible with our config parser, api.example.com will not use a /webroot/ folder, it contains just static html files. To make things even more tricky, api.example.com and book.example.com will not listen on same ip like their neighbour application product.com. cd /usr/local/etc/lighttpd From now on, we will continue our work in this directory. Lets say that we want to use ip 1.2.3.4 for domains example.com, api.example.com and book.example.com. sudo mkdir -p ./1.2.3.4:80/company sudo ln -s /home/company/www/cake ./1.2.3.4:80/company/cake sudo ln -s /home/company/www/vendors ./1.2.3.4:80/company/vendors sudo ln -s /home/company/www/mainsite ./1.2.3.4:80/company/example.com sudo mkdir ./1.2.3.4:80/product sudo ln -s /home/product/www/cake ./1.2.3.4:80/product/cake sudo ln -s /home/product/www/vendors ./1.2.3.4:80/product/vendors sudo ln -s /home/product/www/api ./1.2.3.4:80/product/api.example.com sudo ln -s /home/product/www/book ./1.2.3.4:80/product/book.example.com What exactly did we just do? We created a folder named 1.2.3.4:80, containing 2 subfolders company and product. These will be used as groups of virtual hosts - their names should be the same as the name of their home directory (by default, path for logs can be adjusted). We will use them for setting paths to log files later. Both company and product have a symlinked cake and vendors folders and symlinks named as real domains and pointing to our app folders. Lets continue - ip 2.3.4:5:80 will be used for rest of the group product. sudo mkdir -p ./2.3.4.5:80/product sudo ln -s /home/product/www/cake ./2.3.4.5:80/product/cake sudo ln -s /home/product/www/vendors ./2.3.4.5:80/product/vendors sudo ln -s /home/product/www/product ./2.3.4.5:80/product/product.com That means only one virtual host for now. Ok, ip 3.4.5.6 is going to be used for the www subdomains. No symlinks to existing applications are necessary here, because lighttpd will redirect requests coming to www.example.com to example.com automatically. sudo mkdir -p ./3.4.5.6:80/company/www.example.com ./3.4.5.6:80/product/www.product.com We just had to create ip:port directory for the socket, group(s) of www virtualhosts and some domain-based directories just to have something to point default virtual host of this group at. Staging and development checkouts will all share one ip 4.5.6.7. sudo mkdir -p ./4.5.6.7:80/development sudo ln -s /home/development/www/cake ./4.5.6.7:80/development/cake sudo ln -s /home/development/www/vendors ./4.5.6.7:80/development/vendors sudo ln -s /home/development/www/mainsite ./4.5.6.7:80/development/dev-main.example.com sudo ln -s /home/development/www/product ./4.5.6.7:80/development/dev-product.example.com sudo mkdir ./4.5.6.7:80/staging sudo ln -s /home/staging/www/cake ./4.5.6.7:80/staging/cake sudo ln -s /home/staging/www/vendors ./4.5.6.7:80/staging/vendors sudo ln -s /home/staging/www/mainsite ./4.5.6.7:80/staging/stage-main.example.com sudo ln -s /home/staging/www/product ./4.5.6.7:80/staging/stage-product.example.com Four virtual hosts on one ip from different home folders (therefore placed in different groups). The hard part is complete. Lets go through the bothering part of this custom setup. Did I said already that everything is a file? Don't be scared from amount of necessary steps, it will all be worth it in the future. Lets look what we have done in directory /usr/local/etc/lighttpd/: 1.2.3.4:80/ company/ cake/ <-- /home/company/www/cake example.com/ <-- /home/company/www/mainsite vendors/ <-- /home/company/www/vendors product/ api.example.com/ <-- /home/product/www/api book.example.com/ <-- /home/product/www/book cake/ <-- /home/product/www/cake vendors/ <-- /home/product/www/vendors 2.3.4.5:80/ product/ cake/ <-- /home/product/www/cake product.com/ <-- /home/product/www/product vendors/ <-- /home/product/www/vendors 3.4.5.6:80/ company/www.example.com/ <-- empty directory (redirected), necessary for default virtual host product/www.product.com/ <-- empty directory (redirected), necessary for default virtual host 4.5.6:7:80/ development/ cake/ <-- /home/development/www/cake dev-main.example.com/ <-- /home/development/www/mainsite dev-product.example.com/ <-- /home/development/www/product vendors/ <-- /home/development/www/vendors staging/ cake/ <-- /home/staging/www/cake stage-main.example.com/ <-- /home/staging/www/mainsite stage-product.example.com/ <-- /home/staging/www/product vendors/ <-- /home/staging/www/vendors Some new folders with symlinks. Are you still with me? For those who know mod_simple_vhost, you should be already be pretty clear where we are going. Besides the accesslog path and compress folder path, we will also switch simple-vhost.server-root and simple-vhost.default-host in dependency of used socket and some hostname condition for virthost group. Actually, there is a bit more as well that I will show you. The above directory structure shows that we have 7 groups of virtual hosts in 4 sockets, so lets create 7 simple configuration files for our groups of virtual hosts. Configuration file for group is not required in very special case - no regex pattern for this group, only one virtual host inside and - either only group in socket, or (alphabetically) last one. <?php # /usr/local/etc/lighttpd/1.2.3.4:80/company/config.php $config['group'] = array( 'host' => '^example\.com', 'default' => 'example.com' ); ?> <?php # /usr/local/etc/lighttpd/1.2.3.4:80/product/config.php $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => 'book.example.com' ); ?> <?php # /usr/local/etc/lighttpd/2.3.4.5:80/product/config.php $config['group'] = array( 'host' => '^product\.com', 'default' => 'product.com' ); ?> <?php # /usr/local/etc/lighttpd/3.4.5.6:80/company/config.php $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => 'www.example.com' ); ?> <?php # /usr/local/etc/lighttpd/3.4.5.6:80/product/config.php $config['group'] = array( 'host' => '^(.*)\.product\.com', 'default' => 'www.product.com' ); ?> <?php # /usr/local/etc/lighttpd/4.5.6:7:80/development/config.php $config['group'] = array( 'host' => '^dev-(.*)\.example\.com', 'default' => 'dev-main.example.com' ); ?> <?php # /usr/local/etc/lighttpd/4.5.6:7:80/staging/config.php $config['group'] = array( 'host' => '^stage-(.*)\.example\.com', 'default' => 'stage-main.example.com' ); ?> And that's it. Every group (subfolder of ip.ad.dr.es:80 socket folder) has the required minimal configuration, and everything is properly set up. So lets see what we can take off from it.

Dynamic configuration

Extract this file in folder /usr/local/etc/lighttpd. sudo chmod a+x ./simple_config.php Make simple_config.php executable for everyone. Now run it as a non-privileged user. ./simple_config.php | more You should see a basic generated configuration for your sockets, virthosts and virthosts groups. Now we are already looking at a snippet of the generated configuration. # # Simple configuration parser output # # ERROR logfile /home/company/logs/example-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/example.com/ can not be created, SKIPPING # ERROR logfile /home/product/logs/api-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/api.example.com/ can not be created, SKIPPING # ERROR logfile /home/product/logs/book-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/book.example.com/ can not be created, SKIPPING # ERROR logfile /home/product/logs/product-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/product.com/ can not be created, SKIPPING # ERROR logfile /home/company/logs/www-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/www.example.com/ can not be created, SKIPPING # ERROR logfile /home/product/logs/www-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/www.product.com/ can not be created, SKIPPING # ERROR logfile /home/development/logs/dev-main-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/dev-main.example.com/ can not be created, SKIPPING # ERROR logfile /home/development/logs/dev-product-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/dev-product.example.com/ can not be created, SKIPPING # ERROR logfile /home/staging/logs/stage-main-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/stage-main.example.com/ can not be created, SKIPPING # ERROR logfile /home/staging/logs/stage-product-access_log can not be created, SKIPPING # ERROR compress cache /var/cache/lighttpd/compress/stage-product.example.com/ can not be created, SKIPPING # $SERVER["socket"] == "1.2.3.4:80" { $HTTP["host"] =~ "^example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/company/" simple-vhost.default-host = "example.com" $HTTP["host"] == "example.com" { .... You can see which files this script is trying to create. It will create all of them when you will run it as root once. But there are two things we would like to fix first: access logs /home/company/logs/www-access_log and /home/product/logs/www-access_log are generated for our redirected domains. Lets redirect these logs to those used by domains example.com and product.com: <?php # /usr/local/etc/lighttpd/3.4.5.6:80/company/config.php $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => 'www.example.com' ); $config['virthosts'] = array( 'www.example.com' => array( 'log' => 'example' ) ); ?> <?php # /usr/local/etc/lighttpd/3.4.5.6:80/product/config.php $config['group'] = array( 'host' => '^(.*)\.product\.com', 'default' => 'www.product.com' ); $config['virthosts'] = array( 'www.product.com' => array( 'log' => 'product' ) ); ?> Running ./simple_config.php as unprivileged user again shows this script is no longer trying to create any www-access_log files. We will not care about directories for compressed content, they can be used later, but we will never serve different content on example.com and www.example.com, so it is logical that they share one log file. Every decent logfile parser can handle several domains in one log file. Now, you can run this script as root: sudo ./simple_config.php and result will look much better now: # # Simple configuration parser output # # NOTICE created logfile /home/company/logs/example-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/example.com/ # NOTICE created logfile /home/product/logs/api-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/api.example.com/ # NOTICE created logfile /home/product/logs/book-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/book.example.com/ # NOTICE created logfile /home/product/logs/product-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/product.com/ # NOTICE created compress cache /var/cache/lighttpd/compress/www.example.com/ # NOTICE created compress cache /var/cache/lighttpd/compress/www.product.com/ # NOTICE created logfile /home/development/logs/dev-main-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/dev-main.example.com/ # NOTICE created logfile /home/development/logs/dev-product-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/dev-product.example.com/ # NOTICE created logfile /home/staging/logs/stage-main-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/stage-main.example.com/ # NOTICE created logfile /home/staging/logs/stage-product-access_log # NOTICE created compress cache /var/cache/lighttpd/compress/stage-product.example.com/ # $SERVER["socket"] == "1.2.3.4:80" { $HTTP["host"] =~ "^example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/company/" simple-vhost.default-host = "example.com" $HTTP["host"] == "example.com" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/example.com/" } } else $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/product/" simple-vhost.default-host = "book.example.com" $HTTP["host"] == "api.example.com" { accesslog.filename = "/home/product/logs/api-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/api.example.com/" } else $HTTP["host"] == "book.example.com" { accesslog.filename = "/home/product/logs/book-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/book.example.com/" } } } $SERVER["socket"] == "2.3.4.5:80" { $HTTP["host"] =~ "^product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/2.3.4.5:80/product/" simple-vhost.default-host = "product.com" $HTTP["host"] == "product.com" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/product.com/" } } } $SERVER["socket"] == "3.4.5.6:80" { $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/company/" simple-vhost.default-host = "www.example.com" $HTTP["host"] == "www.example.com" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/www.example.com/" } } else $HTTP["host"] =~ "^(.*)\.product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/product/" simple-vhost.default-host = "www.product.com" $HTTP["host"] == "www.product.com" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/www.product.com/" } } } $SERVER["socket"] == "4.5.6.7:80" { $HTTP["host"] =~ "^dev-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/development/" simple-vhost.default-host = "dev-main.example.com" $HTTP["host"] == "dev-main.example.com" { accesslog.filename = "/home/development/logs/dev-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/dev-main.example.com/" } else $HTTP["host"] == "dev-product.example.com" { accesslog.filename = "/home/development/logs/dev-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/dev-product.example.com/" } } else $HTTP["host"] =~ "^stage-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/staging/" simple-vhost.default-host = "stage-main.example.com" $HTTP["host"] == "stage-main.example.com" { accesslog.filename = "/home/staging/logs/stage-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/stage-main.example.com/" } else $HTTP["host"] == "stage-product.example.com" { accesslog.filename = "/home/staging/logs/stage-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/stage-product.example.com/" } } } Getting close to what we need from this setup. I will process several steps now, and then I will paste here final output of config parser for you to compare with above one. We have another domain manual.example.com (with no virthost set) and we want to redirect it to api.example.com with configuration only, it will be using its own manual-access_log. Furthermore, we want book.example.com condition happen sooner then the condition on api.example.com, because book is gaining more traffic, and attach domain aliases bibliotheca.example.com and bookstore.example.com to book.example.com. Also, expire headers for book should be set for 2 years and as previously mentioned api.example.com is not using /webroot/ folder. <?php # /usr/local/etc/lighttpd/1.2.3.4:80/product/config.php $config['group'] = array( 'host' => '^(.*)\.example\.com', 'default' => 'book.example.com' ); $config['virthosts'] = array( 'book.example.com' => array( 'expire' => array( '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 2 years' ), 'aliases' => array( 'bibliotheca.example.com', 'bookstore.example.com' ) ), 'api.example.com' => array( 'webroot' => '/' ), 'manual.example.com' => array( 'redirect' => 'http://api.example.org/' ) ); ?> All of it is fixed now. We even do not need folder/symlink for manual.example.com in this case. Important note: we do not have to create folders for domains bibliotheca.example.com and bookstore.example.com, because they are aliases for book.example.com and it is used as default virtual host for this group! If you will set alias for non-default virtual host, you have to symlink aliased application several times to group folder - every time with a different domain name. We want all staging sites to store logs in /home/development/logs. Also all staging and development sites should use expire headers for 5 minutes only and have to use http auth (one common file for now). <?php # /usr/local/etc/lighttpd/4.5.6:7:80/development/config.php $config['group'] = array( 'host' => '^dev-(.*)\.example\.com', 'default' => 'dev-main.example.com', 'expire' => array( '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 5 minutes' ), 'auth' => array( 'backend' => 'htpasswd', 'file' => '/var/projects/company/.trac.htpasswd', 'protect' => array( '/' => array( 'realm' => 'Development Access', 'require' => 'valid-user' ) ) ) ); ?> <?php # /usr/local/etc/lighttpd/4.5.6:7:80/staging/config.php $config['group'] = array( 'host' => '^stage-(.*)\.example\.com', 'default' => 'stage-main.example.com', 'expire' => array( '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 5 minutes' ), 'logs' => '/home/development/logs', 'auth' => array( 'backend' => 'htpasswd', 'file' => '/var/projects/company/.trac.htpasswd', 'protect' => array( '/' => array( 'realm' => 'Staging Access', 'require' => 'valid-user' ) ) ) ); ?> This has all been fixed now. Now our simple_config.php returns this: # # Simple configuration parser output # $SERVER["socket"] == "1.2.3.4:80" { $HTTP["host"] =~ "^example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/company/" simple-vhost.default-host = "example.com" $HTTP["host"] == "example.com" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/example.com/" } } else $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/product/" simple-vhost.default-host = "book.example.com" $HTTP["host"] =~ "^(book\.example\.com|bibliotheca\.example\.com|bookstore\.example\.com)" { accesslog.filename = "/home/product/logs/book-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/book.example.com/" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ("" => "access 2 years") } } else $HTTP["host"] == "api.example.com" { accesslog.filename = "/home/product/logs/api-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/api.example.com/" simple-vhost.document-root = "/" } else $HTTP["host"] == "manual.example.com" { accesslog.filename = "/home/product/logs/manual-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/manual.example.com/" url.redirect = ( ".*" => "http://api.example.org/" ) } } } $SERVER["socket"] == "2.3.4.5:80" { $HTTP["host"] =~ "^product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/2.3.4.5:80/product/" simple-vhost.default-host = "product.com" $HTTP["host"] == "product.com" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/product.com/" } } } $SERVER["socket"] == "3.4.5.6:80" { $HTTP["host"] =~ "^(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/company/" simple-vhost.default-host = "www.example.com" $HTTP["host"] == "www.example.com" { accesslog.filename = "/home/company/logs/example-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/www.example.com/" } } else $HTTP["host"] =~ "^(.*)\.product\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/product/" simple-vhost.default-host = "www.product.com" $HTTP["host"] == "www.product.com" { accesslog.filename = "/home/product/logs/product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/www.product.com/" } } } $SERVER["socket"] == "4.5.6.7:80" { $HTTP["host"] =~ "^dev-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/development/" simple-vhost.default-host = "dev-main.example.com" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ("" => "access 5 minutes") } auth.backend = "htpasswd" auth.backend.htpasswd.userfile = "/var/projects/company/.trac.htpasswd" auth.require = ( "/" => ( "method" => "basic", "realm" => "Development Access", "require" => "valid-user" ) ) $HTTP["host"] == "dev-main.example.com" { accesslog.filename = "/home/development/logs/dev-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/dev-main.example.com/" } else $HTTP["host"] == "dev-product.example.com" { accesslog.filename = "/home/development/logs/dev-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/dev-product.example.com/" } } else $HTTP["host"] =~ "^stage-(.*)\.example\.com" { simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/staging/" simple-vhost.default-host = "stage-main.example.com" $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" { expire.url = ("" => "access 5 minutes") } auth.backend = "htpasswd" auth.backend.htpasswd.userfile = "/var/projects/company/.trac.htpasswd" auth.require = ( "/" => ( "method" => "basic", "realm" => "Staging Access", "require" => "valid-user" ) ) $HTTP["host"] == "stage-main.example.com" { accesslog.filename = "/home/development/logs/stage-main-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/stage-main.example.com/" } else $HTTP["host"] == "stage-product.example.com" { accesslog.filename = "/home/development/logs/stage-product-access_log" compress.cache-dir = "/var/cache/lighttpd/compress/stage-product.example.com/" } } } Now it looks like we are set with everything we needed. One last line for /usr/local/etc/lighttpd.conf is: include_shell "/usr/local/etc/lighttpd/simple_config.php" And that's all. Before you will start or restart lighttpd, try and see if it can parse the new configuration (with our include) without errors, or inspect how it sees configuration after parsing: lighttpd -t -f /usr/local/etc/lighttpd.conf lighttpd -p -f /usr/local/etc/lighttpd.conf It is better to run the above commands as root, off course.

Now what?

Think twice about patterns for groups - don't be surprised if you get 'It works' page or default virthost of another group, if you are too lazy to read the generated configuration! Groups are processed in alphabetical order - just so you know which patterns are going to be checked first. Well, it is possible to change order of groups - change name of some company group folder to xxx_company and: $config['group'] = array( 'name' => 'company', Now you should be fine - this group in folder named xxx_company instead of company, and everything will still work. Everything that is necessary should be up and running now. Lighttpd should serve all virtual hosts from groups in sockets from now on. Read how to clear cache for mod_compress too. Smart brain should ask now, why we are using mod_simple_vhost, if our parser generates configuration for every virtual host it founds in our configuration files and directory structure. We don't do it, but you can - read code. Note for these who do not want or can not follow our default logs location, home directories, cache directories, user account lighttpd will use, or want to store directory structure with sockets/groups/virthosts somewhere else - read code too ;-) Reason why we set mod_simple_vhost for this example as default is simple - to get some domain serving some application, we need only one simple thing: symlink to app directory with domain name, placed in some virtual group in proper socket. This virtual host will be accessible immediately - although, restart of webserver is still necessary to have configuration for access logfile and compress directory for this virtual host (otherwise default accesslog and compress dir will be used), but not required. A few questions remain, what and how needs to be done in obvious use cases - adding new ip addresses, groups, virthosts, or moving whole groups over sockets, moving virthosts over sockets, etc... I assume this part will be sweet piece of cake for you. Definitely - feel free to call simple_config.php as often as you want to. It is highly reccommended to save functional configuration to a backup file by redirecting the output. Sure, one can use include "/some/path/generated_output.conf" exclusively, instead of include_shell - it is up to you. Backup, backup, backup. This is nothing more then a functional example, but the entire code lives in one class, so feel free to change or extend it for your needs. It is released under MIT license and is provided as it is, so you can do anything you want with it (except for removing license and copyright note). Keep in mind it was not tested in all possible situations and some of things I did not mention in this tutorial (but they are implemented in code) were not intensively tested yet. If you feel that some of the subdomains used in this tutorial sound familiar to you, you are probably right. I didn't said it was going to be a fairy tale. I said, I will tell you a story. To be continued...

Basic CakePHP markup for designers

This article assumes you practice semantic markup and have a fairly good understanding of CSS. CakePHP does a number of things automatically that designers might not expect, which can cause confusion for designers and difficulty for developers in implementing markup. I hope to provide some general purpose information and guidelines to designing for a CakePHP site. I would like to make a couple quick notes first however. CakePHP is a very flexible platform where markup can be re-used very easily. The more you as a designer try to make reusable markup, the easier your developers life will be. One of the most important things that you can do is to avoid id's other than for top level wrappers such as columns. This allows the developer to change content to address shifting specifications without having to rewrite chunks of CSS in order to avoid Id conflicts in the generated markup. CakePHP has a flexible output system that lets developers easily specify how output is generated. Unlike some applications such as older CMS systems which have a specified header and footer that are called before and after the content, Cake applications render their (x)html into a layout. Two main types of output are rendered into a layout, Views and Elements. Views are the page specific content, such as a blog post, or an entry form. Elements are blocks of markup that can be used across multiple pages, and can be used in the layout, in the view, or even from inside another element. Form related tags can appear in any of the output region types, but it is helpful if you as a designer try to avoid having forms span multiple elements or different sections. Doing so makes it much easier to ensure that the form will work correctly every time it appears. By default most form elements are wrapped in a div with descriptor classes such as “input text” for a text field. Check boxes, and radio buttons are wrapped in a fieldset instead of a div. Also by default the form helper automatically generates id's for the form input widget, and a label that matches it. This is another situation where it is easier to avoid an id attribute and use a class name as a designer. If you specify and ID, your developer has to check that it has not been used in CSS to specify a style, or choose to override it with your specified value and loose flexibility in the future. Examples of form widgets: Select box:   Blog blog. Text field:   Title Check Boxes:   Status Published Allow Comments Moderate   The other CakePHP core utility that generates a fair amount of HTML is the paginator helper. While the helper itself has been covered on the bakery before and is worthy of several blog posts itself, I'd at least like to touch on the default markup generated by the helper.   You can easily specify a string that will appear in between the spans, and each of the other elements are called separately, so they can be in other spots on the page, or have markup between. Unfortunately it is not easily possible to remove the span from the markup, but it would be fairly easy to change the pagination output to something similar to this:  

Meet the CakeDC

The articles section of our site will include best practices, business advice, and technical advice, updated weekly or more often by our talented developers. Our articles will touch upon a wide range of topics from our experiences and expertise. Check back frequently for fresh thoughts from our seasoned talent, including Mark Story who was at CakeFest in Argentina providing updates daily on the talks. As a founder of and lead developer at the Cake Development Corporation, I am proud to provide the inaugural article for the all new CakeDC.com. It has been a long and wonderful last 12 months, and CakeDC has helped bring reality to the limitless potential of our clients' projects. Like any great recipe, the ingredients of Cake Development Corporation have been hand–selected and carefully measured to create the very best blend of talent, imagination, and sophistication of any team in the world. It is my great pleasure to introduce you to our outstanding development team, as well as provide a little background about where our company has come from and where we are headed. After spending nearly 2 years working exclusively on CakePHP and seeing its popularity growing daily, Garrett Woodworth and I realized the need for a team of experts devoted to building the best applications for clients. Therefore, in 2007, Garrett and I formed Cake Development Corporation – a company dedicated to bringing the potential of CakePHP to life while supporting its further development and aiding clients in the rapid development of web applications. In this venture, Garrett and I recognized that we had a unique and rarely realized opportunity to do what we love as our full–time jobs – and, unlike during those initial two years of development, actually get paid to do it. As a business partner, I could not have asked for anyone better than Garrett. A key player in the development of CakePHP, Garrett continues to develop the code and drive the machines that keep users coming back for more. He is extremely passionate about the work he does and takes pride in delivering quality products that are the best of their kind, a true leader the team members can look up to. In fact, I have come to think of him as a younger brother – someone whom I respect greatly, but with whom I also spar from time to time (in a healthy way, of course). After a short time, it became necessary to bring on some new talent. Garrett and I welcomed a third developer Jitka Koukalová, to our small family in 2007. Jitka, a developer who was active in the CakePHP community and was vital contributor to its code, was an amazing addition to our team from the beginning, and exhibited great skill in, and equally great appreciation for, our open source platform. With great attention to detail, she became instrumental in server security and maintenance. Her ability to find an issue and fix it fast along with her classy and professional demeanor would make her a mentor to future members of the Cake development team. Florian Krämer, the next to join CakeDC, soon became our second great recruit. Florian has continually proven himself to be extremely knowledgeable when it comes to developing applications. Florian's meticulous attention to detail, outstanding work ethic, and dogged pursuit of excellence continue to be hugely important assets of our company. But more than a colleague, I consider Florian a good friend. Apart from his great skill, his fun and personable demeanor make him a great team member and someone I genuinely enjoy working with. He has a great sense of humor – or, at least, he appreciates mine! Having seen a great deal of growth in 2007, we sought to expand our development team in early 2008 by welcoming aboard a whole new crop of distinguished developers with a great working knowledge of CakePHP and a passion for its vast capabilities. The first of this group was Yevgeny Tomenko. Yevgeny knows an astounding amount about the inner workings of application development, and I am continually amazed at his dependability and sheer speed. Since joining our team, he has become a great "go–to" person who uses his years of experience to help guide the others on the team. I really love how passionate he is about our product, and it’s great to see such a talented and focused guy enjoying his work at CakeDC. Next, after spending some time watching Niles Rowland help others in the CakePHP IRC channel, we realized he would be a true asset to the CakeDC team. We recruited him in early 2008, and since then he has been a very knowledgeable and dependable part of the development team with a vast understanding of CakePHP and programming in general. Soon to follow Niles were Erin McCargar and Daniel Feinberg, both of whom came aboard in May of 2008. Erin has a long history working with CakePHP and is looked up to as an advisor by those on her team. Daniel, a key person when it comes to machine learning, is very knowledgeable with CakePHP. With outstanding attention to detail and a great skill in their fields of expertise, both are willing and able to go the extra mile to make a project shine. Finally, our most recent addition to the family is Mark Story who joined the team in November of 2008. Mark brings a lot to CakeDC; he is a core developer of CakePHP with design experience that is second to none. Mark is all about getting things done right the first time, and is great at working with others. The rest of the team truly looks up to him for both his skill and amicable personality. This eclectic group of talent is the heart and soul of CakeDC, the family unit that makes our company stand a head and shoulders above the rest. After seeing how far we have come in the last two years, I am very excited to see what this team can accomplish next! We are looking forward to all the challenges that lay ahead and can’t wait to help our clients create a brighter, more innovative future.

Felix Geisendörfer - Jquery and CakePHP

Felix started off by baking his application. While baking his controller he added in the Javascript Helper and RequestHandler Component. This will save adding them in later. Because the demo was based on Jquery the Ajax helper could not be used. However, Felix raised a very good issue of using a helper vs. writing Javascript. His opinion is that if you want to build a heavy AJAX application you should write all your javascript. If you just need a bit of AJAX sprinkled on you can use the helper. However, helpers are difficult to use in wide applications as they abstract one language into another which is tricky. Felix talked briefly about how he organizes his javascript. He likes to have specific javascript files for each controller/view placed in js/views as well as a few utility libraries. This allows his projects to reuse general js code as well as keep the Javascript separate for each view.

Adding jquery.form

Felix recommends using the form jquery plugin for working with forms. It allows you to easily add ajax behavior to your forms. After adding jquery.forms to the layout and creating his view js file. He was able to quickly make his form Ajax-ified. A question was asked about using JSON with Ajax in CakePHP. Felix then demonstrated how you could create a JSON Ajax view. By adding Router::parseExtensions('json'); Felix then created his json layout and his json view. Extension based views need to go into a directory that shares the name with the extension. By adding a 2nd and 4th parameter to $.get() you can force a json return. $.get('/cakefest/view/2.json', {}, function(response, status){ //handle response here }, 'json'); Is an example of how to do this. A question about pagination was asked. Felix's solution was to use a selector and attach an event to all your pagination buttons.   This concludes CakeFest Argentina. I had a really great time and would like to thank everyone who came out, and everyone who presented. I would also like to thank all of our sponsors, SaniSoft, Zeadoo, WidgetPress and the CakeDC. Lastly, thanks to Mariano and Claudio for orgnanizing and hosting the event.

Garret Woodworth - Advanced console

Since we got through bake on the last talk, we are going to look at some advanced usage of bake and other consoles. The interactive console was demonstrated. With the interactive console, you can examine the routes, interact with models. Following the console demonstration, Garrett demonstrated creation of a custom shell. Building on a previous example application, garret started building a shell to generate a menu. This covered creating methods in your shell, using in() and out() as well as how to access args and params. A demo of Cakebot was next. Cakebot is the IRC bot used in #cakephp, Cakebot was also written as a CakePHP shell.

Mariano Iglesias - Internationalizatio...

Why internationalize? You can attract a larger market by making content available in additional languages. When offering international content, you need to translate both the fixed and dynamic content in the database. In CakePHP you can facilitate translation with __() and __n(). In addition content stored in the database can be translated with TranslateBehavior. Multibyte characters exist in many languages, characters outside the traditional latin character set are represented with multibyte characters. This allows for the creation of additional characters and idiomatic expresssion. There is a PHP extentsion for using multibyte strings, it provides the mb_ functions. CakePHP also provides a MultiByte class which provides all the mb_ functions for PHP4 or PHP5 installations lacking them.

Using translation in CakePHP

By using __() we can create translatable strings in our applications. You can use placeholders as well in your translation strings. It is important to use place holders as parameters such as a name can move around in a sentance based on the language. Once you have added all the __() calls, you can use cake i18n extract -output app/locale. The generated file will be placed in app/locale/default.pot Mariano then gave an example and quick walkthrough of translation and i18n. He added translated elements to the views. Then extracted the strings with the shell and created translated values with poEdit.

Translating database content

Translate behavior helps to translate any database content. All translations in CakePHP are stored in one table. Translate behavior takes a number of fields when included on a model. This indicates which fields are going to be translated. Translated fields are not needed on the table schema. The translate behavior also allows you to specify a number of languages and language specific values or you can simply add one language. Later on another language can be added. The rest of Translate Behavior works seamlessly on find() and save(). When using elements with caching. You can use a key parameter when calling element() to ensure that different languages are cached separately. Mariano's talk brought the third day of CakeFest to an end. This has been an excellent conference so far and the last day promises to be great as well.

Martin Radosta - Record level security...

Martin's presentation was based around a behavior that he wrote to provide access control using SQL. In designing a solution, martin came up with a few criteria. The solution must be generic, it only requires 4 fields on any table that will be using the behavior. It should perform quickly and not create a lot of extra queries. The solution he searched for also needed to provide a few features. It should provide permissions for read write and delete. Permissions are assigned by role, with users having many roles. This system is similar to ACL but different, in that it stores the permissions for each record. Martin's behavior uses a permission system similar to the unix file system, with a owner, group, world access. This was implemented as series of bit masks. Unlike the unix filesystems, these permissions are summed and stored as one field. The 4 fields mentioned earlier are user_id, role_id, group_id, and permissions. These four fields allow the behavior to work, both the roles and groups also use binary values to reduce the number of columns. The binary values for roles and groups are compared to those in the role and group id for records . In addition the requested permission is combined with group and role values and checked against the permissions field. Since binary values are used, roles can be combined and will always be unique. Permissions in a system like this permissions are done via a bitmask system. User values for group, and role are compared to record permissions. If the value of the bitmask meets the expectation, the record is returned. In addition to a controller, an element, action in Appcontroller and a model are used. Martin gave a quick demo of an application using his behavior. The permissions checks are all done in the SQL of the behavior. Another interesting part of the behavior is that in the behavior's afterFind() extra values are added to indicate whether or not a user can write or delete. This allows for your interface to display the correct icons. Which is a nice added bonus. I personally was really impressed with how his system was designed and how it worked. He demonstrated how his fine grained access control group. He even had a root user that was not bound by the permissions system. The permissionYou can find this project at Sourceforge the project is licensed under the MIT license.

Fabian Andres - CakePHP in big compani...

Why talk about large companies. Larger companies have large demands and often they involve money over the internet. Large companies require fast and efficient applications, which in turn requires efficient tools. This is where CakePHP steps in. Current solutions for large merchant solutions include Java, and .Net solutions. In addition to these solutions, there are alternatives in PHP, ruby, python. However, rapid development is difficult in some of the popular solutions. By using an alternate solution you can tap into a large pool of developers worldwide. CakePHP helps in this area as well, as with a large community allows you to find developers to fill specific roles. CakePHP offers connectors for many databases. CakePHP also offers easy to use and efficient caching methods which help improve performance. There are many CakePHP sites run by large companies including Mozilla, Sony, Diesel, TV mallorca. Fabian then showed many of the websites that were mentioned in the presentation. Following this presentation there was a good discussion on caching and performance, and how to deal with high traffic situations.

Jeff Loiselle - Authentication & Autho...

Why authorize and authenticate? Authorization ensures someone is who they say they are. Restricting access is a separate process. Authorization is done in CakePHP with Auth Component. Auth works with a database by default, but it can be changed to work with anything. LDAP, Radius, and OpenID are some examples of alternate authentication sources. Auth component has a number of benefits. It hashes passwords, and it provides one API for all authentication sources.

Setting up AuthComponent

Need to setup a users table and create a user model. You should have a username and password field. This will help you make the most of the intelligent defaults built into Auth. After that you need to add Auth to you controllers var $components.

Acl component

Has an easy to use API, identifies, create nodes, checks permissions. The ACL component handles hierarchies with ease, which allows inheritance permission systems.

Auth modes and simple auth.

Auth component has a number of modes, each handles authorization in a different situation. Jeff did a live demo of setting up the ACL tables. Jeff setup the acl tables with cake acl initdb. After the acl tables were set up Jeff covered creating a simple login form and easy Auth.

Using Auth and Acl's together.

Before you can start using Auth and Acl together, you need some aros and acos. Jeff used his Acl management plugin to quickly create some aros and acos, as well as set the permissions for his new user and controllers. Jeff then demonstrated how to use to the AuthComponent in crud mode. Jeff ran out of time, but covered a few good options for handling Auth and ACL.

Jim Lerza - Origami - Automating the p...

Jim is from Expungement Assitance Services. They run clearMyRecord.com which helps people get their criminal records expunged. Origami is a proposed solution for automating criminal records relief in the United States. The application helps manage all the different types of forms that need to be filled out. All these forms need to be collected and standardized so that the data can be normalized. This helps to automate future applications, and streamline the the user experience. There was an existing application that had a pile of legacy code. This legacy code was partly written in mandarin, and had a difficult to understand naming convention. The database was a mess, and the identification system was jumbled and difficult to follow. There were many challenges and requirements that led them to choose CakePHP, including needed features, time, funding and other constraints. Their team faced a few Cake specific challenges. They had trouble with OOP practices, getting sandboxes setup, and some members of their team had challenges with the API documentation and the contents of the book. They also had difficulties figuring out how to implement specific features. Afterwards there was a quick demonstration of the clearMyRecord application and a sample form was filled out. Even though the team at EAS seemed to face a lot of challenges. Jim seemed to be fairly happy with their choice in frameworks, and would make the same choice again.

Ryan Petersen - Collective Intelligence

Collective intelligence is a shared or group intelligence that emerges from the collaboration and competition of many individual. Some examples would be Amazon's book recommendations. Netflix uses collective intelligence to track their stock and supply changes, and more accurately allocate their resources. Recommendations requires something to track like sales, or user preferences. A group of users is also required, the larger the group the better. A research survey found that a random sample of less that 1500 people will be the ideal and most efficient sample size. There are many ways to estimate preferences. The most simple and basic one is Euclidean distance scoring. Pearson Correlation Score, also uses 2 axis to plot out the points, this method is used by amazon and delicious.

Stochastic Optimization

Theory of being able to optimize things. Requires a cost function, it is the most important part of optimization as it gathers the dataset used. Ie. Execution time of a request, time used for data integrity, error handling/logging, client bandwidth, data queries/filtering. When doing metrics and calculating cost you can use a unit cost and parts to unit. This will allow you to control and manage the final output. Depending on what you are measuring, you need to control different weighting. Random searching is an inefficient process, as it is difficult to determine whether you are moving toward or away from where you want to be. Writing unit costing functions is a specific to the task you want to achieve. When doing optimization with cost optimization, you need to be aware of local minimums. Once you have reached a new minimum or optimized level you need to do additional testing by expanding the range in either direction. By expanding your test range you help to ensure that you have found a more accurate low cost point, and aim to ensure that you haven't found a local low point. Genetic Algorithms, and Hierarchal Grouping are other statistical methods to look into. Ryan is going to be releasing both Pearson Correlation and Euclid distance scoring will be released on the The bakery and SerenitySoft So check back for some code.

Martín Sarsale - Death of Apache Long ...

Lighttpd runs as one thread there are no forks, and only one thread. Lighttp is not good for large virtualhosts. It also bottlenecks on the disk speed. It is however good, when you have a single large application or need to total control. Lighttpd has several modules that are analogous to Apache such as Rewrite, Auth, Expires etc. Configuration files in lighttpd are very dynamic and allow for variables, regular expressions and if blocks. By using caching well you can reduce the number of requests made. By using Asset.timestamp = force you will get query strings that contain a querystring with the last modified datetime. You can use a configuration script to set the expiry time for these requests.

Lighttpd and PHP.

Lighttpd doesn't use a PHP module like Apache. Instead it connects to PHP via a socket or TCP/IP. Often this is done with FastCGI. This decoupling of PHP processor and webserver allows you to scale horizontally easily by adding more PHP servers. You can implement this with static resources as well. Static files can be easily routed to alternate machines based on regular expressions done in the config files. Unlike apache which handles and processes each request Lighttpd works more like a pipe connecting inputs to outputs, or between the client and the servers. LUA scripting is a simple scripting language that is built into lightttp. For example sending a random image. This can be done with raw PHP, PHP and X-Sendfile, or by using a LUA script. The LUA script is ~400% faster when compared to PHP. This demonstrates one of the many advantages that Lighttpd can offer over Apace There are similar performance gains to be made when using LUA scripts with full page caching. Lighttpd gives you a lot more control on how you configure your severs. You can easily fix problems with CPU use or HDD disk by adding more resources to the appropriate area. And since you can have multiple file servers attached to one Lighttpd process. Swapping out or adding extra disk server is not a difficult thing to do. Lighttpd seems like a viable and exciting alternative to Apache that I would consider using in the future.

Garrett Woodworth - CakePHP console ba...

Garret's talk focused on getting command line tools setup. Before getting started on that he covered how to setup a virtual host for a cake project. Once he got into the cake shell, he covered the basic core shells and how to call shells. Garret then used bake to create a new project and a database config file. Using cake bake project and cake bake db_config. Up next was creating a schema file with cake schema generate. The schema file was populated with a few tables to store some information about restaurants. A question was raised about adding and removing columns. Garret demonstrated that as well. Baking models controllers and views was up next. Using the previously created restaurants table, garret walked through baking a model, controller, and some views. At first a scaffold was used but the views were made concrete very quickly. The talk ended with a quick tour of the API and TestSuite shell, and how to look up different methods on different classes. Testing was up last and garret ran the freshly baked tests from the CLI. Garret will be continuing this talk later this week.

Mariano Iglesias - CakeFest.org: Theme...

Mariano is presenting about creating a complete website with a framework. Specifically Mariano will be talking the CakeFest site. This should help to show how to apply the theories that have been talked about up to today and throughout the rest of the week. cakefest.org was developed for the first Cakefest, it is multi-user, multi-event, multi-language application, that was needed to scale for mulitple events. It was built with Themes, Behaviors, pagination, security, Acl, Auth and i18n.

Using themes to 'reskin' a site

Themes are set by using Controller::$view = 'Theme'; and setting Controller::$theme = 'themeName'. The theme views allow you to override specific views and leave others as they are in the core application. In the CakeFest site the theme was tied to a url parameter. This allowed search engines to crawl and consume all of the content. A cookie would not allow that.

Behaviors

Behaviors were used to reduce and abstract much of the model code. Behaviors allow you to reuse code not only between models in this application but others as well. Mariano provided an example with the tokenaable behavior used in resetting passwords.

Security

You can use security to stop form manipulation, and force actions to only accept specific HTTP requests. For example using requireGet('login') will not allow a POST or any other request type other than GET to login. Security component automatically secures all forms built with form helper. A hash key is added to each form, and if the hash key doesn't match after post, the request is sent to a blackHole. Using the security component is an easy way to keep forms secure and safe.

Email Component

Email component was used to send emails from the CakeFest site. It used templates and attachments which are both native to the EmailComponent.

RSS and RequestHandler

By using RequestHandler and Router::parseExtensions() we don't need to add a separate action for the RSS and non RSS versions of the news listings. With this approach we also use a separate view file for the RSS format. Another benefit is that we are not required to set the headers manually or switch the layout. This helps to save time and effort, as the layout switching and header content types are generated automatically.

Nate Abele - PHP is dying

Nate started with a brief history of computation, starting with the abacus. And progressing to PHP and covering the presence of bad PHP and Perl code.

The good and bad of frameworks

Frameworks can help inexperienced developers write better code, however a framework can also be used in a black box context. As a tool that you use, you should try to understand how and why it works. Having a blackbox is not going to help you get better. If you want to use a tool and use it well you have to open it up. PHP still faces a number of challenges even after inexperienced developers are not included. These include a lack of namespaces, anonymous functions, closures. Many of these issues are solved by PHP 5.3. However, PHP is still dying becuase you can't overload operators. And it is just too simple. It requires only a few lines to generate output. Unlike a real language like Java or C++. It was a very tongue in cheek presentation, but it was well received and fun to watch.

We Bake with CakePHP