CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

Lighty Story

Written by Rafael on December 20, 2008
27912 VIEWS

I will tell you a story. Once upon a time... Seriously though, it was not too long ago in the past - but it happened and it is possible you can benefit from it.

What?

This tutorial will show how to make lighttpd 1.4.20 serve virtual hosts with CakePHP applications. Our scenario is quite simple:

  1. For admin purposes, lighttpd will listen on localhost, it will serve several CakePHP applications on several external ip addresses, without SSL.
  2. Virtual hosts will be organized in groups and every group will use one CakePHP core checkout for its virtual hosts.
  3. Every virtual host will have it own access log (this server will not run hundreds of virtual hosts, so we can afford to waste one file descriptor for each) and its own directory for caching of compressed static files.
  4. Management of virtual hosts, their default and custom settings should be as easy as possible, so we can delegate the management of some ip addresses or just groups of virthosts to someone else and sleep well, because nobody will have to touch our precious configuration files.

However, our scenario has some special requirements which we need to solve. By the way, I will be showing you how to do things the hard way from the start. In hopes to spare you a lot of headaches in future. Lighttpd is sweet piece of software, and is under active development. Unfortunately, there are things that are not easy to set up. For example - when using any of provided virtual host modules, it is impossible to set up different access logs and cache directories for compressed content etc. dynamically in a pure lighty config file without external scripts. Everything (except for per virtual host errorlog) is possible by writing necessary configuration by hand. But we willing to work more now, so we can be lazy later!

There are several approaches for bash, Ruby etc. However, nothing usable in PHP as far as I know. I will show you how easy it could be. Take this as a working example, I am sharing ideas here, not bullet-proof all-mighty solutions. Lets go for it - and utilize PHP and the include_shell command in our lighttpd configuration file. The motto of this article is: it is easier read generated configuration, then write it by hand.

How? Lighty!

Don't think this is not a good answer. Lets set up a decent lighttpd installation. We'll assume you have it compiled and installed. Lets also assume that you have PHP prepared for lighttpd's ModFastCGI and are just waiting for configuration and the first test run. Also, for shell commands which need to be executed under root account, I'll use sudo in following examples.

    sudo mkdir /usr/local/etc/lighttpd

First of all, we need a directory for our custom configuration. When in doubt, a fast look into its contents will tell you everything one should know about virtual hosts configuration.

    sudo mkdir -p /usr/local/www/data/default/webroot
    echo "<html><head><title>It works<body>It works" > /usr/local/www/data/default/webroot/index.html

Next we created a directory for our default webroot. It will be used on localhost only, with index.html.

    sudo touch /var/log/lighttpd.error.log /var/log/lighttpd.access.log
    sudo chown www:www /var/log/lighttpd.error.log /var/log/lighttpd.access.log

Now we need to create error and access log files. The first one will be common for whole server, the second will be used for localhost only.

    sudo mkdir -p /var/cache/lighttpd/compress/default
    sudo chown -R www:www /var/cache/lighttpd

The last thing we had to prepare was the default directory for caching of compressed static files.

In /usr/local/etc/lighttpd.conf we will setup a simple config file containing the common configuration we will utilize later:

    server.modules = (
        "mod_simple_vhost",
        "mod_magnet",
        "mod_redirect",
        "mod_access",
        "mod_auth",
        "mod_expire",
        "mod_compress",
        "mod_fastcgi",
        "mod_accesslog"
    )
    
    server.document-root = "/usr/local/www/data/default/webroot/"
    server.errorlog = "/var/log/lighttpd.error.log"
    accesslog.filename = "/var/log/lighttpd.access.log"
    server.port = 80
    server.bind = "127.0.0.1"
    server.username = "www"
    server.groupname = "www"
    server.pid-file = "/var/run/lighttpd.pid"
    index-file.names = ( "index.php", "index.html", "index.htm", "default.htm" )
    
    # shortened !!!
    mimetype.assign = (
        ...
    )
    
    url.access-deny = ( "~", ".inc" )
    
    static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
    
    dir-listing.activate = "disable"
    
    etag.use-mtime = "enable"
    static-file.etags = "enable"
    
    $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" {
        expire.url = ( "" => "access 7 days" )
    }
    
    compress.cache-dir = "/var/cache/lighttpd/compress/default/"
    compress.filetype = ( "text/plain", "text/html", "text/xml", "text/javascript", "text/css" )
    
    fastcgi.server = (
        ".php" => ((
            "bin-path" => "/usr/local/bin/php-cgi -c /usr/local/etc/php.ini",
            "socket" => "/tmp/lighttpd_php5.socket",
            "min-procs" => 1,
            "max-procs" => 1,
            "bin-environment" => (
                "FCGI_WEB_SERVER_ADDRS" => "127.0.0.1",
                "PHP_FCGI_CHILDREN" => "4",
                "PHP_FCGI_MAX_REQUESTS" => "1000"
            ),
            "bin-copy-environment" => ( "PATH", "SHELL", "USER"),
            "broken-scriptfilename" => "enable"
        ))
    )
    
    simple-vhost.server-root = "/usr/local/www/data/"
    simple-vhost.document-root = "webroot"
    simple-vhost.default-host = "default"
    
    $HTTP["host"] =~ "^www\.(.*)" {
        url.redirect = ( "^/(.*)" => "http://%1/$1" )
    }

How far along are we? So far we have a configured webserver with few preloaded modules and simple common configuration.

Our sever is currently:

  1. Listening on localhost:80.
  2. Refusing directory listing or sending some filetypes as plain text.
  3. Using etags and sending expiration headers for a set of static resources to 7 days by default. This allows us to schedule an upgrade of any virtual host just a week before it will happen.
  4. Using compression and caching of compressed static files for several mimetypes.
  5. Starting PHP as FastCGI, with only one parent process (we are going to use opcode cache). We are allowing only few child processes for this example tutorial and killing fcgi child processes after every 1000 requests
  6. Using mod_simple_vhost for name-based virtual hosting (preconfigured for fallback to default webroot).
  7. Redirecting all domains using www subdomain to the shorter version.

You will probably want to tweak some other settings. I am not going to describe all the server.max* configuration options, or talk about other pretty obvious things like mod_evasive, mod_status, mod_rrdtool etc, don't worry. Two things you should consider if some of your visitors will use one of the major browsers.

    $HTTP["url"] =~ "\.pdf$" {
        server.range-requests = "disable"
    }

You do not want to cut off IE users from your pdf documents, right?

    compress.filetype = ( "text/plain", "text/html", "text/xml" )
    $HTTP["useragent"] =~ "Firefox" {
        compress.filetype  += ("text/javascript", "text/css" )
    }

If your visitors are using an old (and/or above mentioned undesirable) internet browser, you can control compression settings per useragent in this way. Instead of the above example, compressing all 5 crucial mimetypes.

Ready to go? Ok, start lighttpd and make sure you see what you expect at http://localhost/

    echo "<?php phpinfo(); ?>" > /usr/local/www/data/default/webroot/phpinfo.php

Just to be sure that fcgi works as expected, try to see info about your current PHP setup at http://localhost/phpinfo.php and watch /var/log/lighttpd.error.log.

Url rewriting

It is possible to use lighttpd's mod_rewrite and create pattern for our static files if we are sure they exist. This approach has downsides though. We want to setup this part of webserver up and forget it exists. This is not possible with mod_rewrite, because for example, we are not going to force our developers to forget about /js/something.js as url for some of application controllers. Instead, we will use mod_magnet and custom Lua script. Visit this thread at CakePHP Google Group. Save the provided script to /usr/local/etc/lighttpd/cleanurl-v6.lua and add the following line to bottom of /usr/local/etc/lighttpd.conf:

    magnet.attract-physical-path-to = ( "/usr/local/etc/lighttpd/cleanurl-v6.lua" )

After restarting lighttpd, we are ready to remove all the .htaccess files from our filesystem and forget they exist. All requests for non-existing static files will be rewritten to /index.php?url=xxx like CakePHP requires.

Virtual hosts

Now we want to set up a directory structure and custom configuration for our virtual hosts and their groups. We will design a directory structure that can be used for dynamic configuration later, with no need to repeat anything obvious in configuration files. In this case, only logs folder matters (make sure it is writable by webserver). We will symlink everything else. Lets use the following directory structure with CakePHP core and our applications checkouts like our standard:

    # example.com (with redirect from www.example.com)
    /home/company/
                  logs/
                  www/
                      cake/
                      mainsite/
                               ...
                               webroot/
                      vendors/
    # dev-main.example.com and dev-product.example.com
    /home/development/
                  logs/
                  www/
                      cake/
                      mainsite/
                               ...
                               webroot/
                      product/
                               ...
                               webroot/
                      vendors/
    # stage-main.example.com and stage-product.example.com
    /home/staging/
                  logs/
                  www/
                      cake/
                      mainsite/
                               ...
                               webroot/
                      product/
                               ...
                               webroot/
                      vendors/
    # api.example.com, book.example.com, product.com ( with redirect from www.product.com)
    /home/product/
                  logs/
                  www/
                      api/
                          ...
                          index.html
                      book/
                               ...
                               webroot/
                      cake/
                      product/
                               ...
                               webroot/
                      vendors/

If you think the above directory tree is overcomplicated, or it seems too long for simple tutorial example, stop reading please, and feel free to come back any time later. It was nice to meet you :-) Things are only getting worse from here on in. For those brave enough to read on, you should have an idea of which domains will use which applications, and which applications will share one CakePHP core and folder for logs (not necessarily, read more).

Now we are getting somewhere - we need tell our webserver on which external ip addresses it has to listen for incoming connections, and which virtual hosts map to each ip address. Our www subdomains (redirected) should listen on a different ip address then their short versions. This allows us to use different SSL certificates for them later, if there is a need for secure connections. To show what is possible with our config parser, api.example.com will not use a /webroot/ folder, it contains just static html files. To make things even more tricky, api.example.com and book.example.com will not listen on same ip like their neighbour application product.com.

    cd /usr/local/etc/lighttpd

From now on, we will continue our work in this directory.

Lets say that we want to use ip 1.2.3.4 for domains example.com, api.example.com and book.example.com.

    sudo mkdir -p ./1.2.3.4:80/company
    sudo ln -s /home/company/www/cake ./1.2.3.4:80/company/cake
    sudo ln -s /home/company/www/vendors ./1.2.3.4:80/company/vendors
    
    sudo ln -s /home/company/www/mainsite ./1.2.3.4:80/company/example.com
    
    sudo mkdir ./1.2.3.4:80/product
    sudo ln -s /home/product/www/cake ./1.2.3.4:80/product/cake
    sudo ln -s /home/product/www/vendors ./1.2.3.4:80/product/vendors
    
    sudo ln -s /home/product/www/api ./1.2.3.4:80/product/api.example.com
    sudo ln -s /home/product/www/book ./1.2.3.4:80/product/book.example.com

What exactly did we just do? We created a folder named 1.2.3.4:80, containing 2 subfolders company and product. These will be used as groups of virtual hosts - their names should be the same as the name of their home directory (by default, path for logs can be adjusted). We will use them for setting paths to log files later. Both company and product have a symlinked cake and vendors folders and symlinks named as real domains and pointing to our app folders.

Lets continue - ip 2.3.4:5:80 will be used for rest of the group product.

    sudo mkdir -p ./2.3.4.5:80/product
    sudo ln -s /home/product/www/cake ./2.3.4.5:80/product/cake
    sudo ln -s /home/product/www/vendors ./2.3.4.5:80/product/vendors
    
    sudo ln -s /home/product/www/product ./2.3.4.5:80/product/product.com

That means only one virtual host for now.

Ok, ip 3.4.5.6 is going to be used for the www subdomains. No symlinks to existing applications are necessary here, because lighttpd will redirect requests coming to www.example.com to example.com automatically.

    sudo mkdir -p ./3.4.5.6:80/company/www.example.com ./3.4.5.6:80/product/www.product.com

We just had to create ip:port directory for the socket, group(s) of www virtualhosts and some domain-based directories just to have something to point default virtual host of this group at.

Staging and development checkouts will all share one ip 4.5.6.7.

    sudo mkdir -p ./4.5.6.7:80/development
    sudo ln -s /home/development/www/cake ./4.5.6.7:80/development/cake
    sudo ln -s /home/development/www/vendors ./4.5.6.7:80/development/vendors
    
    sudo ln -s /home/development/www/mainsite ./4.5.6.7:80/development/dev-main.example.com
    sudo ln -s /home/development/www/product ./4.5.6.7:80/development/dev-product.example.com
    
    sudo mkdir ./4.5.6.7:80/staging
    sudo ln -s /home/staging/www/cake ./4.5.6.7:80/staging/cake
    sudo ln -s /home/staging/www/vendors ./4.5.6.7:80/staging/vendors
    
    sudo ln -s /home/staging/www/mainsite ./4.5.6.7:80/staging/stage-main.example.com
    sudo ln -s /home/staging/www/product ./4.5.6.7:80/staging/stage-product.example.com

Four virtual hosts on one ip from different home folders (therefore placed in different groups).

The hard part is complete. Lets go through the bothering part of this custom setup. Did I said already that everything is a file? Don't be scared from amount of necessary steps, it will all be worth it in the future.

Lets look what we have done in directory /usr/local/etc/lighttpd/:

    1.2.3.4:80/
               company/
                        cake/        <-- /home/company/www/cake
                        example.com/ <-- /home/company/www/mainsite
                        vendors/     <-- /home/company/www/vendors
               product/
                        api.example.com/  <-- /home/product/www/api
                        book.example.com/ <-- /home/product/www/book
                        cake/             <-- /home/product/www/cake
                        vendors/          <-- /home/product/www/vendors
    2.3.4.5:80/
               product/
                        cake/        <-- /home/product/www/cake
                        product.com/ <-- /home/product/www/product
                        vendors/     <-- /home/product/www/vendors
    3.4.5.6:80/
               company/www.example.com/ <-- empty directory (redirected), necessary for default virtual host 
               product/www.product.com/ <-- empty directory (redirected), necessary for default virtual host
    4.5.6:7:80/
               development/
                        cake/                    <-- /home/development/www/cake
                        dev-main.example.com/    <-- /home/development/www/mainsite
                        dev-product.example.com/ <-- /home/development/www/product
                        vendors/                 <-- /home/development/www/vendors
               staging/
                        cake/                      <-- /home/staging/www/cake
                        stage-main.example.com/    <-- /home/staging/www/mainsite
                        stage-product.example.com/ <-- /home/staging/www/product
                        vendors/                   <-- /home/staging/www/vendors

Some new folders with symlinks.

Are you still with me? For those who know mod_simple_vhost, you should be already be pretty clear where we are going. Besides the accesslog path and compress folder path, we will also switch simple-vhost.server-root and simple-vhost.default-host in dependency of used socket and some hostname condition for virthost group. Actually, there is a bit more as well that I will show you.

The above directory structure shows that we have 7 groups of virtual hosts in 4 sockets, so lets create 7 simple configuration files for our groups of virtual hosts. Configuration file for group is not required in very special case - no regex pattern for this group, only one virtual host inside and - either only group in socket, or (alphabetically) last one.

<?php # /usr/local/etc/lighttpd/1.2.3.4:80/company/config.php
    $config['group'] = array(
        'host' => '^example\.com',
        'default' => 'example.com'
    );
?>
<?php # /usr/local/etc/lighttpd/1.2.3.4:80/product/config.php
    $config['group'] = array(
        'host' => '^(.*)\.example\.com',
        'default' => 'book.example.com'
    );
?>
<?php # /usr/local/etc/lighttpd/2.3.4.5:80/product/config.php
    $config['group'] = array(
        'host' => '^product\.com',
        'default' => 'product.com'
    );
?>
<?php # /usr/local/etc/lighttpd/3.4.5.6:80/company/config.php
    $config['group'] = array(
        'host' => '^(.*)\.example\.com',
        'default' => 'www.example.com'
    );
?>
<?php # /usr/local/etc/lighttpd/3.4.5.6:80/product/config.php
    $config['group'] = array(
        'host' => '^(.*)\.product\.com',
        'default' => 'www.product.com'
    );
?>
<?php # /usr/local/etc/lighttpd/4.5.6:7:80/development/config.php
    $config['group'] = array(
        'host' => '^dev-(.*)\.example\.com',
        'default' => 'dev-main.example.com'
    );
?>
<?php # /usr/local/etc/lighttpd/4.5.6:7:80/staging/config.php
    $config['group'] = array(
        'host' => '^stage-(.*)\.example\.com',
        'default' => 'stage-main.example.com'
    );
?>

And that's it. Every group (subfolder of ip.ad.dr.es:80 socket folder) has the required minimal configuration, and everything is properly set up. So lets see what we can take off from it.

Dynamic configuration

Extract this file in folder /usr/local/etc/lighttpd.

    sudo chmod a+x ./simple_config.php

Make simple_config.php executable for everyone.

Now run it as a non-privileged user.

    ./simple_config.php | more

You should see a basic generated configuration for your sockets, virthosts and virthosts groups.

Now we are already looking at a snippet of the generated configuration.

    #
    # Simple configuration parser output
    #
    # ERROR logfile /home/company/logs/example-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/example.com/ can not be created, SKIPPING
    # ERROR logfile /home/product/logs/api-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/api.example.com/ can not be created, SKIPPING
    # ERROR logfile /home/product/logs/book-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/book.example.com/ can not be created, SKIPPING
    # ERROR logfile /home/product/logs/product-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/product.com/ can not be created, SKIPPING
    # ERROR logfile /home/company/logs/www-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/www.example.com/ can not be created, SKIPPING
    # ERROR logfile /home/product/logs/www-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/www.product.com/ can not be created, SKIPPING
    # ERROR logfile /home/development/logs/dev-main-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/dev-main.example.com/ can not be created, SKIPPING
    # ERROR logfile /home/development/logs/dev-product-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/dev-product.example.com/ can not be created, SKIPPING
    # ERROR logfile /home/staging/logs/stage-main-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/stage-main.example.com/ can not be created, SKIPPING
    # ERROR logfile /home/staging/logs/stage-product-access_log can not be created, SKIPPING
    # ERROR compress cache /var/cache/lighttpd/compress/stage-product.example.com/ can not be created, SKIPPING
    #
    
    $SERVER["socket"] == "1.2.3.4:80" {
            $HTTP["host"] =~ "^example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/company/"
                    simple-vhost.default-host = "example.com"
                    $HTTP["host"] == "example.com" {
                    ....

You can see which files this script is trying to create. It will create all of them when you will run it as root once. But there are two things we would like to fix first: access logs /home/company/logs/www-access_log and /home/product/logs/www-access_log are generated for our redirected domains.

Lets redirect these logs to those used by domains example.com and product.com:

<?php # /usr/local/etc/lighttpd/3.4.5.6:80/company/config.php
    $config['group'] = array(
        'host' => '^(.*)\.example\.com',
        'default' => 'www.example.com'
    );
    $config['virthosts'] = array(
        'www.example.com' => array(
            'log' => 'example'
        )
    );
?>
<?php # /usr/local/etc/lighttpd/3.4.5.6:80/product/config.php
    $config['group'] = array(
        'host' => '^(.*)\.product\.com',
        'default' => 'www.product.com'
    );
    $config['virthosts'] = array(
        'www.product.com' => array(
            'log' => 'product'
        )
    );
?>

Running ./simple_config.php as unprivileged user again shows this script is no longer trying to create any www-access_log files. We will not care about directories for compressed content, they can be used later, but we will never serve different content on example.com and www.example.com, so it is logical that they share one log file. Every decent logfile parser can handle several domains in one log file.

Now, you can run this script as root:

    sudo ./simple_config.php

and result will look much better now:

#
# Simple configuration parser output
#
# NOTICE created logfile /home/company/logs/example-access_log
# NOTICE created compress cache /var/cache/lighttpd/compress/example.com/
# NOTICE created logfile /home/product/logs/api-access_log
# NOTICE created compress cache /var/cache/lighttpd/compress/api.example.com/
# NOTICE created logfile /home/product/logs/book-access_log
# NOTICE created compress cache /var/cache/lighttpd/compress/book.example.com/
# NOTICE created logfile /home/product/logs/product-access_log
# NOTICE created compress cache /var/cache/lighttpd/compress/product.com/
# NOTICE created compress cache /var/cache/lighttpd/compress/www.example.com/
# NOTICE created compress cache /var/cache/lighttpd/compress/www.product.com/
# NOTICE created logfile /home/development/logs/dev-main-access_log
# NOTICE created compress cache /var/cache/lighttpd/compress/dev-main.example.com/
# NOTICE created logfile /home/development/logs/dev-product-access_log
# NOTICE created compress cache /var/cache/lighttpd/compress/dev-product.example.com/
# NOTICE created logfile /home/staging/logs/stage-main-access_log
# NOTICE created compress cache /var/cache/lighttpd/compress/stage-main.example.com/
# NOTICE created logfile /home/staging/logs/stage-product-access_log
# NOTICE created compress cache /var/cache/lighttpd/compress/stage-product.example.com/
#

    $SERVER["socket"] == "1.2.3.4:80" {
            $HTTP["host"] =~ "^example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/company/"
                    simple-vhost.default-host = "example.com"
                    $HTTP["host"] == "example.com" {
                            accesslog.filename = "/home/company/logs/example-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/example.com/"
                    }
            }
            else $HTTP["host"] =~ "^(.*)\.example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/product/"
                    simple-vhost.default-host = "book.example.com"
                    $HTTP["host"] == "api.example.com" {
                            accesslog.filename = "/home/product/logs/api-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/api.example.com/"
                    }
                    else $HTTP["host"] == "book.example.com" {
                            accesslog.filename = "/home/product/logs/book-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/book.example.com/"
                    }
            }
    }
    $SERVER["socket"] == "2.3.4.5:80" {
            $HTTP["host"] =~ "^product\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/2.3.4.5:80/product/"
                    simple-vhost.default-host = "product.com"
                    $HTTP["host"] == "product.com" {
                            accesslog.filename = "/home/product/logs/product-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/product.com/"
                    }
            }
    }
    $SERVER["socket"] == "3.4.5.6:80" {
            $HTTP["host"] =~ "^(.*)\.example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/company/"
                    simple-vhost.default-host = "www.example.com"
                    $HTTP["host"] == "www.example.com" {
                            accesslog.filename = "/home/company/logs/example-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/www.example.com/"
                    }
            }
            else $HTTP["host"] =~ "^(.*)\.product\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/product/"
                    simple-vhost.default-host = "www.product.com"
                    $HTTP["host"] == "www.product.com" {
                            accesslog.filename = "/home/product/logs/product-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/www.product.com/"
                    }
            }
    }
    $SERVER["socket"] == "4.5.6.7:80" {
            $HTTP["host"] =~ "^dev-(.*)\.example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/development/"
                    simple-vhost.default-host = "dev-main.example.com"
                    $HTTP["host"] == "dev-main.example.com" {
                            accesslog.filename = "/home/development/logs/dev-main-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/dev-main.example.com/"
                    }
                    else $HTTP["host"] == "dev-product.example.com" {
                            accesslog.filename = "/home/development/logs/dev-product-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/dev-product.example.com/"
                    }
            }
            else $HTTP["host"] =~ "^stage-(.*)\.example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/staging/"
                    simple-vhost.default-host = "stage-main.example.com"
                    $HTTP["host"] == "stage-main.example.com" {
                            accesslog.filename = "/home/staging/logs/stage-main-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/stage-main.example.com/"
                    }
                    else $HTTP["host"] == "stage-product.example.com" {
                            accesslog.filename = "/home/staging/logs/stage-product-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/stage-product.example.com/"
                    }
            }
    }

Getting close to what we need from this setup.

I will process several steps now, and then I will paste here final output of config parser for you to compare with above one.

We have another domain manual.example.com (with no virthost set) and we want to redirect it to api.example.com with configuration only, it will be using its own manual-access_log. Furthermore, we want book.example.com condition happen sooner then the condition on api.example.com, because book is gaining more traffic, and attach domain aliases bibliotheca.example.com and bookstore.example.com to book.example.com. Also, expire headers for book should be set for 2 years and as previously mentioned api.example.com is not using /webroot/ folder.

<?php # /usr/local/etc/lighttpd/1.2.3.4:80/product/config.php
    $config['group'] = array(
        'host' => '^(.*)\.example\.com',
        'default' => 'book.example.com'
    );
    $config['virthosts'] = array(
        'book.example.com' => array(
            'expire' => array(
                '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 2 years'
            ),
            'aliases' => array(
                'bibliotheca.example.com',
                'bookstore.example.com'
            )
        ),
        'api.example.com' => array(
            'webroot' => '/'
        ),
        'manual.example.com' => array(
             'redirect' => 'http://api.example.org/'
        )
    );
?>

All of it is fixed now. We even do not need folder/symlink for manual.example.com in this case.

Important note: we do not have to create folders for domains bibliotheca.example.com and bookstore.example.com, because they are aliases for book.example.com and it is used as default virtual host for this group! If you will set alias for non-default virtual host, you have to symlink aliased application several times to group folder - every time with a different domain name.

We want all staging sites to store logs in /home/development/logs. Also all staging and development sites should use expire headers for 5 minutes only and have to use http auth (one common file for now).

<?php # /usr/local/etc/lighttpd/4.5.6:7:80/development/config.php 
    $config['group'] = array(
        'host' => '^dev-(.*)\.example\.com', 
        'default' => 'dev-main.example.com', 
        'expire' => array(
             '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 5 minutes' 
        ), 
        'auth' => array( 
            'backend' => 'htpasswd', 
            'file' => '/var/projects/company/.trac.htpasswd', 
            'protect' => array( 
                '/' => array( 
                    'realm' => 'Development Access', 
                    'require' => 'valid-user' 
                ) 
            )
        ) 
    );
?>
<?php # /usr/local/etc/lighttpd/4.5.6:7:80/staging/config.php 
    $config['group'] = array(
        'host' => '^stage-(.*)\.example\.com', 
        'default' => 'stage-main.example.com', 
        'expire' => array( 
            '^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)' => 'access 5 minutes' 
        ),
        'logs' => '/home/development/logs', 
        'auth' => array( 
            'backend' => 'htpasswd', 
            'file' => '/var/projects/company/.trac.htpasswd', 
            'protect' => array( 
                '/' => array( 
                    'realm' => 'Staging Access', 
                    'require' => 'valid-user' 
                ) 
            )
        ) 
    ); 
?>

This has all been fixed now.

Now our simple_config.php returns this:

    #
    # Simple configuration parser output
    #
    
    $SERVER["socket"] == "1.2.3.4:80" {
            $HTTP["host"] =~ "^example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/company/"
                    simple-vhost.default-host = "example.com"
                    $HTTP["host"] == "example.com" {
                            accesslog.filename = "/home/company/logs/example-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/example.com/"
                    }
            }
            else $HTTP["host"] =~ "^(.*)\.example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/1.2.3.4:80/product/"
                    simple-vhost.default-host = "book.example.com"
                    $HTTP["host"] =~ "^(book\.example\.com|bibliotheca\.example\.com|bookstore\.example\.com)" {
                            accesslog.filename = "/home/product/logs/book-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/book.example.com/"
                            $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" {
                                    expire.url = ("" => "access 2 years")
                            }
                    }
                    else $HTTP["host"] == "api.example.com" {
                            accesslog.filename = "/home/product/logs/api-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/api.example.com/"
                            simple-vhost.document-root = "/"
                    }
                    else $HTTP["host"] == "manual.example.com" {
                            accesslog.filename = "/home/product/logs/manual-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/manual.example.com/"
                            url.redirect = (
                                    ".*" => "http://api.example.org/"
                            )
                    }
            }
    }
    $SERVER["socket"] == "2.3.4.5:80" {
            $HTTP["host"] =~ "^product\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/2.3.4.5:80/product/"
                    simple-vhost.default-host = "product.com"
                    $HTTP["host"] == "product.com" {
                            accesslog.filename = "/home/product/logs/product-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/product.com/"
                    }
            }
    }
    $SERVER["socket"] == "3.4.5.6:80" {
            $HTTP["host"] =~ "^(.*)\.example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/company/"
                    simple-vhost.default-host = "www.example.com"
                    $HTTP["host"] == "www.example.com" {
                            accesslog.filename = "/home/company/logs/example-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/www.example.com/"
                    }
            }
            else $HTTP["host"] =~ "^(.*)\.product\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/3.4.5.6:80/product/"
                    simple-vhost.default-host = "www.product.com"
                    $HTTP["host"] == "www.product.com" {
                            accesslog.filename = "/home/product/logs/product-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/www.product.com/"
                    }
            }
    }
    $SERVER["socket"] == "4.5.6.7:80" {
            $HTTP["host"] =~ "^dev-(.*)\.example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/development/"
                    simple-vhost.default-host = "dev-main.example.com"
                    $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" {
                            expire.url = ("" => "access 5 minutes")
                    }
                    auth.backend = "htpasswd"
                    auth.backend.htpasswd.userfile = "/var/projects/company/.trac.htpasswd"
                    auth.require = (
                            "/" => (
                                    "method" => "basic",
                                    "realm" => "Development Access",
                                    "require" => "valid-user"
                            )
                    )
                    $HTTP["host"] == "dev-main.example.com" {
                            accesslog.filename = "/home/development/logs/dev-main-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/dev-main.example.com/"
                    }
                    else $HTTP["host"] == "dev-product.example.com" {
                            accesslog.filename = "/home/development/logs/dev-product-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/dev-product.example.com/"
                    }
            }
            else $HTTP["host"] =~ "^stage-(.*)\.example\.com" {
                    simple-vhost.server-root = "/usr/local/etc/lighttpd/4.5.6.7:80/staging/"
                    simple-vhost.default-host = "stage-main.example.com"
                    $HTTP["url"] =~ "^(/css/|/files/|/img/|/js/|/images/|/themed/|/favicon.ico)" {
                            expire.url = ("" => "access 5 minutes")
                    }
                    auth.backend = "htpasswd"
                    auth.backend.htpasswd.userfile = "/var/projects/company/.trac.htpasswd"
                    auth.require = (
                            "/" => (
                                    "method" => "basic",
                                    "realm" => "Staging Access",
                                    "require" => "valid-user"
                            )
                    )
                    $HTTP["host"] == "stage-main.example.com" {
                            accesslog.filename = "/home/development/logs/stage-main-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/stage-main.example.com/"
                    }
                    else $HTTP["host"] == "stage-product.example.com" {
                            accesslog.filename = "/home/development/logs/stage-product-access_log"
                            compress.cache-dir = "/var/cache/lighttpd/compress/stage-product.example.com/"
                    }
            }
    }

Now it looks like we are set with everything we needed.

One last line for /usr/local/etc/lighttpd.conf is:

    include_shell "/usr/local/etc/lighttpd/simple_config.php"

And that's all.

Before you will start or restart lighttpd, try and see if it can parse the new configuration (with our include) without errors, or inspect how it sees configuration after parsing:

    lighttpd -t -f /usr/local/etc/lighttpd.conf
    lighttpd -p -f /usr/local/etc/lighttpd.conf

It is better to run the above commands as root, off course.

Now what?

Think twice about patterns for groups - don't be surprised if you get 'It works' page or default virthost of another group, if you are too lazy to read the generated configuration! Groups are processed in alphabetical order - just so you know which patterns are going to be checked first. Well, it is possible to change order of groups - change name of some company group folder to xxx_company and:

    $config['group'] = array(
        'name' => 'company',

Now you should be fine - this group in folder named xxx_company instead of company, and everything will still work.

Everything that is necessary should be up and running now. Lighttpd should serve all virtual hosts from groups in sockets from now on. Read how to clear cache for mod_compress too. Smart brain should ask now, why we are using mod_simple_vhost, if our parser generates configuration for every virtual host it founds in our configuration files and directory structure. We don't do it, but you can - read code. Note for these who do not want or can not follow our default logs location, home directories, cache directories, user account lighttpd will use, or want to store directory structure with sockets/groups/virthosts somewhere else - read code too ;-) Reason why we set mod_simple_vhost for this example as default is simple - to get some domain serving some application, we need only one simple thing: symlink to app directory with domain name, placed in some virtual group in proper socket. This virtual host will be accessible immediately - although, restart of webserver is still necessary to have configuration for access logfile and compress directory for this virtual host (otherwise default accesslog and compress dir will be used), but not required.

A few questions remain, what and how needs to be done in obvious use cases - adding new ip addresses, groups, virthosts, or moving whole groups over sockets, moving virthosts over sockets, etc... I assume this part will be sweet piece of cake for you. Definitely - feel free to call simple_config.php as often as you want to. It is highly reccommended to save functional configuration to a backup file by redirecting the output. Sure, one can use include "/some/path/generated_output.conf" exclusively, instead of include_shell - it is up to you.

Backup, backup, backup. This is nothing more then a functional example, but the entire code lives in one class, so feel free to change or extend it for your needs. It is released under MIT license and is provided as it is, so you can do anything you want with it (except for removing license and copyright note). Keep in mind it was not tested in all possible situations and some of things I did not mention in this tutorial (but they are implemented in code) were not intensively tested yet.

If you feel that some of the subdomains used in this tutorial sound familiar to you, you are probably right. I didn't said it was going to be a fairy tale. I said, I will tell you a story. To be continued...

Latest articles

48 Views

The Generational Perception of Work and Productivity in the Remote-Work Era

Generational Work Illustration

The Generational Perception of Work and Productivity in the Remote-Work Era

In the year 2020, everything changed when the world stopped completely during COVID-19. The perception of safety, health, mental health, work, and private life completely turned around and led to a different conception of the world we knew. As the global pandemic thrived, we saw how many jobs could be done from home, because people had to reinvent themselves as we were not able to go to our workplaces. And it settled a statement, changing the perception of work dramatically. Before it, and for older generations, work was associated with physical presence, rigid schedules, and productivity measured by visible hours. But after it, younger generations saw the potential of working from home or being a so-called digital nomad, giving more priority to flexibility, emotional well-being, and measuring efficiency through results. This change reflects a social evolution guided by new technologies, new expectations, and a more connected workforce. Remote work has been key in this transformation. For thousands of professionals, the ability to work from home meant reclaiming personal time, reducing stress, and achieving a healthier work--life balance (for example, by reducing commuting time most people get almost 2 extra hours of personal time). Productivity did not decrease --- in many cases, it actually improved --- because the focus shifted from "time spent" to "goals achieved." This model has also shown that trust and autonomy can lead to more engaged teams. However, despite all of the perks, many companies are apparently eager to return to traditional workplaces. Maybe it is the fear of losing control or a lack of understanding of the new work dynamics, but this tendency threatens to undo meaningful progress for generations that have already experienced the freedom and effectiveness of remote work. Going back to the old-fashioned way of work feels like a step backward. So now, the challenge is to find a middle ground that acknowledges the cultural and technological changes of our time, passing the torch to a new generation of workers. Because productivity is no longer measured by how many people are sitting in a chair, but by the value of the final results. And if we want organizations truly prepared for the future, we must listen to younger generations and build work models that prioritize both results and workers' well-being. In CakeDC we do believe in remote work! Proving through the years that work can be done remotely no matter the timezone or language.

Written by Rcardoso on December 10, 2025

199 Views

Scaling Task Processing in CakePHP: Achieving Concurrency with Multiple...

This article is part of the CakeDC Advent Calendar 2025 (December 9th 2025)

Introduction: need of Concurrency

While offloading long-running tasks to an asynchronous queue solves the initial web request bottleneck, relying on a single queue worker introduces a new, serious point of failure and bottleneck. This single-threaded approach transfers the issue from the web server to the queue system itself.

Bottlenecks of Single-Worker Queue Processing

The fundamental limitation in the standard web request lifecycle is its synchronous, single-threaded architecture. This design mandates that a user's request must wait for all associated processing to fully complete before a response can be returned. The Problem: Single-Lane Processing Imagine a queue worker as a single cashier at a very busy bank . Each item in the queue (the "job") represents a customer.
  1. Job Blocking (The Long Transaction): If the single cashier encounters a customer with an extremely long or slow transaction (e.g., generating a massive report, bulk sending 100,000 emails, or waiting for a slow API), every other customer must wait for that transaction to complete.
  2. Queue Backlog Accumulation: New incoming jobs (customers) pile up rapidly in the queue. This is known as a queue backlog. The time between a job being put on the queue and it starting to execute (Job Latency) skyrockets.
  3. Real-Time Failure: If a job requires an action to happen now (like sending a password reset email), the backlog means that action is critically delayed, potentially breaking the user experience or application logic.
  4. Worker Vulnerability and Downtime: If this single worker crashes (due to a memory limit or unhandled error) or is temporarily taken offline for maintenance, queue processing stops entirely. The application suddenly loses its entire asynchronous capability until the worker is manually restarted, resulting in a complete system freeze of all background operations.
To eliminate this bottleneck, queue consumption must be handled by multiple concurrent workers, allowing the system to process many jobs simultaneously and ensuring no single slow job can paralyze the entire queue.

Improved System Throughput and Reliability with Multiple Workers

While introducing a queue solves the initial issue of synchronous blocking, scaling the queue consumption with multiple concurrent workers is what unlocks significant performance gains and reliability for the application's background processes.

Key Benefits of Multi-Worker Queue Consumption

  • Consistent, Low Latency: Multiple workers process jobs in parallel, preventing any single slow or heavy job (e.g., report generation) from causing a queue backlog. This ensures time-sensitive tasks, like password resets, are processed quickly, maintaining instant user feedback.
  • Enhanced Reliability and Resilience: If one worker crashes, the other workers instantly take over** the remaining jobs. This prevents a complete system freeze and ensures queue processing remains continuous.
  • Decoupling and Effortless Scaling: The queue facilitates decoupling. When background load increases, you simply deploy more CakePHP queue workers. This horizontal scaling is simple, cost-effective, and far more efficient than scaling the entire web server layer.

Workflows that Benefit from Multi-Worker Concurrency

These examples show why using multiple concurrent workers with the CakePHP Queue plugin (https://github.com/cakephp/queue) is essential for performance and reliability:
  • Mass Email Campaigns (Throughput): Workers process thousands of emails simultaneously, drastically cutting the time for large campaigns and ensuring the entire list is delivered fast.
  • Large Media Processing (Parallelism): Multiple workers handle concurrent user uploads or divide up thumbnail generation tasks. This speeds up content delivery by preventing one heavy image from blocking all others.
  • High-Volume API Synchronization (Consistency): Workers ensure that unpredictable external API latency from one service doesn't paralyze updates to another. This maintains a consistent, uninterrupted flow of data across all integrations.

The Job

Lets say that you have the queue job like this: <?php declare(strict_types=1); namespace App\Job; use Cake\Mailer\Mailer; use Cake\ORM\TableRegistry; use Cake\Queue\Job\JobInterface; use Cake\Queue\Job\Message; use Interop\Queue\Processor; /** * SendBatchNotification job */ class SendBatchNotificationJob implements JobInterface { /** * The maximum number of times the job may be attempted. * * @var int|null */ public static $maxAttempts = 10; /** * We need to set the shouldBeUnique to true to avoid race condition with multiple queue workers * * @var bool */ public static $shouldBeUnique = true; /** * Executes logic for SendBatchNotificationJob * * @param \Cake\Queue\Job\Message $message job message * @return string|null */ public function execute(Message $message): ?string { // 1. Retrieve job data from the message object $data = $message->getArgument('data'); $userId = $data['user_id'] ?? null; if (!$userId) { // Log error or skip, but return ACK to remove from queue return Processor::ACK; } try { // 2. Load user and prepare email $usersTable = TableRegistry::getTableLocator()->get('Users'); $user = $usersTable->get($userId); $mailer = new Mailer('default'); $mailer ->setTo($user->email) ->setSubject('Your batch update is complete!') ->setBodyString("Hello {$user->username}, \n\nThe recent batch process for your account has finished."); // 3. Send the email (I/O operation that can benefit from concurrency) $mailer->send(); } catch (\Exception $e) { // If the email server fails, we can tell the worker to try again later // The queue system will handle the delay and retry count. return Processor::REQUEUE; } // Success: Acknowledge the job to remove it from the queue return Processor::ACK; } } Setting $shouldBeUnique = true; in a CakePHP Queue Job class is crucial for preventing a race condition when multiple queue workers consume the same queue, as it ensures only one instance of the job is processed at any given time, thus avoiding duplicate execution or conflicting updates. In another part of the application you have code that enqueues the job like this: // In a Controller, Command, or Service Layer: use Cake\ORM\TableRegistry; use Cake\Queue\QueueManager; use App\Job\SendBatchNotificationJob; // Our new Job class // Find all users who need notification (e.g., 500 users) $usersToNotify = TableRegistry::getTableLocator()->get('Users')->find()->where(['is_notified' => false]); foreach ($usersToNotify as $user) { // Each loop iteration dispatches a distinct, lightweight job $data = [ 'user_id' => $user->id, ]; // Dispatch the job using the JobInterface class name QueueManager::push(SendBatchNotificationJob::class, $data); } // Result: 500 jobs are ready in the queue. By pushing 500 separate jobs, you allow 10, 20, or even 50 concurrent workers to pick up these small jobs and run the email sending logic in parallel, drastically reducing the total time it takes for all 500 users to receive their notification.

Implementing Concurrency with multiple queue workers

In modern Linux distributions, systemd is the preferred init and service manager. By leveraging User Sessions and the Lingering feature, we can run the CakePHP worker as a dedicated, managed service without needing root privileges for the process itself, offering excellent stability and integration.

SystemD User Sessions

Prerequisite: The Lingering User Session

For a service to run continuously in the background, even after the user logs out, we must enable the lingering feature for the user account that will run the workers (e.g., a service user named appuser). Enabling Lingering: Bash sudo loginctl enable-linger appuser This ensures the appuser's systemd user session remains active indefinitely, allowing the worker processes to survive server reboots and user logouts.

Creating the Systemd User Unit File

We define the worker service using a unit file, placed in the user's systemd configuration directory (~/.config/systemd/user/).
  • File Location: ~appuser/.config/systemd/user/[email protected]
  • Purpose of @: The @ symbol makes this a template unit. This allows us to use a single file to create multiple, distinct worker processes, which is key to achieving concurrency.
[email protected] Content: Ini, TOML [Unit] Description=CakePHP Queue Worker #%i After=network.target [Service] # We use the full path to the PHP executable ExecStart=/usr/bin/php /path/to/your/app/bin/cake queue worker # Set the current working directory to the application root WorkingDirectory=/path/to/your/app # Restart the worker if it fails (crashes, memory limit exceeded, etc.) Restart=always # Wait a few seconds before attempting a restart RestartSec=5 # Output logs to the systemd journal StandardOutput=journal StandardError=journal # Ensure permissions are correct and process runs as the user User=appuser [Install] WantedBy=default.target

Achieving Concurrency (Scaling the Workers)

Concurrency is achieved by enabling multiple instances of this service template, distinguished by the suffix provided in the instance name (e.g., -1, -2, -3). Reload and Start Instances: After creating the file, the user session must be reloaded, and the worker instances must be started and enabled: Reload Daemon (as appuser): Bash systemctl --user daemon-reload Start and Enable Concurrent Workers (as appuser): To run three workers concurrently: Bash # Start Worker Instance 1 systemctl --user enable --now [email protected] # Start Worker Instance 2 systemctl --user enable --now [email protected] # Start Worker Instance 3 systemctl --user enable --now [email protected] Result: The system now has three independent and managed processes running the bin/cake queue worker command, achieving a concurrent processing pool of three jobs.

Monitoring and Management

systemd provides powerful tools for managing and debugging the worker pool: Check Concurrency Status: Bash systemctl --user status 'cakephp-worker@*' This command displays the status of all concurrent worker instances, showing which are running or if any have failed and been automatically restarted. Viewing Worker Logs: All output is directed to the systemd journal: Bash journalctl --user -u 'cakephp-worker@*' -f This allows developers to inspect errors and task completion messages across all concurrent workers from a single, centralized log. Using systemd and lingering is highly advantageous as it eliminates the need for a third-party tool, integrates naturally with system logging, and provides reliable process management for a robust, concurrent task environment.

Summary

Shifting from a single worker to multiple concurrent workers is essential to prevent bottlenecks and system freezes caused by slow jobs, ensuring high reliability and low latency for asynchronous tasks. One robust way to achieve this concurrency in CakePHP applications is by using Systemd User Sessions and template unit files (e.g., [email protected]) to easily manage and horizontally scale the worker processes. This article is part of the CakeDC Advent Calendar 2025 (December 9th 2025)

Written by Arusinowski on December 08, 2025

84 Views

Notifications That Actually Work

This article is part of the CakeDC Advent Calendar 2025 (December 8th 2025) Building a modern application without notifications is like running a restaurant without telling customers their food is ready. Users need to know what's happening. An order shipped. A payment went through. Someone mentioned them in a comment. These moments matter, and how you communicate them matters even more. I've built notification systems before. They always started simple. Send an email when something happens. Easy enough. Then someone wants in-app notifications. Then someone needs Slack alerts. Then the mobile team wants push notifications. Before you know it, you're maintaining five different notification implementations, each with its own bugs and quirks. That's exactly why the CakePHP Notification plugin exists. It brings order to the chaos by giving you one consistent way to send notifications, regardless of where they're going or how they're being delivered. The core notification system (crustum/notification) provides the foundation with database and email support built in.

Two Worlds of Notifications

Notifications naturally fall into two categories, and understanding this split helps you architect your system correctly. The first category is what I call presence notifications. These are for users actively using your application. They're sitting there, browser open, working away. You want to tell them something right now. A new message arrived. Someone approved their request. The background job finished. These notifications need to appear instantly in the UI, update the notification bell, and maybe play a sound. They live in your database and get pushed to the browser through WebSockets. The second category is reach-out notifications. These go find users wherever they are. Email reaches them in their inbox. SMS hits their phone. Slack pings them in their workspace. Telegram messages appear on every device they own. These notifications cross boundaries, reaching into other platforms and services to deliver your message. Understanding this distinction is crucial because these two types of notifications serve different purposes and require different technical approaches. Presence notifications need a database to store history and WebSocket connections for real-time delivery. Reach-out notifications need API integrations and reliable delivery mechanisms.

The Beautiful Part: One Interface

Here's where it gets good. Despite these two worlds being completely different, you write the same code to send both types. Your application doesn't care whether a notification goes to the database, WebSocket, email, or Slack. You just say "notify this user" and the system handles the rest. $user = $this->Users->get($userId); $user->notify(new OrderShipped($order)); That's it. The OrderShipped notification might go to the database for the in-app notification bell, get broadcast via WebSocket for instant delivery, and send an email with tracking information. All from that one line of code.

Web interface for notifications

Let's talk about the in-app notification experience first. This is what most users interact with daily. That little bell icon in the corner of your application. Click it, see your notifications. It's so common now that users expect it. The NotificationUI plugin (crustum/notification-ui) provides a complete notification interface out of the box. There's a bell widget that you drop into your layout, and it just works. It shows the unread count, displays notifications in a clean interface, marks them as read when clicked, and supports actions like buttons in the notification. You have two display modes to choose from. Dropdown mode gives you the traditional experience where clicking the bell opens a menu below it. Panel mode creates a sticky side panel that slides in from the edge of your screen, similar to what you see in modern admin panels. Setting it up takes just a few lines in your layout template. <?= $this->element('Crustum/NotificationUI.notifications/bell_icon', [ 'mode' => 'panel', 'pollInterval' => 30000, ]) ?> The widget automatically polls the server for new notifications every 30 seconds by default. This works perfectly fine for most applications. Users see new notifications within a reasonable time, and your server isn't overwhelmed with requests. But sometimes 30 seconds feels like forever. When someone sends you a direct message, you want to see it immediately. That's where real-time broadcasting comes in.

Real-Time Broadcasting for Instant Delivery

Adding real-time broadcasting transforms the notification experience. Instead of polling every 30 seconds, new notifications appear instantly through WebSocket connections. The moment someone triggers a notification for you, it pops up in your interface. The beautiful thing is you can combine both approaches. Keep database polling as a fallback, add real-time broadcasting for instant delivery. If the WebSocket connection drops, polling keeps working. When the connection comes back, broadcasting takes over again. Users get reliability and instant feedback. <?php $authUser = $this->request->getAttribute('identity'); ?> <?= $this->element('Crustum/NotificationUI.notifications/bell_icon', [ 'mode' => 'panel', 'enablePolling' => true, 'broadcasting' => [ 'userId' => $authUser->getIdentifier(), 'userName' => $authUser->username, 'pusherKey' => 'app-key', 'pusherHost' => '127.0.0.1', 'pusherPort' => 8080, ], ]) ?> This hybrid approach gives you the best of both worlds. Real-time when possible, reliable fallback always available. Behind the scenes, this uses the Broadcasting (crustum/broadcasting) and BroadcastingNotification (crustum/notification-broadcasting) plugins working together. When you broadcast a notification, it goes through the same WebSocket infrastructure. The NotificationUI plugin handles subscribing to the right channels and updating the interface when broadcasts arrive.

Creating Your Notification Classes

Notifications in CakePHP are just classes. Each notification type gets its own class that defines where it goes and what it contains. This keeps everything organized and makes notifications easy to test. namespace App\Notification; use Crustum\Notification\Notification; use Crustum\Notification\Message\DatabaseMessage; use Crustum\Notification\Message\MailMessage; use Crustum\BroadcastingNotification\Message\BroadcastMessage; use Crustum\BroadcastingNotification\Trait\BroadcastableNotificationTrait; class OrderShipped extends Notification { use BroadcastableNotificationTrait; public function __construct( private $order ) {} public function via($notifiable): array { return ['database', 'broadcast', 'mail']; } public function toDatabase($notifiable): DatabaseMessage { return DatabaseMessage::new() ->title('Order Shipped') ->message("Your order #{$this->order->id} has shipped!") ->actionUrl(Router::url(['controller' => 'Orders', 'action' => 'view', $this->order->id], true)) ->icon('check'); } public function toMail($notifiable): MailMessage { return MailMessage::create() ->subject('Your Order Has Shipped') ->greeting("Hello {$notifiable->name}!") ->line("Great news! Your order #{$this->order->id} has shipped.") ->line("Tracking: {$this->order->tracking_number}") ->action('Track Your Order', ['controller' => 'Orders', 'action' => 'track', $this->order->id]); } public function toBroadcast(EntityInterface|AnonymousNotifiable $notifiable): BroadcastMessage|array { return new BroadcastMessage([ 'title' => 'Order Shipped', 'message' => "Your order #{$this->order->id} has shipped!", 'order_id' => $this->order->id, 'order_title' => $this->order->title, 'tracking_number' => $this->order->tracking_number, 'action_url' => Router::url(['controller' => 'Orders', 'action' => 'view', $this->order->id], true), ]); } public function broadcastOn(): array { return [new PrivateChannel('users.' . $notifiable->id)]; } } The via method tells the system which channels to use. The toDatabase method formats the notification for display in your app. The toMail method creates an email. The toBroadcast method formats the notification for broadcast. The broadcastOn method specifies which WebSocket channels to broadcast to. One notification class, three different formats, all sent automatically when you call notify. That's the power of this approach.

Reach-Out Notifications

Now let's talk about reaching users outside your application. This is where the plugin really shines because there are so many channels available. Email is the classic. Everyone has email. The base notification plugin gives you a fluent API for building beautiful transactional emails. You describe what you want to say using simple methods, and it generates a responsive HTML email with a plain text version automatically. Slack integration (crustum/notification-slack) lets you send notifications to team channels. Perfect for internal alerts, deployment notifications, or monitoring events. You get full support for Slack's Block Kit, so you can create rich, interactive messages with buttons, images, and formatted sections. Telegram (crustum/notification-telegram) reaches users on their phones. Since Telegram has a bot API, you can send notifications directly to users who've connected their Telegram account. The messages support formatting, buttons, and even images. SMS through Seven.io (crustum/notification-seven) gets messages to phones as text messages. This is great for critical alerts, verification codes, or appointment reminders. Things that need immediate attention and work even without internet access. RocketChat (crustum/notification-rocketchat) is perfect if you're using RocketChat for team communication. Send notifications to channels or direct messages, complete with attachments and formatting. The plugin system allows you to add new notification channels easily. You can create a new plugin for a new channel and install it like any other plugin. The brilliant part is that adding any of these channels to a notification is just adding a string to the via array and implementing one method. Want to add Slack to that OrderShipped notification? Add 'slack' to the array and implement toSlack. Done. public function via($notifiable): array { return ['database', 'broadcast', 'mail', 'slack']; } public function toSlack($notifiable): BlockKitMessage { return (new BlockKitMessage()) ->text('Order Shipped') ->headerBlock('Order Shipped') ->sectionBlock(function ($block) { $block->text("Order #{$this->order->id} has shipped!"); $block->field("*Customer:*\n{$notifiable->name}"); $block->field("*Tracking:*\n{$this->order->tracking_number}"); }); } Now when someone's order ships, they get an in-app notification with real-time delivery, an email with full details, and your team gets a Slack message in the orders channel. All automatic.

The Database as Your Notification Store

Every notification sent through the database channel gets stored in a notifications table. This gives you a complete history of what users were notified about and when. The NotifiableBehavior adds methods to your tables for working with notifications. $user = $usersTable->get($userId); $unreadNotifications = $usersTable->unreadNotifications($user)->all(); $readNotifications = $usersTable->readNotifications($user)->all(); $usersTable->markNotificationAsRead($user, $notificationId); $usersTable->markAllNotificationsAsRead($user); The UI widget uses these methods to display notifications and mark them as read. But you can use them anywhere in your application. Maybe you want to show recent notifications on a user's dashboard. Maybe you want to delete old notifications. The methods are there.

Queuing for Performance

Sending notifications, especially external ones, takes time. Making API calls to Slack, Seven.io, or Pusher adds latency to your request. If you're sending to multiple channels, that latency multiplies. The solution is queuing. Implement the ShouldQueueInterface on your notification class, and the system automatically queues notification sending as background jobs. use Crustum\Notification\ShouldQueueInterface; class OrderShipped extends Notification implements ShouldQueueInterface { protected ?string $queue = 'notifications'; } Now when you call notify, it returns immediately. The actual notification sending happens in a background worker. Your application stays fast, users don't wait, and notifications still get delivered reliably.

Testing Your Notifications

Testing notification systems used to be painful. You'd either send test notifications to real services (annoying) or mock everything (fragile). The NotificationTrait makes testing clean and simple. use Crustum\Notification\TestSuite\NotificationTrait; class OrderTest extends TestCase { use NotificationTrait; public function testOrderShippedNotification() { $user = $this->Users->get(1); $order = $this->Orders->get(1); $user->notify(new OrderShipped($order)); $this->assertNotificationSentTo($user, OrderShipped::class); $this->assertNotificationSentToChannel('mail', OrderShipped::class); $this->assertNotificationSentToChannel('database', OrderShipped::class); } } The trait captures all notifications instead of sending them. You can assert that the right notifications were sent to the right users through the right channels. You can even inspect the notification data to verify it contains the correct information. There are many diferent assertions you can use to test your notifications. You can assert that the right notifications were sent to the right users through the right channels. You can even inspect the notification data to verify it contains the correct information.

Localization

Applications serve users in different languages, and your notifications should respect that. The notification system integrates with CakePHP's localization system. $user->notify((new OrderShipped($order))->locale('es')); Even better, users can have a preferred locale stored on their entity. Implement a preferredLocale method or property, and notifications automatically use it. class User extends Entity { public function getPreferredLocale(): string { return $this->locale; } } Now you don't even need to specify the locale. The system figures it out automatically and sends notifications in each user's preferred language.

Bringing It Together

What I like about this notification system is how it scales with your needs. Start simple. Just database notifications. Add real-time broadcasting when you want instant delivery. Add email when you need to reach users outside your app. Add Slack when your team wants internal alerts. Add SMS for critical notifications. Each addition is incremental. You're not rewriting your notification system each time. You're adding channels to the via array and implementing format methods. The core logic stays the same. The separation between presence notifications and reach-out notifications makes architectural sense. They serve different purposes, use different infrastructure, but share the same interface. This makes your code clean, your system maintainable, and your notifications reliable. Whether you're building a small application with basic email notifications or a complex system with real-time updates, database history, email, SMS, and team chat integration, you're using the same patterns. The same notification classes. The same notify method. That consistency is what makes the system powerful. You're not context switching between different notification implementations. You're just describing what should be notified, who should receive it, and how it should be formatted. The system handles the rest. This article is part of the CakeDC Advent Calendar 2025 (December 8th 2025)

Written by Yevgeny on December 08, 2025

We Bake with CakePHP