CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

Playing with the new CakePHP Queue

One of the topics discussed in the community is the benefit of a unified, officially supported, CakePHP Queue plugin. Queues are used in most of the projects nowadays and are a central utility plugin.

During the CakeFest 2020 event, there were also a couple direct references from speakers: (https://speakerdeck.com/josegonzalez/building-and-releasing-a-cakephp-plugin/?slide=15c) and this comment from Mark Story: https://cakesf.slack.com/archives/C172CS4TE/p1602257791377500.

 

This motivated me to take a deeper look at the cakephp/queue plugin and write this blog post.

 

Here at CakeDC we've been using queues for a looooong time. Initially in CakePHP 2, we've used plugins like CakeResque with redis or custom workers tied to Amazon SQS queues. Then in CakePHP 3 & 4 we've been using mostly https://github.com/josegonzalez/cakephp-queuesadilla with redis or mongodb backends.

https://github.com/cakephp/queue

 

First thing would be setting up the plugin in your project, we are going to use the example project we used in CakeFest 2020: https://github.com/cakephp/cakefest2020/#using-docker

 

So after setting up the project and running it via docker compose, we can proceed to setup the plugin via composer. We will need to add it as a repository and set

 

Install via composer

 

After the first release it'll be much easier, but for now you'll need to add the package to your composer.json

 

"repositories": [

        {

            "type": "vcs",

            "url": "https://github.com/cakephp/queue.git"

        }

    ]

 

Then do

composer require cakephp/queue -W

 

And install some transport as stated in the official documentation https://book.cakephp.org/queue/1/en/index.html#installation

composer require enqueue/redis:^0.9

composer require predis/predis:^1

 

Ensure your redis server is up and running, you can check the commands sent to your local redis server using redis-cli monitor

 

Now we are ready to configure the queue, we'll create 1 default queue adding this to the config/app.php file

 

    'Queue' => [

        'default' => [

            'url' => 'redis:',

        ],

    ],

 

Add a Job using `bin/cake bake job Example` and add some code to the execute method

 

    public function execute(Message $message): string

    {

        $data = $message->getArgument('data');

 

        // do some long operation with data

        Debugger::log($data);

        sleep(2);

 

        return Processor::ACK;

    }

 

I've added a command utility to enqueue a test message bin/cake bake command addJob

 

    public function execute(Arguments $args, ConsoleIo $io)

    {

        $callable = [ExampleJob::class, 'execute'];

        $arguments = ['id' => 1, 'data' => ['some' => 'data']];

 

        QueueManager::push($callable, $arguments);

    }

 

And finally we can start our worker using bin/cake worker to pull jobs from Redis and process them using the ExampleJob::execute method

 

Here's all the example code created: https://github.com/cakephp/cakefest2020/tree/feature/cakephp-queue - for your reference.

 

Please note the plugin is still a work in progress and there is no stable release as of now. 

It's looking great so far and we plan to include it in our next projects!

 

Latest articles

A CakePHP Docker Development Environment

We sponsor a monthly CakePHP training session (register here https://training.cakephp.org ) where we cover different topics about the framework. One of our sessions, the "Getting Started with CakePHP 4" is aimed to help developers starting a new project quickly and following the best practices.   Our previous "recommended" quick setting for a CakePHP development environment was using a vagrant box. See details here:  https://www.cakedc.com/jorge_gonzalez/2018/01/17/using-a-vagrant-box-as-quick-environment-for-the-getting-started-with-cakephp-training-session. However, we've switched internally to use docker as our primary development environment and also we started using docker in our training sessions.   Here's a quick overview of a simple docker based development environment for CakePHP.  

1. Create a new CakePHP project skeleton using 

composer create-project cakephp/app myproject   A new folder "myproject" will be created with a CakePHP project skeleton inside. Go to this new directory and proceed.  

2. Create a new "docker-compose.yaml" file with the following contents

version: '3' services:   mysql8:     image: mysql:8     restart: always     container_name: mysql     environment:         MYSQL_ROOT_PASSWORD: root         MYSQL_DATABASE: my_app         MYSQL_USER: my_app         MYSQL_PASSWORD: secret     volumes:       - ./:/application     ports:       - '9306:3306'     cakephp:     image: webdevops/php-apache:8.0     container_name: cakephp     working_dir: /application/webroot     volumes:       - ./:/application     environment:       - WEB_DOCUMENT_ROOT=/application/webroot       - DATABASE_URL=mysql://my_app:secret@mysql/my_app     ports:       - "8099:80"
 

3. Run "docker-compose up"

You'll create 2 containers named mysql and cakephp -  check the docker-compose configuration to see default database and users created in the mysql container, and the same environment params passed to the cakephp container via DATABASE_URL to allow the cakephp container to connect with the mysql database.   NOTE: the ports exposed are 9306 for mysql and 8099 for cakephp webserver. You can list them using docker-compose ps.  

4. Access your database and cakephp shell

  • To access the database you can use the command:
mysql --port 9306 -umy_app -psecret my_app   To restore a database dump for example, you can use the command: curl -L https://raw.githubusercontent.com/CakeDC/cakephp4-getting-started-session/master/my_app.sql |mysql --port 9306 -umy_app -psecret my_app   You can also configure any database tool to access the database in: localhost:9306  
  • To access the cakephp environment and shell you can use the command:
docker exec -it --user application cakephp bash   You'll go to the webroot folder, so in order to run the cake shell you'll need to: cd .. bin/cake 
  Now you have a working environment to play with the training session contents.   In this previous article, we covered another approach to setting up a local docker environment: https://www.cakedc.com/rochamarcelo/2020/07/20/a-quick-cakephp-local-environment-with-docker    We hope to see you in our next training session! https://training.cakephp.org   

Updating Model Layer

One reason to migrate from CakePHP 2.x to newer versions, is the very powerful ORM system that was introduced in CakePHP 3.x.  

Improved ORM Objects

The CakePHP model layer in CakePHP 3.x uses the Data Mapper pattern. Model classes in CakePHP 3.x ORM are split into two separate objects. Entity represents a single row in the database and it is responsible for keeping record state. Table class provides access to a collection of database records and describe associations, and provides api to work with a database. One notable change is afterFind callback removal. In CakePHP 3.x, it is possible to use Entity level getters to provide calculated fields on entity level.  

Association Upgrade 

In CakePHP 2.x associations are defined as arrays properties like this:     public $belongsTo = [         'Profile' => [             'className' => 'Profile',             'foreignKey' => 'profile_id',         ]     ];   In CakePHP 3.x and 4.x associations are declared in the initialize method. This gives much more flexibility in association configuration.     public function initialize(): void     {         $this->belongsTo('Profile', [                 'className' => 'Profile',                 'foreignKey' => 'profile_id',         ]);              }   Or using setters syntax, it could be done this way: public function initialize(): void     {         $this->belongsTo('Profile')         ->setForeignKey('profile_id')     }  

Behavior Upgrade

In CakePHP 2.x, behaviors are initialized as arrays properties:     public $actsAs = [         'Sluggable' => [             'label' => 'name',         ],     ];   In CakePHP 3.x and 4.x,  behaviors are configured in the initialize method. This gives much more flexibility in configuration, as in params it's possible to pass anonymous functions.     public function initialize(): void     {         $this->addBehavior('Sluggable', [             'label' => 'name',         ]);              }  

Validation Upgrade

In CakePHP 2.x, behaviors are  initialized as arrays properties:     public $validation = [         'title' => [             'notBlank' => [                 'rule' => ['notBlank'],                 'allowEmpty' => false,                 'required' => true,                 'message' => 'Title is required'             ],         ],     ];   In CakePHP 3.x and 4.x, validation is defined in validationDefault method which builds validation rules.     public function validationDefault(Validator $validator): Validator     {         $validator             ->scalar('title')             ->requirePresence('title', 'create')             ->notEmptyString('title');           return $validator;     }   Additionally, CakePHP introduced the buildRules method, which is where  described foreign keys constraints, uniqueness, or business level rules.     public function buildRules(RulesChecker $rules): RulesChecker     {         $rules->add($rules->existsIn(['user_id'], 'Users'));         $rules->add($rules->isUnique(['username'], __('username should be unique')));                            return $rules;     }  

Finder Methods

In CakePHP 2.x, the custom finder method is called twice - before and after fetching data from the database, which is defined by the $state parameter. Parameter $query contains current query state, and in $results passed data returned from database.     protected function _findIndex($state, $query, $results = array()) {         if ($state == 'before') {             $query['contain'] = ['User'];         } else {             // ...         }     }       In CakePHP 3.x, custom finder method accepts query object and some options passed from client code and returns an updated query. This allows for combining multiple finder methods in the same call, and has better grained finder logic.     public function findIndex(Query $query, array $options): Query     {         return $query->contain(['Users']);     }   The afterFind method could be implemented with the Query::formatResults method, which accepts an anonymous function to map each collection item.

Why Database Compression?

Nowadays people are not concerned about how large their database is in terms of MB. Storage is cheap. Even getting cheap SSD storage is not a big deal.    However, this is true if we are talking about hundreds of MB or even several GB, but sometimes we get into a situation where we have massive amounts of data (i.e Several tables with lots of longtext columns). At this point it becomes a concern because we need to increase the hard disk size, and find ourselves checking to see  if the hard disk is full several times per day or week, etc.   Now, if you have faced a situation like this before, it's time to talk about database compression. Compression is a technique, developed theoretically back in the 1940s but actually implemented in the 1970s. For this post we will focus on MySQL compression, which is performed using the open-source ZLib library. This library implements the LZ77 dictionary-based compression algorithm.   Before going into MySQL compression details, lets name some of the main DBMS and their compression techniques:

  • MySQL: ZLib (LZ77) [1]
  • Oracle: Oracle Advanced Compression (Proprietary)[2]
  • Postgres: PGLZ or LZ4 (if added this option at compilation level) [3]
  • DB2: Fixed-length compression or Huffman in some systems [4]
  So, now that we know this useless information, lets learn how to implement this in MySQL.   Firstly, you need to know that you CAN'T enable compression if:
  • Your table lives into `system` tablespace, or
  • Your tablespace was created with the option `innodb_file_per_table` disabled.
  It is important to test if the compression is the best solution for you.  If you have a table with a lot of small columns, you will probably end up with a larger-size table after "compressing" because of the headers and compression information. Compression is always great when you have longtext columns which can be heavily compressed.   Then, to enable compression for a table, you just need to include the following option when your table is created, or execute it as part of an alter statement: ROW_FORMAT=COMPRESSED These are the basics but you may find more useful information in MySQL manual.   You can also take a look at Percona which implements a Column level compression. This is interesting if you have a table with a lot of small fields and one large column, or if you have to optimize your database as much as possible. [6]   Finally, just say that even that storage is cheaper than ever, the amount of information has increased as well and we are now using and processing an incredible amount of data... so it looks like compression will always be a requirement.   I hope you find this information useful and please let me know if you have any questions or suggestions below in the comments section.

  [1]:https://dev.mysql.com/doc/internals/en/zlib-directory.html  [2]:https://www.oracle.com/technetwork/database/options/compression/advanced-compression-wp-12c-1896128.pdf  [3]:https://www.postgresql.org/docs/devel/runtime-config-client.html  [4]:https://www.ibm.com/docs/en/db2-for-zos/12?topic=performance-compressing-your-data  [6]:https://www.percona.com/doc/percona-server/8.0/flexibility/compressed_columns.html

We Bake with CakePHP