CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

Introduction to CakeDC Api plugin

 

The CakeDC API plugin was created with the goal to prepare abstract solutions that solve generic tasks -in case of creating a rest API. It provides such features like automatic rest API generation based on db schema, support nested routes. It also allows the use of different formats like json or xml, and easily adds their own custom format. It helps to solve generic tasks appearing in development of any API, like pagination, data validation, adding common data to response, or building metadata, about data objects.

Dependencies

The CakeDC API plugin hardly depends on the CakeDC Users Plugin. For authentication it is highly recommended to use CakePHP Authentication plugin configured as middleware.

Installation

You can install this plugin into your CakePHP application using composer.

The recommended way to install composer packages is:

composer require cakedc/cakephp-api

 

 Load the Plugin

Ensure  The CakeDC API Plugin is loaded in your src/Aplication.php in bootstrap method.

php

    $this->addPlugin(\CakeDC\Users\Plugin::class);

    $this->addPlugin(\CakeDC\Api\Plugin::class, ['bootstrap' => true, 'routes' => true]);

Configuration

Minimal configuration to allow non authorized requests require you to copy file: ./vendor/cakedc/cakephp-api/config/api_permissions.php.default to ./config/api_permissions.php

Examples

Lets bake table blogs with two fields id and name.

After that, the next requests would be possible to perform to api. Requests would be performed using curl.

Request:

curl http://localhost:8765/api/blogs

Response:

{

    "status": "success",

    "data": [

        {

            "id": 1,

            "name": "blog001"

        }

    ],

    "pagination": {

        "page": 1,

        "limit": 20,

        "pages": 1,

        "count": 1

    },

    "links": [

        {

            "name": "self",

            "href": "http:\/\/localhost:8765\/api\/blogs",

            "rel": "\/api\/blogs",

            "method": "GET"

        },

        {

            "name": "blogs:add",

            "href": "http:\/\/localhost:8765\/api\/blogs",

            "rel": "\/api\/blogs",

            "method": "POST"

        }

    ]

}

Request:

curl -d "name=blog001" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://localhost:8765/api/blogs

Response:

{

    "status": "success",

    "data": {

        "name": "blog001",

        "id": 1

    },

    "links": []

}

Request:

curl -d "name=blog002" -H "Content-Type: application/x-www-form-urlencoded" -X PUT http://localhost:8765/api/blogs/1

Response:

{

    "status": "success",

    "data": {

        "id": 1,

        "name": "blog002"

    },

    "links": []

}

Request:

curl -X DELETE http://localhost:8765/api/blogs/1

Response:

{

    "status": "success",

    "data": true,

    "links": []

}

For more complex features about plugin initialization and configuration based on routes middlewares, we plan to create an additional article.

Services and Actions

In the REST recommendations documents names defined as a noun. Here, services come into play.

It describes business entities. From other side actions define the verbs that describe the operations that should be performed on the actions.

Common and difference between controller classes and services.

The common part is the service is the managing the choosing action to execute.

The primary difference is that service could be nested, if this is defined by request url.

Common and difference between controller actions and service actions.

The common part is the action defined logic of the request.

The primary is that each service’s action is defined as a separate class.

This means that generic actions could be defined as common class and reused in many services.

From the other side, an action class is able to extend if the system has slightly different actions.

This way it is possible to build actions hierarchy.

Both service and actions define an event during this execution flow. 

Main service events:

* Service.beforeDispatch

* Service.beforeProcess

* Service.afterDispatch

Main action events:

* Action.beforeProcess

* Action.onAuth

* Action.beforeValidate

* Action.beforeValidateStopped

* Action.validationFailed

* Action.beforeExecute

* Action.beforeExecuteStopped

* Action.afterProcess

Crud actions define events that depend on the type of action, and more details could be checked in documentation.

* Action.Crud.onPatchEntity

* Action.Crud.onFindEntities

* Action.Crud.afterFindEntities 

* Action.Crud.onFindEntity

Nested services

Consider we have request with method POST /blogs/1/posts with data like {"title": "...", "body": "..."}

As it is possible to see there is nothing in the given data about the blog_id to which the newly created post should belong to.

In the case of controllers we should define custom logic to parse a route, and to consume the blog_id from url.

For nested service all checks and records updates are automatically executed. This will happen for any crud operations, when detected by the route parent service. So for example: GET /blogs/1/posts, will return only posts for the blog with id 1.

Logical checks are also performed, so for request: DELETE /blogs/1/posts/2, a user gets an error if the post with id 2 belongs to the blog with id 2.

Action inheritance

As each action can be defined as a separate class, it is possible to use class inheritance to define common logic. For example:  Add and Edit actions.

Extending services and actions with shared functionality

The alternative way for defining common logic actions is using action extensions. Action extension is a more powerful feature and could be used for global tasks like search or pagination.

It is also possible to create service level extensions. Those extensions work on the top level of the execution process, and could be used for things like adding cors feature, or to append some counter into response.

Add service actions from service::initialize

This is a recommended way to register non crud actions. The mapAction uses the Router class syntax for parsing routes. So on any special use cases well described in cakephp core.

    public function initialize()

    {

        parent::initialize();

        $this->mapAction('view_edit', ViewEditAction::class, [

            'method' => ['GET'],

            'path' => 'view_edit/:id'

        ]);

    }

Configure actions using action class map.

Each action class uses $_actionsClassMap for defining a map between crud (and non crud) actions on the name of the action class.

Non crud actions should be additionally mapped, which is described in the previous step.

use App\Service\Protocols\IndexAction;

class ProtocolsService extends AppFallbackService

{

    /**

     * Actions classes map.

     *

     * @var array

     */

    protected $_actionsClassMap = [

        'index' => IndexAction::class,

    ];

Configure service and action in config file

Service options are defined in the config/api.php in Api.Service section.

Let's consider configuration options for ArticlesService.

Configuration are hierarchical in the next sense: 

  • define default options for any service within the application in the Api.Service.default.options section.
  • define options for any service within the application in Api.Service.articles.options section.

All defined options are overridden from up to down in described order.

This allows common service settings, and the ability to overwrite them in bottom level.

  •  Api.Service.classMap - defines name map, that allows defining services action classes with custom location logic.
    Any action, that could be loaded as default action defined in fallback class, or specific action class could be configured using configuration file.
    Let's consider how one can configure options for IndexAction of ArticlesService.
    Configuration are hierarchical in the next sense: 
  • one can define default options for any action for all services in the application in the Api.Service.default.Action.default section.
  • one can define default options for index action for all services in the application in the Api.Service.default.Action.index section.
  • one can define options for any action in the specific (articles) service in the Api.Service.articles.Action.default section.
  • one can define options for index action in the specific (articles) service in the  Api.Service.articles.Action.index section.

Crud and non crud methods. Mapping non-crud actions.

Crud services mapped automatically in two levels routing by FallbackService.

Index and view. Formatting output

The CakeDC Api Plugin is flexible and provides multiple ways to prepare result data for the response objects.

There is a list of main options.

Use Entity serialization

The most trivial way to convert data is using entity serialization.

When converting an entity to a JSON, the virtual and hidden field lists are applied. 

Entities are recursively converted to JSON as well. 

This means that if you eager, and loading entities and their associations, CakePHP will correctly handle converting the associated data into the correct format.

Additional fields could be defined using Entity::$_virtual and hidden using Entity::$$_hidden.

Build object manually from Action::execute

In this case users manually perform mapping of requests received from model layer to output array.

public function process()

{

    $entity = $this->getTable()->get($this->getId());

    return [

        'id' => $entity->id,

        'name' => $entity->name,

    ];

}

Use Query::formatResults in model layer

The request could be formatted in model layer using: Query::formatResults.

So in this case, the process action just calls for a needed finder from the model layer and returns the result.

public function findApiFormat(Query $query, array $options)

{

    return $query

        ->select(['id', 'body', 'created', 'modified', 'author_id'])

        ->formatResults(function ($results) use ($options) {

            return $results->map(function ($row) use ($options) {

                $row['author'] = $this->Authors->getFormatted($row['author_id']);

                unset($row['author_id']);

 

                return $row;

            });

        });

Use Action extensions to format output

In index action defined callback Action.Crud.afterFindEntities, which called after data fetched,  could be used to extend or overload results coming from the database.

Callbacks are catch-in-action extensions and could be applied to multiple endpoints.

For view action defined Action.Crud.afterFindEntity, which called after single record fetched.

Use Action extensions to append additional data to output

Sometimes there is some additional information needed to be presented in some group of endpoints. In this case it is possible to implement an action extension to append additional data.

For example, pagination provides information about number of pages, records count, and current page number.

Another example for additional data is some execution statistics about the query.

Here you see main parts of appending such data from extension.

class PaginateExtension extends Extension implements EventListenerInterface

{

    public function implementedEvents(): array

    {

        return [

            'Action.Crud.afterFindEntities' => 'afterFind',

        ];

    }

...

    public function afterFind(EventInterface $event): void

    {

        ...

        $pagination = [

            'page' => $this->_page($action),

            'limit' => $limit,

            'pages' => ceil($count / $limit),

            'count' => $count,

        ];

        $result->appendPayload('pagination', $pagination);

    }  

 

The renderer class describes how to handle payload data.

For example in JSend renderer, all payload records appended to the root of the resulting json object.

Rendering output. Renderers.

Renderers perform final mapping of response records to output format. 

Such formats like xml, json, or file are provided by  The CakeDC API plugin.

JSend is the json extension with some additional agreements about returning results.

 

 

Latest articles

A CakePHP Docker Development Environment

We sponsor a monthly CakePHP training session (register here https://training.cakephp.org ) where we cover different topics about the framework. One of our sessions, the "Getting Started with CakePHP 4" is aimed to help developers starting a new project quickly and following the best practices.   Our previous "recommended" quick setting for a CakePHP development environment was using a vagrant box. See details here:  https://www.cakedc.com/jorge_gonzalez/2018/01/17/using-a-vagrant-box-as-quick-environment-for-the-getting-started-with-cakephp-training-session. However, we've switched internally to use docker as our primary development environment and also we started using docker in our training sessions.   Here's a quick overview of a simple docker based development environment for CakePHP.  

1. Create a new CakePHP project skeleton using 

composer create-project cakephp/app myproject   A new folder "myproject" will be created with a CakePHP project skeleton inside. Go to this new directory and proceed.  

2. Create a new "docker-compose.yaml" file with the following contents

version: '3' services:   mysql8:     image: mysql:8     restart: always     container_name: mysql     environment:         MYSQL_ROOT_PASSWORD: root         MYSQL_DATABASE: my_app         MYSQL_USER: my_app         MYSQL_PASSWORD: secret     volumes:       - ./:/application     ports:       - '9306:3306'     cakephp:     image: webdevops/php-apache:8.0     container_name: cakephp     working_dir: /application/webroot     volumes:       - ./:/application     environment:       - WEB_DOCUMENT_ROOT=/application/webroot       - DATABASE_URL=mysql://my_app:secret@mysql/my_app     ports:       - "8099:80"
 

3. Run "docker-compose up"

You'll create 2 containers named mysql and cakephp -  check the docker-compose configuration to see default database and users created in the mysql container, and the same environment params passed to the cakephp container via DATABASE_URL to allow the cakephp container to connect with the mysql database.   NOTE: the ports exposed are 9306 for mysql and 8099 for cakephp webserver. You can list them using docker-compose ps.  

4. Access your database and cakephp shell

  • To access the database you can use the command:
mysql --port 9306 -umy_app -psecret my_app   To restore a database dump for example, you can use the command: curl -L https://raw.githubusercontent.com/CakeDC/cakephp4-getting-started-session/master/my_app.sql |mysql --port 9306 -umy_app -psecret my_app   You can also configure any database tool to access the database in: localhost:9306  
  • To access the cakephp environment and shell you can use the command:
docker exec -it --user application cakephp bash   You'll go to the webroot folder, so in order to run the cake shell you'll need to: cd .. bin/cake 
  Now you have a working environment to play with the training session contents.   In this previous article, we covered another approach to setting up a local docker environment: https://www.cakedc.com/rochamarcelo/2020/07/20/a-quick-cakephp-local-environment-with-docker    We hope to see you in our next training session! https://training.cakephp.org   

Updating Model Layer

One reason to migrate from CakePHP 2.x to newer versions, is the very powerful ORM system that was introduced in CakePHP 3.x.  

Improved ORM Objects

The CakePHP model layer in CakePHP 3.x uses the Data Mapper pattern. Model classes in CakePHP 3.x ORM are split into two separate objects. Entity represents a single row in the database and it is responsible for keeping record state. Table class provides access to a collection of database records and describe associations, and provides api to work with a database. One notable change is afterFind callback removal. In CakePHP 3.x, it is possible to use Entity level getters to provide calculated fields on entity level.  

Association Upgrade 

In CakePHP 2.x associations are defined as arrays properties like this:     public $belongsTo = [         'Profile' => [             'className' => 'Profile',             'foreignKey' => 'profile_id',         ]     ];   In CakePHP 3.x and 4.x associations are declared in the initialize method. This gives much more flexibility in association configuration.     public function initialize(): void     {         $this->belongsTo('Profile', [                 'className' => 'Profile',                 'foreignKey' => 'profile_id',         ]);              }   Or using setters syntax, it could be done this way: public function initialize(): void     {         $this->belongsTo('Profile')         ->setForeignKey('profile_id')     }  

Behavior Upgrade

In CakePHP 2.x, behaviors are initialized as arrays properties:     public $actsAs = [         'Sluggable' => [             'label' => 'name',         ],     ];   In CakePHP 3.x and 4.x,  behaviors are configured in the initialize method. This gives much more flexibility in configuration, as in params it's possible to pass anonymous functions.     public function initialize(): void     {         $this->addBehavior('Sluggable', [             'label' => 'name',         ]);              }  

Validation Upgrade

In CakePHP 2.x, behaviors are  initialized as arrays properties:     public $validation = [         'title' => [             'notBlank' => [                 'rule' => ['notBlank'],                 'allowEmpty' => false,                 'required' => true,                 'message' => 'Title is required'             ],         ],     ];   In CakePHP 3.x and 4.x, validation is defined in validationDefault method which builds validation rules.     public function validationDefault(Validator $validator): Validator     {         $validator             ->scalar('title')             ->requirePresence('title', 'create')             ->notEmptyString('title');           return $validator;     }   Additionally, CakePHP introduced the buildRules method, which is where  described foreign keys constraints, uniqueness, or business level rules.     public function buildRules(RulesChecker $rules): RulesChecker     {         $rules->add($rules->existsIn(['user_id'], 'Users'));         $rules->add($rules->isUnique(['username'], __('username should be unique')));                            return $rules;     }  

Finder Methods

In CakePHP 2.x, the custom finder method is called twice - before and after fetching data from the database, which is defined by the $state parameter. Parameter $query contains current query state, and in $results passed data returned from database.     protected function _findIndex($state, $query, $results = array()) {         if ($state == 'before') {             $query['contain'] = ['User'];         } else {             // ...         }     }       In CakePHP 3.x, custom finder method accepts query object and some options passed from client code and returns an updated query. This allows for combining multiple finder methods in the same call, and has better grained finder logic.     public function findIndex(Query $query, array $options): Query     {         return $query->contain(['Users']);     }   The afterFind method could be implemented with the Query::formatResults method, which accepts an anonymous function to map each collection item.

Why Database Compression?

Nowadays people are not concerned about how large their database is in terms of MB. Storage is cheap. Even getting cheap SSD storage is not a big deal.    However, this is true if we are talking about hundreds of MB or even several GB, but sometimes we get into a situation where we have massive amounts of data (i.e Several tables with lots of longtext columns). At this point it becomes a concern because we need to increase the hard disk size, and find ourselves checking to see  if the hard disk is full several times per day or week, etc.   Now, if you have faced a situation like this before, it's time to talk about database compression. Compression is a technique, developed theoretically back in the 1940s but actually implemented in the 1970s. For this post we will focus on MySQL compression, which is performed using the open-source ZLib library. This library implements the LZ77 dictionary-based compression algorithm.   Before going into MySQL compression details, lets name some of the main DBMS and their compression techniques:

  • MySQL: ZLib (LZ77) [1]
  • Oracle: Oracle Advanced Compression (Proprietary)[2]
  • Postgres: PGLZ or LZ4 (if added this option at compilation level) [3]
  • DB2: Fixed-length compression or Huffman in some systems [4]
  So, now that we know this useless information, lets learn how to implement this in MySQL.   Firstly, you need to know that you CAN'T enable compression if:
  • Your table lives into `system` tablespace, or
  • Your tablespace was created with the option `innodb_file_per_table` disabled.
  It is important to test if the compression is the best solution for you.  If you have a table with a lot of small columns, you will probably end up with a larger-size table after "compressing" because of the headers and compression information. Compression is always great when you have longtext columns which can be heavily compressed.   Then, to enable compression for a table, you just need to include the following option when your table is created, or execute it as part of an alter statement: ROW_FORMAT=COMPRESSED These are the basics but you may find more useful information in MySQL manual.   You can also take a look at Percona which implements a Column level compression. This is interesting if you have a table with a lot of small fields and one large column, or if you have to optimize your database as much as possible. [6]   Finally, just say that even that storage is cheaper than ever, the amount of information has increased as well and we are now using and processing an incredible amount of data... so it looks like compression will always be a requirement.   I hope you find this information useful and please let me know if you have any questions or suggestions below in the comments section.

  [1]:https://dev.mysql.com/doc/internals/en/zlib-directory.html  [2]:https://www.oracle.com/technetwork/database/options/compression/advanced-compression-wp-12c-1896128.pdf  [3]:https://www.postgresql.org/docs/devel/runtime-config-client.html  [4]:https://www.ibm.com/docs/en/db2-for-zos/12?topic=performance-compressing-your-data  [6]:https://www.percona.com/doc/percona-server/8.0/flexibility/compressed_columns.html

We Bake with CakePHP