CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

CakePHP Migrations plugin: easily version and deploy whole applications

This article is a quick introduction to the Migrations plugin, open sourced a few weeks ago by our company. You will see how simple it is to use the plugin and what you could do with it. I hope this article will show you the benefits of using migrations in your CakePHP applications and make you give it a try right after the reading!

Here is a one-sentence description of the plugin: the Migrations plugin allows developers to easily version and automate the creation / update process of any database schema and application data from the command line.

For information, CakeDC uses this plugin on its project since several years to make team collaboration and deployment easier. The plugin has been entirely rewritten a few months ago and fully tested (code coverage >95% as always at CakeDC) before being open sourced under the MIT license. It is now available to the community along with its documentation... and it is free!

Why is it useful?

It has been a while since companies integrated Source Code Management in their development process and CVS, SVN, Mercurial or Git are now common tools. Inspired from the open source movement it is also a good practice for single developers to version application source code.

As you might know, an application almost always depends of the database schema it is aimed to use... however it is not easy to version both the source code and database schema with a SCM. Let's take the example of a CakePHP application: until now the only way to do was to version a single file, either a sql dump or a CakePHP schema.php file generated with the cake schema shell. These two approaches are not very convenient to use on a daily basis, the first one forcing the developer to drop and recreate the whole database every time!

Moreover, a web application development is never really finished (there are always new features to add, software updates or bug fixing to do...) and deploying these change on a test or production server is always a delicate task.

Here comes the Migrations plugin! It provides a simple and easy way to version a database... and to perform many other different tasks thanks to its callback system. Here are some features:

  • keep a local database schema up-to-date: you just have to run all non applied migrations to update the local database schema to the latest version
  • make team work easier: when several developers work on the same application it is important that all of them work with the same database schema during all the development cycle. With migrations every commit is tied to the database schema at this precise instant, which makes easy switching branches and resetting a branch to a specific commit.
  • make installation and updates easier: ready to push the new version of your application live? You will only have to push the sources on the server and run all non applied migrations!
  • migrate more than database schema: the callback system allows you to do everything you want before (or after) applying (or reverting) each migration. Here are some examples: creating an initial admin account, add initial or test data to the application (lorem ipsums, categories, content...), update values from the database, send an email if debug > 0... The only limit may be your imagination ;)

Where can I find the code?

Announced a few weeks ago, a packaged version of the plugin can be downloaded from the "Plugins" section of CakeDC.com. This page contains a link to download the 1.0 version, the plugin documentation and the Github project for tickets and direct Git access to the repository.

To make people aware of the need to show their support to the Cake Software Fundation by donating a few bucks (this is unfortunately not done enough), the plugin was first available to donors only. The "Download without donation" button was added later, when the repository was made public! However, if you find this plugin useful please consider making a donation to the CSF... that is the best thing you could do for thanking us.

Click here to lend your support to: cakephp1x and make a donation at www.pledgie.com !

Even better! A sample application was also released for those who want to see how migrations could be used and integrated in an application. To play with it, Download the code or git clone the project using:

git clone git://codaset.com/cakedc/sample-migrations-application.git sample_migrations

You will only need to create a database.php configuration file and update CakePHP's core location to make the application work. Git users, run

git submodule init
git submodule update

to automatically add the migrations plugin as a submodule!

What do I need to use it in my application?

Note: the packaged plugin is for the CakePHP 1.3 version only. You can either download the 1.3-beta package of the framework, or use the 1.2 branch available in the Git repository.

Adding the plugin to an existing application is very simple. If you downloaded the archive containing the plugin code, unzip it in the "/plugins/migrations" folder of your application. Git users can add it as a submodule with the following command:

git submodule add git://codaset.com/cakedc/migrations.git plugins/migrations

To check that it is installed correctly, execute the following command from your application root (it will display the available command to use the plugin):

cake migration help

If you encounter any problem here, please read the official documentation about CakePHP's console usage.

How does it work?

This post is not aimed at providing a comprehensive tutorial on how to use the plugin, thus I will just introduce the most useful commands along with some use cases.

For a complete documentation, please read the official documentation provided on the plugin page. For a simple (but useful for understanding purpose) use case you can take a look at the sample application introduced above. Going through the commit history will allow you to understand how migrations could be used in a development process.

Create a migration

To generate a new migration, type the following command

cake migration generate

The tool will ask you to give a name to the migration and suggest to do a dump of the current database schema. If a "schema.php" file is found in the application, it will ask you if you want to generate a diff between this schema and your current database one.

Generated migration files will be added to the "/config/migrations" application directory.

Apply / Revert migrations

When you pull an application containing migrations, several commands are available to apply or revert migrations. The simplest one is:

cake migration

It will display all the found migrations along with their status (applied or not applied) and id number. Just enter a migration number to update your database to the correct version. Some convenience commands are also available. You can use:

cake migration up, down, all or reset

These commands will respectively:

  • apply the next migration
  • revert the latest applied migration
  • apply all non applied migrations (and thus update the schema to the most recent version)
  • revert all applied migrations (and empty the database)

Migrations for plugins

Adding plugins to an existing application often implies adding new tables to the database or altering existing ones. The Migrations plugin brings a quick and efficient way to automate this installation. On the one hand developers can easily add necessary migrations to their plugin (making upgrades easier), on the other hand users can apply them as easily.

The only difference compared with commands introduced above is the parameter "-plugin pluginname" that needs to be added. Here is how the user will install the database for the newly added / updated plugin "test":

cake migration run all -plugin test

I would like to highlight the fact that callbacks allow the developer to do everything they want before / after each migration. It is convenient for adding initial data, and one can even implement a callback method opening the bootstrap.php file to append plugin's configuration entries there (it is just an example ;)).

... going further

Of course, feel free to add any remark or example of migrations use in the comments.

As this post is not aimed at providing support for the plugin, I recommend you to use the official tools available:

  • If you found a bug or want to suggest enhancements: open a ticket!
  • An installation problem or a question about the plugin usage? Ask your question to the community!
  • You would like a custom version of this plugin, or professional related services... contact us, it is our job ;)

I hope you enjoyed this post, it is now time for you to start playing with the Migrations plugin...

Latest articles

Upgrading to CakePHP 4

As you know, CakePHP announced the version 4.x last December.I recommend that you consider upgrading your applications to the next version, to keep up to date and get all the benefits. Now, let's see how to bake!  

Step 1: Upgrade PHP

First things first, if you are not running on PHP 7.2 or higher, you will need to upgrade PHP before updating CakePHP. CakePHP 4.0 requires a minimum of PHP 7.2.  

Step 2: Upgrade Templates and Resources

There is an upgrade CLI tool for rename and moving the templates and resources:   Templates and Resources must have been moved and renamed, check the result below: * This project doesn't have Resources files   Now, let's create a new constant for Resources on /config/paths.php: Finally, update the paths on config/app.php:  

Step 3: Upgrade CakePHP

The next step is optional (and the Migration Guide included this) - run the rector command to automatically fix many deprecated method calls: The rector applied on codebase some return type declarations: https://github.com/rafaelqueiroz/cakephp-upgrade-sample/commit/d7e5c2ecc5dc28045700a270721f07098a8e189c?branch=d7e5c2ecc5dc28045700a270721f07098a8e189c&diff=split Pay attention: It is important to apply rector before you upgrade your dependencies.   Upgrade CakePHP and PHPUnit: PHPUnit can be upgraded easily. Most of the time, the --update-with-dependencies doesn’t work with me for CakePHP: The root of the issue is the packages using Caret Version Range, so let’s update debug_kit, migrations and bake using editor:   Here we go:   Now, let see how the project looks: Here, we have few deprecations and warnings. Do you remember I mentioned the rector is optional? So, the question is the rector and it's not always able to handle these issues.   I will use the PHPStan to fix this - we will install with composer: Now, we can run the phpstan analyse and fix the issues:   It's up to you how much effort you will put in with PHPStan issues. I recommend fixing everything. For this post, I did fix only what was needed to run the project after the update, you can check the fixes on this commit.   After the last fixes, the project is running well:  That’s all? No. But we upgraded CakePHP? Yes. Real applications probably use many plugins, and if these plugins don't have a version for CakePHP 4, you will need to update. Depending on the size and level of complexity of the project, the upgrade could be hard, but never impossible.    If you do not feel confident or your company would like to outsource support for this, don't hesitate to contact us at Cake Development Corporation. Our team is offering a full upgrade from CakePHP 2/3 to CakePHP 4. This will be a migration of your current application code to make it compatible with CakePHP 4 features, plugins, security settings, etc. We will be doing these migration services for a special rate - something we have never done before! Learn more about our Upgrade Services You can check the codebase of the examples on this repository. The branch upgrade has all steps by commit.  With every release CakePHP gets better, and version 4.x is no exception. There are many benefits that come with upgrading, and it makes baking a lot easier.

Using Postgres as default database- hints, tricks and tips

SQL language for different databases has some differences, which could cause problems after migrations between these databases. Here, we collected some hints, problems could appear during migration from MySQL to PostgreSQL. We focus on the CakePHP model layer and query generation layer.  

Tables join in where expression

Often we want to join two tables using a condition like $query->where([‘Author.id = Article.author_id’]) which works fine till we dont need field alias quotes. In the case of migration to postgres, we might want to enable autoQuotes. In this case, we can use $query->newExpr()->equalFields(‘Author.idArticle.author_id’).   

Case sensitivity in like expressions

By default mysql does case insensitive search. Switching to postgres, you can note that some functionality works differently. Hopefully you have tests, which covers your code, and this will be detected during migration. Postgres uses custom syntax for such queries named ILIKE.The case of old style conditions where method arrays straight forward,  you’d just go with ILIKE instead of LIKE. But what if we want to use ILIKE in builder methods... Here is that example:     return $query->where(         function (QueryExpression $exp) use ($field, $value): QueryExpression {             return $exp->add(new \Cake\Database\Expression\Comparison($field, $value, 'string', 'ILIKE'));         });   

Type casing

As postgres is much more strict with types, type casing is not a rare operation and may be needed. Here is an example of how to perform it using FunctionExpression$expr = (new FunctionExpression('CAST'))->setConjunction(' AS ')->add([$id, 'varchar' => 'literal']); which generates expression like :id AS varchar there :id is the placeholder for variable $id. This trick, used with literal, allows you to cast to any postgres type.  

 Quotes of tables and fields

Sometimes it is critical to inform CakePHP that the field should be quoted. One recommendation is to avoid using plain strings in case of table joins, or using IS NULL as string. So if array syntax is used, all CakePHP conventions must be followed. However,  sometimes we should help the ORM and obviously wrap a field name with IdentifiedExpression. Let's take a look back to the previous example, but now we want to type cast not value, but a field. The only solution to quote field name correctly is using this code:     $id = new IdentifierExpression($this->aliasField($field));     $expr = (new FunctionExpression('CAST'))->setConjunction(' AS ')->add([$id, 'varchar' => 'literal']);   

Building complex arithmetic expressions

In case we want to generate expressions in query fields, and we don’t want to overcomplicate logic, we could use these next tricks. Here, I have created ListExpression, which could be used as a collection of expressions. Each of these are corrected, and generates a query with correct handling of each element. See: https://gist.github.com/skie/f6e4f1a1b61e0f902a507f7907c3bbf2 So, say we want to generate expressions like this: “Events”.”time_to” - “Events”.”time_from”... With ListExpression, it can be done quite easy:  $diff = new ListExpression([new IdentifierExpression('Events.time_to'), '-', new IdentifierExpression('Events.time_from')]);   Hopefully these tricks will be as useful for your baking as they have been for mine! 

CakePHP Common Errors: Saving HasMany relations

The Cake Development Corporation team performs many code reviews. In fact, that is our starting point with every new client, as we offer free quick reviews. This is a good way to see where code stands, and how much work will need to be done to get it functioning properly.  One of the common errors we have found while doing Code Reviews of existing applications or just working with inherited code, it’s the way HasMany relations data is saved.  We have noticed that to save HasMany relations, some developers save the target relation, and then when the ID is retrieved after saving, they save one per one each item of the ‘many’ relation. There is no need to do this, as CakePHP can do all this in one single ‘save’! You won’t have any issue related to inconsistent data, because everything will be stored in one single transaction and your code will look much more clean. Let’s see a quick and easy to follow example - We will have the following relations: ‘Users’ and one User could have many ‘Addresses’.  We wish to save one user and this user will have two addresses.  First, you need to build the form in the proper way, the request data should follow the structure of your entities. The key in the form is the fieldName for the hasMany inputs. They must follow this format: {entityname}.{position}.{property}, for example: adddress.0.street_1, adddress.0.street_2, etc for the first item so store, for the second one: : adddress.1.street_1, adddress.1.street_2, and so on. More examples can be found here:  https://book.cakephp.org/4/en/views/helpers/form.html#creating-inputs-for-associated-data.
<?= $this->Form->create($user) ?> <fieldset>    <legend><?= __('Add User') ?></legend>    <?php        echo $this->Form->control('first_name');        echo $this->Form->control('last_name');        echo $this->Form->control('phone');    ?>    <legend><?= __('Address 1') ?></legend>    <?php    echo $this->Form->control('addresses.0.street_1');    echo $this->Form->control('addresses.0.street_2');    echo $this->Form->control('addresses.0.zip');    echo $this->Form->control('addresses.0.city');    echo $this->Form->control('addresses.0.state');    ?>    <legend><?= __('Address 2') ?></legend>    <?php    echo $this->Form->control('addresses.1.street_1');    echo $this->Form->control('addresses.1.street_2');    echo $this->Form->control('addresses.1.zip');    echo $this->Form->control('addresses.1.city');    echo $this->Form->control('addresses.1.state');    ?> </fieldset> <?= $this->Form->button(__('Submit')) ?> <?= $this->Form->end() ?>
Now that we have the form, we need to convert the request data. The Table class provides an easy and efficient way to convert one or many entities from request data. It’s needed to define which associations should be marshalled, using associated
public function add() {    $user = $this->Users->newEmptyEntity();    if ($this->request->is('post')) {        $user = $this->Users->patchEntity($user, $this->request->getData(), ['associated' => ['Addresses']]);        if ($this->Users->save($user)) {            $this->Flash->success(__('The user has been saved.'));              return $this->redirect(['action' => 'index']);        }        $this->Flash->error(__('The user could not be saved. Please, try again.'));    }    $this->set(compact('user')); } In this example we are saving one user and two addresses for the given user.  Associated data is validated by default, If you wish to bypass data validation, pass the validate => false option, for example: $this->Users->patchEntity($user, $this->request->getData(), ['associated' => ['Addresses' => [‘validate’ => false]]]).    We are all about working smarter and in less time, so I hope this information will be useful! Take a look here for more information: https://book.cakephp.org/4/en/orm/saving-data.html#converting-request-data-into-entities  We will be posting more Common Errors while using CakePHP.  Keep checking back!

We Bake with CakePHP