CakeDC Blog

TIPS, INSIGHTS AND THE LATEST FROM THE EXPERTS BEHIND CAKEPHP

File uploading, file storage and CakePHPs MediaView class

This article includes how to upload and store files, because I've seen a lot of discussion about that too, but if you're just interested in how to use the MediaView class scroll down.

Handling file uploads in CakePHP

First let's start with the required form, to create a file upload form all you have to do is this:

echo $form->create('Media', array('action' => 'upload', 'type' => 'file'));
echo $form->file('file');
echo $form->submit(__('Upload', true));

 

The "type" in the options of Form::create() takes post, get or file. To configure the form for file uploading it has to be set to file which will render the form as a multipart/form-data form.

When you submit the form now, you'll get data like this in $this->data of your controller:

Array
(
	[Media] => Array
	(
		[file] => Array
		(
			[name] => cake.jpg
			[type] => image/jpeg
			[tmp_name] => /tmp/hp1083.tmp
			[error] => 0
			[size] => 24530
		)
	)
)

Ok, now the big question with a simple answer is where the file data should be processed, guess where. Right – in the model because it's data to deal with and validation to do against it. Because it's a recurring task to upload files I suggest you to write a behaviour for it or convert your existing component to a behaviour.

If you keep it generic you can extend it with a CsvUpload, VideoUpload or ImageUpload behaviour to process the file directly after its upload or do special stuff with it, like resizing the image or parsing the csv file and store its data in a (associated) model.

We're not going to show you our own code here for obvious reasons, but I'll give you a few hints what you can or should do inside of the behavior:

  1. Validate the uploaded field, the field itself contains already an error code if something was wrong with the upload. Here is a link to the php manual page that shows you the list of the errors that you can get from the form data. http://www.php.net/manual/en/features.file-upload.errors.php
  2. Validate the uploaded file, is it really the kind of file you want and does it really contain the data structure you want?
  3. Check if the target destination of the file is writeable, create directories, whatever is needed and error handling for it, I suggest you to use CakePHP's File and Folder classes for that.
  4. Add a callback like beforeFileSave() and afterFileSave() to allow possible extending behaviors to use them.

Database vs file system storage

Feel free to skip that part if you already store the files in the file system.

Storing files in the database is in nearly all cases a bad solution because when you get the file it has to go its way through the database connection, which can, specially on servers that are not in the same network, cause performance problems.

Advantages of storage in the file system:

  1. Easy and direct file access, to parse them (csv, xml...) or manipulate them (images)
  2. You don't need to install any additional software to manage them
  3. Easy to move and mount on other machines
  4. Smaller then stored in a DB

The suggested solution is to store meta data of the file like size, hash, maybe path and other related info in a DB table and save the file in the file system.

Some people come up with the security and want to store a file because of that in the database which is wrong. You should not store the file in a public accessible directory like the webroot of the application. Store it in another location like APP/media. You control the access to the file by checking the permissions against the DB records of your meta data and sending it by using the CakePHP MediaView class, I'll explain later how to use it.

I don't say that storage of files inside the DB is in general a bad idea but for web based applications it is in nearly every case a bad idea.

File system Performance

A bottleneck in the long run on every file system is a large amount of files in a single directory. Imagine just 10.000 users and each has an individual avatar image. Further ext3 for example is limited to 32000 sub folders, other file systems have maybe similar restrictions. You can find a list of file system limitations here: http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

To avoid performance problems caused by that you should store your files in a pseudo-random directory structure like APP/media/32/a5/3n/. This will also allow you to easily mount some of the semi-random created directories on another machine in the case you run out of disk space.

/**
 * Builds a semi random path based on the id to avoid having thousands of files
 * or directories in one directory. This would result in a slowdown on most file systems.
 *
 * Works up to 5 level deep
 *
 * @see http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits
 * @param mixed $string
 * @param integer $level
 * @return mixed
 * @access protected
 */
	protected function _randomPath($string, $level = 3) {
		if (!$string) {
			throw new Exception(__('First argument is not a string!', true));
		}

		$string = crc32($string);
		$decrement = 0;
		$path = null;
		
		for ($i = 0; $i < $level; $i++) {
			$decrement = $decrement -2;
			$path .= sprintf("%02d" . DS, substr('000000' . $string, $decrement, 2));
		}

		return $path;
	}

You should also know that php running in safe mode does not allow you to create more then one directory deep in one call. You have to take this in consideration, the above function does not cover that because safe mode is basically deprecated and will be also removed in php6

Sending a file to the client – or the unknown MediaView class

From what I've seen in the ruins of outsourced projects that asked us for rescue and also in the CakePHP googlegroup I think not many people are aware that CakePHP has a view that is thought to be used for downloads and display (images, text...) of files. It's called the MediaView class.

I'll now explain you how to use this class to send files to the client.

/**
 * Sends a file to the client
 *
 * @param string $id UUID
 * @access public
 */
	public function download($id = null) {
		$this->Media->recursive = -1;
		$media = $this->Media->read(null, $id);

		if (empty($media)) {
		$this->redirect('/', 404, true);
		}
		
		$this->set('cache', '3 days');
		$this->set('download', true);
		$this->set('name', $media['Media']['slug']);
		$this->set('id', $media['Media']['filename']);
		$this->set('path', APP . 'media' . DS . $media['Media']['path']);
		$this->set('modified', $media['Media']['modified']);
		$this->set('mimeType', $media['Media']['mime_type']);
		$this->set('extension', $media['Media']['extension']);

		$this->view = 'Media';
		$this->autoLayout = false;
		if ($this->render() !== false) {
			$this->Media->updateAll(
				array('Media.downloads' => 'Media.downloads + 1'),
				array('Media.id' => $id));
		}
	}

You simply have to set autoLayout to false and the view class to media.

$this->view = 'Media';
$this->autoLayout = false;

There are a few view variables to set to “configure” the file download or display. To control if you want to make the client downloading the file or to display it, in the case of images for example, you simply set 'download' to true or false;

	$this->set('download', true);

You can control the browser caching of the file by setting cache. Please not that you do not have to use caching if download is set to true! Downloads do not need caching.

	$this->set('cache', '3 days');

The next part might be a little confusing, you have “id” and “name”. Id is the actual file on your server you want to send while name is the filename under which you want to send the file to the client. “path” is the path to the file on the server.

	$this->set('name', $media['Media']['slug']);
$this->set('id', $media['Media']['filename']);
$this->set('path', APP . 'media' . DS . $media['Media']['path']);

If you want to send a mime type that does not already in the MediaView class you can set it.

	$this->set('mimeType', $media['Media']['mime_type']);

If you don't set it, the class will try to determine the mime type by the extension.

	$this->set('extension', $media['Media']['extension']);

Note that you have to set the extension to make it work and that the extension is attached to the filename! If you store the filename with an extension you have to break it up.

When everything is set you can check if render() was successfully and do whatever you want after that, for example count the download.

	if ($this->render() !== false) {
	$this->Media->updateAll(
	array('Media.downloads' => 'Media.downloads + 1'),
	array('Media.id' => $id));
}

 

Closing words

I hope you enjoyed reading the article and it helped you improving your knowledge about CakePHP. Feel free to ask further questions by using the comment functionality. Have fun coding!

Latest articles

How to push Docker image to Container Registry and create App on...

The title speaks for itself, let’s jump right in! As a preliminary step, we start from a user registered in DigitalOcean with a validated account. Use the doctl tool for the entire communication process with DigitalOcean.

Step 1: Install doctl

$ cd ~
$ wget https://github.com/digitalocean/doctl/releases/download/v1.71.0/doctl-1.71.0-linux-amd64.tar.gz
$ tar xvf doctl-1.71.0-linux-amd64.tar.gz
$ sudo mv doctl /usr/local/bin
 

Step 2: Create an API token

  Go to https://cloud.digitalocean.com/account/api/tokens Generate new token for read and write and save apart the value of token generated <TOKEN NAME>: personaltoken 
<TOKEN VALUE>: 6e981fc2a674dbb7a610b9b85d0c8b00

Step 3: Use the API token to grant account access to doctl

$ doctl auth init --context personaltoken
Validating token... OK
Prompt for <TOKEN VALUE>, then enter it and press return 
 

Step 4: Validate that doctl is working

$ doctl auth init  Validating token... OK
  Prompt for <TOKEN VALUE>, then enter it and press return  Validate by obtaining the account information
$ doctl account get Email      Droplet Limit    Email Verified    UUID         Status email@cakedc.com    10     true     5415bbf8-d501-4096-9b75-ab781c017948    active
 

Step 5: Create a Container Registry with doctl

<MY-REGISTRY-NAME> : container-nyc-795 <REGION> : nyc3
$ doctl registry create container-nyc-795 --region nyc3 Name       Endpoint        Region slug container-nyc-795    registry.digitalocean.com/container-nyc-795    nyc3

Important: the region of the Container registry and Kubernetes cluster MUST be the same Keep in mind that container names must be unique, must be lowercase, and only accepts alphanumeric characters and hyphens.  

Step 6: Login to authenticate docker with your registry

$ doctl registry login
Logging Docker in to registry.digitalocean.com
 

Step 7: Create kubernetes cluster

$ doctl kubernetes cluster create cluster-static-example --region nyc3 Notice: Cluster is provisioning, waiting for cluster to be running ................................................................... Notice: Cluster created, fetching credentials Notice: Adding cluster credentials to kubeconfig file found in "/home/andres/.kube/config" Notice: Setting current-context to do-nyc3-cluster-static-example ID                                      Name                      Region    Version        Auto Upgrade    Status     Node Pools d24f180b-6007-4dbc-a2fe-3952801570aa    cluster-static-example    nyc3      1.22.7-do.0    false           running    cluster-static-example-default-pool
Important: the region of the Container registry and Kubernetes cluster MUST be the same here as well.  This operation isn’t a fast process.  

Step 8: Integrate kubernetes cluster in Container register

$ doctl kubernetes cluster registry add cluster-static-example
 

Step 9: Get token certificate and connect to cluster

$ doctl kubernetes cluster kubeconfig save cluster-static-example Notice: Adding cluster credentials to kubeconfig file found in "/home/andres/.kube/config" Notice: Setting current-context to do-nyc3-cluster-static-example
To validate this, use the kubectl tool to get context. If is not installed, get the last version you find - for example in googleapis
$ wget https://storage.googleapis.com/kubernetes-release/release/v1.23.5/bin/linux/amd64/kubectl
$ chmod +x kubectl
$ sudo mv kubectl /usr/local/bin/
Check with
$ kubectl config current-context
do-nyc3-cluster-static-example
Here, you see: bash prompt do-ny3-cluster-sttic-example. That is the context you created in Step 7
 

Step 10: Generate docker image, tag and push to DigitalOcean

We assume that the user already has docker installed
$ mkdir myapp
$ mkdir myapp/html
$ nano myapp/Dockerfile
  Inside Dockerfile put   
FROM nginx:latest
COPY ./html/hello.html /usr/share/nginx/html/hello.htm
l
  We create simple Dockerfile with NGINX Server and copy the file hello.html in the default html of nginx  
$ nano myapp/html/hello.html
  Inside hello.html put  
<!DOCTYPE html> <html>  <head>    <title>Hello World!</title>  </head>  <body>    <p>This is an example of a simple HTML page served from the Nginx container.</p>  </body> </html>
Then build a docker image file tag with repository and push with docker
 
$ cd myapp
$ docker build -t registry.digitalocean.com/container-nyc-795/static-app .
Sending build context to Docker daemon  3.584kB Step 1/2 : FROM nginx:latest latest: Pulling from library/nginx c229119241af: Pull complete 2215908dc0a2: Pull complete 08c3cb2073f1: Pull complete 18f38162c0ce: Pull complete 10e2168f148a: Pull complete c4ffe9532b5f: Pull complete Digest: sha256:2275af0f20d71b293916f1958f8497f987b8d8fd8113df54635f2a5915002bf1 Status: Downloaded newer image for nginx:latest  ---> 12766a6745ee Step 2/2 : COPY ./html/hello.html /usr/share/nginx/html/hello.html  ---> 2b9be913c377 Successfully built 2b9be913c377 Successfully tagged registry.digitalocean.com/container-nyc-795/static-app:latest   $ docker images REPOSITORY                                               TAG       IMAGE ID       CREATED         SIZE registry.digitalocean.com/container-nyc-795/static-app   latest    2b9be913c377   2 minutes ago   142MB   $ docker push registry.digitalocean.com/container-nyc-795/static-app Using default tag: latest The push refers to repository [registry.digitalocean.com/container-nyc-795/static-app] ac03ae036a53: Pushed ea4bc0cd4a93: Pushed fac199a5a1a5: Pushed 5c77d760e1f4: Pushed 33cf1b723f65: Pushed ea207a4854e7: Pushed 608f3a074261: Pushed latest: digest: sha256:22615ad4c324ca5dc13fe2c3e1d2d801bd166165e3809f96ed6a96a2b2ca2748 size: 1777


Step 11: Create app

Create file example-static-app.yaml and insert: alerts:                                 
- rule: DEPLOYMENT_FAILED               
- rule: DOMAIN_FAILED                   
name: example-static-app                
region: nyc                             
services:                               
- http_port: 80                         
  image:                                
    registry_type: DOCR                 
    repository: static-app              
    tag: latest                         
  instance_count: 2                     
  instance_size_slug: professional-xs   
  name: static-service                  
  routes:                               
  - path: /                             
  source_dir: /             
              Validate file with
$  doctl apps spec validate example-static-app.yaml
Create app
$ doctl apps create --spec example-static-app.yaml
Notice: App created
ID        Spec Name             Default Ingress    Active Deployment ID    In Progress Deployment ID    Created At     Updated At
55a7cb68-65b7-4ff1-b6af-388cdb1df507    example-static-app   2022-03-30 09:34:01.288257225 +0000 UTC    2022-03-30 09:34:01.288257225 +0000 UTC
If you access https://cloud.digitalocean.com/apps And in the live url + /hello.html you can see:

Step 12: Deploy app

If you modify the code of your app, you need to generate a new image with docker and push  (see step 10). Then you don’t need to create a new app, you need to deploy the image in the already created app, with the id executing the command to deploy in bash.
$  doctl apps create-deployment 55a7cb68-65b7-4ff1-b6af-388cdb1df507

 

How to generate deploy of Docker image to Container Registry on DigitalOcean Platform Apps

Once you have created the application (and if you have the code in gitlab), you can create a direct deployment of your code in the DigitalOcean container and deploy on top of your application.

Step 1: Define variables

First two variables are defined in gitlab. You can find these inside project in the left menu - enter in Settings > CI/CD > Variables $DIGITALOCEAN_API_KEY = token generated in DigitalOcean dashboard $APP_ID = previously generated application identifier   More can be defined as the name of the repository. The value of these variables will be injected into the file that we will create below in the Step 3  

 

Step 2: Register runner in gitlab

In your project in the left menu, go to Settings > CI/CD > Runners Create a specific runner for the project in the URL with registration token. Register GitLab Runner from the command line. It is important to use docker and privileged
# Download the binary for your system
sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64

# Give it permission to execute
sudo chmod +x /usr/local/bin/gitlab-runner

# Register runner
sudo gitlab-runner register -n --url https://git.cakedc.com --registration-token GR1348941gx7sgV3pZFQgRqg5qUR_ --executor docker --description "My Docker Runner" --docker-image "docker:19.03.12" --docker-privileged --docker-volumes "/certs/client"
 

Step 3: Create file gitlab-ci.yml 

The gitlab-ci.yml file takes care of

  • Authentication and identification using doctl in DigitalOcean
  • Generating and sending the docker image to the DigitalOcean container
  • Deploying the container image to an existing app
Create the file gitlab-ci.yml in the root of your project as
image: docker:20-dind

variables:
  DOCKER_HOST: tcp://docker:2375
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""
  REPOSITORY_URL: registry.digitalocean.com/container-nyc-795/static-app
  CONTAINER_NAME: static-app

services:
  - name: docker:20-dind
    alias: docker
    command: ["--tls=false"]

before_script:
  - apk update
  - apk upgrade
  - apk add doctl --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community
  - docker info

build:
  stage: build
  script:
    - doctl auth init --access-token $DIGITALOCEAN_API_KEY
    - doctl account get     
    - echo $DIGITALOCEAN_API_KEY | docker login -u $DIGITALOCEAN_API_KEY --password-stdin registry.digitalocean.com
    - docker build -t $REPOSITORY_URL:latest .
    - docker push $REPOSITORY_URL
    - doctl apps create-deployment $APP_ID
Now, every time you do a git push in your project, the runner will automatically inject the variables defined previously in the gitlab-ci.yml file. Then, it will generate a docker image with docker (docker-in-docker) to create an image of your project, send it to the digitalocean repository and deploy it in the app configured.   That’s it!  

CakeFest 2021 Recap

Here we are again coming off of the CakeFest sugar high! I don’t even know where to begin.    Unfortunately, or fortunately - I haven’t decided, we had to do another virtual event. The safety of speakers, staff and attendees is very important to us, so a physical event was not the best option in our opinion with traveling.    However, after this event, I started thinking about the people who were able to attend from the comfort of their own homes or offices. These people may not have been able to travel or attend otherwise, and that gives me our silver lining. Not to mention that we had more ticket sales this year than any of our previous events (at least that I can remember).    The theme, for me anyway, ha ha, was traveling the world, ironically. We started in the Canary Islands, traveled to Germany, to Canada, to England and Austria. We had new faces from the US, the Czech Republic and even Japan - and more! This is, as I’ve mentioned, one of the best things about the CakePHP community, we have community members all over the world. This was our chance to come together.    So let’s get to the event. Here’s what you may have missed: 

Workshops:

Workshop 1 Jorge González (Twitter: @steinkelz) Topics covered included: 0:00 - Docker development environment for CakePHP 15:56 - Middlewares  30:05 - Security 1:31:36 - Performance optimization 2:04:49 - Events   Workshop 2 Michael Hoffmann (Twitter: @cleptric) Topics covered included: 0:00:00 -Setup login action in CakePHP 0:29:10 - Vite with hot reloading Vue.js tailwind css   Workshop 3 Mark Story (Twitter: @mark_story) Topics covered included: 0:04:25 - Leveraging new style fixtures 0:48:26 - Using the DI container 1:30:13 - Browser automation testing with Panther. 2:17:13 - Helpers you may need.
 

Talks:

* Juan Pablo Ramirez (Twitter: @jpramidev) gave the keynote talk on behalf of Passbolt. * Sho Ito (Twitter: @itosho) taught us all about Components * Yuki Kanazawa (Twitter: @yakitori009) and this talk about Automatically Distributing Reference Queries to    Read Replica in CakePHP4 * Mark Scherer (Twitter: @dereuromark) schooled attendees on IDE in CakePHP development * Jiri Havlicek (Twitter: @Jerryhavl) played a big role in fighting COVID-19 by helping create a  contact tracing app (developed with CakePHP) in Czech Republic * Chris Miller (Twitter: @ccmiller2019) explained standards and why we use them * Kevin Phifer (Twitter: @lordsimal)  joined in to explain how to re-use code - utility classes and PHP namespaces * Paul Henriks created a plugin with attendees LIVE * Ed Barnard (Twitter: @ewbarnard) brought the dragons! He talked about finding the Joy in Software Development * Chris Hartjes (Twitter: @grmpyprogrammer) delivered a Grumpy Programmer's Guide to being a senior developer  * Joe Ferguson (Twitter: @joepferguson) shared his knowledge on Modern Infrastructure as code with Ansible * Timo Stark (Twitter: @linux_lenny) shared details about NGINX Unit - and how to modernize your CakePHP deployments

Trivia and giveaways 

Cake ceremony dedicated to Mark Story

We took this time to thank and acknowledge Mark Story for all of his hard work and dedication that he puts into CakePHP. He then headed the cake cutting ceremony (virtually of course) as speakers and attendees enjoyed their own treats!   See the full archive here: https://cakefest.org/archive/virtual-2021  

So what’s to come? 

First!  Videos are starting to be released. With the help of community member Aroop Roelofs, we will be releasing these videos faster than expected. Ticket holders have been receiving access, and they will be released publicly in the coming days.  In regards to future events, it’s up in the air. We will have some internal discussions about safety measures and restrictions, then we will weigh the option between another virtual or physical event. We will, of course, reach out to the community for their input.  I will close by just saying THANK YOU. Thank you for making my job worth it. When an event runs smoothly and gets so much great feedback, that is a direct reflection from the community support. We hope you all will continue to join us in years to come!    Thanks for baking!  

Dependency Injection with CakePHP

Let’s talk about Dependency Injection!

SOLID principles

As you know SOLID is an acronym for the  five object-oriented design principles. In this topic, we will focus on Interface segregation principle and Dependency inversion principle. Interface segregation principle states that a client must not be forced to implement an interface that they do not use, or clients shouldn’t be forced to depend on methods they do not use. In other words, having  many client-specific interfaces is better than one general-purpose interface. From the other side, Dependency inversion principle states that objects must depend on abstractions, not on concretions. It states that the high-level module must not depend on the low-level module, but they should depend on abstractions. To follow Dependency inversion principle, we need to construct low-level modules and pass them to constructors, and that might create a lot of manual work for developers. The dependency injection container is created specifically for solving the problem with manual construction of an object, before creating a specific object. If we follow interface segregation principle when developing application modules, it would be easy to configure a container and switch module dependency. This is where the interface shows its incredible power.  

Few words about CakePHP Events System

CakePHP Events System was created to allow injecting some logic using listeners. However, in some cases, it is used to get results from code that will be created by the module user. When an event is dispatched by the listener, it can return the result. Callback injection through the event system has some drawbacks. First of all, parameters passed to the event need to pass as a hash array. So unfortunately, there is no way to check that all params are really passed or to be sure that all passed params have correct types. Is there a way to solve this problem? Yes, and containers could help with that. Instead of passing events, we can get the required object from the container and call it method. But you could say: wait, we don't know what object could be used in client code within the developed plugin. That's fine, and this  is where interface segregation principle can help. In our plugin, we define an interface for each such case, and instead of dispatching an event, we can easily get an object from the container by interface.       $updater = $container->get(AfterLoginInterface::class);     if ($updater !== null) {         $user = $updater->afterLogin($user);     }   In the Application::services method, users link the interface with the specific class.       public function services(ContainerInterface $container): void     {         $container->add(AfterLoginInterface::class, MyAfterLogin::class);     }   In some of default behavior needed we can map service class for container to default implementation using Plugin::services method.       public function services(ContainerInterface $container): void     {         if (!$container->has(AfterLoginInterface::class)) {             $container->add(AfterLoginInterface::class, NullAfterLogin::class);         }     }  

Container propagation

Dependency injection is an experimental feature. Initial implementation limited by Controllers constructors and methods, and Commands constructors. If we want to access the container in other parts of the application, we may want to propagate it from app level. The most logical way would be to implement middleware and store the container inside the request attribute.   <?php declare(strict_types=1);   namespace App\Middleware;   use Cake\Core\ContainerInterface; use Psr\Http\Message\ResponseInterface; use Psr\Http\Message\ServerRequestInterface; use Psr\Http\Server\MiddlewareInterface; use Psr\Http\Server\RequestHandlerInterface; use RuntimeException;   /**  * Container Injector Middleware  */ class ContainerInjectorMiddleware implements MiddlewareInterface {     /**      * @var \Cake\Core\ContainerInterface      */     protected $container;       /**      * Constructor      *      * @param \Cake\Core\ContainerInterface $container The container to build controllers with.      */     public function __construct(ContainerInterface $container)     {         $this->container = $container;     }       /**      * Serve assets if the path matches one.      *      * @param \Psr\Http\Message\ServerRequestInterface $request The request.      * @param \Psr\Http\Server\RequestHandlerInterface $handler The request handler.      * @return \Psr\Http\Message\ResponseInterface A response.      */     public function process(ServerRequestInterface $request, RequestHandlerInterface $handler): ResponseInterface     {         return $handler->handle($request->withAttribute('container', $this->container));     }   That’s it! I hope that this will help you when you are baking with dependency injections. If you run into any problems, there are many support channels that allow the CakePHP community to help  You can check them out under the community tab at CakePHP.org.

We Bake with CakePHP