Running Composer and npm scripts with deployment via SCP in GitLab CI/CD

{{< details >}}

• Tier: Free, Premium, Ultimate
• Offering: GitLab.com, GitLab Self-Managed, GitLab Dedicated

{{< /details >}}

This guide covers the building of dependencies of a PHP project while compiling assets via an npm script using GitLab CI/CD.

It is possible to create your own image with custom PHP and Node.js versions. For brevity, this guide uses an existing Docker image with both PHP and Node.js installed.

image: tetraweb/php

The next step is to install zip/unzip packages and make composer available. Place these in the before_script section:

before_script:
  - apt-get update
  - apt-get install zip unzip
  - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
  - php composer-setup.php
  - php -r "unlink('composer-setup.php');"

This ensures all requirements are ready. Next, run composer install to fetch all PHP dependencies and npm install to load Node.js packages. Then run the npm script. Append the commands to the before_script section:

before_script:
  # ...
  - php composer.phar install
  - npm install
  - npm run deploy

In this particular case, the npm deploy script is a Gulp script that does the following:

1. Compile CSS & JS
2. Create sprites
3. Copy various assets (images, fonts) around
4. Replace some strings

All these operations put all files into a build folder, which is ready to be deployed to a live server.

How to transfer files to a live server

You have multiple options such as rsync, SCP, or SFTP. For now, use SCP.

To make this work, you must add a GitLab CI/CD Variable (accessible on gitlab.example/your-project-name/variables). Name this variable STAGING_PRIVATE_KEY and set it to the private SSH key of your server.

Security tip

Create a user that has access only to the folder that needs to be updated.

After you create that variable, make sure that key is added to the Docker container on run:

before_script:
  # - ....
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  - mkdir -p ~/.ssh
  - eval $(ssh-agent -s)
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

This script performs the following actions:

1. Check if the ssh-agent is available and install it if it's not.
2. Create the ~/.ssh folder.
3. Make sure the script execution environment is running bash.
4. Disable host checking. Every connection occurs in a new environment, so disabling host checking ensures GitLab doesn't ask the user to verify and accept the server's identity before connecting each time.

And this is basically all you need in the before_script section.

How to deploy

To deploy the build folder from the Docker image to your server, create a new job:

stage_deploy:
  artifacts:
    paths:
      - build/
  rules:
    - if: $CI_COMMIT_BRANCH == "dev"
  script:
    - ssh-add <(echo "$STAGING_PRIVATE_KEY")
    - ssh -p22 server_user@server_host "mkdir htdocs/wp-content/themes/_tmp"
    - scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/_tmp
    - ssh -p22 server_user@server_host "mv htdocs/wp-content/themes/live htdocs/wp-content/themes/_old && mv htdocs/wp-content/themes/_tmp htdocs/wp-content/themes/live"
    - ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/_old"

Here's the breakdown:

1. rules:if: $CI_COMMIT_BRANCH == "dev" means that this build runs only when something is pushed to the dev branch. You can remove this block completely and have everything run on every push (but probably this is something you don't want).
2. ssh-add ... adds that private key you added on the web UI to the Docker container.
3. Connect using ssh and create a new _tmp folder.
4. Connect using scp and upload the build folder (which was generated by a npm script) to the previously created _tmp folder.
5. Connect again using ssh and move the live folder to an _old folder, then move _tmp to live.
6. Connect to SSH and remove the _old folder.

The artifacts section instructs GitLab CI/CD to keep the build directory (later on, you can download that as needed).

Why do it this way

If you're using this only for stage server, you could do this in two steps:

- ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/live/*"
- scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/live

The problem is that there's a small period of time when you don't have the app on your server.

Therefore, for a production environment, the additional steps ensure that a functional app is in place at any given time.

Where to go next

Because this was a WordPress project, it includes real code snippets. Some further ideas you can pursue:

• Having a slightly different script for the default branch allows you to deploy to a production server from that branch and to a stage server from any other branches.
• Instead of pushing it live, you can push it to WordPress official repository.
• You could generate i18n text domains on the fly.

The final .gitlab-ci.yml looks like this:

stage_deploy:
  image: tetraweb/php
  artifacts:
    paths:
      - build/
  rules:
    - if: $CI_COMMIT_BRANCH == "dev"
  before_script:
    - apt-get update
    - apt-get install zip unzip
    - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
    - php composer-setup.php
    - php -r "unlink('composer-setup.php');"
    - php composer.phar install
    - npm install
    - npm run deploy
    - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
    - mkdir -p ~/.ssh
    - eval $(ssh-agent -s)
    - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
  script:
    - ssh-add <(echo "$STAGING_PRIVATE_KEY")
    - ssh -p22 server_user@server_host "mkdir htdocs/wp-content/themes/_tmp"
    - scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/_tmp
    - ssh -p22 server_user@server_host "mv htdocs/wp-content/themes/live htdocs/wp-content/themes/_old && mv htdocs/wp-content/themes/_tmp htdocs/wp-content/themes/live"
    - ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/_old"