ContextQMD
Back to Codeceptjs

Continuous Integration

docs/continuous-integration.md

4.0.012.1 KB
Original Source

Continuous Integration

CodeceptJS runs in any CI that can install Node.js. This page covers the setup, then provides ready-to-use configs for the major CI systems.

Setup

  • Node.js — install it on the runner (actions/setup-node, the node:20 image, NodeTool@0 on Azure). Examples below use Node 20.

  • Headless — codecept.conf.js must contain setHeadlessWhen(process.env.HEADLESS || process.env.CI). codeceptjs init adds it; since CI sets CI=true, the suite runs headless automatically.

    js
    import { setHeadlessWhen } from '@codeceptjs/configure'

setHeadlessWhen(process.env.HEADLESS || process.env.CI)

    Override the browser or viewport per run with the browser plugin: -p browser:browser=firefox:windowSize=1280x1024.

  • Artifacts — enable the on-failure capture plugins:

    js
    plugins: {
  screenshot: { enabled: true },          // screenshot on failure
  pageInfo:   { enabled: true },          // page URL, HTML errors, console logs
  aiTrace:    { enabled: true, on: 'fail' }, // AI-friendly trace.md
}

    screencast records video of failed tests (Playwright). Everything lands in output/ — upload that directory as a build artifact. Every example below does.

Browsers and drivers

Check before running

npx codeceptjs check loads the config, opens the browser, and counts tests. Prepend it so a broken environment fails fast with a clear message:

bash
npx codeceptjs check
npx codeceptjs run

Every CI example below does this.

Parallel execution

  • npx codeceptjs run — one process.
  • npx codeceptjs run-workers 4 — parallel on one machine, tests handed to workers dynamically.
  • npx codeceptjs run --shard 1/4 — split the suite across CI matrix jobs (one shard per machine).

Shards and workers combine. Full reference: Parallel Execution.

Reporting

Use @testomatio/reporter. It ships pipes that publish results into GitHub PR checks, GitLab merge-request widgets, and Bitbucket pipeline reports. See Reporting.

CI examples

Each example uses Playwright by default; a WebDriver variant follows where it differs. WebdriverIO 9 downloads its own browser and driver, so the WebDriver variants run on a plain node:20 image with no Selenium service and no browser-install step. For Playwright, a node:20 base image plus npx playwright install --with-deps keeps these configs free of version pins.

GitHub Actions — Playwright

.github/workflows/tests.yml:

yaml
name: Tests
on:
  push:
    branches: [main]
  pull_request:

env:
  FORCE_COLOR: 1

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx codeceptjs check
      - run: npx codeceptjs run-workers 4 --by pool
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: codeceptjs-output
          path: output/

GitHub Actions — WebDriver

WebdriverIO 9 downloads its own browser and driver, so the job is just npm ci + run — no Selenium service, no browser-install step. This mirrors WebdriverIO's own boilerplate CI.

yaml
name: WebDriver Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      FORCE_COLOR: 1
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx codeceptjs check
      - run: npx codeceptjs run-workers 2 --by pool
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: codeceptjs-output
          path: output/

GitHub Actions — sharded matrix

Each shard runs on its own runner in parallel:

yaml
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: ['1/4', '2/4', '3/4', '4/4']
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx codeceptjs check
      - run: npx codeceptjs run --shard ${{ matrix.shard }}
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: output-${{ strategy.job-index }}
          path: output/

GitHub Actions — browser matrix

Run the same suite across browsers and viewports via the browser plugin (npm i --save-dev @codeceptjs/configure):

yaml
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        include:
          - { browser: chromium, size: 1920x1080 }
          - { browser: firefox,  size: 1366x768 }
          - { browser: webkit,   size: 414x896 }
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx codeceptjs check
      - run: npx codeceptjs run -p browser:browser=${{ matrix.browser }}:windowSize=${{ matrix.size }}
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: output-${{ matrix.browser }}-${{ matrix.size }}
          path: output/

GitLab CI

.gitlab-ci.yml:

yaml
stages: [test]

playwright:
  stage: test
  image: node:20
  variables:
    FORCE_COLOR: "1"
  parallel: 4
  before_script:
    - npm ci
    - npx playwright install --with-deps chromium
  script:
    - npx codeceptjs check
    - npx codeceptjs run --shard $CI_NODE_INDEX/$CI_NODE_TOTAL
  artifacts:
    when: on_failure
    paths: [output/]
    expire_in: 1 week

webdriver:
  stage: test
  image: node:20  # WebdriverIO 9 downloads its own browser and driver
  script:
    - npm ci
    - npx codeceptjs check
    - npx codeceptjs run-workers 2 --by pool
  artifacts:
    when: on_failure
    paths: [output/]

$CI_NODE_INDEX is 1-based — it maps directly to CodeceptJS's --shard index.

Bitbucket Pipelines

bitbucket-pipelines.yml:

yaml
image: node:20

definitions:
  caches:
    playwright: ~/.cache/ms-playwright

pipelines:
  default:
    - parallel:
        - step:
            name: Playwright — shard 1/2
            caches: [node, playwright]
            script:
              - npm ci
              - npx playwright install --with-deps chromium
              - npx codeceptjs check
              - npx codeceptjs run --shard 1/2
            artifacts: [output/**]
        - step:
            name: Playwright — shard 2/2
            caches: [node, playwright]
            script:
              - npm ci
              - npx playwright install --with-deps chromium
              - npx codeceptjs check
              - npx codeceptjs run --shard 2/2
            artifacts: [output/**]

For WebDriver, no Selenium service or browser image is needed — WebdriverIO 9 downloads its own browser and driver:

yaml
image: node:20

pipelines:
  default:
    - step:
        script:
          - npm ci
          - npx codeceptjs check
          - npx codeceptjs run-workers 2 --by pool
        artifacts: [output/**]

Jenkins

Jenkinsfile (declarative pipeline):

groovy
pipeline {
  agent {
    docker {
      image 'node:20'
      args '-u root'
    }
  }
  environment {
    FORCE_COLOR = '1'
  }
  stages {
    stage('Install') {
      steps {
        sh 'npm ci'
        sh 'npx playwright install --with-deps chromium'
      }
    }
    stage('Test') {
      parallel {
        stage('Shard 1/4') { steps { sh 'npx codeceptjs check && npx codeceptjs run --shard 1/4' } }
        stage('Shard 2/4') { steps { sh 'npx codeceptjs check && npx codeceptjs run --shard 2/4' } }
        stage('Shard 3/4') { steps { sh 'npx codeceptjs check && npx codeceptjs run --shard 3/4' } }
        stage('Shard 4/4') { steps { sh 'npx codeceptjs check && npx codeceptjs run --shard 4/4' } }
      }
    }
  }
  post {
    failure {
      archiveArtifacts artifacts: 'output/**', allowEmptyArchive: true
    }
  }
}

For WebDriver, keep the same node:20 agent — WebdriverIO 9 downloads its own browser and driver, so no Selenium container is needed:

groovy
stage('Test') {
  steps {
    sh 'npx codeceptjs check'
    sh 'npx codeceptjs run-workers 2 --by pool'
  }
}

CircleCI

.circleci/config.yml:

yaml
version: 2.1

jobs:
  test:
    docker:
      - image: cimg/node:20.18-browsers
    parallelism: 4
    steps:
      - checkout
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx codeceptjs check
      - run:
          name: Run shard
          command: |
            INDEX=$((CIRCLE_NODE_INDEX + 1))
            npx codeceptjs run --shard ${INDEX}/${CIRCLE_NODE_TOTAL}
      - store_artifacts:
          path: output

  webdriver:
    docker:
      # WebdriverIO 9 downloads its own browser and driver
      - image: cimg/node:20.18
    steps:
      - checkout
      - run: npm ci
      - run: npx codeceptjs check
      - run: npx codeceptjs run-workers 2 --by pool
      - store_artifacts:
          path: output

workflows:
  test:
    jobs:
      - test
      - webdriver

CIRCLE_NODE_INDEX is 0-based, so add 1 to match CodeceptJS's 1-based --shard index.

Azure Pipelines

azure-pipelines.yml:

yaml
trigger: [main]

pool:
  vmImage: ubuntu-latest

strategy:
  parallel: 4

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'
  - script: npm ci
    displayName: Install dependencies
  - script: npx playwright install --with-deps chromium
    displayName: Install browsers
  - script: npx codeceptjs check
    displayName: Check setup
  - script: npx codeceptjs run --shard $(System.JobPositionInPhase)/$(System.TotalJobsInPhase)
    displayName: Run shard $(System.JobPositionInPhase)/$(System.TotalJobsInPhase)
    env:
      FORCE_COLOR: 1
  - task: PublishBuildArtifacts@1
    condition: failed()
    inputs:
      pathToPublish: output
      artifactName: codeceptjs-output-$(System.JobPositionInPhase)

For WebDriver, no extra setup is needed — WebdriverIO 9 downloads its own browser and driver:

yaml
  - script: |
      npx codeceptjs check
      npx codeceptjs run-workers 2 --by pool
    displayName: Run tests

Docker

The official codeceptjs/codeceptjs image runs Playwright, Puppeteer, and WebDriver suites with no extra setup. Pass runner flags through CODECEPT_ARGS and the worker count through NO_OF_WORKERS. See Docker.

See also

Community recipes

Got a setup that works for you? Share your recipe and we will list it here.