Back to Tuist

Xcode test sharding {#xcode-test-sharding}

server/priv/docs/en/guides/features/test-sharding/xcode.md

4.204.018.1 KB
Original Source

Xcode test sharding {#xcode-test-sharding}

[!WARNING] Requirements

  • A <.localized_link href="/guides/server/accounts-and-projects">Tuist account and project</.localized_link>
  • <.localized_link href="/guides/features/test-insights">Test Insights</.localized_link> configured (for optimal shard balancing)

Test sharding for Xcode projects uses tuist xcodebuild build-for-testing to create a shard plan and tuist xcodebuild test to execute each shard.

How it works {#how-it-works}

Test sharding follows a two-phase workflow:

  1. Build phase: Tuist reads the test modules from the built .xctestrun file and creates a shard plan on the server. The server uses historical test timing data from the last 30 days to distribute tests across shards so each shard takes roughly the same amount of time. The build phase outputs a shard matrix that your CI system uses to spawn parallel runners.
  2. Test phase: Each continuous integration runner receives a shard index and the exact shard plan identifier, then executes that plan's test selection.

With suite granularity, the server chooses known suites per module: for each module in the current .xctestrun, it uses the latest CI run on the build branch that included that module, falling back to the project's default branch for modules without branch history. This keeps selective testing runs from hiding modules that were skipped in the latest branch run.

When --shard-granularity suite is used, Tuist balances known suites across the requested shard count and uses the final shard as the catch-all. For example, --shard-total 5 produces shard indexes 0 through 4, with shard index 4 as the catch-all. Regular shards run with -only-testing for their assigned suites. The final shard runs without -only-testing and passes -skip-testing for every suite assigned to the earlier shards, so it runs its planned suites plus newly added suites or suites missing from historical inventory. Module granularity does not need a catch-all because the .xctestrun file provides the module list.

Build phase {#build-phase}

Build your tests and create a shard plan:

sh
tuist xcodebuild build-for-testing \
  -scheme MyScheme \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  --shard-total 5

This command:

  1. Builds your tests with xcodebuild build-for-testing
  2. Creates a shard plan on the Tuist server using historical timing data
  3. Uploads the .xctestproducts bundle or writes a shard archive for use by shard runners
  4. Outputs a shard matrix for your CI system

Build options {#build-options}

FlagEnvironment variableDescription
--shard-max <N>TUIST_TEST_SHARD_MAXMaximum number of shards. Used with --shard-max-duration to cap the shard count
--shard-min <N>TUIST_TEST_SHARD_MINMinimum number of shards
--shard-total <N>TUIST_TEST_SHARD_TOTALExact number of shards (mutually exclusive with --shard-min/--shard-max). With suite granularity, the final shard is the catch-all
--shard-max-duration <MS>TUIST_TEST_SHARD_MAX_DURATIONTarget maximum duration per shard in milliseconds
--shard-granularity <LEVEL>TUIST_TEST_SHARD_GRANULARITYmodule (default) distributes entire test modules across shards; suite distributes individual test classes for finer-grained balancing
--shard-reference <REF>TUIST_SHARD_REFERENCEUnique identifier for the shard plan (auto-derived on supported CI providers)
--shard-archive-path <PATH>TUIST_TEST_SHARD_ARCHIVE_PATHPath where Tuist writes the optimized shard archive instead of uploading test products to remote storage

Test phase {#test-phase}

Each shard runner executes its assigned tests:

sh
tuist xcodebuild test \
  -scheme MyScheme \
  -destination 'platform=iOS Simulator,name=iPhone 16'

Test options {#test-options}

FlagEnvironment variableDescription
--shard-index <N>TUIST_SHARD_INDEXZero-based index of the shard to execute
--shard-plan-id <IDENTIFIER>TUIST_SHARD_PLAN_IDExact shard plan identifier emitted by the build phase. Generated provider outputs include this value automatically
--shard-reference <REF>TUIST_SHARD_REFERENCEUnique identifier for the shard plan (auto-derived on supported CI providers)
--shard-archive-path <PATH>TUIST_TEST_SHARD_ARCHIVE_PATHPath to a locally managed shard archive; Tuist extracts it instead of downloading test products from remote storage

Tuist downloads the .xctestproducts bundle and filters it to include only the tests assigned to that shard.

Continuous integration {#continuous-integration}

Tuist automatically detects the following CI providers:

For other providers, refer to the .tuist-shard-matrix.json file to set up parallel jobs.

GitHub Actions {#github-actions}

Use a matrix strategy to run shards in parallel:

yaml
name: Tests
on: [pull_request]

jobs:
  build:
    name: Build test shards
    runs-on: macos-latest
    outputs:
      matrix: ${{ steps.build.outputs.matrix }}
    steps:
      - uses: actions/checkout@v4
      - uses: jdx/mise-action@v2
      - run: tuist auth login
      - id: build
        run: |
          tuist xcodebuild build-for-testing \
            -scheme MyScheme \
            -destination 'platform=iOS Simulator,name=iPhone 16' \
            --shard-total 5

  test:
    name: "Shard #${{ matrix.shard }}"
    needs: build
    runs-on: macos-latest
    strategy:
      fail-fast: false
      matrix: ${{ fromJson(needs.build.outputs.matrix) }}
    env:
      TUIST_SHARD_INDEX: ${{ matrix.shard }}
      TUIST_SHARD_PLAN_ID: ${{ matrix.shard_plan_id }}
    steps:
      - uses: actions/checkout@v4
      - uses: jdx/mise-action@v2
      - run: tuist auth login
      - run: |
          tuist xcodebuild test \
            -scheme MyScheme \
            -destination 'platform=iOS Simulator,name=iPhone 16'

GitLab CI {#gitlab-ci}

Tuist generates a .tuist-shard-child-pipeline.yml that you trigger as a child pipeline. Define a .tuist-shard template job that the generated shard jobs extend:

yaml
# .gitlab-ci.yml
stages:
  - build
  - test

build-shards:
  stage: build
  tags: [macos]
  script:
    - tuist auth login
    - |
      tuist xcodebuild build-for-testing \
        -scheme MyScheme \
        -destination 'platform=iOS Simulator,name=iPhone 16' \
        --shard-total 5
  artifacts:
    paths:
      - .tuist-shard-child-pipeline.yml

test-shards:
  stage: test
  needs: [build-shards]
  trigger:
    include:
      - artifact: .tuist-shard-child-pipeline.yml
        job: build-shards
    strategy: depend
yaml
# .gitlab/shard-template.yml
.tuist-shard:
  tags: [macos]
  script:
    - tuist auth login
    - |
      tuist xcodebuild test \
        -scheme MyScheme \
        -destination 'platform=iOS Simulator,name=iPhone 16'

CircleCI {#circleci}

Tuist generates a .tuist-shard-continuation.json with parameters for the continuation orb. The shard-plan-id parameter must be declared in your continued configuration:

yaml
# .circleci/config.yml
version: 2.1
setup: true

orbs:
  continuation: circleci/continuation@1

jobs:
  build-shards:
    macos:
      xcode: "16.0"
    steps:
      - checkout
      - run:
          name: Build and plan shards
          command: |
            tuist auth login
            tuist xcodebuild build-for-testing \
              -scheme MyScheme \
              -destination 'platform=iOS Simulator,name=iPhone 16' \
              --shard-total 5
      - continuation/continue:
          configuration_path: .circleci/continue-config.yml
          parameters: .tuist-shard-continuation.json

workflows:
  setup:
    jobs:
      - build-shards
yaml
# .circleci/continue-config.yml
version: 2.1

parameters:
  shard-indices:
    type: string
    default: ""
  shard-count:
    type: integer
    default: 0
  shard-plan-id:
    type: string
    default: ""

jobs:
  test-shard:
    macos:
      xcode: "16.0"
    parameters:
      shard-index:
        type: integer
    steps:
      - checkout
      - run:
          name: Run shard
          command: |
            export TUIST_SHARD_INDEX=<< parameters.shard-index >>
            export TUIST_SHARD_PLAN_ID=<< pipeline.parameters.shard-plan-id >>
            tuist auth login
            tuist xcodebuild test \
              -scheme MyScheme \
              -destination 'platform=iOS Simulator,name=iPhone 16'

workflows:
  test:
    jobs:
      - test-shard:
          matrix:
            parameters:
              shard-index: [<< pipeline.parameters.shard-indices >>]

Buildkite {#buildkite}

Tuist generates a .tuist-shard-pipeline.yml with one step per shard. Upload it with buildkite-agent pipeline upload:

yaml
# pipeline.yml
steps:
  - label: "Build test shards"
    command: |
      tuist auth login
      tuist xcodebuild build-for-testing \
        -scheme MyScheme \
        -destination 'platform=iOS Simulator,name=iPhone 16' \
        --shard-total 5
      buildkite-agent pipeline upload .tuist-shard-pipeline.yml
    agents:
      queue: macos

Each generated step has TUIST_SHARD_INDEX and TUIST_SHARD_PLAN_ID set in its environment. Add the test command to each shard step using a shared script:

bash
# .buildkite/shard-step.sh
#!/bin/bash
tuist auth login
tuist xcodebuild test \
  -scheme MyScheme \
  -destination 'platform=iOS Simulator,name=iPhone 16'

Codemagic {#codemagic}

Codemagic does not support dynamic matrix jobs, so define a separate workflow per shard. Tuist writes TUIST_SHARD_MATRIX and TUIST_SHARD_COUNT to the CM_ENV file for use within the build workflow. Codemagic's CM_ENV is workflow-scoped, so the exact shard plan identifier does not cross workflow boundaries; test workflows fall back to reference-based shard lookup:

yaml
# codemagic.yaml
workflows:
  build-shards:
    name: Build test shards
    instance_type: mac_mini_m2
    environment:
      xcode: latest
    scripts:
      - name: Build and plan shards
        script: |
          tuist auth login
          tuist xcodebuild build-for-testing \
            -scheme MyScheme \
            -destination 'platform=iOS Simulator,name=iPhone 16' \
            --shard-total 5

  test-shard-0: &shard-workflow
    name: "Shard #0"
    instance_type: mac_mini_m2
    environment:
      xcode: latest
      vars:
        TUIST_SHARD_INDEX: 0
    scripts:
      - name: Run shard
        script: |
          tuist auth login
          tuist xcodebuild test \
            -scheme MyScheme \
            -destination 'platform=iOS Simulator,name=iPhone 16'

  test-shard-1:
    <<: *shard-workflow
    name: "Shard #1"
    environment:
      xcode: latest
      vars:
        TUIST_SHARD_INDEX: 1

  test-shard-2:
    <<: *shard-workflow
    name: "Shard #2"
    environment:
      xcode: latest
      vars:
        TUIST_SHARD_INDEX: 2

  test-shard-3:
    <<: *shard-workflow
    name: "Shard #3"
    environment:
      xcode: latest
      vars:
        TUIST_SHARD_INDEX: 3

  test-shard-4:
    <<: *shard-workflow
    name: "Shard #4"
    environment:
      xcode: latest
      vars:
        TUIST_SHARD_INDEX: 4

Bitrise {#bitrise}

On Bitrise, Tuist writes .tuist-shard-matrix.json (including shard_plan_id) to the BITRISE_DEPLOY_DIR, making it available as a build artifact for downstream pipeline stages. Each test workflow extracts the plan ID from the downloaded artifact using envman so it reaches the test command's environment:

yaml
# bitrise.yml
pipelines:
  test-pipeline:
    stages:
      - build-stage: {}
      - test-stage: {}

stages:
  build-stage:
    workflows:
      - build-shards: {}
  test-stage:
    workflows:
      - test-shard-0: {}
      - test-shard-1: {}
      - test-shard-2: {}
      - test-shard-3: {}
      - test-shard-4: {}

workflows:
  build-shards:
    steps:
      - script:
          title: Build and plan shards
          inputs:
            - content: |
                tuist auth login
                tuist xcodebuild build-for-testing \
                  -scheme MyScheme \
                  -destination 'platform=iOS Simulator,name=iPhone 16' \
                  --shard-total 5
      - deploy-to-bitrise-io: {}

  test-shard-0: &shard-workflow
    envs:
      - TUIST_SHARD_INDEX: 0
    steps:
      - pull-intermediate-files: {}
      - script:
          title: Set shard plan ID from artifact
          inputs:
            - content: |
                PLAN_ID=$(jq -r '.shard_plan_id' "$BITRISE_DEPLOY_DIR/.tuist-shard-matrix.json")
                envman add --key TUIST_SHARD_PLAN_ID --value "$PLAN_ID"
      - script:
          title: Run shard
          inputs:
            - content: |
                tuist auth login
                tuist xcodebuild test \
                  -scheme MyScheme \
                  -destination 'platform=iOS Simulator,name=iPhone 16'
  test-shard-1:
    <<: *shard-workflow
    envs:
      - TUIST_SHARD_INDEX: 1
  test-shard-2:
    <<: *shard-workflow
    envs:
      - TUIST_SHARD_INDEX: 2
  test-shard-3:
    <<: *shard-workflow
    envs:
      - TUIST_SHARD_INDEX: 3
  test-shard-4:
    <<: *shard-workflow
    envs:
      - TUIST_SHARD_INDEX: 4

[!TIP] Bitrise does not support dynamic parallel job creation at runtime. Define a fixed number of shard workflows in your pipeline stages — workflows within a stage run in parallel automatically.

Shared volumes {#shared-volumes}

By default, the build phase uploads the .xctestproducts bundle to remote storage, and each shard runner downloads it. If your CI provider supports shared volumes (persistent storage mounted across jobs), you can skip this upload/download entirely by passing the test products through a shared filesystem.

This can significantly reduce shard startup time, especially for large test bundles.

To use shared volumes:

  1. In the build phase, pass -testProductsPath pointing to a shared volume and add --shard-skip-upload to skip the remote upload:
sh
tuist xcodebuild build-for-testing \
  -scheme MyScheme \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  --shard-total 5 \
  --shard-skip-upload \
  -testProductsPath /path/to/shared/volume/$UNIQUE_ID/MyScheme.xctestproducts
  1. In the test phase, pass the same -testProductsPath so Tuist reads the test products locally instead of downloading them:
sh
tuist xcodebuild test \
  -scheme MyScheme \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  -testProductsPath /path/to/shared/volume/$UNIQUE_ID/MyScheme.xctestproducts

[!IMPORTANT] Use a unique path per workflow run (e.g. include the CI run ID) to avoid collisions between concurrent runs. You should also clean up the test products after sharding completes to avoid accumulating stale data on the volume.

FlagEnvironment variableDescription
--shard-skip-uploadTUIST_TEST_SHARD_SKIP_UPLOADSkip uploading the test products bundle to remote storage

Self-managed artifacts {#self-managed-artifacts}

If your CI provider already has artifact upload and download steps, you can let Tuist handle archive and extraction while your CI handles transport.

  1. In the build phase, pass --shard-archive-path so Tuist writes its optimized shard archive locally instead of uploading test products:
sh
tuist xcodebuild build-for-testing \
  -scheme MyScheme \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  --shard-total 5 \
  --shard-archive-path /tmp/shards/${UNIQUE_ID}/bundle.aar
  1. Upload that archive using your CI's native artifact step.

  2. In each test phase job, download the archive and pass the same path back to Tuist:

sh
tuist xcodebuild test \
  -scheme MyScheme \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  --shard-archive-path /tmp/shards/${UNIQUE_ID}/bundle.aar

When --shard-archive-path is set, Tuist skips remote test-products transfer and uses the local archive instead. If you also pass --shard-skip-upload, the archive path takes precedence.

Namespace {#namespace}

Namespace runners work well with GitHub Actions artifacts. Set TUIST_TEST_SHARD_ARCHIVE_PATH once so the build job writes the shard archive locally, upload it, and download it in each shard job before running tuist xcodebuild test:

yaml
name: Tests
on: [pull_request]

env:
  TUIST_TEST_SHARD_ARCHIVE_PATH: /tmp/shards/${{ github.run_id }}/bundle.aar

jobs:
  build:
    name: Build test shards
    runs-on: namespace-profile-default-macos
    outputs:
      matrix: ${{ steps.build.outputs.matrix }}
    steps:
      - uses: actions/checkout@v4
      - uses: jdx/mise-action@v2
      - run: tuist auth login
      - id: build
        run: |
          tuist xcodebuild build-for-testing \
            -scheme MyScheme \
            -destination 'platform=iOS Simulator,name=iPhone 16' \
            --shard-total 5
      - uses: actions/upload-artifact@v4
        with:
          name: test-shard-archive
          path: ${{ env.TUIST_TEST_SHARD_ARCHIVE_PATH }}
      - if: always()
        run: rm -rf /tmp/shards/${{ github.run_id }}

  test:
    name: "Shard #${{ matrix.shard }}"
    needs: build
    runs-on: namespace-profile-default-macos
    strategy:
      fail-fast: false
      matrix: ${{ fromJson(needs.build.outputs.matrix) }}
    env:
      TUIST_SHARD_INDEX: ${{ matrix.shard }}
      TUIST_SHARD_PLAN_ID: ${{ matrix.shard_plan_id }}
    steps:
      - uses: actions/checkout@v4
      - uses: jdx/mise-action@v2
      - run: tuist auth login
      - uses: actions/download-artifact@v4
        with:
          name: test-shard-archive
          path: /tmp/shards/${{ github.run_id }}
      - run: |
          tuist xcodebuild test \
            -scheme MyScheme \
            -destination 'platform=iOS Simulator,name=iPhone 16'
      - if: always()
        run: rm -rf /tmp/shards/${{ github.run_id }}