Back to Tuist

Data Export Documentation

server/data-export.md

4.203.037.8 KB
Original Source

Data Export Documentation

This document outlines all data that Tuist can export for customers upon legal request (GDPR, CCPA). Data is exported for the specific account associated with the requesting user or organization.

Export Format

Data is provided in a single compressed archive containing:

  • Database data: JSON files with account information, projects, command events, and analytics
  • Binary files: All uploaded files (cache artifacts, app previews, icons)
  • Manifest: Index of all included files and data

Sensitive authentication data (passwords, tokens) are excluded from exports.

Exportable Data

Account & User Data

  • User profiles (email, active/inactive status, account settings, preferred locale)

  • Organization records (account handle/name, creator relationship, and timestamps)

  • Organization memberships and roles (user, organization, role, and timestamps)

  • Account billing information and subscriptions

  • Account-level cache settings, including storage region, custom cache endpoint enablement, and cache write policy (members_and_tokens or tokens_only)

  • API tokens, SCIM-scoped account tokens, and project tokens (existence, scopes, names, timestamps, and last-used metadata; token values, token hashes, and generated token secret previews are excluded)

  • Agent registration audit records (agent_registrations, agent_registration_events, agent_auth_jtis, and agent_auth_credentials tables): registration type/status, requested credential type, verified email address, claim attempt id, claim and user-code expiry timestamps, last claim-poll timestamp, registration / claim request / completion Internet Protocol address metadata, claimed user relationship, linked account-token id or access-token identifier, Identity Assertion Authorization Grant issuer/subject/audience/client metadata, replay-protection token identifiers, each issued access token's identifier, expiry, and revocation timestamp, append-only state-change events (created, assertion_issued, token_issued, token_revoked, claim_requested, user_code_minted, claim_confirmed, claim_resent, otp_failed, claimed, expired, revoked), event Internet Protocol address metadata, event metadata, and timestamps. The claim token hash, claim-attempt token hash, user-code hash, issued application programming interface key value, signed identity assertion, and signed access token are excluded from exports as authentication secrets.

  • Custom cache endpoint configurations (account_cache_endpoints table): account-specific custom cache endpoints, active regional Kura endpoint mirrors, and the internal peer (mTLS) addresses of enrolled self-hosted Kura nodes (rows with technology = :kura_self_hosted_peer, written at enrollment from each node's KURA_NODE_URL). These peer addresses are customer infrastructure hostnames used only for mesh peering and are never returned to the CLI as cache endpoints. Self-hosted peer rows whose node stops sending mesh heartbeats carry a deactivated_at timestamp (the peer is withheld from the mesh but kept; the node's recovery re-enrollment reactivates it) and are deleted once deactivated past the peer-certificate lifetime (30 days). Legacy account-level Kura global endpoint rows matching https://<lowercase-account-handle>.kura.tuist.dev are no longer stored separately; they are removed by the Kura global-endpoint cleanup migration.

  • Organization SSO configuration metadata, including the configured SSO provider, provider URL, and full OAuth2 endpoint URLs

  • Kura server records (kura_servers table): per-account Kura server configuration including region, image tag, public URL, status, the warm-handoff move state (move_phase, target_node) recording whether the server is a steady-state instance or a transient move source/target and which box a move target is pinned to, and the observed-state projection (observed_image_tag, last_observed_at, last_ready_at) recording which image the backing cluster reports running, when it was last observed, and when its private endpoint was last observed ready

  • Kura deployment history (kura_deployments table): rollout attempts for the account's Kura servers including image tag, status, error messages, and start/finish timestamps

  • Self-hosted Kura credentials (kura_self_hosted_clients table): tenant-scoped credentials a customer uses to run self-hosted Kura nodes, including the public client_id, friendly name, last-used timestamp, and secret_last_four (the trailing four characters of the secret, kept as a masked-preview hint). The encrypted_secret_hash (Bcrypt hash of the client secret) is excluded from exports as an authentication secret, and the plaintext secret is never stored.

  • Registered Kura endpoints (kura_registered_endpoints table): client-facing cache endpoints reported by a customer's self-hosted Kura nodes via registration heartbeats, including the node id, region label, advertised HTTP URL, readiness, runtime version, traffic state, last-heartbeat timestamp, and lease expiry. These rows are lease-based operational state, refreshed on each heartbeat and removed when a node stops heartbeating.

  • GitHub App installation metadata (github_app_installations table): the installation ID GitHub assigned, the GitHub instance the App lives on (client_url, e.g. https://github.com or a customer's GitHub Enterprise Server host), the App's app_id/app_slug/client_id, and the GitHub-side management html_url. The accompanying client_secret, private_key (PEM), and webhook_secret are stored encrypted at rest and are excluded from exports as authentication secrets.

  • VCS connections (vcs_connections table): the link between a Tuist project and an external repository handle (provider, repository full name, the originating GitHub App installation, and the user who created the connection)

  • Artifact retention cursors (artifact_retention_cursors table): per-account cleanup progress for database-backed artifact families. Exports include the artifact type plus the last processed metadata cursor (after_inserted_at, after_id) used to avoid re-processing blobs that have already been purged from object storage. The run_session cursor covers all run artifact blobs stored under an expired run's artifact prefix.

  • (Internal Tuist-team JIT elevation tables previously documented here moved out of this server's Postgres entirely. They now live in the standalone tuist-ops service on its own CNPG cluster in the mgmt cluster. The data is operator-side audit about Tuist staff only — never customer data — and is out of scope for this server's data export. See tuist-ops/AGENTS.md for where it lives now.)

Projects & Development

  • Project information (account relationship, handle/name, build system, default branch, visibility/settings, repositories, and timestamps)
  • Command events (CLI usage, build data, performance metrics)
  • Cache events and cache action items
  • Test cases and test execution results
  • Build system data (Xcode graphs, projects, targets)
  • Cacheable tasks (Xcode cache analytics: type, status, keys)

App Previews & Builds

  • Preview metadata (versions, platforms, git info)
  • App build information

Alerts & Monitoring

  • Alert rules (name, category, metric, deviation thresholds, Slack channel configuration, baseline established timestamp)
  • Alert history (triggered alerts with current/previous values, timestamps)
  • Automation alerts (automation_alerts table): per-project test automations, including name, enabled flag, monitor type (flakiness_rate / flaky_run_count / reliability_rate / test_updated), evaluation cadence, baseline established timestamp, scoped evaluation cursor timestamp, and the trigger / recovery configuration as JSON. The configuration includes the comparison threshold, comparison operator, window_type (last_days for calendar windows or rolling for count-based windows), the day-string window (e.g. 30d) used in last_days mode, and the integer rolling_window_size (up to 1000 runs) used in rolling mode. Trigger and recovery action lists are stored alongside the configuration (state changes, label adds/removes, Slack channel references).
  • Automation alert events (automation_alert_events table): per-test-case trigger and recovery records produced by automation alerts (alert id, test case id, status, timestamps).

Managed GitHub Actions Runners

  • Per-account availability and concurrency configuration: whether runners are enabled for an account is gated by the :runners feature flag (Tuist.FeatureFlags.runners_enabled?/1). PostgreSQL runner_concurrency_limits stores one row per account and runner platform (linux | macos) with the aggregate vcpus and memory_gb that active runners may consume concurrently. These configuration rows do not contain observed usage history and are deleted with their account.
  • Runner profiles (Postgres runner_profiles): account-scoped named bundles that customers reference from runs-on: as tuist-<name>. Columns: id (PK), account_id, name, platform (enum linux | macos — the runner OS the profile dispatches to), vcpus, memory_gb, xcode_version (string, macOS-only; pins the runner image's Xcode tag), protected (boolean), inserted_at, updated_at. The dispatch path resolves (account, requested-label) through these rows to the matching shape pool. The shape pool itself is operator-managed in Kubernetes and not customer data. Every account is auto-bootstrapped at sign-up with one protected linux row and one protected macos row; protected rows cannot be deleted, but their shape (and the macOS row's Xcode version) remains customer-editable.
  • Active claim coordination (Postgres runner_claims): one row per currently-claimed workflow_job. Columns: workflow_job_id (GitHub's job id, PK), account_id, fleet_name (the RunnerPool name the claim is bound to), pod_name (the SA / Pod that won the claim), claimed_at, platform (linux | macos), vcpus, memory_gb, lifecycle_state (claimed during the JIT-mint window, running once the runner has registered with GitHub), runner_name (the GitHub-side runner label, populated when lifecycle_state flips to running), and executed_workflow_job_id (the job GitHub actually ran on this runner, learned from the workflow_job.in_progress / completed webhook's runner_name; NULL until GitHub proves the runner ran something, and may differ from workflow_job_id because GitHub assigns queued jobs to any label-eligible runner regardless of which one the server claimed the job for), and pod_missing_since (operational reconciliation state, not customer content: the first tick on which the claim's Pod was observed absent from a complete cluster read, NULL whenever the Pod is present. Tuist.Runners.Workers.PodClaimReconciliationWorker requires the absence to persist before releasing the claim, so the column exists only between the first missed observation and the release, and is deleted with the row). The resource columns power atomic per-platform concurrency enforcement and current usage display. Rows are deleted when the executing runner's job completes, on release / stale recovery, or when the Pod stops; steady-state size is bounded by the number of in-flight runners.
  • Runner job completions (Postgres runner_job_completions): one row per workflow_job that reached a completed GitHub state. Columns: workflow_job_id (GitHub's job id, PK), account_id, conclusion, completed_at, inserted_at, and updated_at. Used as an atomic webhook ordering guard so late queued/waiting redeliveries cannot resurrect completed or cancelled runner work.
  • Runner billing sessions (Postgres runner_sessions): append-only record of every runner Pod we provisioned, keyed off the Pod lifecycle rather than the workflow_job's GitHub-reported runtime. Columns: id (PK), account_id, workflow_job_id (the job the Pod was claimed/minted for), executed_workflow_job_id (the job GitHub actually ran on this Pod's runner, learned from the workflow_job.in_progress / completed webhook; NULL until GitHub proves execution — its absence is how we identify a runner that was never handed work — and may differ from workflow_job_id. This is the durable binding that outlives the Pod, and it is what attributes runner machine metrics to the job that actually ran), fleet_name, pod_name, runner_name, repository (denormalized owner/name handle from the workflow_job for billing-page scope filters), workflow_name (denormalized for the same), started_at (claim-win — proxy for Pod creation), ended_at (set by the runners-controller via POST /api/internal/runners/pods/stopped when it observes the Pod's container terminate; NULL while still in flight). Drives metered-compute invoicing via Tuist.Runners.Billing. Retries (Jobs.record_queued/1) produce additional rows so the customer is billed for every Pod they actually held.
  • Runner interactive sessions (Postgres runner_interactive_sessions): session-scoped access grants for interactive runner debugging. Columns: id (PK), account_id, workflow_job_id, pod_name, fleet_name, kind (vnc or shell), state (requested, ready, active, or closed), token_hash (SHA-256 hash of the short-lived browser/gateway session token; the raw token is not persisted), requested_by_user_id (nullable if the user is deleted), connected_at, closed_at, expires_at, last_activity_at, relay_host, relay_port, relay_ready_at, and close_reason. Stores account/user/workflow/pod access metadata for authorization, lifecycle, audit, and support. Relay host/port are transient server-side infrastructure coordinates for the WebSocket bridge; the table does not store Tart VNC passwords.
  • Runner interactive session connections (Postgres runner_interactive_session_connections): per-browser WebSocket connection lifecycle rows for interactive runner sessions. Columns: id (PK), interactive_session_id, connection_id (opaque server-generated connection marker), connected_at, disconnected_at, inserted_at, and updated_at. Used to keep a VNC grant open until all viewers disconnect; does not store VNC credentials or raw browser session tokens.
  • Runner volume affinity (Postgres runner_volume_affinities): dispatch-time hint recording which Mac host an account's runner jobs last ran on, so dispatch can prefer hosts that already hold the account's cache volume. Columns: id (PK), node_name (the cluster Node / Mac host name — infrastructure identifier, not customer content), account_id, volume_name (tuist-cache reserved), last_run_at, inserted_at, and updated_at. One row per (host, account, volume); rows are pruned 14 days after last_run_at. Operational scheduling metadata only — no build content, cache artifacts, or credentials.
  • Runner volume HEAD (Postgres runner_volume_heads): the canonical pointer to an account's current warm cache set for the Mac runner fleet, so a host whose local cache master lags can converge to the latest. Columns: id (PK), account_id, volume_name (tuist-cache reserved), generation (monotonic version counter advanced on each accepted promote), tree_digest (the sorted cache entry-name hash that fingerprints the warm set — a digest, not cache content), node_name (the host that last published the HEAD — infrastructure identifier, observability only), inserted_at, and updated_at. One row per (account, volume), fast-forwarded on promote (a promote advances the HEAD only if it built on the current generation). Coordination metadata only — no build content, cache artifacts, or credentials.
  • Runner cache volume master orphans (Postgres runner_volume_master_orphans): transient bookkeeping for cache-volume master objects a runner uploaded but whose fast-forward promote the server rejected (a stale base another host advanced past). Because the guest uploads the content-addressed image before the compare-and-swap bump, a rejected promote leaves an object with no HEAD pointing at it; a row here lets PruneVolumeMasterOrphanWorker reclaim it after a grace window instead of leaving it until account deletion. Columns: id (PK), account_id, volume_name (tuist-cache reserved), tree_digest (the sorted cache entry-name hash — a digest, not cache content), inserted_at, and updated_at. A row is deleted the moment the same digest is accepted as HEAD or its object is reclaimed. Coordination/cleanup metadata only — no build content, cache artifacts, or credentials.
  • Runner cache volume master images (S3 runner-volume-masters/{account_id}/tuist-cache/{inventory_digest}.image): a per-account snapshot of the account's warm Tuist cache set — compiled Manifests / ProjectDescriptionHelpers / Plugins and the binary-cache index a runner materializes to start a build warm. The object is a sparse APFS disk image (preserving the artifact-signature xattrs, symlinks, and modes natively). Keyed by the set's content (inventory digest) so it is immutable and a concurrent promote never overwrites the object a published HEAD points at; an account may therefore hold several such objects across cache generations. Published by a runner after a successful, cache-changing job and downloaded by a lagging host to converge toward the HEAD above. This IS customer-derived build cache content (no credentials). It is account-scoped under the runner-volume-masters/{account_id}/ prefix. Because it is keyed by account_id (not the account handle), it is NOT swept by the handle-based artifact retention; instead Accounts.delete_account!/1 explicitly deletes the whole prefix on account deletion (best-effort), so it does not outlive the account.
  • Workflow_job lifecycle (ClickHouse runner_jobs, ReplacingMergeTree on workflow_job_id): one logical row per workflow_job carrying the full lifecycle from queuedclaimedrunningcompleted. Columns include the GitHub correlation fields (workflow_job_id, workflow_run_id, run_attempt, workflow_name, job_name, head_branch, head_sha, repository), selected runner resources (platform, vcpus, memory_gb), lifecycle state (status, conclusion), timestamps (enqueued_at, claimed_at, started_at, completed_at, updated_at), binding (pod_name, runner_name), and the downloadable-archive marker (log_archived_at, set once the gzipped log archive is uploaded — see the runner job log archives entry below). Per-step data lives in runner_job_steps. Powers the customer-facing "queued / running / recent runs" surfaces and preserves the resource shape selected when the job was enqueued.
  • Runner job steps (ClickHouse runner_job_steps, ReplacingMergeTree on (workflow_job_id, number)): one row per workflow_job step, captured from the workflow_job.completed webhook. Columns: workflow_job_id, account_id, number (the step's 1-based position), name, status, conclusion, started_at, completed_at, and inserted_at (the RMT version). Powers the job detail page's Steps card and step-level analytics (failure rates per step name, p95 of Build duration, slowest steps in a workflow).
  • Runner job logs (ClickHouse runner_job_logs, ReplacingMergeTree on (workflow_job_id, line_number)): the runner container's captured stdout, one row per line. Columns: workflow_job_id, account_id, line_number, ts (the per-line timestamp GitHub stamps in the log payload), message (the log text), and inserted_at (the RMT version). Populated by Tuist.Runners.Workers.FetchLogsWorker, which streams the job's log from GitHub's Actions Logs API after the workflow_job.completed webhook and inserts batched lines. Surfaced on the job detail page's Logs tab (per-step slicing via ##[group]Run markers, full-log search). Retained for 90 days.
  • Runner job machine metrics (ClickHouse runner_job_machine_metrics, ReplacingMergeTree on (workflow_job_id, timestamp)): per-sample resource usage of the runner Pod/VM while a workflow_job executes, one row per snapshot. Columns: workflow_job_id, account_id, timestamp (epoch seconds the sample was taken), cpu_usage_percent, cpu_iowait_percent (0 on macOS, which has no iowait accounting), memory_used_bytes, memory_total_bytes, network_bytes_in, network_bytes_out, disk_used_bytes, disk_total_bytes, and inserted_at (the RMT version). Unlike logs, these have no GitHub-side source — they describe our infrastructure's runner; the runner metrics collector samples each running job's Pod and POSTs batches to POST /api/internal/runners/jobs/:workflow_job_id/metrics (authenticated with the runners-controller's in-cluster ServiceAccount token). Surfaced on the job detail page's Overview charts and Metrics tab. Retained for 90 days.
  • Runner job log archives (S3 runners/{account_id}/{workflow_job_id}/runner.log.gz): once FetchLogsWorker finishes ingesting a job's lines, Tuist.Runners.Workers.ArchiveLogsWorker stream-gzips them into a single object (multipart-uploaded so a large log never materialises in memory) and stamps log_archived_at on the runner_jobs row. The "Download logs" action redirects to a presigned URL for that object; the button is hidden while log_archived_at is NULL. Same content as runner_job_logs, rendered to plain text (<ISO timestamp> <message> per line). Retained for 90 days, matched to the row-level TTL by Tuist.Runners.Workers.PruneArchivedLogsWorker (a daily Oban cron) which both deletes the S3 object and clears log_archived_at.
  • Kubernetes-side state (RunnerPool CR + Pods / ServiceAccounts in the tuist-runners namespace): operational metadata only — pool name, dispatch label, image, replica count, owner labels on Pods. Reconciled by the runners-controller.
  • JIT runner configs: the GitHub-issued JIT credential the dispatch endpoint mints for each runner registration is an ephemeral authentication secret and is never persisted server-side. Only the resulting runner_name (the GitHub-side runner label) is recorded in runner_jobs.

Slack Integration

  • Account-level Slack installation records (workspace id/name, bot user id; bot access tokens are excluded as authentication secrets)
  • Per-channel Slack webhook destinations stored on projects, alert rules, and project flaky-test settings (channel id, channel name; the encrypted webhook URL itself is excluded as an authentication secret)
  • Per-action Slack webhook destinations stored on automation alerts (channel id, channel name; the encrypted webhook URL embedded in the action payload is excluded as an authentication secret)

Outbound Webhooks

  • Webhook endpoints (webhook_endpoints table): per-account HTTPS destinations subscribed to Tuist events. Exports include endpoint name, subscribed event types (test_case.created, test_case.updated, preview.created, preview.deleted), and timestamps. The endpoint URL and signing secret are stored Vault-encrypted at rest and are excluded from exports as authentication material.
  • Webhook delivery attempts (webhook_delivery_attempts table, ClickHouse): per-attempt audit log generated by Tuist.Webhooks.Workers.DeliveryWorker. Exports include the event id and type, attempt number, delivery status (delivered / failed), the JSON payload Tuist sent (request_body), the response status / headers / body returned by the upstream, any error string, the duration in milliseconds, and the attempt timestamp. Stored in a MergeTree partitioned monthly; retained indefinitely so the dashboard can surface historical deliveries for debugging.

Analytics Data (ClickHouse)

The following data is stored in ClickHouse for analytics purposes:

  • Build runs (build_runs table): Complete build execution data including duration, status, cache statistics, CI metadata, git information, and custom tags
  • Build issues (build_issues table): Compilation warnings and errors from builds
  • Build files (build_files table): Individual file compilation metrics
  • Build targets (build_targets table): Target/module build performance
  • Cacheable tasks (cacheable_tasks table): Xcode cache task analytics with hit/miss status
  • CAS outputs (cas_outputs table): Content-addressable storage upload/download records, including the denormalized project id used for project-scoped analytics
  • Module cache outputs (module_cache_outputs table): Per-artifact module (binary) cache download/upload records for a command run, used for module cache network analytics. Columns: command_event_id, project_id, operation (download/upload), name (target name), hash (content hash), size (on-disk bytes), compressed_size (bytes transferred over the wire), duration (ms), and inserted_at. Keyed by command_event_id because module cache artifacts are fetched during tuist generate, before any build run exists.
  • Command events by name and run time (command_events_by_name_ran_at materialized view + target table): A full copy of command_events re-sorted by (project_id, name, ran_at) that backs the name-filtered recent-runs lists (cache runs, generate runs, the /runs API with a name filter). Carries the same columns as command_events; contains no data not already covered by the source command_events table.
  • Shard plans (shard_plans table): Test sharding plan data including reference, shard count, and granularity
  • Shard plan modules (shard_plan_modules table): Per-shard module assignments with estimated durations
  • Shard plan test suites (shard_plan_test_suites table): Per-shard test suite assignments with estimated durations
  • Shard runs (shard_runs table): Per-shard execution results with status and duration
  • Test runs (test_runs table): Includes shard_plan_id linking test results to their shard plan. The proj_by_project_ran_at projection stores project_id, run id, duration, status, and ran_at, ordered by project and run time for recent-run dashboard queries. It contains no data beyond the source test_runs rows and is exported through those source records rather than as a separate dataset.
  • Test run errors (test_run_errors table): Run/target-level errors where the test runner itself errored (e.g. a target whose .xctest bundle could not be loaded), modelled separately from test failures. Columns: id, test_run_id, project_id, module_name (the test target, empty for run-level), message, and inserted_at.
  • Bundles (bundles table): App bundle metadata (name, app bundle id, version, install/download size, supported platforms, type, git ref/branch/commit).
  • Bundle artifacts (artifacts table): App bundle artifact tree (paths, sizes, SHA hashes, parent/child hierarchy) per uploaded bundle.
  • Active test cases daily stats (test_case_runs_active_daily_stats materialized view): Exact daily presence rows per (project_id, date, is_ci, test_case_id) derived from test_case_runs. Powers the Test Cases analytics chart; contains no data not already covered by the source test_case_runs table.
  • Test case runs by commit (test_case_runs_by_commit materialized view): Slim projection of test_case_runs ordered by (project_id, git_commit_sha, scheme, is_ci, status, id). Powers the cross-run flakiness lookup; contains no data not already covered by the source test_case_runs table.
  • Per-test-case daily run stats (test_case_run_daily_stats_per_case materialized view): AggregatingMergeTree keyed on (project_id, date, test_case_id) with count, sumState(toUInt8(is_flaky)), and sumState(toUInt8(status = 'success')) aggregate states. Powers the test automation engine's per-test windowed comparisons; contains no data not already covered by the source test_case_runs table.
  • Per-test-case recent run stats (test_case_runs_recent_per_case and test_case_runs_recent_{100,250,500,750}_per_case materialized views): AggregatingMergeTree tables keyed on (project_id, test_case_id) with recent-run aggregate states derived from test_case_runs. The 1000-run table powers larger rolling windows and stores both recent flaky-run and recent successful-run aggregate states; the smaller bucket tables power common flaky-test rolling-window comparisons without reading the larger aggregate state. Contains no data not already covered by the source test_case_runs table.
  • Per-environment last-run timestamps (test_cases.last_ran_at_ci, test_cases.last_ran_at_local columns): Denormalized timestamps tracking the most recent CI and local run per test case. Maintained by the ingestion path on every test run; contains no data not already covered by the source test_case_runs table.
  • Test case state (test_case_states table): Control-plane state per test case — state (enabled/muted/skipped) and is_flaky — ordered by (project_id, test_case_id, inserted_at). Derived data: it is a projection of test_case_events, maintained by the test_case_states_mv materialized view, and contains no information not already in that table. Each row records only the column its source event affected and leaves the other null, so the current value of each column is the latest non-null one. A test case with no row here is enabled and not flagged. The state and is_flaky columns on test_cases are legacy leftovers of the same data and are no longer read.
  • Test case events (test_case_events table): Audit log of state changes on a test case — first_run, marked_flaky/unmarked_flaky, muted/unmuted, skipped/unskipped. Each row records the test_case_id, the project_id (denormalized from test_cases so the state projection can be scoped per project), the event_type, the inserted_at timestamp, and attribution columns: actor_id (the account that performed the change when initiated by a user; null for system/automation writes) and alert_id (the automation_alerts.id whose action produced the change; null otherwise). Powers the test case history timeline and is the source of truth for test case state.
  • Build performance metrics

Non-Exportable Data

  • Encrypted passwords and authentication secrets
  • Account, SCIM-scoped account, and project token values and encrypted token hashes
  • Agent registration claim token hashes, claim-view token hashes, OTP hashes, issued API key values, and signed JWT values
  • Encrypted SSO client secrets for Okta and custom OAuth2 providers
  • Internal replication bookkeeping (e.g., bundles.artifacts_replicated_to_ch) used to drive the PG → ClickHouse artifacts backfill
  • Internal Kura shared secrets used by the control plane and Kura runtime extensions
  • Managed Kura operational traces retained in Tempo: service/resource metadata (including kura.tenant_id, kura.region, serving-node geo.country.iso_code / geo.region.iso_code) and request spans (including route, status, duration, and best-effort client geo.country.iso_code / geo.region.iso_code). This telemetry is used for platform observability and incident response, and is not part of the customer export archive today.
  • Encrypted GitHub App credentials (client_secret, private_key, webhook_secret on github_app_installations)
  • Slack bot access tokens and incoming-webhook URLs (treated as bearer credentials)
  • Outbound webhook endpoint URLs and signing secrets on webhook_endpoints (treated as bearer credentials — path/query tokens often appear in destination URLs)
  • GitHub-issued JIT runner configs (minted on demand for runner Pods at dispatch time and never persisted server-side)
  • Swift package registry mirror data: public package source archives, manifests, and package metadata stored in the registry S3 bucket under registry/swift/ and registry/metadata/, plus service-level sync coordination state held in S3 (registry/locks/, registry/state/sync_cursor.json) and server Oban queue records for the swift_registry_sync and swift_registry_release workers. These records are used to serve and maintain the public registry mirror through the stateless standalone registry service, are not tied to Tuist accounts, and are excluded from customer account exports.
  • Operator project-access audit trail: the reason-gated, time-boxed grants a Tuist operator obtains to access a customer account (project_access_requests / project_access_grants, storing the operator's email, the customer account handle, the access tier, the stated reason, return URL, approver, and lifecycle timestamps). This lives in the separate tuist-ops Postgres (not the customer-facing server database), is internal security/audit data rather than customer-owned content, and is retained for accountability. It is not part of the standard customer export archive, but the access history for a given account can be surfaced on a legal/transparency request.

Binary Files

All uploaded files associated with the account are included:

  • Cache artifacts: Build caches and compiled binaries, including Xcode, legacy CAS, module, and Gradle artifacts
    • Xcode cache branch tags: each Xcode cache action-cache entry may additionally carry the name of the git branch that produced it, stored on the entry's manifest in the account's Kura RocksDB store and replicated to that account's other Kura nodes. It is recorded so a cache view can be scoped to the project's trunk (entries from feature branches are kept out of the trunk view), and it is supplied by the client with the publication rather than read from the customer's repository. It has no retention of its own: it lives and dies with the entry it tags, under the same cache eviction, and is exported alongside that entry. Branch names routinely carry ticket ids or people's names, so they are treated as customer content rather than operational metadata.
  • App previews: iOS app bundles (.app/.ipa files) and icons
  • Run artifacts: uploaded result bundles, invocation records, result-bundle objects, and session archives stored under {account}/{project}/runs/{run_id}/
  • Shard bundles: Shared .xctestproducts bundles stored at {account_id}/{project_id}/shards/{shard_plan_id}/
  • Runner job log archives: gzipped runner logs stored at runners/{account_id}/{workflow_job_id}/runner.log.gz

Data Retention

The customer-facing summary of these windows lives in the public data retention guide at server/priv/docs/en/guides/server/data-retention.md.

On the hosted Tuist server, stored artifact blobs are subject to plan-based retention capped at 30 days. The active account plan determines the applicable window, with Air used when an account has no active subscription. Retention windows, in days, by plan:

ArtifactAir / Open SourceProEnterprise
Cache artifacts (Xcode compilation, legacy CAS, module, Gradle)143030
App preview builds and icons303030
Build archives303030
Run artifacts303030
Test run attachments303030
Shard bundles71430

Self-hosted artifact cleanup is configured independently for each artifact family:

Artifact familyEnvironment variable
Cache artifacts, including Xcode compilation, legacy content-addressable storage, module, and Gradle filesTUIST_CACHE_ARTIFACT_RETENTION_DAYS
App preview builds and iconsTUIST_APP_PREVIEW_RETENTION_DAYS
Current and legacy build archivesTUIST_BUILD_ARCHIVE_RETENTION_DAYS
Run artifactsTUIST_RUN_ARTIFACT_RETENTION_DAYS
Test run attachmentsTUIST_TEST_ATTACHMENT_RETENTION_DAYS
Shard bundlesTUIST_SHARD_BUNDLE_RETENTION_DAYS

Each variable accepts a positive integer day window. Leaving a variable unset or blank disables cleanup only for that artifact family. Self-hosted windows do not have the hosted 30-day cap, and these variables do not override the hosted policy.

Once an artifact is older than the applicable window, the cleanup process removes its binary from object storage only. The associated PostgreSQL and ClickHouse metadata rows, including build runs, test runs, command events, preview records, and shard plans, are kept so analytics and dashboards remain intact. The configurable policy does not change database retention rules.

Retention status is computed when cleanup runs. Cache artifacts use the object storage last_modified timestamp, while previews, current build archives, test attachments, and shard bundles use their database inserted_at timestamp. Run artifacts use the command event ran_at timestamp. Legacy build artifacts use the object storage last_modified timestamp. On the hosted Tuist server, legacy build artifacts whose account prefix no longer resolves to a live account use the Air build archive window.

Cache artifact cleanup scans instance-managed cache buckets and skips accounts configured with account-specific custom cache storage. Matching cache objects whose prefix no longer resolves to a current account are cleaned with the configured window. Database-backed cleanup for app previews, current build archives, run artifacts, test attachments, and shard bundles follows the account's current storage configuration. Legacy build archive cleanup scans the instance-managed artifact bucket. Package registry mirror objects and runner log archives are not covered by these configurable self-hosted retention variables. Runner log archives retain their separate 90-day policy.

Tuist stores per-account cleanup progress for database-backed artifact families so daily retention jobs can resume after previously-purged metadata rows without issuing repeated object-storage deletes. This is not a per-artifact purge ledger; retention is still derived from the timestamps and the applicable hosted or self-hosted policy above. An export reflects the artifacts present at export time; binaries already purged under these windows are no longer available, though their metadata and the account-level cleanup cursor are still exported.

Export Process

  1. Verify user identity and account ownership
  2. Collect all database records for the account and associated organizations
  3. Collect all binary files owned by the account
  4. Create compressed archive with JSON data files and binary files
  5. Provide secure download link

The archive contains everything needed to understand the account's complete data footprint within Tuist.