server/data-export.md
This document outlines all data that Tuist can export for customers upon legal request (GDPR, CCPA). Data is exported for the specific account associated with the requesting user or organization.
Data is provided in a single compressed archive containing:
Sensitive authentication data (passwords, tokens) are excluded from exports.
User profiles (email, active/inactive status, account settings, preferred locale)
User account lifecycle timestamps (users table): last_sign_in_at, the moment the user most recently authenticated by any means, recorded on session sign-in, application programming interface token use, and remember-me resumption, and written at most once every twelve hours per user; and disabled_at, the moment the account was suspended, which is set only for suspended accounts and is what the inactivity control measures its retirement window from. Both are exportable and are retained for the life of the account.
Organization records (account handle/name, creator relationship, and timestamps)
Organization memberships and roles (user, organization, role, and timestamps)
Account billing information and subscriptions
Account-level cache settings, including storage region, custom cache endpoint enablement, and cache write policy (members_and_tokens or tokens_only)
Free-tier reset timestamp (accounts.free_tier_reset_at): the moment an account's free-tier usage counter was last restarted, which is the point the monthly billable-run count is measured from instead of the start of the calendar month. It is set only for accounts on the Air plan, and only when a reset is applied by a Tuist operator or a deployment-time backfill, and it stops having any effect once the calendar month rolls past it. Exportable, holds no request or artifact detail, and is retained for the life of the account.
Account dashboard visibility (accounts.visibility, private or public): whether the account's non-administrative dashboards (projects list, runners, and usage) are readable by signed-out visitors. Defaults to private and is changed only by a Tuist operator from the internal operations panel. Members, webhooks, cache, billing, and settings remain members-only regardless of this setting.
API tokens, SCIM-scoped account tokens, and project tokens (existence, scopes, names, timestamps, and last-used metadata; token values, token hashes, and generated token secret previews are excluded)
Agent registration audit records (agent_registrations, agent_registration_events, agent_auth_jtis, and agent_auth_credentials tables): registration type/status, requested credential type, verified email address, claim attempt id, claim and user-code expiry timestamps, last claim-poll timestamp, registration / claim request / completion Internet Protocol address metadata, claimed user relationship, linked account-token id or access-token identifier, Identity Assertion Authorization Grant issuer/subject/audience/client metadata, replay-protection token identifiers, each issued access token's identifier, expiry, and revocation timestamp, append-only state-change events (created, assertion_issued, token_issued, token_revoked, claim_requested, user_code_minted, claim_confirmed, claim_resent, otp_failed, claimed, expired, revoked), event Internet Protocol address metadata, event metadata, and timestamps. The claim token hash, claim-attempt token hash, user-code hash, issued application programming interface key value, signed identity assertion, and signed access token are excluded from exports as authentication secrets.
Custom cache endpoint configurations (account_cache_endpoints table): account-specific custom cache endpoints, active regional Kura endpoint mirrors, and the internal peer (mTLS) addresses of enrolled self-hosted Kura nodes (rows with technology = :kura_self_hosted_peer, written at enrollment from each node's KURA_NODE_URL). These peer addresses are customer infrastructure hostnames used only for mesh peering and are never returned to the CLI as cache endpoints. Self-hosted peer rows whose node stops sending mesh heartbeats carry a deactivated_at timestamp (the peer is withheld from the mesh but kept; the node's recovery re-enrollment reactivates it) and are deleted once deactivated past the peer-certificate lifetime (30 days). Legacy account-level Kura global endpoint rows matching https://<lowercase-account-handle>.kura.tuist.dev are no longer stored separately; they are removed by the Kura global-endpoint cleanup migration.
Organization single sign-on configuration metadata, including the configured provider, provider URL, full OAuth2 endpoint URLs, login email domain, domain verification timestamp, automatic-enrollment policy, and the temporary legacy-domain compatibility marker used for configurations created before domain verification was available. The domain verification challenge token is excluded as authentication material.
Kura server records (kura_servers table): per-account Kura server configuration including region, image tag, public URL, status (including drain_pending and archived, the demand-driven lifecycle states for an instance whose cache is being reclaimed after a full inactivity window and one whose cache has been reclaimed), the warm-handoff move state (move_phase, target_node) recording whether the server is a steady-state instance or a transient move source/target and which box a move target is pinned to, the claim the instance is built at (storage_claim_size, the filesystem quota each replica holds, resolved from the account's plan or its claim override), and the observed-state projection (observed_image_tag, last_observed_at, last_ready_at) recording which image the backing cluster reports running, when it was last observed, and when its private endpoint was last observed ready
Kura sized claims (kura_placer_claims table): the disk claim automatic sizing chose for an account, which takes precedence over the plan constant in resolution order. One row per account holding the claim and its timestamps; the update timestamp marks the last resize, after which sizing evaluates only newer telemetry. Configuration derived from aggregate telemetry, retained until the account is deleted.
Kura claim sizing proposals (kura_claim_proposals table): the audit trail of automatic disk claim sizing. Each row records the account, the evidence region, the direction (grow or shrink), the claim at proposal time, the recommended claim, an evidence map of aggregate telemetry (window length, median shed age, occupancy), status (open, applied, dismissed, superseded), who resolved it (an operator handle or automatic), and timestamps. No cache contents; retained until the account is deleted.
Kura storage rollups (kura_storage_rollups table): day-grain per-account-region aggregates of the ClickHouse eviction and snapshot telemetry (eviction counts and bytes, shed-age and ring-span medians, snapshot counts, peak occupancy, ring budget), maintained hourly as the inputs of claim sizing. Aggregate storage accounting only; retained until the account is deleted.
Kura egress limit overrides (kura_egress_limits table): an optional per-account, per-region override of the egress floor and ceiling, in Mbit/s, the account's Kura cache pods are shaped at on a shared box, set by an operator when the region's numbers suit the account poorly. One row per account per region, holding the region identifier and either or both rates (a null column means that number still comes from the region) plus its creation and update timestamps. It is configuration rather than customer content, records no traffic and no cache contents, and is retained until the account is deleted.
Kura account-region policy history (kura_account_region_policies table): versioned explicit service-region assignments for accounts whose region cannot be derived from their plan and storage-region restriction. Each version includes the selected Kura region, assigning user when still present, operational reason, creation timestamp, and the timestamp when a newer assignment superseded it. The history is retained until the account is deleted so an assignment can be audited or restored.
Kura account-region lifecycle records (kura_account_region_lifecycles table): one row per account and Kura service region recording the timestamp of the account's most recent cache request that would route to Kura (last_cache_demand_at), the keep-warm exception flag, the drain and teardown timestamps of an archival in progress, and the outcome of the last completed archival (when it archived, how many bytes of enforced warm quota were reclaimed, how long the drain took, and when the account last returned from archive). The demand timestamp is derived from cache-endpoint resolution requests and is what decides whether the account's cache instance is provisioned or reclaimed. No request contents, artifact identifiers, or per-project detail are stored; the row is a single timestamp per account-region. Rows are retained until the account is deleted, because the timestamp has to outlive the reclaimed instance for a returning account to be brought back.
Kura deployment history (kura_deployments table): rollout attempts for the account's Kura servers including image tag, status, error messages, start/finish timestamps, and (for deployments minted by a fleet rollout) a reference to the fleet-wide rollout record that scheduled them
Kura fleet rollout wave assignments (kura_rollout_wave_assignments table): which wave of a fleet-wide Kura runtime rollout the account's servers were assigned to. The rollout records themselves (kura_rollouts, kura_rollout_events) and the per-server rollout scope (kura_rollout_servers: health baselines, soak eligibility, convergence timestamps) are Tuist-internal fleet-operations state — they carry no customer content beyond the server/account references above, but the account-linked rows are exportable on request
Self-hosted Kura credentials (kura_self_hosted_clients table): tenant-scoped credentials a customer uses to run self-hosted Kura nodes, including the public client_id, friendly name, last-used timestamp, and secret_last_four (the trailing four characters of the secret, kept as a masked-preview hint). The encrypted_secret_hash (Bcrypt hash of the client secret) is excluded from exports as an authentication secret, and the plaintext secret is never stored.
Registered Kura endpoints (kura_registered_endpoints table): client-facing cache endpoints reported by a customer's self-hosted Kura nodes via registration heartbeats, including the node id, region label, advertised HTTP URL, readiness, runtime version, traffic state, last-heartbeat timestamp, and lease expiry. These rows are lease-based operational state, refreshed on each heartbeat and removed when a node stops heartbeating.
GitHub App installation metadata (github_app_installations table): the installation ID GitHub assigned, the GitHub instance the App lives on (client_url, e.g. https://github.com or a customer's GitHub Enterprise Server host), the App's app_id/app_slug/client_id, and the GitHub-side management html_url. The accompanying client_secret, private_key (PEM), and webhook_secret are stored encrypted at rest and are excluded from exports as authentication secrets.
VCS connections (vcs_connections table): the link between a Tuist project and an external repository handle (provider, repository full name, the originating GitHub App installation, and the user who created the connection)
Federated authentication identities (oauth2_identities table): provider, provider-side subject identifier, issuer or organization identifier when applicable, and the linked user relationship
Artifact retention cursors (artifact_retention_cursors table): per-account cleanup progress for database-backed artifact families. Exports include the artifact type plus the last processed metadata cursor (after_inserted_at, after_id) used to avoid re-processing blobs that have already been purged from object storage. The run_session cursor covers all run artifact blobs stored under an expired run's artifact prefix.
(Internal Tuist-team JIT elevation tables previously documented here moved out of this server's Postgres entirely. They now live in the standalone tuist-ops service on its own CNPG cluster in the mgmt cluster. The data is operator-side audit about Tuist staff only — never customer data — and is out of scope for this server's data export. See tuist-ops/AGENTS.md for where it lives now.)
automation_alerts table): per-project test automations, including name, enabled flag, monitor type (flakiness_rate / flaky_run_count / reliability_rate / test_updated), evaluation cadence, baseline established timestamp, baseline generation, scoped evaluation cursor timestamp, and the trigger / recovery configuration as JSON. The configuration includes the comparison threshold, comparison operator, window_type (last_days for calendar windows or rolling for count-based windows), the day-string window (e.g. 30d) used in last_days mode, and the integer rolling_window_size used in rolling mode. Rolling trigger and recovery windows both support up to 1000 runs. Trigger and recovery action lists are stored alongside the configuration (state changes, label adds/removes, Slack channel references).automation_alert_revisions table): the append-only edit history for automation configuration. Each row records the automation, the user responsible when available, the source of the change, the fields changed with their previous and new values, a sanitized configuration snapshot, and the timestamp. Encrypted Slack webhook URLs are excluded from revision data.automation_alert_baseline_attempts table): durable per-generation progress for the first evaluation of an automation alert. Each row stores the alert relationship, fixed evaluation time, workflow state, keyset evaluation cursor, last published test case identifier, and timestamps. Rows are deleted with their alert.automation_alert_baseline_results table): temporary test case identifiers that matched an automation alert while its initial baseline is being evaluated and published. Rows are deleted after the baseline commits or with their baseline attempt.automation_alert_events table): per-test-case trigger and recovery records produced by automation alerts (alert id, baseline generation, test case id, status, timestamps).test_case_run_flaky_corrections table): durable operational ledger keyed by test case run identifier. Each row stores the project, test case, commit, correction state, and timestamps so retries cannot create additional correction work after the background job history is pruned.:runners feature flag (Tuist.FeatureFlags.runners_enabled?/1). PostgreSQL runner_concurrency_limits stores one row per account and runner platform (linux | macos) with the aggregate vcpus and memory_gb that active runners may consume concurrently. These configuration rows do not contain observed usage history and are deleted with their account.runner_profiles): account-scoped named bundles that customers reference from runs-on: as tuist-<name>. Columns: id (PK), account_id, name, platform (enum linux | macos — the runner OS the profile dispatches to), vcpus, memory_gb, xcode_version (string, macOS-only; pins the runner image's Xcode tag), protected (boolean), inserted_at, updated_at. The dispatch path resolves (account, requested-label) through these rows to the matching shape pool. The shape pool itself is operator-managed in Kubernetes and not customer data. Every account is auto-bootstrapped at sign-up with one protected linux row and one protected macos row; protected rows cannot be deleted, but their shape (and the macOS row's Xcode version) remains customer-editable.runner_claims): one row per currently-claimed workflow_job. Columns: workflow_job_id (GitHub's job id, PK), account_id, fleet_name (the RunnerPool name the claim is bound to), pod_name (the SA / Pod that won the claim), claimed_at, platform (linux | macos), vcpus, memory_gb, lifecycle_state (claimed during the JIT-mint window, running once the runner has registered with GitHub), runner_name (the GitHub-side runner label, populated when lifecycle_state flips to running), and executed_workflow_job_id (the job GitHub actually ran on this runner, learned from the workflow_job.in_progress / completed webhook's runner_name; NULL until GitHub proves the runner ran something, and may differ from workflow_job_id because GitHub assigns queued jobs to any label-eligible runner regardless of which one the server claimed the job for), and pod_missing_since (operational reconciliation state, not customer content: the first tick on which the claim's Pod was observed absent from a complete cluster read, NULL whenever the Pod is present. Tuist.Runners.Workers.PodReconciliationWorker requires the absence to persist before releasing the claim, so the column exists only between the first missed observation and the release, and is deleted with the row). The resource columns power atomic per-platform concurrency enforcement and current usage display. Rows are deleted when the executing runner's job completes, on release / stale recovery, or when the Pod stops; steady-state size is bounded by the number of in-flight runners.runner_job_completions): one row per workflow_job that reached a completed GitHub state. Columns: workflow_job_id (GitHub's job id, PK), account_id, conclusion, completed_at, inserted_at, and updated_at. Used as an atomic webhook ordering guard so late queued/waiting redeliveries cannot resurrect completed or cancelled runner work.runner_workflow_jobs): the control-plane twin of the ClickHouse runner_jobs view — one row per workflow_job, mutated in place through guarded status transitions (queued → claimed → running → completed | cancelled, with claim releases moving rows back to queued). Columns: workflow_job_id (GitHub's job id, PK), account_id, fleet_name, status, conclusion, the dispatch-candidate metadata carried from the webhook (platform, vcpus, memory_gb, repository, workflow_run_id, workflow_name, run_attempt, job_name, head_branch, head_sha, requested_dispatch_label), lifecycle timestamps (enqueued_at, claimed_at, started_at, completed_at), binding (pod_name, runner_name, executed_workflow_job_id), the downloadable-archive marker (log_archived_at), inserted_at, and updated_at. Same job metadata the ClickHouse runner_jobs entry below describes, held in Postgres so dispatch-path state transitions are transactional; rows are deleted with their account.runner_workflow_job_transition_events): short-lived outbox rows created in the same transaction as a runner_workflow_jobs status transition and deleted once a batch flusher replays them into ClickHouse runner_jobs. Columns: id (PK), workflow_job_id, account_id, payload (the ClickHouse row snapshot — the same fields as the lifecycle row above), and inserted_at. Transient replication state only; steady-state size is bounded by flusher lag (minutes), and rows cascade-delete with their account so no payload outlives it.runner_sessions): append-only record of every runner Pod we provisioned, keyed off the Pod lifecycle rather than the workflow_job's GitHub-reported runtime. Columns: id (PK), account_id, workflow_job_id (the job the Pod was claimed/minted for), executed_workflow_job_id (the job GitHub actually ran on this Pod's runner, learned from the workflow_job.in_progress / completed webhook; NULL until GitHub proves execution — its absence is how we identify a runner that was never handed work — and may differ from workflow_job_id. This is the durable binding that outlives the Pod, and it is what attributes runner machine metrics to the job that actually ran), fleet_name, selected runner resources (platform, vcpus, memory_gb; NULL on historical rows created before resource-aware billing), billing_multiplier (the machine factor in basis points, relative to its own platform's baseline machine, frozen when the session opened so a later rate-card change cannot reprice usage that already happened; NULL on rows created before compute-unit metering, which fall back to the current catalog), job_started_at / job_ended_at (GitHub's own execution window for the workflow job, recorded from the workflow_job.completed webhook; this is the billable window, since the Pod's own started_at / ended_at also cover VM boot before the job and teardown after it. NULL when no completion webhook evidenced execution, in which case the session bills nothing), pod_name, runner_name, repository (denormalized owner/name handle from the workflow_job for billing-page scope filters), workflow_name (denormalized for the same), started_at (claim-win — proxy for Pod creation), ended_at (normally set by the runners-controller via POST /api/internal/runners/pods/stopped when it observes the Pod's container terminate; NULL while still in flight. When that report is missed, Tuist.Runners.Workers.PodReconciliationWorker closes the row once the Pod is confirmed absent from a complete cluster read. It prefers the workflow_job's terminal completed_at from ClickHouse runner_jobs — the runner's real end — bounded as GREATEST(started_at, LEAST(completed_at, now, started_at + 6h)); when the job never reached a terminal state it falls back to LEAST(now, started_at + 6h), the same bound the billing query already clamps an open session to. Either way the estimate never bills more than the open row would have). Drives metered-compute invoicing via Tuist.Runners.Billing. A job that is dispatched again after a retry opens another session row, so the customer is billed for every Pod they actually held.runner_interactive_sessions): session-scoped access grants for interactive runner debugging. Columns: id (PK), account_id, workflow_job_id, pod_name, fleet_name, kind (vnc or shell), state (requested, ready, active, or closed), token_hash (SHA-256 hash of the short-lived browser/gateway session token; the raw token is not persisted), requested_by_user_id (nullable if the user is deleted), connected_at, closed_at, expires_at, last_activity_at, relay_host, relay_port, relay_ready_at, and close_reason. Stores account/user/workflow/pod access metadata for authorization, lifecycle, audit, and support. Relay host/port are transient server-side infrastructure coordinates for the WebSocket bridge; the table does not store Tart VNC passwords.runner_interactive_session_connections): per-browser WebSocket connection lifecycle rows for interactive runner sessions. Columns: id (PK), interactive_session_id, connection_id (opaque server-generated connection marker), connected_at, disconnected_at, inserted_at, and updated_at. Used to keep a VNC grant open until all viewers disconnect; does not store VNC credentials or raw browser session tokens.runner_volume_affinities): dispatch-time hint recording which Mac host an account's runner jobs last ran on, so dispatch can prefer hosts that already hold the account's cache volume. Columns: id (PK), node_name (the cluster Node / Mac host name — infrastructure identifier, not customer content), account_id, volume_name (tuist-cache reserved), last_run_at, inserted_at, and updated_at. One row per (host, account, volume); rows are pruned 14 days after last_run_at. Operational scheduling metadata only — no build content, cache artifacts, or credentials.runner_volume_heads): the canonical pointer to an account's current warm cache set for the Mac runner fleet, so a host whose local cache master lags can converge to the latest. Columns: id (PK), account_id, volume_name (tuist-cache reserved), generation (monotonic version counter advanced on each accepted promote), tree_digest (the sorted cache entry-name hash that fingerprints the warm set — a digest, not cache content), node_name (the host that last published the HEAD — infrastructure identifier, observability only), inserted_at, and updated_at. One row per (account, volume), fast-forwarded on promote (a promote advances the HEAD only if it built on the current generation). Coordination metadata only — no build content, cache artifacts, or credentials.runner_volume_master_orphans): transient bookkeeping for cache-volume master objects a runner uploaded but whose fast-forward promote the server rejected (a stale base another host advanced past). Because the guest uploads the content-addressed image before the compare-and-swap bump, a rejected promote leaves an object with no HEAD pointing at it; a row here lets PruneVolumeMasterOrphanWorker reclaim it after a grace window instead of leaving it until account deletion. Columns: id (PK), account_id, volume_name (tuist-cache reserved), tree_digest (the sorted cache entry-name hash — a digest, not cache content), inserted_at, and updated_at. A row is deleted the moment the same digest is accepted as HEAD or its object is reclaimed. Coordination/cleanup metadata only — no build content, cache artifacts, or credentials.runner-volume-masters/{account_id}/tuist-cache/{inventory_digest}.image): a per-account snapshot of the account's warm Tuist cache set — compiled Manifests / ProjectDescriptionHelpers / Plugins and the binary-cache index a runner materializes to start a build warm. The object is a sparse APFS disk image (preserving the artifact-signature xattrs, symlinks, and modes natively). Keyed by the set's content (inventory digest) so it is immutable and a concurrent promote never overwrites the object a published HEAD points at; an account may therefore hold several such objects across cache generations. Published by a runner after a successful, cache-changing job and downloaded by a lagging host to converge toward the HEAD above. This IS customer-derived build cache content (no credentials). It is account-scoped under the runner-volume-masters/{account_id}/ prefix. Because it is keyed by account_id (not the account handle), it is NOT swept by the handle-based artifact retention; instead Accounts.delete_account!/1 explicitly deletes the whole prefix on account deletion (best-effort), so it does not outlive the account.runner_jobs, ReplacingMergeTree on workflow_job_id): one logical row per workflow_job carrying the full lifecycle from queued → claimed → running → completed. Columns include the GitHub correlation fields (workflow_job_id, workflow_run_id, run_attempt, workflow_name, job_name, head_branch, head_sha, repository), selected runner resources (platform, vcpus, memory_gb), lifecycle state (status, conclusion), timestamps (enqueued_at, claimed_at, started_at, completed_at, updated_at), binding (pod_name, runner_name), and the downloadable-archive marker (log_archived_at, set once the gzipped log archive is uploaded — see the runner job log archives entry below). Per-step data lives in runner_job_steps. Powers the customer-facing "queued / running / recent runs" surfaces and preserves the resource shape selected when the job was enqueued.runner_job_steps, ReplacingMergeTree on (workflow_job_id, number)): one row per workflow_job step, captured from the workflow_job.completed webhook. Columns: workflow_job_id, account_id, number (the step's 1-based position), name, status, conclusion, started_at, completed_at, and inserted_at (the RMT version). Powers the job detail page's Steps card and step-level analytics (failure rates per step name, p95 of Build duration, slowest steps in a workflow).runner_job_logs, ReplacingMergeTree on (workflow_job_id, line_number)): the runner container's captured stdout, one row per line. Columns: workflow_job_id, account_id, line_number, ts (the per-line timestamp GitHub stamps in the log payload), message (the log text), and inserted_at (the RMT version). Populated by Tuist.Runners.Workers.FetchLogsWorker, which streams the job's log from GitHub's Actions Logs API after the workflow_job.completed webhook and inserts batched lines. Surfaced on the job detail page's Logs tab (per-step slicing via ##[group]Run markers, full-log search). Retained for 90 days.runner_job_machine_metrics, ReplacingMergeTree on (workflow_job_id, timestamp)): per-sample resource usage of the runner Pod/VM while a workflow_job executes, one row per snapshot. Columns: workflow_job_id, account_id, timestamp (epoch seconds the sample was taken), cpu_usage_percent, cpu_iowait_percent (0 on macOS, which has no iowait accounting), memory_used_bytes, memory_total_bytes, network_bytes_in, network_bytes_out, disk_used_bytes, disk_total_bytes, and inserted_at (the RMT version). Unlike logs, these have no GitHub-side source — they describe our infrastructure's runner; the runner metrics collector samples each running job's Pod and POSTs batches to POST /api/internal/runners/jobs/:workflow_job_id/metrics (authenticated with the runners-controller's in-cluster ServiceAccount token). Surfaced on the job detail page's Overview charts and Metrics tab. Retained for 90 days.runner_concurrency_sessions, ReplacingMergeTree on ingested_at): analytics replica of the interval each runner session occupied its account's concurrency budget, powering the Concurrency card on the runners dashboard. Columns: id (the Postgres runner_sessions primary key), account_id, the resolved machine shape (platform, vcpus, memory_gb), the claim interval (started_at, released_at — the moment the slot was freed, or the six-hour runner-session ceiling past which it cannot still be held), source_updated_at (mirrors the Postgres row's updated_at, and is the tailer's resume point), and ingested_at. Derived entirely from Postgres runner_sessions and runner_job_completions by Tuist.Runners.SessionReplication; contains no field those two do not already hold, and no GitHub correlation metadata. Rebuilt by truncating and letting the tailer re-copy, so it is a read model rather than a system of record.runners/{account_id}/{workflow_job_id}/runner.log.gz): once FetchLogsWorker finishes ingesting a job's lines, Tuist.Runners.Workers.ArchiveLogsWorker stream-gzips them into a single object (multipart-uploaded so a large log never materialises in memory) and stamps log_archived_at on the runner_workflow_jobs lifecycle row, which reaches the ClickHouse runner_jobs row through the transition outbox. The "Download logs" action redirects to a presigned URL for that object; the button is hidden while log_archived_at is NULL. Same content as runner_job_logs, rendered to plain text (<ISO timestamp> <message> per line). Retained for 90 days, matched to the row-level TTL by Tuist.Runners.Workers.PruneArchivedLogsWorker (a daily Oban cron) which both deletes the S3 object and clears log_archived_at. Jobs archived before runner_workflow_jobs existed have no lifecycle row, so for those the stamp is written straight to runner_jobs.RunnerPool CR + Pods / ServiceAccounts in the tuist-runners namespace): operational metadata only — pool name, dispatch label, image, replica count, owner labels on Pods. Reconciled by the runners-controller.accounts.runner_trial_started_at, accounts.runner_trial_ended_at): whether an account is on a runner trial, meaning its runner usage is metered and reported but not billed. A trial is the absence of a runner item on the account's Stripe subscription rather than a credit balance, so these two timestamps are the whole record: runner_trial_started_at set with runner_trial_ended_at NULL means the trial is running, and a set runner_trial_ended_at records when it was cancelled and runner usage became billable. Retained for the life of the account as the audit trail for why a period of runner usage went unbilled.billing.credit_grant, no Tuist-side table): prepaid runner time is held as a money-denominated Stripe billing credit grant scoped to the runner metered Prices, not as a minutes ledger in Tuist. Tuist stores no balance of its own: it creates the grant when the minutes are sold and reads the remaining balance back from Stripe on demand, so an export of Tuist's databases contains no prepaid balance and the authoritative record is the Stripe customer. Runner trials are not credit grants; they are the account timestamps described in the entry above. Selling minutes creates two Stripe objects together: a pending invoice item, so the charge rides the customer's ordinary monthly invoice rather than arriving as a separate bill, and the credit grant itself, so the account can run on the minutes immediately rather than waiting for that invoice to be paid. If the grant fails the charge is withdrawn with it, and Tuist.Billing.Workers.CreateRunnerPrepaidGrantWorker remains as a backstop on invoice.paid for a prepaid line that reaches an invoice with no grant against it. Each grant carries Tuist-set metadata identifying it (tuist_runner_credit = prepaid), the Stripe object it was funded from (tuist_prepaid_invoice_line_id, holding the invoice item id for a grant made at sale time or the invoice line id for one made by the backstop, with tuist_prepaid_invoice_id set only in the latter case), and the terms applied (tuist_prepaid_paid_cents, tuist_prepaid_funding_ratio_bp). Grants expire at the end of the billing period they were bought for, capped at a month, and do not roll over; setting an account's minutes replaces what it holds, voiding the existing grants and deleting their pending charges before creating the new one. Runner usage is always reported to Stripe gross at the on-demand rate for prepaid and pay-as-you-go customers alike, so runner_sessions above is unaffected by whether an account is prepaid.runner_name (the GitHub-side runner label) is recorded in runner_jobs.webhook_endpoints table): per-account HTTPS destinations subscribed to Tuist events. Exports include endpoint name, subscribed event types (test_case.created, test_case.updated, preview.created, preview.deleted), and timestamps. The endpoint URL and signing secret are stored Vault-encrypted at rest and are excluded from exports as authentication material.webhook_delivery_attempts table, ClickHouse): per-attempt audit log generated by Tuist.Webhooks.Workers.DeliveryWorker. Exports include the event id and type, attempt number, delivery status (delivered / failed), the JSON payload Tuist sent (request_body), the response status / headers / body returned by the upstream, any error string, the duration in milliseconds, and the attempt timestamp. Stored in a MergeTree partitioned monthly; retained indefinitely so the dashboard can surface historical deliveries for debugging.The following data is stored in ClickHouse for analytics purposes:
build_runs table): Complete build execution data including duration, status, cache statistics, CI metadata, git information, and custom tagsbuild_issues table): Compilation warnings and errors from buildsbuild_files table): Individual file compilation metricsbuild_targets table): Target/module build performancecacheable_tasks table): Xcode cache task analytics with hit/miss statuskura_usage_events table): per-window cache traffic rollups pushed by Kura nodes, one row per (tenant, project namespace, traffic plane, direction, operation, protocol, artifact kind, window). Columns: event_id (deterministic dedup key), resolved account_id/project_id, node_id, region, the rollup dimensions, bytes, request_count, window_start, window_seconds, and inserted_at. Aggregate traffic accounting only; no artifact identifiers or request contents.kura_eviction_events table): one row per cache segment a managed Kura node evicted under ring size pressure, pushed with the node's usage batches. Columns: event_id (deterministic per evicted segment), resolved account_id, node_id, region, segment_id (an internal storage identifier, not customer content), reason, evicted_at, segment_created_at, newest_content_at, artifact_count, bytes, and inserted_at. Drives automatic disk claim sizing (the difference evicted_at - newest_content_at is how soon after being written cached content was shed). Aggregate storage accounting only; no artifact identifiers or cache contents. Exportable as account records, bounded by a 180-day table TTL: an export covers the trailing 180 days and nothing earlier, since the rows are gone. Rows whose tenant handle did not resolve to an account land with account_id 0 and belong to no account, so they appear in no export.kura_storage_snapshots table): periodic ring-occupancy snapshots from managed Kura nodes. Columns: event_id, resolved account_id, node_id, region, captured_at, ring_budget_bytes, desired_segment_count, live_segment_count, live_segment_bytes, oldest_segment_created_at, newest_content_at, and inserted_at. The shrink signal of automatic claim sizing and the per-pod disk view on the operator account page. Aggregate storage accounting only. Exportable on the same terms as the eviction events above: a 180-day table TTL bounds the window, and unresolved-tenant rows carry account_id 0 and belong to no account.cas_outputs table): Content-addressable storage upload/download records, including the denormalized project id used for project-scoped analyticsmodule_cache_outputs table): Per-artifact module (binary) cache download/upload records for a command run, used for module cache network analytics. Columns: command_event_id, project_id, operation (download/upload), name (target name), hash (content hash), size (on-disk bytes), compressed_size (bytes transferred over the wire), duration (ms), and inserted_at. Keyed by command_event_id because module cache artifacts are fetched during tuist generate, before any build run exists.command_events_by_name_ran_at materialized view + target table): A full copy of command_events re-sorted by (project_id, name, ran_at) that backs the name-filtered recent-runs lists (cache runs, generate runs, the /runs API with a name filter). Carries the same columns as command_events; contains no data not already covered by the source command_events table.shard_plans table): Test sharding plan data including reference, shard count, and granularityshard_plan_modules table): Per-shard module assignments with estimated durationsshard_plan_test_suites table): Per-shard test suite assignments with estimated durationsshard_runs table): Per-shard execution results with status and durationtest_runs table): Includes shard_plan_id linking test results to their shard plan. The proj_by_project_ran_at projection stores project_id, run id, duration, status, and ran_at, ordered by project and run time for recent-run dashboard queries. It contains no data beyond the source test_runs rows and is exported through those source records rather than as a separate dataset.test_run_errors table): Run/target-level entries modelled separately from test failures. Two categories are stored: errors where the test runner itself errored (e.g. a target whose .xctest bundle could not be loaded), and Swift Testing issues recorded while no test was running, which xcresult reports under an "Issues recorded without an associated test or suite" case. For the latter, message carries the recorded issue and can therefore contain a source file path, a line number, and the asserted expression from the customer's test code. Columns: id, test_run_id, project_id, module_name (the test target, empty for run-level), message, and inserted_at.test_case_failures table): Per-failure detail for a failing test case run. Columns: id, test_case_run_id, test_case_run_argument_id, message, path, line_number, issue_type (error_thrown, assertion_failure, issue_recorded, unknown), and inserted_at. message, path, and line_number are taken from the xcresult, so they can contain a source file path and the asserted expression from the customer's test code.bundles table): App bundle metadata (name, app bundle id, version, install/download size, supported platforms, type, git ref/branch/commit).artifacts table): App bundle artifact tree (paths, sizes, SHA hashes, parent/child hierarchy) per uploaded bundle.bundle_size_approvers table): Per-project allowlist of GitHub usernames permitted to accept a bundle size increase from a pull request check run. Columns: id (PK), project_id, github_handle (stored lowercased, kept for display), github_id (GitHub's numeric account id, what authorization compares), inserted_at, updated_at. Rows are only consulted while the project's bundle_size_approval_policy is the selected-users policy, but they are kept when it changes, so a project that switches away and back keeps the list it had.bundle_size_approvals table): Record of who accepted a bundle size increase that exceeded a threshold. Columns: id (PK), bundle_id, project_id, approved_by_handle (the GitHub username that pressed the button), approved_by_user_id (the Tuist user, when that GitHub account is linked to one; null otherwise), inserted_at, updated_at. Lives in Postgres rather than on the ClickHouse bundles row because updating a MergeTree row is an asynchronous mutation.test_case_runs_active_daily_stats materialized view): Exact daily presence rows per (project_id, date, is_ci, test_case_id) derived from test_case_runs. Powers the Test Cases analytics chart; contains no data not already covered by the source test_case_runs table.test_case_runs_by_commit materialized view): Slim projection of test_case_runs ordered by (project_id, git_commit_sha, scheme, is_ci, status, id), including the latest is_flaky value used to avoid repeating a historical run correction. Powers the cross-run flakiness lookup; contains no data not already covered by the source test_case_runs table.test_case_run_daily_stats_per_case materialized view): AggregatingMergeTree keyed on (project_id, date, test_case_id) with count, sumState(toUInt8(is_flaky)), and sumState(toUInt8(status = 'success')) aggregate states. Powers the test automation engine's per-test windowed comparisons; contains no data not already covered by the source test_case_runs table.test_case_duration_daily_stats_per_case materialized view): AggregatingMergeTree keyed on (project_id, test_case_id, date, is_ci) with uniqExactState(id), avgState(duration), and quantileState(0.5|0.9|0.99)(duration) aggregate states. Powers the Test Cases listing's duration column and its ordering; contains no data not already covered by the source test_case_runs table.test_case_runs.is_default_branch column): Boolean recording whether a run's git_branch matched the project's default_branch at the time the run was ingested. Derived at ingestion from data already stored in test_case_runs.git_branch and projects.default_branch; contains no data not already covered by those columns.test_case_runs_recent_per_case and test_case_runs_recent_{100,250,500,750}_per_case tables): AggregatingMergeTree tables keyed on (project_id, test_case_id) with recent-run aggregate states derived from test_case_runs. The 100-run table is still maintained by its materialized views but is no longer read by automations, which now serve every rolling window from the packed aggregate below. The 250, 500, 750, and 1,000-run tables are retained as historical derived data; their materialized views are removed, so they receive no new rows, and automatic background merges are disabled. These tables contain no data not already covered by the source test_case_runs table.test_case_run_daily_stats_per_case_default_branch materialized view): AggregatingMergeTree keyed on (project_id, date, test_case_id) with count, sumState(toUInt8(is_flaky)), and sumState(toUInt8(status = 'success')) aggregate states, restricted to runs whose is_default_branch is set. Same grain and columns as test_case_run_daily_stats_per_case; it exists so a reliability or flakiness automation scoped to the default branch narrows its read rather than filtering after it. Not seeded; populated forward by its materialized view from the point it was created. Contains no data not already covered by the source test_case_runs table.test_case_runs_default_branch_recent_window_per_case table): AggregatingMergeTree keyed on (project_id, test_case_id), partitioned on a hash of project_id, holding a groupArraySorted(2000) state of the latest default-branch runs per test case, encoded identically to test_case_runs_recent_window_per_case. Serves rolling automation windows scoped to the default branch, which cannot be read out of the all-branch bucket because the default branch is a minority of most projects' runs. Not seeded; populated forward by its materialized view from the point it was created. Contains no data not already covered by the source test_case_runs table.test_case_runs_recent_window_per_case table): AggregatingMergeTree keyed on (project_id, test_case_id), partitioned on a hash of project_id, holding a groupArraySorted(2000) state of the latest runs per test case. Each run is encoded as one Int64 combining its ran_at timestamp in microseconds with its is_flaky and success flags. Serves every rolling automation trigger window; seeded from the 100-run table and maintained forward by its materialized view. Contains no data not already covered by the source test_case_runs table.test_cases.last_ran_at_ci, test_cases.last_ran_at_local columns): Denormalized timestamps tracking the most recent CI and local run per test case. Maintained by the ingestion path on every test run; contains no data not already covered by the source test_case_runs table.test_case_states table): Control-plane state per test case — state (enabled/muted/skipped) and is_flaky — ordered by (project_id, test_case_id, inserted_at). Derived data: it is a projection of test_case_events, maintained by the test_case_states_mv materialized view, and contains no information not already in that table. Each row records only the column its source event affected and leaves the other null, so the current value of each column is the latest non-null one. A test case with no row here is enabled and not flagged. The state and is_flaky columns on test_cases are legacy leftovers of the same data and are no longer read.test_case_current_states table): AggregatingMergeTree keyed on (project_id, test_case_id) holding argMaxIf aggregate states for state (enabled/muted/skipped) and is_flaky. Derived data: it is the pre-aggregated read form of test_case_states, maintained by the test_case_current_states_mv materialized view off that ledger, and contains no information not already in test_case_states (and transitively test_case_events). The current value of each column is the latest non-null one; a test case with no row here is enabled and not flagged. Retention and export behavior match the source ledger.test_case_events table): Audit log of state changes on a test case — first_run, marked_flaky/unmarked_flaky, muted/unmuted, skipped/unskipped. Each row records the test_case_id, the project_id (denormalized from test_cases so the state projection can be scoped per project), the event_type, the inserted_at timestamp, and attribution columns: actor_id (the account that performed the change when initiated by a user; null for system/automation writes) and alert_id (the automation_alerts.id whose action produced the change; null otherwise). Powers the test case history timeline and is the source of truth for test case state.Application logs are shipped off the originating host to a third-party telemetry platform and are retained there for thirty days, after which they are deleted by the platform's retention policy. They are operational records rather than account records, so they are not part of the standard export; they can be searched and extracted on request for the retention window while they exist.
Each request record carries the acting account handle, the selected account and project where one is in scope, the request method, path, response status and duration, a correlation identifier, and the originating client network address. The network address is recorded so that authentication failures, which carry no account, can be attributed to a source at all; it is the only personal data in these records beyond the account handle. Sign-in attempts additionally carry an explicit outcome. Logs deliberately exclude credentials, tokens, and request bodies.
bundles.artifacts_replicated_to_ch) used to drive the PG → ClickHouse artifacts backfillkura.tenant_id, kura.region, serving-node geo.country.iso_code / geo.region.iso_code) and request spans (route, status, and duration; Kura does not geolocate client IPs, so no client location is derived or retained). This telemetry is used for platform observability and incident response, and is not part of the customer export archive today.{job="tuist-macos-tart-kubelet"}, with labels instance — the Mac host's cluster Node / CAPI Machine name — plus env and level). These are tart-kubelet's own launchd stdout, shipped off each Mac mini by the host-side agent in infra/macos-log-shipper because a Pod on a macOS Node is a Tart VM and no in-cluster collector can read the host's filesystem. The lines describe our infrastructure's behavior — Pod-to-VM reconciliation, node registration, VM garbage collection, and per-account cache-volume materialize / promote / converge decisions — and the account-correlated fields they carry are: the Tuist account id (account, read from the tuist.dev/runner-account label the server stamps at dispatch — the id, not the handle), the Tart VM name (vm, formed as <namespace>-<pod name>, both Kubernetes identifiers), and cache-volume coordination values (generation counters and cache inventory digests, which are hashes of entry names rather than cache content). No build output, cache artifacts, source, repository or branch names, or credentials are logged. Retained under the Grafana Cloud Logs retention configured for the observability stack (the same window as the clusters' pod logs; a stack-level setting, not configured in this repository). Like the Kura traces above, this is operational telemetry rather than customer-owned content and is not part of the standard customer export archive, but because lines carry the account id, the entries relating to a given account can be surfaced on a legal or transparency request.client_secret, private_key, webhook_secret on github_app_installations)webhook_endpoints (treated as bearer credentials — path/query tokens often appear in destination URLs)registry/swift/ and registry/metadata/, plus service-level sync coordination state held in S3 (registry/locks/, registry/state/sync_cursor.json) and server Oban queue records for the swift_registry_sync and swift_registry_release workers. These records are used to serve and maintain the public registry mirror through the stateless standalone registry service, are not tied to Tuist accounts, and are excluded from customer account exports.open-graph-images/{content_hash}.jpg. These are public, content-addressed derivatives of Tuist-owned marketing and documentation content. They contain no account or customer data, are retained as a reusable rendering cache until operationally evicted, and are excluded from customer account exports.project_access_requests / project_access_grants, storing the operator's email, the customer account handle, the access tier, the stated reason, return URL, approver, and lifecycle timestamps). This lives in the separate tuist-ops Postgres (not the customer-facing server database), is internal security/audit data rather than customer-owned content, and is retained for accountability. It is not part of the standard customer export archive, but the access history for a given account can be surfaced on a legal/transparency request.All uploaded files associated with the account are included:
{account}/{project}/runs/{run_id}/.xctestproducts bundles stored at {account_id}/{project_id}/shards/{shard_plan_id}/runners/{account_id}/{workflow_job_id}/runner.log.gzThe customer-facing summary of these windows lives in the public data retention
guide at server/priv/docs/en/guides/server/data-retention.md.
On the hosted Tuist server, stored artifact blobs are subject to plan-based retention capped at 30 days. The active account plan determines the applicable window, with Air used when an account has no active subscription. Retention windows, in days, by plan:
| Artifact | Air / Open Source | Pro | Enterprise |
|---|---|---|---|
| Cache artifacts (Xcode compilation, legacy CAS, module, Gradle) | 14 | 30 | 30 |
| App preview builds and icons | 30 | 30 | 30 |
| Build archives | 30 | 30 | 30 |
| Run artifacts | 30 | 30 | 30 |
| Test run attachments | 30 | 30 | 30 |
| Shard bundles | 7 | 14 | 30 |
Self-hosted artifact cleanup is configured independently for each artifact family:
| Artifact family | Environment variable |
|---|---|
| Cache artifacts, including Xcode compilation, legacy content-addressable storage, module, and Gradle files | TUIST_CACHE_ARTIFACT_RETENTION_DAYS |
| App preview builds and icons | TUIST_APP_PREVIEW_RETENTION_DAYS |
| Current and legacy build archives | TUIST_BUILD_ARCHIVE_RETENTION_DAYS |
| Run artifacts | TUIST_RUN_ARTIFACT_RETENTION_DAYS |
| Test run attachments | TUIST_TEST_ATTACHMENT_RETENTION_DAYS |
| Shard bundles | TUIST_SHARD_BUNDLE_RETENTION_DAYS |
Each variable accepts a positive integer day window. Leaving a variable unset or blank disables cleanup only for that artifact family. Self-hosted windows do not have the hosted 30-day cap, and these variables do not override the hosted policy.
Once an artifact is older than the applicable window, the cleanup process removes its binary from object storage only. The associated PostgreSQL and ClickHouse metadata rows, including build runs, test runs, command events, preview records, and shard plans, are kept so analytics and dashboards remain intact. The configurable policy does not change database retention rules.
Test attachments created before attachment metadata included a test-run identifier are excluded from automated deletion. Their binary and metadata remain available and are included in an export. This avoids resolving account ownership by scanning the full test-case-run history during retention cleanup.
Retention status is computed when cleanup runs. Cache artifacts use the object
storage last_modified timestamp, while previews, current build archives, test
attachments, and shard bundles use their database inserted_at timestamp. Run
artifacts use the command event ran_at timestamp. Legacy build artifacts use
the object storage last_modified timestamp. On the hosted Tuist server, legacy
build artifacts whose account prefix no longer resolves to a live account use the
Air build archive window.
Cache artifact cleanup scans instance-managed cache buckets and skips accounts configured with account-specific custom cache storage. Matching cache objects whose prefix no longer resolves to a current account are cleaned with the configured window. Database-backed cleanup for app previews, current build archives, run artifacts, test attachments, and shard bundles follows the account's current storage configuration. Legacy build archive cleanup scans the instance-managed artifact bucket. Package registry mirror objects and runner log archives are not covered by these configurable self-hosted retention variables. Runner log archives retain their separate 90-day policy.
Tuist stores per-account cleanup progress for database-backed artifact families so daily retention jobs can resume after previously-purged metadata rows without issuing repeated object-storage deletes. This is not a per-artifact purge ledger; retention is still derived from the timestamps and the applicable hosted or self-hosted policy above. An export reflects the artifacts present at export time; binaries already purged under these windows are no longer available, though their metadata and the account-level cleanup cursor are still exported.
The archive contains everything needed to understand the account's complete data footprint within Tuist.