docs/tasks/remote-cache-protocol.md
[!WARNING] Remote build caching is experimental. This document defines protocol version 1, which generalizes the original task-only store contract into a shared action-cache protocol.
The protocol is a secure, content-addressed cache protocol for build actions and their outputs. Tasks, compiler invocations, build-system operations, and future adapters share the same transport, storage, authentication, and integrity model. It does not expose mise's local cache directories, manifests, or archive formats. Local storage is an implementation detail and may use archives or packs without changing the remote protocol.
Version 1 separates immutable build data from mutable discovery state:
This separation deduplicates content between actions, permits partial and parallel transfers, and allows a server to verify all referenced content before publishing a cache hit.
Version 1 uses HTTPS and HTTP semantics. Requests carrying authorization credentials require HTTPS,
except for loopback development servers (localhost, 127.0.0.0/8, and ::1). Clients may connect
to an unauthenticated non-loopback HTTP service after emitting a visible warning. This mode provides
neither confidentiality nor server authenticity: an on-path attacker can replace an action result
and its internally consistent CAS graph. Implementations may use HTTP/1.1, HTTP/2, or HTTP/3.
Every API request sends:
| Header | Value |
|---|---|
mise-cache-protocol | 1 |
mise-cache-namespace | The namespace for the operation, except on discovery endpoints |
The URL prefix /v1 is the protocol's major version. Compatible additions are advertised as
capabilities and do not require a new URL prefix. An incompatible wire or integrity change requires
a new major protocol; version 1 must not be used as an alias for an incompatible implementation.
Servers must ignore unknown JSON response fields. Clients must not send unknown request fields unless a negotiated capability permits them.
mise activates the Rust action-cache adapter only inside a task whose effective rust_cache
configuration enables it. Shell activation and commands run outside mise run do not
inject compiler wrappers, cache programs, or local proxy endpoints. One top-level mise run owns
the in-process cache session: it serves adapter requests, flushes uploads, and reports exact hits,
misses, and bytes before a successful run exits. Release builds never read or write action cache
entries. Compiler adapters define their prefetch inputs when they are introduced; the protocol does
not require unused prediction fields in task metadata.
Outside CI, mise action-cache sessions may read local and remote entries but do not upload. CI write authorization remains a server decision based on verified workload identity; a client-side mode is only defense in depth.
GitHub Actions protected-branch push jobs and GitLab protected-branch push pipelines may use the
configured write mode. Pull requests, tags/releases, unprotected branches, unknown CI systems, and
local runs are restricted to reads; a configured write-only client disables its remote rather than
silently broadening to read access. Tag and release pipelines do not activate compiler caching.
The JSON representation of a digest is:
{
"algorithm": "blake3",
"hash": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"size": 1234
}
Protocol version 1 defines blake3 and sha256. Servers advertise the algorithms they accept and
must support BLAKE3. Action descriptors and action-result keys always use BLAKE3; SHA-256 may be used
for other CAS objects when the server advertises it. A digest always covers the exact, uncompressed
bytes and includes their length. A server must reject malformed hashes, unsupported algorithms,
negative sizes, and content that does not match its declared digest.
Digest URL components use /v1/blobs/{algorithm}/{hash}/{size}. The algorithm and hash must match
the JSON representation, and size is an unsigned decimal integer.
GET /v1/capabilities requires no namespace and returns the protocol and server limits:
{
"protocol": { "major": 1, "minor": 0 },
"digest_algorithms": ["blake3", "sha256"],
"compressors": ["identity", "zstd"],
"action_kinds": {
"task": { "action_schema": 1, "metadata_schema": 1 }
},
"features": {
"batch": true,
"resumable_uploads": true,
"delegated_transfers": true
},
"limits": {
"max_batch_items": 1000,
"max_inline_blob_bytes": 1048576,
"max_blob_bytes": 107374182400
}
}
Each action_kinds entry advertises the action-descriptor and client-metadata schema versions the
server validates for that kind. Clients must not read or publish a non-task action unless the
server advertises the kind and the exact schema versions the client implements. Servers must reject
unadvertised kinds and unsupported schema versions. Compatible protocol additions may add kinds or
new schema versions without changing the major protocol version.
Clients must honor advertised limits and fall back from optional features. Servers return 426 Upgrade Required for unsupported major versions and include their supported major version in
mise-cache-protocol.
GET /v1/status is an operational health endpoint. A successful response means the API process is
live; it is not a substitute for capability negotiation or an authorization check.
Protocol JSON objects use UTF-8 and the JSON Canonicalization Scheme (RFC 8785) whenever their bytes are hashed. Duplicate object keys, invalid UTF-8, non-canonical encodings, and values that cannot be represented by the declared schema must be rejected.
An action descriptor contains a stable action kind and everything declared to affect its result.
Action schema version 1 defines task; compatible capabilities may add compiler and build-system
kinds without changing the CAS or action-result APIs:
{
"version": 1,
"kind": "task",
"task": "build",
"phase": "normal",
"run": [{ "task": "cargo build --release" }],
"args": [],
"shell": null,
"outputs": ["target/release/widget"],
"root": "crates/widget",
"source_hash": "blake3:...",
"dependency_keys": [],
"environment": { "PROFILE": "release" },
"command_inputs": [],
"vars": {},
"tools": ["core:[email protected]"],
"os": "linux",
"arch": "x86_64"
}
Arrays whose order has no action meaning must be sorted by the field defined by their schema. Version strings are opaque and are never semantically ordered. Secrets must not appear in an action descriptor. A task includes environment variables only when it declares them as cache inputs. Every action kind defines its own canonical fields and cacheability rules; a server may reject kinds it does not advertise.
source_hash binds the declared source paths and contents without uploading task inputs that are not
needed for cache-only operation. The canonical descriptor is stored in CAS. Its digest is the action
digest and the action-result URL key. Two clients that describe the same action must produce
identical canonical bytes.
A directory object has media type application/vnd.mise.cache-directory.v1+json:
{
"version": 1,
"directories": [
{
"name": "assets",
"digest": { "algorithm": "blake3", "hash": "...", "size": 321 },
"mode": 493
}
],
"files": [
{
"name": "widget",
"digest": { "algorithm": "blake3", "hash": "...", "size": 123456 },
"executable": true,
"mode": 493
}
],
"symlinks": [{ "name": "current", "target": "widget", "mode": 511 }]
}
Each node list is sorted by the UTF-8 bytes of name. Names must be a single path component and
must not be empty, ., .., contain a slash or NUL, or collide with another node. Absolute symlink
targets and targets that escape the declared output root must be rejected during restoration.
The portable metadata set is file contents, directory structure, symbolic links, executable state,
and the portable permission bits represented by mode. Owners, groups, timestamps, devices,
sockets, FIFOs, platform ACLs, and extended attributes are not restored. Hard links may be restored
as independent files. Unsupported source objects make the task result ineligible for remote caching
rather than being silently changed.
An action-result response and commit body have media type
application/vnd.mise.cache-action-result.v1+json:
{
"version": 1,
"action": { "algorithm": "blake3", "hash": "...", "size": 789 },
"output_root": { "algorithm": "blake3", "hash": "...", "size": 456 },
"metadata": { "algorithm": "blake3", "hash": "...", "size": 234 }
}
Only successful, cacheable action executions may be published. output_root is absent when an
action has no output files. metadata references canonical
application/vnd.mise.cache-client-metadata.v1+json containing typed client metadata. Task metadata
contains output roots, captured output, task identity, restored-byte estimate, and execution
duration. The metadata schema is part of the remote protocol and is independent of mise's local
cache manifests.
{
"version": 1,
"kind": "task",
"task_identity": "build:crates/widget",
"roots": ["target/release/widget"],
"output": [{ "stream": "stdout", "line": "built widget" }],
"restored_bytes": 123456,
"execution_duration_ns": 900000000
}
Each metadata kind has a versioned schema. Task root paths use forward slashes, are relative to the task working directory, and must satisfy the same path-safety rules as directory nodes. Task output entries preserve their declared order.
The metadata kind must equal the referenced action descriptor's kind. Servers reject a commit
with mismatched kinds before publication, even when both objects independently satisfy their schemas.
The action descriptor and every object reachable from the result must exist and validate before the result becomes readable.
Retention, last-access time, quota accounting, internal storage location, and server annotations are not part of the immutable action result.
POST /v1/blobs:missing accepts application/vnd.mise.cache-digests.v1+json:
{ "digests": [{ "algorithm": "blake3", "hash": "...", "size": 1234 }] }
It returns 200 OK with the subset not present in verified CAS:
{ "missing": [{ "algorithm": "blake3", "hash": "...", "size": 1234 }] }
The server must not disclose whether objects exist outside the request's readable namespaces or CAS visibility domain.
GET /v1/blobs/{algorithm}/{hash}/{size} returns 200 OK, or 404 Not Found when the caller cannot
read the object. The response includes Digest and Content-Length metadata. Servers may honor
Range and may return a negotiated Content-Encoding: zstd; the URL digest always describes the
uncompressed bytes.
A server advertising delegated transfers may return 307 Temporary Redirect to a short-lived HTTPS
URL. The redirect must grant access only to the requested immutable object. Clients must not forward
the cache service's Authorization header to the delegated host.
Clients verify the complete uncompressed digest before using downloaded content. A mismatch is a cache miss, emits a visible integrity warning, and must be reported to server telemetry when the reporting capability is enabled.
Small blobs may be sent directly with
PUT /v1/blobs/{algorithm}/{hash}/{size} and If-None-Match: *. The server returns:
201 Created after verifying and publishing new content;204 No Content when identical verified content already exists;400 Bad Request when the bytes do not match the digest;412 Precondition Failed when an immutable precondition fails;413 Content Too Large when an advertised limit is exceeded.Large or resumable uploads use an upload session:
POST /v1/uploads declares one or more digests.POST /v1/uploads/{id}/finalize verifies complete content and promotes it into CAS.Delegated uploads always target an isolated staging key, never a readable CAS key. A presigned S3 upload is therefore insufficient by itself: finalization must validate the declared digest before publication. Expired or abandoned staging objects are removed asynchronously.
GET /v1/action-results/{algorithm}/{hash}/{size} returns a committed action result or 404 Not Found. The namespace identifies the single read scope for that request. Clients configured with
multiple read scopes query them in policy order rather than sending an ambiguous multi-namespace
request.
PUT /v1/action-results/{algorithm}/{hash}/{size} commits an action result. It requires
If-None-Match: *. The server must atomically:
The response is 201 Created, 204 No Content for an identical committed result, 409 Conflict
when a different result already owns the action key, or 412 Precondition Failed when the immutable
precondition is absent or fails. Concurrent valid writers may upload identical CAS data, but only one
action-result commit wins.
GET /v1/action-manifests/{algorithm}/{hash}/{size} returns the latest canonical task action
manifest and a strong BLAKE3 ETag, or 404 Not Found. The URL key is the digest of the canonical
selector {"kind":"task_action_manifest","task":"<task identity>","version":1}; the server
rejects a manifest whose task identity does not produce that key.
PUT /v1/action-manifests/{algorithm}/{hash}/{size} creates a manifest with If-None-Match: * or
updates the version named by If-Match: "<etag>". The server returns 201 Created for a new
manifest, 204 No Content for an update, 412 Precondition Failed for a stale writer, and 428 Precondition Required when neither conditional header is present. After 412, clients read the
current manifest, merge predictions by invocation digest, and retry. This mutable index never
changes the immutability of action results or CAS objects.
Ordinary cache writers do not receive delete permission. Administrative deletion uses a separately authorized endpoint and must remove the action-result mapping before unreachable CAS data is garbage collected. A client-side cache clear operation must not imply authority to delete shared remote data.
The protocol supports bearer tokens, OIDC-derived tokens, mTLS, and trusted reverse-proxy identity. Authentication mechanism discovery is deployment configuration rather than CAS object metadata. Credentials must be scoped and redacted from diagnostics.
Servers authorize reads and writes independently. The standard CI policy is one shared namespace: protected branches may write it, while pull-request jobs may only read it. The server enforces this from verified OIDC claims such as repository, ref, event, and workflow identity; a client-provided remote mode is defense in depth, not the authorization boundary.
Immutable storage does not prevent cache poisoning by the first writer. OIDC-backed namespace authorization is therefore required even when the backing object store rejects overwrites. A single bucket credential shared by trusted and untrusted jobs is not a conforming security boundary.
401 Unauthorized means authentication is missing or invalid.403 Forbidden means the identity lacks permission for the requested namespace or operation.404 Not Found is a cache miss and must not reveal inaccessible objects.409 Conflict is an immutable action-result conflict.412 Precondition Failed is a missing or failed conditional-write requirement.422 Unprocessable Content is a validly encoded object with an invalid reference graph.426 Upgrade Required is a major-version mismatch.429 Too Many Requests and 5xx responses may be retried with bounded exponential backoff and
jitter, honoring Retry-After.Cache unavailability, malformed objects, missing referenced objects, and integrity failures normally degrade to a cache miss so local task execution can continue. Authentication, authorization, and integrity failures must still produce visible warnings; clients must not silently label them as ordinary misses. Deployments may enable a strict mode that makes selected failures fatal.
Idempotency keys may be sent for upload-session creation and other retryable POST operations.
Servers must bound their retention and scope them to the authenticated identity and namespace.
A conforming self-hosted server may use a filesystem, S3-compatible object storage, or another blob store. Clients communicate with the cache service rather than receiving general object-store credentials.
The official reference server is maintained separately at
jdx/mise-cache. It provides filesystem and S3-compatible blob
storage, PostgreSQL metadata, namespace-scoped authorization, Docker Compose, and a Helm chart. The
server remains a separate deployment and release lifecycle from the mise client while this document
is the canonical protocol specification.
A server using S3 should:
Object-store versioning, retention locks, and encryption are useful defense in depth but do not replace application authorization or digest verification.
The repository's compatibility suite is the executable definition of required version 1 behavior. It must cover capability negotiation, canonical object validation, namespace isolation, independent read/write authorization, missing-blob batches, streamed and resumable transfers, digest rejection, atomic action-result commits, immutable conflicts, delegated-transfer credential isolation, corruption handling, and retry semantics.
Servers may implement additional administrative, metrics, and health APIs outside /v1. Those APIs
must not weaken the version 1 cache invariants.