docs/tasks/remote-cache-protocol.md
[!WARNING] Remote task caching is experimental and is not yet configurable. This document defines the first public protocol, version 1. There is no earlier remote-cache protocol to preserve.
The protocol is a secure, content-addressed cache protocol for task inputs and outputs. It does not expose mise's local cache directory, manifest, or archive format. Local storage is an implementation detail and may use archives or packs without changing the remote protocol.
Version 1 separates two kinds of immutable data:
This separation deduplicates content between tasks, permits partial and parallel transfers, and allows a server to verify all referenced content before publishing a cache hit.
Version 1 uses HTTPS and HTTP semantics. Requests carrying authorization credentials require HTTPS,
except for loopback development servers (localhost, 127.0.0.0/8, and ::1). Clients may connect
to an unauthenticated non-loopback HTTP service after emitting a visible warning. This mode provides
neither confidentiality nor server authenticity: an on-path attacker can replace an unsigned action
result and its internally consistent CAS graph. Implementations may use HTTP/1.1, HTTP/2, or HTTP/3.
Every API request sends:
| Header | Value |
|---|---|
Mise-Cache-Protocol | 1 |
Mise-Cache-Namespace | The namespace for the operation, except on discovery endpoints |
The URL prefix /v1 is the protocol's major version. Compatible additions are advertised as
capabilities and do not require a new URL prefix. An incompatible wire or integrity change requires
a new major protocol; version 1 must not be used as an alias for an incompatible implementation.
Servers must ignore unknown JSON response fields. Clients must not send unknown request fields unless a negotiated capability permits them.
The JSON representation of a digest is:
{
"algorithm": "blake3",
"hash": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"size": 1234
}
Version 1 defines blake3 and sha256. Servers advertise the algorithms they accept. Mise clients
prefer BLAKE3 but may use SHA-256 for deployments that require it. A digest always covers the exact,
uncompressed bytes and includes their length. A server must reject malformed hashes, unsupported
algorithms, negative sizes, and content that does not match its declared digest.
Digest URL components use /v1/blobs/{algorithm}/{hash}/{size}. The algorithm and hash must match
the JSON representation, and size is an unsigned decimal integer.
GET /v1/capabilities requires no namespace and returns the protocol and server limits:
{
"protocol": { "major": 1, "minor": 0 },
"digest_algorithms": ["blake3", "sha256"],
"compressors": ["identity", "zstd"],
"features": {
"batch": true,
"resumable_uploads": true,
"delegated_transfers": true,
"signed_results": ["ed25519"]
},
"limits": {
"max_batch_items": 1000,
"max_inline_blob_bytes": 1048576,
"max_blob_bytes": 107374182400
}
}
Clients must honor advertised limits and fall back from optional features. Servers return
426 Upgrade Required for unsupported major versions and include their supported major version in
Mise-Cache-Protocol.
GET /v1/status is an operational health endpoint. A successful response means the API process is
live; it is not a substitute for capability negotiation or an authorization check.
Protocol JSON objects use UTF-8 and the JSON Canonicalization Scheme (RFC 8785) whenever their bytes are hashed or signed. Duplicate object keys, invalid UTF-8, non-canonical encodings, and values that cannot be represented by the declared schema must be rejected.
An action descriptor contains everything declared to affect a task result:
{
"version": 1,
"task": "build",
"phase": "run",
"run": [{ "task": "cargo build --release" }],
"args": [],
"shell": null,
"outputs": ["target/release/widget"],
"root": "crates/widget",
"source_hash": "blake3:...",
"dependency_keys": [],
"environment": { "PROFILE": "release" },
"command_inputs": [],
"vars": {},
"tools": ["core:[email protected]"],
"os": "linux",
"arch": "x86_64"
}
Arrays whose order has no task meaning must be sorted by the field defined by their schema. Version strings are opaque and are never semantically ordered. Secrets must not appear in an action descriptor. Environment variables are included only when the task declares them as cache inputs.
source_hash binds the declared source paths and contents without uploading task inputs that are not
needed for cache-only operation. The canonical descriptor is stored in CAS. Its digest is the action
digest and the action-result URL key. Two clients that describe the same action must produce
identical canonical bytes.
A directory object has media type application/vnd.mise.cache-directory.v1+json:
{
"version": 1,
"directories": [
{
"name": "assets",
"digest": { "algorithm": "blake3", "hash": "...", "size": 321 },
"mode": 493
}
],
"files": [
{
"name": "widget",
"digest": { "algorithm": "blake3", "hash": "...", "size": 123456 },
"executable": true,
"mode": 493
}
],
"symlinks": [{ "name": "current", "target": "widget", "mode": 511 }]
}
Each node list is sorted by the UTF-8 bytes of name. Names must be a single path component and
must not be empty, ., .., contain a slash or NUL, or collide with another node. Absolute symlink
targets and targets that escape the declared output root must be rejected during restoration.
The portable metadata set is file contents, directory structure, symbolic links, executable state,
and the portable permission bits represented by mode. Owners, groups, timestamps, devices,
sockets, FIFOs, platform ACLs, and extended attributes are not restored. Hard links may be restored
as independent files. Unsupported source objects make the task result ineligible for remote caching
rather than being silently changed.
An action-result response and commit body have media type
application/vnd.mise.cache-action-result.v1+json. The wire object is always an envelope so an
unsigned result and a signed result have the same schema:
{
"result": {
"version": 1,
"action": { "algorithm": "blake3", "hash": "...", "size": 789 },
"output_root": { "algorithm": "blake3", "hash": "...", "size": 456 },
"metadata": { "algorithm": "blake3", "hash": "...", "size": 234 }
},
"signatures": []
}
Only successful, cacheable task executions may be published. output_root is absent when a task has
no declared output files. metadata references canonical
application/vnd.mise.cache-client-metadata.v1+json containing the output roots, captured output,
task identity, restored-byte estimate, and execution duration needed by mise clients. The metadata
schema is part of the remote protocol and is independent of mise's local cache manifest.
{
"version": 1,
"task_identity": "build:crates/widget",
"roots": ["target/release/widget"],
"output": [{ "stream": "stdout", "line": "built widget" }],
"restored_bytes": 123456,
"execution_duration_ns": 900000000
}
Root paths use forward slashes, are relative to the task working directory, and must satisfy the same path-safety rules as directory nodes. Output entries preserve their declared order.
The action descriptor and every object reachable from the result must exist and validate before the result becomes readable. Authenticated producer identity, repository, revision, CI run, and commit time are recorded by the server alongside the immutable result and included in signed receipts; a client cannot establish its own trusted provenance by placing claims in the result body.
Retention, last-access time, quota accounting, internal storage location, and server annotations are not part of the immutable action result.
POST /v1/blobs:missing accepts application/vnd.mise.cache-digests.v1+json:
{ "digests": [{ "algorithm": "blake3", "hash": "...", "size": 1234 }] }
It returns 200 OK with the subset not present in verified CAS:
{ "missing": [{ "algorithm": "blake3", "hash": "...", "size": 1234 }] }
The server must not disclose whether objects exist outside the request's readable namespaces or CAS visibility domain.
GET /v1/blobs/{algorithm}/{hash}/{size} returns 200 OK, or 404 Not Found when the caller cannot
read the object. The response includes Digest and Content-Length metadata. Servers may honor
Range and may return a negotiated Content-Encoding: zstd; the URL digest always describes the
uncompressed bytes.
A server advertising delegated transfers may return 307 Temporary Redirect to a short-lived HTTPS
URL. The redirect must grant access only to the requested immutable object. Clients must not forward
the cache service's Authorization header to the delegated host.
Clients verify the complete uncompressed digest before using downloaded content. A mismatch is a cache miss, emits a visible integrity warning, and must be reported to server telemetry when the reporting capability is enabled.
Small blobs may be sent directly with
PUT /v1/blobs/{algorithm}/{hash}/{size} and If-None-Match: *. The server returns:
201 Created after verifying and publishing new content;204 No Content when identical verified content already exists;400 Bad Request when the bytes do not match the digest;412 Precondition Failed when an immutable precondition fails;413 Content Too Large when an advertised limit is exceeded.Large or resumable uploads use an upload session:
POST /v1/uploads declares one or more digests.POST /v1/uploads/{id}/finalize verifies complete content and promotes it into CAS.Delegated uploads always target an isolated staging key, never a readable CAS key. A presigned S3 upload is therefore insufficient by itself: finalization must validate the declared digest before publication. Expired or abandoned staging objects are removed asynchronously.
GET /v1/action-results/{algorithm}/{hash}/{size} returns a committed action result or 404 Not Found. The namespace identifies the single read scope for that request. Clients configured with
multiple read scopes query them in policy order rather than sending an ambiguous multi-namespace
request.
PUT /v1/action-results/{algorithm}/{hash}/{size} commits an action result. It requires
If-None-Match: *. The server must atomically:
The response is 201 Created, 204 No Content for an identical committed result, 409 Conflict
when a different result already owns the action key, or 412 Precondition Failed when the immutable
precondition is absent or fails. Concurrent valid writers may upload identical CAS data, but only one
action-result commit wins.
Ordinary cache writers do not receive delete permission. Administrative deletion uses a separately authorized endpoint and must remove the action-result mapping before unreachable CAS data is garbage collected. A client-side cache clear operation must not imply authority to delete shared remote data.
CAS digests provide integrity but do not prove who associated an action with an output. The action-result envelope may therefore contain signatures:
{
"result": {
"version": 1,
"action": { "algorithm": "blake3", "hash": "...", "size": 789 }
},
"signatures": [
{
"algorithm": "ed25519",
"key_id": "cache.example.com/2026-08",
"signature": "..."
}
]
}
The signature input is the ASCII domain separator mise-cache-action-result-v1, one NUL byte, and
the canonical JSON bytes of result. Version 1 defines Ed25519 signatures. Servers may also retain
CI identity attestations and issue a signed commit receipt.
Clients configured to require signatures treat an absent, unknown, or invalid signature as a cache miss and display a warning. Public verification keys are configuration; private signing keys must not be distributed to untrusted clients. Shared-secret HMAC is not the default because any verifier holding the secret could forge results.
The protocol supports bearer tokens, OIDC-derived tokens, mTLS, and trusted reverse-proxy identity. Authentication mechanism discovery is deployment configuration rather than CAS object metadata. Credentials must be scoped and redacted from diagnostics.
Servers authorize reads and writes independently. A secure deployment should provide at least:
Immutable storage does not prevent cache poisoning by the first writer. Namespace authorization, producer provenance, and signature policy are required even when the backing object store rejects overwrites. A single bucket credential shared by trusted and untrusted jobs is not a conforming security boundary.
401 Unauthorized means authentication is missing or invalid.403 Forbidden means the identity lacks permission for the requested namespace or operation.404 Not Found is a cache miss and must not reveal inaccessible objects.409 Conflict is an immutable action-result conflict.412 Precondition Failed is a missing or failed conditional-write requirement.422 Unprocessable Content is a validly encoded object with an invalid reference graph.426 Upgrade Required is a major-version mismatch.429 Too Many Requests and 5xx responses may be retried with bounded exponential backoff and
jitter, honoring Retry-After.Cache unavailability, malformed objects, missing referenced objects, and integrity failures normally degrade to a cache miss so local task execution can continue. Authentication, authorization, and integrity failures must still produce visible warnings; clients must not silently label them as ordinary misses. Deployments may enable a strict mode that makes selected failures fatal.
Idempotency keys may be sent for upload-session creation and other retryable POST operations.
Servers must bound their retention and scope them to the authenticated identity and namespace.
A conforming self-hosted server may use a filesystem, S3-compatible object storage, or another blob store. Clients communicate with the cache service rather than receiving general object-store credentials.
The official reference server is maintained separately at
jdx/mise-cache. It provides filesystem and S3-compatible blob
storage, PostgreSQL metadata, namespace-scoped authorization, Docker Compose, and a Helm chart. The
server remains a separate deployment and release lifecycle from the mise client while this document
is the canonical protocol specification.
A server using S3 should:
Object-store versioning, retention locks, and encryption are useful defense in depth but do not replace application authorization or digest verification.
The repository's compatibility suite is the executable definition of required version 1 behavior. It must cover capability negotiation, canonical object validation, namespace isolation, independent read/write authorization, missing-blob batches, streamed and resumable transfers, digest rejection, atomic action-result commits, immutable conflicts, delegated-transfer credential isolation, signatures, corruption handling, and retry semantics.
Servers may implement additional administrative, metrics, and health APIs outside /v1. Those APIs
must not weaken the version 1 cache invariants.