website/docs/guides/remote-cache.mdx
import VersionLabel from '@site/src/components/Docs/VersionLabel';
Is your CI pipeline running slower than usual? Are you tired of running the same build over and over although nothing has changed? Do you wish to reuse the same local cache across other machines and environments? These are just a few scenarios that remote caching aims to solve.
Remote caching is a system that shares artifacts to improve performance, reduce unnecessary computation time, and alleviate resources. It achieves this by uploading hashed artifacts to a cloud storage provider, like AWS S3 or Google Cloud, and downloading them on demand when a build matches a derived hash.
To make use of remote caching, we provide 2 solutions.
This solution allows you to host any remote caching service that is compatible with the
Bazel Remote Execution v2 API,
such as bazel-remote. When using this solution, the
following RE API features must be enabled:
When you have chosen (or built) a compatible service, host it and make it available through gRPC or
HTTPS. For example, if you plan to use bazel-remote, you can do something like the following:
bazel-remote --dir /path/to/moon-cache --max_size 10 --storage_mode uncompressed --grpc_address 0.0.0.0:9092
If you've configured the remote.cache.compression setting to
"zstd", you'll need to run the binary with that storage mode as well.
bazel-remote --dir /path/to/moon-cache --max_size 10 --storage_mode zstd --grpc_address 0.0.0.0:9092
:::info
View the official bazel-remote documentation for
all the available options, like storing artifacts in S3, configuring authentication (TLS/mTLS),
proxies, and more.
:::
Once your service is running, you can enable remote caching by configuring the
remote settings in .moon/workspace.*. At
minimum, the only setting that is required is host.
# gRPC
remote:
host: 'grpc://your-host.com:9092'
# HTTPS
remote:
api: 'http'
host: 'https://your-host.com:8080'
:::info
Remote caching can be conditionally enabled by setting the MOON_REMOTE_HOST environment variable.
:::
We have rudimentary support for TLS and mTLS, but it's very unstable, and has not been thoroughly tested. There's also many many issues around authentication in Tonic.
# TLS
remote:
host: 'grpcs://your-host.com:9092'
tls:
cert: 'certs/ca.pem'
domain: 'your-host.com'
# mTLS
remote:
host: 'grpcs://your-host.com:9092'
mtls:
caCert: 'certs/ca.pem'
clientCert: 'certs/client.pem'
clientKey: 'certs/client.key'
domain: 'your-host.com'
If you'd prefer not to host your own solution, you could use Depot Cache, a cloud-based caching solution. To make use of Depot, follow these steps:
DEPOT_TOKEN environment variable to your moon pipelinesOnce these steps have been completed, you can enable remote caching in moon with the following
configuration. If your Depot account has more than 1 organization, you'll need to set the
X-Depot-Org header.
remote:
host: 'grpcs://cache.depot.dev'
auth:
token: 'DEPOT_TOKEN'
headers:
'X-Depot-Org': '<your-org-id>'
In the context of moon and remote caching, an artifact is the outputs of a task, as well as the stdout and stderr of the task that generated the outputs. Artifacts are uniquely identified by the moon generated hash.
No, remote caching is optional. It's intended purpose is to store long lived build artifacts to
speed up CI pipelines, and optionally local development. For the most part,
moon ci does a great job of only running what's affected in pull requests, and
is a great starting point.
No, remote caching does not store source code. It stores the
outputs of a task, which is typically built and compiled code. To
verify this, you can inspect the tar archives in .moon/cache/outputs.
No, moon does not collect any PII as part of the remote caching process.
We do not encrypt on moon's side, as encryption is provided by your cloud storage provider.