Docker integration

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

Using Docker to run your applications? Or build your artifacts? No worries, moon can be utilized with Docker, and supports a robust integration layer.

:::success

Looking to speed up your Docker builds? Want to build in the cloud? Give Depot a try!

:::

Requirements

The first requirement, which is very important, is adding .moon/cache to the workspace root .dockerignore (moon assumes builds are running from the root). Not all files in .moon/cache are portable across machines/environments, so copying these file into Docker will definitely cause interoperability issues.

.moon/cache

The other requirement depends on how you want to integrate Git with Docker. Since moon executes git commands under the hood, there are some special considerations to be aware of when running moon within Docker. There's 2 scenarios to choose from:

1. (recommended) Add the .git folder to .dockerignore, so that it's not COPY'd. moon will continue to work just fine, albeit with some functionality disabled, like caching.
2. Ensure that the git library is installed in the container, and copy the .git folder with COPY. moon will work with full functionality, but it will increase the overall size of the image because of caching.

Creating a Dockerfile

:::info

Our moon docker file command can automatically generate a Dockerfile based on this guide! We suggest generating the file then reading the guide below to understand what's going on.

:::

We're very familiar with how tedious Dockerfiles are to write and maintain, so in an effort to reduce this headache, we've built a handful of tools to make this process much easier. With moon, we'll take advantage of Docker's layer caching and staged builds as much as possible.

With that being said, there's many approaches you can utilize, depending on your workflow (we'll document them below):

• Running moon docker commands before running docker run|build commands.
• Running moon docker commands within the Dockerfile.
• Using multi-staged or non-staged (standard) builds.
• Something else unique to your setup!

:::warning

This guide and our Docker approach is merely a suggestion and is not a requirement for using moon with Docker! Feel free to use this as a starting point, or not at all. Choose the approach that works best for you!

:::

What we're trying to avoid

Before we dive into writing a perfect Dockerfile, we'll briefly talk about the pain points we're trying to avoid. In the context of Node.js and monorepo's, you may be familiar with having to COPY each individual package.json in the monorepo before installing node_modules, to effectively use layer caching. This is very brittle, as each new application or package is created, every Dockerfile in the monorepo will need to be modified to account for this new package.json.

Furthermore, we'll have to follow a similar process for only copying source files necessary for the build or CMD to complete. This is very tedious, so most developers simply use COPY . . and forget about it. Copying the entire monorepo is costly, especially as it grows.

As an example, we'll use moon's official repository. The Dockerfile would look something like the following.

FROM node:latest

WORKDIR /app

# Install moon binary
RUN npm install -g @moonrepo/cli

# Copy moon files
COPY ./.moon ./.moon

# Copy all package.json's and lockfiles
COPY ./packages/cli/package.json ./packages/cli/package.json
COPY ./packages/core-linux-arm64-gnu/package.json ./packages/core-linux-arm64-gnu/package.json
COPY ./packages/core-linux-arm64-musl/package.json ./packages/core-linux-arm64-musl/package.json
COPY ./packages/core-linux-x64-gnu/package.json ./packages/core-linux-x64-gnu/package.json
COPY ./packages/core-linux-x64-musl/package.json ./packages/core-linux-x64-musl/package.json
COPY ./packages/core-macos-arm64/package.json ./packages/core-macos-arm64/package.json
COPY ./packages/core-macos-x64/package.json ./packages/core-macos-x64/package.json
COPY ./packages/core-windows-x64-msvc/package.json ./packages/core-windows-x64-msvc/package.json
COPY ./packages/runtime/package.json ./packages/runtime/package.json
COPY ./packages/types/package.json ./packages/types/package.json
COPY ./package.json ./package.json
COPY ./yarn.lock ./yarn.lock
COPY ./.yarn ./.yarn
COPY ./.yarnrc.yml ./yarnrc.yml

# Install toolchain and dependencies
# In non-moon repos: yarn install
RUN moon docker setup

# Copy project and required files
# Or COPY . .
COPY ./packages/types ./packages/types
COPY ./packages/runtime ./packages/runtime

# Build the target
RUN moon run runtime:build

For such a small monorepo, this already looks too confusing!!! Let's remedy this by utilizing moon itself to the fullest!

Scaffolding the bare minimum

The first step in this process is to only copy the bare minimum of files necessary for installing dependencies (Node.js modules, etc). This is typically manifests (package.json), lockfiles (yarn.lock, etc), and any configuration (.yarnrc.yml, etc).

This can all be achieved with the moon docker scaffold command, which scaffolds a skeleton of the repository structure, with only necessary files (the above). Let's update our Dockerfile usage.

<Tabs groupId="dockerfile" defaultValue="standard" values={[ { label: 'Non-staged', value: 'standard' }, { label: 'Multi-staged', value: 'staged' }, ]}

This assumes moon docker scaffold <project> is ran outside of the Dockerfile.

FROM node:latest
WORKDIR /app

# Install moon binary
RUN npm install -g @moonrepo/cli

# Copy workspace skeleton
COPY ./.moon/docker/workspace .

# Install toolchain and dependencies
RUN moon docker setup

#### BASE
FROM node:latest AS base
WORKDIR /app

# Install moon binary
RUN npm install -g @moonrepo/cli

#### SKELETON
FROM base AS skeleton

# Copy entire repository and scaffold
COPY . .
RUN moon docker scaffold <project>

#### BUILD
FROM base AS build

# Copy workspace skeleton
COPY --from=skeleton /app/.moon/docker/workspace .

# Install toolchain and dependencies
RUN moon docker setup

And with this, our dependencies will be layer cached effectively! Let's now move onto copying source files.

Copying necessary source files

The next step is to copy all source files necessary for CMD or any RUN commands to execute correctly. This typically requires copying all source files for the project and all source files of the project's dependencies... NOT the entire repository!

Luckily our moon docker scaffold <project> command has already done this for us! Let's continue updating our Dockerfile to account for this, by appending the following:

<Tabs groupId="dockerfile" defaultValue="standard" values={[ { label: 'Non-staged', value: 'standard' }, { label: 'Multi-staged', value: 'staged' }, ]}

# Copy source files
COPY ./.moon/docker/sources .

# Build something (optional)
RUN moon run <project>:<task>

# Copy source files
COPY --from=skeleton /app/.moon/docker/sources .

# Build something (optional)
RUN moon run <project>:<task>

:::info

If you need to copy additional files for your commands to run successfully, you can configure the docker.scaffold settings in .moon/workspace.yaml (entire workspace) or moon.* (per project).

:::

Pruning extraneous files

Now that we've ran a command or built an artifact, we should prune the Docker environment to remove unneeded files and folders. We can do this with the moon docker prune command, which must be ran within the context of a Dockerfile!

# Prune workspace
RUN moon docker prune

When ran, this command will do the following, in order:

• Remove extraneous dependencies (node_modules) for unfocused projects.
• Install production only dependencies for the projects that were scaffolded.

:::info

This process can be customized using the docker.prune setting in .moon/workspace.yaml.

:::

Final result

And with this moon integration, we've reduced the original Dockerfile of 35 lines to 18 lines, a reduction of almost 50%. The original file can also be seen as O(n), as each new manifest requires cascading updates, while the moon approach is O(1)!

<Tabs groupId="dockerfile" defaultValue="standard" values={[ { label: 'Non-staged', value: 'standard' }, { label: 'Multi-staged', value: 'staged' }, ]}

FROM node:latest
WORKDIR /app

# Install moon binary
RUN npm install -g @moonrepo/cli

# Copy workspace skeleton
COPY ./.moon/docker/workspace .

# Install toolchain and dependencies
RUN moon docker setup

# Copy source files
COPY ./.moon/docker/sources .

# Build something (optional)
RUN moon run <project>:<task>

# Prune workspace
RUN moon docker prune

# CMD

#### BASE
FROM node:latest AS base
WORKDIR /app

# Install moon binary
RUN npm install -g @moonrepo/cli

#### SKELETON
FROM base AS skeleton

# Copy entire repository and scaffold
COPY . .
RUN moon docker scaffold <project>

#### BUILD
FROM base AS build

# Copy workspace skeleton
COPY --from=skeleton /app/.moon/docker/workspace .

# Install toolchain and dependencies
RUN moon docker setup

# Copy source files
COPY --from=skeleton /app/.moon/docker/sources .

# Build something (optional)
RUN moon run <project>:<task>

# Prune workspace
RUN moon docker prune

# CMD

Running docker commands

When running docker commands, they must be ran from moon's workspace root (typically the repository root) so that the project graph and all moon docker commands resolve correctly.

docker build .

If you're Dockerfiles are located within each applicable project, use the -f argument.

docker run -f ./apps/client/Dockerfile .

Troubleshooting

Supporting node:alpine images

If you're trying to use the node:alpine image with moon's integrated toolchain, you'll need to set the MOON_TOOLCHAIN_FORCE_GLOBALS environment variable in the Docker image to disable moon's toolchain. This is required as Node.js does not provide pre-built binaries for the Alpine target, so installing the Node.js toolchain will fail.

FROM node:alpine

ENV MOON_TOOLCHAIN_FORCE_GLOBALS=true