Services

Dagger Functions support service containers, enabling users to spin up additional long-running services (as containers) and communicate with those services from Dagger Functions.

This makes it possible to:

• Instantiate and return services from a Dagger Function, and then:
  • Use those services in other Dagger Functions (container-to-container networking)
  • Use those services from the calling host (container-to-host networking)
• Expose host services for use in a Dagger Function (host-to-container networking).

Some common scenarios for using services with Dagger Functions are:

• Running a database service for local storage or testing
• Running end-to-end integration tests against a service
• Running sidecar services

Service containers

Services instantiated by a Dagger Function run in service containers, which have the following characteristics:

• Each service container has a canonical, content-addressed hostname and an optional set of exposed ports.
• Service containers are started just-in-time, de-duplicated, and stopped when no longer needed.
• Service containers are health checked prior to running clients.

Bind services in functions

A Dagger Function can create and return a service, which can then be used from another Dagger Function or from the calling host. Services in Dagger Functions are returned using the Service type.

Here is an example of a Dagger Function that returns an HTTP service. This service is used by another Dagger Function, which creates a service binding using the alias www and then accesses the HTTP service using this alias.

Here is an example call for this Dagger Function:

The result will be:

Hello, world!

Expose services returned by functions to the host

Services returned by Dagger Functions can also be exposed directly to the host. This enables clients on the host to communicate with services running in Dagger.

One use case is for testing, where you need to be able to spin up ephemeral databases against which to run tests. You might also use this to access a web UI in a browser on your desktop.

Here is another example call for the Dagger Function shown previously, this time exposing the HTTP service on the host

By default, each service port maps to the same port on the host - in this case, port 8080. The service can then be accessed by clients on the host. Here's an example:

curl localhost:8080

The result will be:

Hello, world!

To specify a different mapping, use the additional --ports argument with a list of host/service port mappings. Here's an example, which exposes the service on host port 9000:

:::note To bind ports randomly, use the --random argument. :::

Expose host services to functions

Dagger Functions can also receive host services as function arguments of type Service, in the form tcp://<host>:<port>. This enables client containers in Dagger Functions to communicate with services running on the host.

:::note This implies that a service is already listening on a port on the host, out-of-band of Dagger. :::

Here is an example of how a container running in a Dagger Function can access and query a MariaDB database service (bound using the alias db) running on the host.

Before calling this Dagger Function, use the following command to start a MariaDB database service on the host:

docker run --rm --detach -p 3306:3306 --name my-mariadb --env MARIADB_ROOT_PASSWORD=secret  mariadb:10.11.2

Here is an example call for this Dagger Function:

The result will be:

Host    User
%       root
localhost       mariadb.sys
localhost       root

Create interdependent services

Global hostnames can be assigned to services. This feature is especially valuable for complex networking configurations, such as circular dependencies between services, by allowing services to reference each other by predefined hostnames, without requiring an explicit service binding.

Custom hostnames follow a structured format (<host>.<module id>.<session id>.dagger.local), ensuring unique identifiers across modules and sessions.

For example, you can now run two services that depend on each other, each using a hostname to refer to the other by name:

In this example, service A and service B are set up with custom hostnames svca and svcb, allowing each service to communicate with the other by hostname. This capability provides enhanced flexibility for managing service dependencies and interconnections within modular workflows, making it easier to handle complex setups in Dagger.

Here is an example call for this Dagger Function:

Use service endpoints

Every service has an endpoint, and this endpoint can be obtained via the Dagger API. This feature is typically useful in two scenarios:

• When a target container (which you bind the service to) is unable to resolve the service with the bound alias. This is because the service binding works at the /etc/hosts level and not at the DNS level. As a result, a small number of applications might fail to resolve the service using the alias.
• When a service needs to be accessible (for example, over standard HTTP) without needing another container to access it.

The following example demonstrates starting an HTTP service manually, retrieving its endpoint, and then querying that service using a standard HTTP GET request:

Here is an example call for this Dagger Function:

The result will be the NGINX welcome page.

Persist service state

Dagger cancels each service run after a 10 second grace period to avoid frequent restarts. To avoid relying on the grace period, use a cache volume to persist a service's data, as in the following example:

This example uses Redis's SAVE command to save the service's data to a cache volume. When a new instance of the service is created, it uses the same cache volume to recreate the original state.

Here is an example of using these Dagger Functions:

The result will be:

123

Start and stop services

Services are designed to be expressed as a Directed Acyclic Graph (DAG) with explicit bindings allowing services to be started lazily, just like every other DAG node. But sometimes, you may need to explicitly manage the lifecycle in a Dagger Function.

For example, this may be needed if the application in the service has certain behavior on shutdown (such as flushing data) that needs careful coordination with the rest of your logic.

The following example explicitly starts the Redis service and stops it at the end, ensuring the 10 second grace period doesn't get in the way, without the need for a persistent cache volume:

Example: MariaDB database service for application tests

The following example demonstrates how services can be used in Dagger Functions, by creating a Dagger Function for application unit/integration testing against a bound MariaDB database service.

The application used in this example is Drupal, a popular open-source PHP CMS. Drupal includes a large number of unit tests, including tests which require an active database connection. All Drupal 10.x tests are written and executed using the PHPUnit testing framework. Read more about running PHPUnit tests in Drupal.

Here is an example call for this Dagger Function:

The result will be:

PHPUnit 9.6.17 by Sebastian Bergmann and contributors.
Runtime:       PHP 8.2.5
Configuration: /opt/drupal/web/core/phpunit.xml.dist
Testing
.....................S                                            22 / 22 (100%)
Time: 00:15.806, Memory: 315.00 MB
There was 1 skipped test:

1) Drupal\Tests\pgsql\Kernel\pgsql\KernelTestBaseTest::testSetUp

This test only runs for the database driver 'pgsql'. Current database driver is 'mysql'.
/opt/drupal/web/core/tests/Drupal/KernelTests/Core/Database/DriverSpecificKernelTestBase.php:44
/opt/drupal/vendor/phpunit/phpunit/src/Framework/TestResult.php:728

OK, but incomplete, skipped, or risky tests!
Tests: 22, Assertions: 72, Skipped: 1.

Reference: How service binding works in Dagger Functions

If you're not interested in what's happening in the background, you can skip this section and just trust that services are running when they need to be. If you're interested in the theory, keep reading.

Consider this example:

Here's what happens on the last line:

1. The client requests the ping container's stdout, which requires the container to run.
2. Dagger sees that the ping container has a service binding, redisSrv.
3. Dagger starts the redisSrv container, which recurses into this same process.
4. Dagger waits for health checks to pass against redisSrv.
5. Dagger runs the ping container with the redis-srv alias magically added to /etc/hosts.

Here's what happens on the last line:

1. The client requests the ping container's stdout, which requires the container to run.
2. Dagger sees that the ping container has a service binding, redis_srv.
3. Dagger starts the redis_srv container, which recurses into this same process.
4. Dagger waits for health checks to pass against redis_srv.
5. Dagger runs the ping container with the redis-srv alias magically added to /etc/hosts.

Here's what happens on the last line:

1. The client requests the ping container's stdout, which requires the container to run.
2. Dagger sees that the ping container has a service binding, redisSrv.
3. Dagger starts the redisSrv container, which recurses into this same process.
4. Dagger waits for health checks to pass against redisSrv.
5. Dagger runs the ping container with the redis-srv alias magically added to /etc/hosts.

Here's what happens on the last line:

1. The client requests the ping container's stdout, which requires the container to run.
2. Dagger sees that the ping container has a service binding, $redisSrv.
3. Dagger starts the $redisSrv container, which recurses into this same process.
4. Dagger waits for health checks to pass against $redisSrv.
5. Dagger runs the ping container with the redis-srv alias magically added to /etc/hosts.

Here's what happens on the last line:

1. The client requests the ping container's stdout, which requires the container to run.
2. Dagger sees that the ping container has a service binding, redisSrv.
3. Dagger starts the redisSrv container, which recurses into this same process.
4. Dagger waits for health checks to pass against redisSrv.
5. Dagger runs the ping container with the redis-srv alias magically added to /etc/hosts.

:::note Dagger cancels each service run after a 10 second grace period to avoid frequent restarts, unless the explicit Start and Stop APIs are used. :::

Services are based on containers, but they run a little differently. Whereas regular containers in Dagger are de-duplicated across the entire Dagger Engine, service containers are only de-duplicated within a Dagger client session. This means that if you run separate Dagger sessions that use the exact same services, they will each get their own "instance" of the service. This process is carefully tuned to preserve caching at each client call-site, while prohibiting "cross-talk" from one Dagger session's client to another Dagger session's service.

Content-addressed services are very convenient. You don't have to come up with names and maintain instances of services; just use them by value. You also don't have to manage the state of the service; you can just trust that it will be running when needed and stopped when not.

:::tip If you need multiple instances of a service, just attach something unique to each one, such as an instance ID. :::

Here's a more detailed client-server example of running commands against a Redis service:

This example relies on the 10-second grace period, which you should try to avoid. Depending on the 10-second grace period is risky because there are many factors which could cause a 10-second delay between calls to Dagger, such as excessive CPU load, high network latency between the client and Dagger, or Dagger operations that require a variable amount of time to process.

It would be better to chain both commands together, which ensures that the service stays running for both, as in the revision below: