ContextQMD
Back to Questdb

Using Docker with QuestDB

documentation/deployment/docker.md

latest10.8 KB
Original Source

import InterpolateReleaseData from "../../src/components/InterpolateReleaseData" import CodeBlock from "@theme/CodeBlock" import Tabs from "@theme/Tabs" import TabItem from "@theme/TabItem"

QuestDB has images for both Linux/macOS and Windows on Docker Hub.

Install Docker

To begin, install Docker. You can find guides for your platform on the official documentation.

Run QuestDB image

Once Docker is installed, you will need to pull QuestDB's image from Docker Hub and create a container.

This can be done with a single command using:

<InterpolateReleaseData renderText={(release) => ( <CodeBlock className="language-shell"> {docker run \\ -p 9000:9000 -p 9009:9009 -p 8812:8812 -p 9003:9003 \\ questdb/questdb:${release.name}} </CodeBlock> )} />

This command starts a Docker container from questdb/questdb image. In addition, it exposes some ports, allowing you to explore QuestDB.

In order to configure QuestDB, it is recommended to mount a volume to allow data persistance. This can be done by adding a -v flag to the above command:

-v "/host/volume/location:/var/lib/questdb"

Below each parameter is described in detail.

-p parameter to expose ports

This parameter will expose a port to the host. You can specify:

All ports are optional, you can pick only the ones you need. For example, it is enough to expose 8812 if you only plan to use Postgres wire protocol.

-v parameter to mount storage

This parameter will make a local directory available to QuestDB Docker container. It will have all data ingested to QuestDB, server logs and configuration.

The QuestDB root_directory is located at the /var/lib/questdb path in the container.

Docker image version

By default, questdb/questdb points to the latest QuestDB version available on Docker. However, it is recommended to define the version used.

<InterpolateReleaseData renderText={(release) => ( <CodeBlock className="language-shell"> {questdb/questdb:${release.name}} </CodeBlock> )} />

Environment variables

Server configuration can be passed to QuestDB running in Docker by using the -e flag to pass an environment variable to a container:

bash
docker run -p 4000:4000 -e QDB_HTTP_BIND_TO=0.0.0.0:4000 questdb/questdb

For a list of configuration options, see Configuration.

Container status

You can check the status of your container with docker ps.

It also lists the exposed ports, container name, uptime and more:

shell
docker ps
shell
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                NAMES
dd363939f261        questdb/questdb     "/app/bin/java -m io…"   3 seconds ago       Up 2 seconds        8812/tcp, 9000/tcp   frosty_gauss

This container:

  • has an id of dd363939f261
  • uses ports 8812 & 9000, for Postgres wire protocol and HTTP respectively
  • is using a questdb/questdb image
  • ran java to start the binary
  • is 3 seconds old
  • has been up for 2 seconds
  • has the unfortunate name of frosty_gauss

For full container status information, see the docker ps manual.

Debugging container logs

Docker may generate a runtime error.

The error may not be accurate, as the true culprit is often indicated higher up in the logs.

To see the full log, retrieve the UUID - also known as the CONTAINER ID - using docker ps:

shell
CONTAINER ID        IMAGE               ...
dd363939f261        questdb/questdb     ...

Now pass the CONTAINER ID - or dd363939f261 - to the docker logs command:

shell
$ docker logs dd363939f261
No arguments found, start with default arguments
Running as questdb user
Log configuration loaded from: /var/lib/questdb/conf/log.conf
...
...

Note that the log will pull from /var/lib/questdb/conf/log.conf by default.

Sharing this log when seeking support for Docker deployments will help us find the root cause.

Importing data and sending queries

When QuestDB is running, you can start interacting with it:

Data persistence

Mounting a volume

Volumes can be mounted to the QuestDB Docker container so that data may be persisted or server configuration settings may be passed to an instance. The following example demonstrated how to mount the current directory to a QuestDB container using the -v flag in a Docker run command:

<InterpolateReleaseData renderText={(release) => ( <CodeBlock className="language-shell" title={"Mounting a volume"}> {docker run -p 9000:9000 \\ -p 9009:9009 \\ -p 8812:8812 \\ -p 9003:9003 \\ -v "$(pwd):/var/lib/questdb" \\ questdb/questdb:${release.name}} </CodeBlock> )} />

The current directory will then have data persisted to disk for convenient migration or backups:

bash
├── conf
│   └── server.conf
├── db
├── log
├── public
└── snapshot (optional)

A server configuration file can also be provided by mounting a local directory in a QuestDB container. Given the following configuration file which overrides the default HTTP bind property:

shell
http.bind.to=0.0.0.0:4000

Running the container with the -v flag allows for mounting the current directory to QuestDB's conf directory in the container. With the server configuration above, HTTP ports for the Web Console and REST API will be available on http://localhost:4000:

bash
docker run -v "$(pwd):/var/lib/questdb/conf" -p 4000:4000 questdb/questdb

:::note

If you wish to use ZFS for your QuestDB deployment, with Docker, then you will need to enable ZFS on the host volume that Docker uses.

Please see the docker documentation for more information.

:::

Upgrade QuestDB version

It is possible to upgrade your QuestDB instance on Docker when a volume is mounted to maintain data persistence.

:::note

  • Check the release notes and ensure that necessary backup is completed.
  • Upgrading an instance is possible only when the original instance has a volume mounted. Without mounting a volume for the original instance, the following steps create a new instance and data in the old instance cannot be retrieved.

:::

  1. Run docker ps to copy the container name or ID:
shell

# The existing QuestDB version is 6.5.2:

CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                NAMES
dd363939f261        questdb/questdb:6.5.2     "/app/bin/java -m io…"   3 seconds ago       Up 2 seconds        8812/tcp, 9000/tcp   frosty_gauss
  1. Stop the instance and then remove the container:
shell
docker stop dd363939f261
docker rm dd363939f261
  1. Download the latest QuestDB image:

<InterpolateReleaseData renderText={(release) => ( <CodeBlock className="language-shell"> {docker pull questdb/questdb:${release.name}} </CodeBlock> )} />

  1. Start a new container with the new version and the same volume mounted:

<InterpolateReleaseData renderText={(release) => ( <CodeBlock className="language-shell"> {docker run -p 8812:8812 -p 9000:9000 -v "$(pwd):/var/lib/questdb" questdb/questdb:${release.name}} </CodeBlock> )} />

Writing logs to disk

When mounting a volume to a Docker container, a logging configuration file may be provided in the container located at /conf/log.conf:

bash
└── conf
    ├── log.conf
    └── server.conf

For example, a file with the following contents can be created:

shell
# list of configured writers
writers=file,stdout,http.min

# file writer
w.file.class=io.questdb.log.LogFileWriter
w.file.location=questdb-docker.log
w.file.level=INFO,ERROR,DEBUG

# stdout
w.stdout.class=io.questdb.log.LogConsoleWriter
w.stdout.level=INFO

# min http server, used monitoring
w.http.min.class=io.questdb.log.LogConsoleWriter
w.http.min.level=ERROR
w.http.min.scope=http-min-server

The current directory can be mounted:

shell
docker run -p 9000:9000 \
 -p 9009:9009 \
 -p 8812:8812 \
 -p 9003:9003 \
 -v "$(pwd):/root/.questdb/" questdb/questdb

The container logs will be written to disk using the logging level and file name provided in the conf/log.conf file, in this case in ./questdb-docker.log:

shell
├── conf
│  ├── log.conf
│  └── server.conf
├── db
│  ├── table1
│  └── table2
├── public
│  ├── ui / assets
│  ├── ...
│  └── version.txt
└── questdb-docker.log

For more information on logging, see the configuration reference documentation.

Restart an existing container

Running the following command will create a new container for the QuestDB image:

shell
docker run -p 9000:9000 \
  -p 9009:9009 \
  -p 8812:8812 \
  -p 9003:9003 \
  questdb/questdb

By giving the container a name with --name container_name, we have an easy way to refer to the container created by run later on:

shell
docker run -p 9000:9000 \
  -p 9009:9009 \
  -p 8812:8812 \
  -p 9003:9003 \
  --name docker_questdb \
  questdb/questdb

If we want to re-use this container and its data after it has been stopped, we can use the following commands:

shell
# bring the container up
docker start docker_questdb
# shut the container down
docker stop docker_questdb

Alternatively, restart it using the CONTAINER ID:

shell
docker start dd363939f261