Back to Questdb

Configuration

documentation/configuration/overview.md

latest17.1 KB
Original Source

import { ConfigTable } from "@theme/ConfigTable" import sharedWorkerConfig from "./configuration-utils/_shared-worker.config.json" import httpConfig from "./configuration-utils/_http.config.json" import cairoConfig from "./configuration-utils/_cairo.config.json" import parallelSqlConfig from "./configuration-utils/_parallel-sql.config.json" import walConfig from "./configuration-utils/_wal.config.json" import csvImportConfig from "./configuration-utils/_csv-import.config.json" import parquetExportConfig from "./configuration-utils/_parquet-export.config.json" import postgresConfig from "./configuration-utils/_postgres.config.json" import tcpConfig from "./configuration-utils/_tcp.config.json" import udpConfig from "./configuration-utils/_udp.config.json" import replicationConfig from "./configuration-utils/_replication.config.json" import iamConfig from "./configuration-utils/_iam.config.json" import oidcConfig from "./configuration-utils/_oidc.config.json" import logConfig from "./configuration-utils/_log.config.json" import matViewConfig from "./configuration-utils/_mat-view.config.json" import configValidationConfig from "./configuration-utils/_config-validation.config.json" import telemetryConfig from "./configuration-utils/_telemetry.config.json"

This page describes methods for configuring QuestDB server settings.

Configuration can be set either:

  • In the server.conf configuration file available in the root directory
  • Using environment variables

When a key is absent from both the config file and the environment variables, the default value is used.

:::note

For Windows users

When entering path values, use either \\ or / instead of the native path separator char \.

  • 👍 C:\\path\\to\\file\\path
  • 👍 C:/path/to/file
  • 👎 C:\path\to\file

The single backslash is interpreted as an escape sequence start within Java properties.

:::

Environment variables

All settings in the configuration file can be set or overridden using environment variables. If a key is set in both the server.conf file and via an environment variable, the environment variable will take precedence and the value in the server configuration file will be ignored.

To make these configuration settings available to QuestDB via environment variables, they must be in the following format:

shell
QDB_<KEY_OF_THE_PROPERTY>

Where <KEY_OF_THE_PROPERTY> is equal to the configuration key name. To properly format a server.conf key as an environment variable it must have:

  1. QDB_ prefix
  2. uppercase characters
  3. all . period characters replaced with _ underscore

For example, the server configuration key for query timeout must be passed as described below:

server.conf keyenv var
query.timeoutQDB_QUERY_TIMEOUT

:::note

QuestDB applies these configuration changes on startup and a running instance must be restarted in order for configuration changes to take effect.

:::

Examples

The following configuration property customizes the query timeout:

shell
query.timeout=120s
shell
export QDB_QUERY_TIMEOUT=120s

Secrets from files

QuestDB supports reading sensitive configuration values from files using the _FILE suffix convention. This is useful in containerized environments like Kubernetes, where secrets are typically mounted as files rather than passed as environment variables.

When a _FILE variant is set, QuestDB reads the secret value from the specified file path. This works with both environment variables and properties in server.conf.

Usage

Environment variable:

shell
QDB_PG_PASSWORD_FILE=/run/secrets/pg-password

Property file:

ini
pg.password.file=/run/secrets/pg-password

Precedence

If both a _FILE variant and the direct value are set, the _FILE variant takes precedence. For example, if both QDB_PG_PASSWORD_FILE and QDB_PG_PASSWORD are set, the value is read from the file.

File requirements

Secret files must meet the following requirements:

  • Maximum size: 64KB
  • Encoding: UTF-8
  • Content handling: Leading and trailing whitespace is automatically trimmed

The following paths are not allowed for security reasons:

  • Paths containing .. (path traversal)
  • Paths starting with /dev/, /proc/, or /sys/
  • Directories (including symlinks to directories)

If a secret file is empty or contains only whitespace, QuestDB logs an advisory warning, as this may weaken authentication.

Error handling

If a secret file cannot be read at startup, QuestDB fails to start. This includes cases where the file does not exist, is too large, or the path is not allowed.

During runtime, if reload_config() cannot read a secret file, the reload fails and the previous value is retained. This ensures the server continues operating if a secret file is temporarily unavailable.

Reloading secrets

Secrets loaded from files support runtime reloading. After updating a secret file, call reload_config() to apply the new value. See Reloadable settings for details.

To verify that a secret was loaded from a file, run SHOW PARAMETERS and check the value_source column, which displays file for secrets loaded from files.

Supported properties

The following properties support the _FILE suffix:

PropertyEnvironment variable
pg.passwordQDB_PG_PASSWORD_FILE
pg.readonly.passwordQDB_PG_READONLY_PASSWORD_FILE
http.passwordQDB_HTTP_PASSWORD_FILE

Enterprise properties

The following additional properties are available in QuestDB Enterprise:

PropertyEnvironment variable
acl.admin.passwordQDB_ACL_ADMIN_PASSWORD_FILE
acl.oidc.tls.keystore.passwordQDB_ACL_OIDC_TLS_KEYSTORE_PASSWORD_FILE
replication.object.storeQDB_REPLICATION_OBJECT_STORE_FILE
cold.storage.object.storeQDB_COLD_STORAGE_OBJECT_STORE_FILE
backup.object.store.*QDB_BACKUP_OBJECT_STORE_*_FILE

For Kubernetes-specific examples, see the Kubernetes deployment guide.

Reloadable settings

Certain configuration settings can be reloaded without having to restart the server. To reload a setting, edit its value in the server.conf file and then run the reload_config SQL function:

questdb-sql
SELECT reload_config();

If the value was reloaded successfully, the reload_config function returns true and a message is printed to the server log:

2025-01-02T09:52:40.833848UTC I i.q.DynamicPropServerConfiguration reloaded config option [update, key=http.net.connection.limit, old=100, new=200]

Each key has a reloadable property that indicates whether the key can be reloaded. If yes, the reload_config function can be used to reload the configuration.

All reloadable properties can be also queried from the server:

questdb-sql
(SHOW PARAMETERS) WHERE reloadable = true;

Keys and default values

This section lists the configuration keys available to QuestDB by topic or subsystem. Parameters for specifying buffer and memory page sizes are provided in the format n<unit>, where <unit> can be one of the following:

  • m for MB
  • k for kB

For example:

ini
http.net.connection.sndbuf=2m

Shared worker

QuestDB uses three specialized worker pools to handle different workloads:

  • Network pool: handles HTTP, PostgreSQL, and ILP server I/O
  • Query pool: executes parallel query operations (filters, group-by)
  • Write pool: manages WAL apply jobs, table writes, materialized view refresh, and housekeeping tasks
<ConfigTable rows={sharedWorkerConfig} />

HTTP server

This section describes configuration settings for the Web Console and the REST API available by default on port 9000. For details on the use of this component, refer to the web console documentation page.

<ConfigTable rows={httpConfig} />

Cairo engine

This section describes configuration settings for the Cairo SQL engine in QuestDB.

<ConfigTable rows={cairoConfig} />

WAL table configurations

The following WAL tables settings on parallel threads are configurable for applying WAL data to the table storage:

<ConfigTable rows={walConfig} />

COPY settings

Import

This section describes configuration settings for using COPY to import large CSV files, or export parquet files.

Settings for COPY FROM (import):

<ConfigTable rows={csvImportConfig} pick={[ "cairo.sql.copy.root", "cairo.sql.copy.work.root", "cairo.iouring.enabled", "cairo.sql.copy.buffer.size", "cairo.sql.copy.log.retention.days", "cairo.sql.copy.max.index.chunk.size", "cairo.sql.copy.queue.capacity", ]} />

CSV import configuration for Docker

For QuestDB instances using Docker:

  • cairo.sql.copy.root must be defined using one of the following settings:
    • The environment variable QDB_CAIRO_SQL_COPY_ROOT.
    • The cairo.sql.copy.root in server.conf.
  • The path for the source CSV file is mounted.
  • The source CSV file path and the path defined by QDB_CAIRO_SQL_COPY_ROOT are identical.
  • It is optional to define QDB_CAIRO_SQL_COPY_WORK_ROOT.

The following is an example command to start a QuestDB instance on Docker, in order to import a CSV file:

shell
docker run -p 9000:9000 \
-v "/tmp/questdb:/var/lib/questdb" \
-v "/tmp/questdb/my_input_root:/var/lib/questdb/questdb_import" \
-e QDB_CAIRO_SQL_COPY_ROOT=/var/lib/questdb/questdb_import \
questdb/questdb

Where:

  • -v "/tmp/questdb/my_input_root:/var/lib/questdb/questdb_import": Defining a source CSV file location to be /tmp/questdb/my_input_root on local machine and mounting it to /var/lib/questdb/questdb_import in the container.
  • -e QDB_CAIRO_SQL_COPY_ROOT=/var/lib/questdb/questdb_import: Defining the copy root directory to be /var/lib/questdb/questdb_import.

It is important that the two path are identical (/var/lib/questdb/questdb_import in the example).

Export

<ConfigTable rows={parquetExportConfig} />

Parquet export is also generally impacted by query execution and parquet conversion parameters.

If not overridden, the following default setting will be used.

<ConfigTable rows={cairoConfig} pick={[ "cairo.partition.encoder.parquet.raw.array.encoding.enabled", "cairo.partition.encoder.parquet.version", "cairo.partition.encoder.parquet.statistics.enabled", "cairo.partition.encoder.parquet.compression.codec", "cairo.partition.encoder.parquet.compression.level", "cairo.partition.encoder.parquet.row.group.size", "cairo.partition.encoder.parquet.data.page.size" ]} />

Parallel SQL execution

This section describes settings that can affect the level of parallelism during SQL execution, and therefore can also have an impact on performance.

<ConfigTable rows={parallelSqlConfig} />

Postgres wire protocol

This section describes configuration settings for client connections using PostgresSQL wire protocol.

<ConfigTable rows={postgresConfig} />

InfluxDB Line Protocol (ILP)

This section describes ingestion settings for incoming messages using InfluxDB Line Protocol.

PropertyDefaultDescription
line.default.partition.byDAYTable partition strategy to be used with tables that are created automatically by InfluxDB Line Protocol. Possible values are: HOUR, DAY, WEEK, MONTH, and YEAR.
line.auto.create.new.columnstrueWhen enabled, automatically creates new columns when they appear in the ingested data. When disabled, messages with new columns will be rejected.
line.auto.create.new.tablestrueWhen enabled, automatically creates new tables when they appear in the ingested data. When disabled, messages for non-existent tables will be rejected.
line.log.message.on.errortrueControls whether malformed ILP messages are printed to the server log when errors occur.

HTTP specific settings

ILP over HTTP is the preferred way of ingesting data.

PropertyDefaultDescription
line.http.enabledtrueEnable ILP over HTTP. Default port is 9000. Enabled by default within open source versions, defaults to false and must be enabled for Enterprise.
line.http.ping.versionv2.2.2Version information for the ping response of ILP over HTTP.
HTTP propertiesVariousSee HTTP settings for general HTTP configuration. ILP over HTTP inherits from HTTP settings.

TCP specific settings

<ConfigTable rows={tcpConfig} />

UDP specific settings

:::note

The UDP receiver is deprecated since QuestDB version 6.5.2. We recommend ILP over HTTP instead, or less frequently ILP over TCP.

:::

<ConfigTable rows={udpConfig} />

Database replication

:::note

Replication is Enterprise only.

:::

Replication enables high availability clusters.

For setup instructions, see the replication operations guide.

For an overview of the concept, see the replication concept page.

For a tuning guide see, the replication tuning guide.

<ConfigTable rows={replicationConfig} />

Identity and Access Management (IAM)

:::note

Identity and Access Management is available within QuestDB Enterprise.

:::

Identity and Access Management (IAM) ensures that data can be accessed only by authorized users. The below configuration properties relate to various authentication and authorization features.

For a full explanation of IAM, see the Identity and Access Management (IAM) documentation.

<ConfigTable rows={iamConfig} />

OpenID Connect (OIDC)

:::note

OpenID Connect is Enterprise only.

:::

OpenID Connect (OIDC) support is part of QuestDB's Identity and Access Management. The database can be integrated with any OAuth2/OIDC Identity Provider (IdP).

For detailed information about OIDC, see the OpenID Connect (OIDC) integration guide.

<ConfigTable rows={oidcConfig} />

Config Validation

The database startup phase checks for configuration issues, such as invalid or deprecated settings. Issues may be classified as advisories or errors. Advisory issues are logged without causing the database to stop its startup sequence: These are usually setting deprecation warnings. Configuration errors can optionally cause the database to fail its startup.

<ConfigTable rows={configValidationConfig} />

We recommended enabling strict validation.

Telemetry

QuestDB sends anonymous telemetry data with information about usage which helps us improve the product over time. We do not collect any personally-identifying information, and we do not share any of this data with third parties.

<ConfigTable rows={telemetryConfig} />

Materialized views

:::info

Materialized View support is now generally available (GA) and ready for production use.

If you are using versions earlier than 8.3.1, we suggest you upgrade at your earliest convenience.

:::

The following settings are available in server.conf:

<ConfigTable rows={matViewConfig} />

Logging & Metrics

The following settings are available in server.conf:

<ConfigTable rows={logConfig} />

Further settings are available in log.conf. For more information, and details of our Prometheus metrics, please visit the Logging & Metrics documentation.