Secret Resolution in Recipes

DataHub recipes support ${SECRET_NAME} syntax to inject secrets at runtime.

Variable Syntax

Secrets are referenced using bash-style variable substitution:

source:
  type: snowflake
  config:
    username: ${SNOWFLAKE_USER}
    password: ${SNOWFLAKE_PASSWORD}

Naming Rules

Variable names must follow these rules:

• Start with a letter (A-Z, a-z) or underscore (_)
• Contain only letters, numbers (0-9), and underscores
• No whitespace or special characters with the exception of underscores

:::caution Hyphens Have Special Meaning The - character is interpreted as a bash default-value operator:

• ${DB-PASSWORD} is parsed as "variable DB, with default value PASSWORD"
• If DB is unset, this resolves to the literal string PASSWORD

This is a common gotcha when using file-based secrets. :::

Bash-Style Defaults

Bash-like parameter expansion is supported:

Secret Backends

All backends are checked and values are merged. If the same secret exists in multiple backends, DataHub takes precedence over File, which takes precedence over Environment.

For example, if DB_PASSWORD is set as an environment variable and also created in the DataHub UI, the value from DataHub is used.

:::note DataHub Cloud On DataHub Cloud, the File and Environment backends are only applicable when using a Remote Executor. :::

Remote Executor: File Secret Store

The file backend reads secrets from files in a directory (default: /mnt/secrets). Each filename is the secret name, and the file contents are the secret value.

/mnt/secrets/
├── SNOWFLAKE_PASSWORD    # contains: my-secret-pw
├── API_KEY               # contains: abc123
└── DB_CONNECTION_STRING  # contains: postgres://...

Reference in recipes:

password: ${SNOWFLAKE_PASSWORD} # reads /mnt/secrets/SNOWFLAKE_PASSWORD

Configuration:

For customers using DataHub Cloud with Remote Executor, see Configuring Secret Mounting for Kubernetes integration examples.