External OAuth Authentication

:::note DataHub Cloud Customers Self-service configuration of external OAuth providers is not yet available on DataHub Cloud.

If you'd like to configure external OAuth for DataHub Cloud, please reach out to your customer support representative with the configuration values outlined below! :::

DataHub supports authenticating API requests using JWT tokens from external identity providers like Okta, Azure AD, Google Identity, and others. This is perfect for service-to-service authentication where your applications need to call DataHub APIs.

Overview

When you configure OAuth authentication, DataHub will:

Accept JWT tokens from your trusted identity provider
Validate the token signature and claims
Automatically create service accounts for authenticated users
Grant API access based on DataHub's permission system

Configuration

Configure OAuth authentication by setting these environment variables in your DataHub deployment:

Set these environment variables for the datahub-gms service:

bash

# Enable OAuth authentication
EXTERNAL_OAUTH_ENABLED=true

# Required: Trusted JWT issuers (comma-separated)
EXTERNAL_OAUTH_TRUSTED_ISSUERS=https://auth.example.com,https://okta.company.com

# Required: Allowed JWT audiences (comma-separated)
EXTERNAL_OAUTH_ALLOWED_AUDIENCES=datahub-api,my-service-id

# Required: JWKS endpoint for signature verification
EXTERNAL_OAUTH_JWKS_URI=https://auth.example.com/.well-known/jwks.json

# Optional: JWT claim containing user ID (default: "sub")
EXTERNAL_OAUTH_USER_ID_CLAIM=sub

# Optional: Signing algorithm (default: "RS256")
EXTERNAL_OAUTH_ALGORITHM=RS256

Docker Compose Example

yaml

version: "3.8"
services:
  datahub-gms:
    image: acryldata/datahub-gms:latest
    environment:
      # External OAuth Configuration
      - EXTERNAL_OAUTH_ENABLED=true
      - EXTERNAL_OAUTH_TRUSTED_ISSUERS=https://my-okta-domain.okta.com/oauth2/default
      - EXTERNAL_OAUTH_ALLOWED_AUDIENCES=0oa1234567890abcdef
      - EXTERNAL_OAUTH_JWKS_URI=https://my-okta-domain.okta.com/oauth2/default/v1/keys
      - EXTERNAL_OAUTH_USER_ID_CLAIM=sub
      - EXTERNAL_OAUTH_ALGORITHM=RS256

      # Standard DataHub settings
      - DATAHUB_GMS_HOST=0.0.0.0
      - DATAHUB_GMS_PORT=8080
      # ... other configurations

Kubernetes Example

yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: datahub-gms
spec:
  template:
    spec:
      containers:
        - name: datahub-gms
          image: acryldata/datahub-gms:latest
          env:
            - name: EXTERNAL_OAUTH_ENABLED
              value: "true"
            - name: EXTERNAL_OAUTH_TRUSTED_ISSUERS
              value: "https://login.microsoftonline.com/tenant-id/v2.0"
            - name: EXTERNAL_OAUTH_ALLOWED_AUDIENCES
              value: "api://datahub-prod"
            - name: EXTERNAL_OAUTH_JWKS_URI
              value: "https://login.microsoftonline.com/tenant-id/discovery/v2.0/keys"
          # ... other environment variables

Multiple Providers

To support multiple OAuth providers, use comma-separated values:

bash

# Multiple issuers and audiences
EXTERNAL_OAUTH_TRUSTED_ISSUERS=https://okta.company.com,https://auth0.company.com
EXTERNAL_OAUTH_ALLOWED_AUDIENCES=datahub-prod,datahub-staging,service-account-id

# Single JWKS URI (if providers share keys) or discovery URI
EXTERNAL_OAUTH_JWKS_URI=https://okta.company.com/.well-known/jwks.json

# Or use discovery URI to auto-derive JWKS
EXTERNAL_OAUTH_DISCOVERY_URI=https://okta.company.com/.well-known/openid-configuration

Discovery URI vs JWKS URI

You can specify either:

JWKS URI: Direct endpoint to signing keys (recommended for production)
Discovery URI: OIDC discovery document URL (DataHub will auto-derive JWKS URI)

bash

# Option 1: Direct JWKS URI (faster, more reliable)
EXTERNAL_OAUTH_JWKS_URI=https://auth.example.com/.well-known/jwks.json

# Option 2: Discovery URI (convenient, auto-derives JWKS)
EXTERNAL_OAUTH_DISCOVERY_URI=https://auth.example.com/.well-known/openid-configuration

Provider Examples

Okta

bash

EXTERNAL_OAUTH_ENABLED=true
EXTERNAL_OAUTH_TRUSTED_ISSUERS=https://your-domain.okta.com/oauth2/default
EXTERNAL_OAUTH_ALLOWED_AUDIENCES=0oa1234567890abcdef
EXTERNAL_OAUTH_JWKS_URI=https://your-domain.okta.com/oauth2/default/v1/keys

Auth0

bash

EXTERNAL_OAUTH_ENABLED=true
EXTERNAL_OAUTH_TRUSTED_ISSUERS=https://your-domain.auth0.com/
EXTERNAL_OAUTH_ALLOWED_AUDIENCES=https://your-api-identifier/
EXTERNAL_OAUTH_JWKS_URI=https://your-domain.auth0.com/.well-known/jwks.json

Azure AD / Microsoft Entra

bash

EXTERNAL_OAUTH_ENABLED=true
EXTERNAL_OAUTH_TRUSTED_ISSUERS=https://login.microsoftonline.com/your-tenant-id/v2.0
EXTERNAL_OAUTH_ALLOWED_AUDIENCES=api://your-app-id
EXTERNAL_OAUTH_JWKS_URI=https://login.microsoftonline.com/your-tenant-id/discovery/v2.0/keys

Google Cloud Identity

bash

EXTERNAL_OAUTH_ENABLED=true
EXTERNAL_OAUTH_TRUSTED_ISSUERS=https://accounts.google.com
EXTERNAL_OAUTH_ALLOWED_AUDIENCES=your-client-id.apps.googleusercontent.com
EXTERNAL_OAUTH_JWKS_URI=https://www.googleapis.com/oauth2/v3/certs

Keycloak

bash

EXTERNAL_OAUTH_ENABLED=true
EXTERNAL_OAUTH_TRUSTED_ISSUERS=https://keycloak.company.com/realms/datahub
EXTERNAL_OAUTH_ALLOWED_AUDIENCES=datahub-client
EXTERNAL_OAUTH_JWKS_URI=https://keycloak.company.com/realms/datahub/protocol/openid-connect/certs

Using OAuth Tokens

Once configured, include your JWT token in the Authorization header when making API requests:

bash

curl -H "Authorization: Bearer YOUR_JWT_TOKEN" \
     -H "Content-Type: application/json" \
     https://your-datahub.com/api/graphql \
     -d '{"query": "{ me { corpUser { urn username }}}"}'

For Python applications:

python

import requests

headers = {
    'Authorization': f'Bearer {your_jwt_token}',
    'Content-Type': 'application/json'
}

response = requests.post(
    'https://your-datahub.com/api/graphql',
    headers=headers,
    json={'query': '{ me { corpUser { urn username }}}'}
)

Authenticating DataHub Clients (CLI, SDK, and Ingestion)

Instead of pasting a static token, the Python SDK, the datahub CLI, and ingestion pipelines can mint short-lived OAuth tokens themselves via a configurable token provider. Tokens are fetched from your IdP, cached, and refreshed automatically before expiry.

Declarative configuration (recipes and SDK)

Add an auth block to any client config — for example in an ingestion recipe:

yaml

sink:
  type: datahub-rest
  config:
    server: https://your-datahub.com/gms
    auth:
      type: oidc_client_credentials
      config:
        token_endpoint: https://your-idp.com/oauth2/token
        client_id: datahub-client
        client_secret: "${IDP_CLIENT_SECRET}"
        audience: datahub-api

The same block works on datahub_api (for sources that read from DataHub) and on DatahubClientConfig in SDK code. auth and token are mutually exclusive — configure one or the other.

Environment-based configuration (`DATAHUB_AUTH_TYPE`)

Processes that resolve their client config from environment variables — the datahub CLI, the default ingestion sink, and Remote Executor recipe subprocesses — can configure OAuth entirely via env vars:

`DATAHUB_AUTH_TYPE`	Required variables	Optional variables
`k8s_oidc`	—	`DATAHUB_AUTH_TOKEN_FILE`, `DATAHUB_AUTH_AUDIENCE`
`azure_entra`	`DATAHUB_AUTH_AZURE_TENANT_ID`, `DATAHUB_AUTH_AZURE_CLIENT_ID`, `DATAHUB_AUTH_AZURE_SCOPE`	`DATAHUB_AUTH_AZURE_CLIENT_SECRET` (omit for workload identity)
`oidc_client_credentials`	`DATAHUB_AUTH_TOKEN_ENDPOINT`, `DATAHUB_AUTH_CLIENT_ID`, `DATAHUB_AUTH_CLIENT_SECRET`	`DATAHUB_AUTH_SCOPE`, `DATAHUB_AUTH_AUDIENCE`

bash

export DATAHUB_GMS_URL=https://your-datahub.com/gms
export DATAHUB_AUTH_TYPE=oidc_client_credentials
export DATAHUB_AUTH_TOKEN_ENDPOINT=https://your-idp.com/oauth2/token
export DATAHUB_AUTH_CLIENT_ID=datahub-client
export DATAHUB_AUTH_CLIENT_SECRET=...
datahub ingest -c recipe.yml

Precedence rules

Explicit credentials win over the environment. A recipe sink or client config that sets token or auth ignores DATAHUB_AUTH_TYPE.
Env OAuth wins over static tokens — over DATAHUB_GMS_TOKEN and over a token stored in ~/.datahubenv. Each override is logged as a warning. Note that this swaps the calling identity: a job that deliberately authenticated with a personal access token will act as the OAuth service identity once DATAHUB_AUTH_TYPE is set in its environment.
Env OAuth applies only to the env-configured server. A credential-less sink: datahub-rest block inherits env OAuth only when its server matches DATAHUB_GMS_URL (same scheme, host, and port); otherwise the merge is skipped with a warning so freshly minted tokens are never sent to a different host.

Custom token providers registered via the provider registry are supported through the declarative auth block only, not through DATAHUB_AUTH_TYPE.

Best Practices

Use HTTPS for all JWKS URIs and discovery endpoints
Use specific audience values (not wildcards) for better security
Use short-lived tokens (< 1 hour recommended)
Separate environments with different audiences (prod/staging/dev)
Enable debug logging during setup: DATAHUB_GMS_LOG_LEVEL=DEBUG

Troubleshooting

Common Issues

"OAuth authenticator is not configured"

Make sure EXTERNAL_OAUTH_ENABLED=true is set
Verify all required environment variables are configured

"No configured OAuth provider matches token issuer"

Check that your JWT issuer exactly matches EXTERNAL_OAUTH_TRUSTED_ISSUERS

"Invalid or missing audience claim"

Verify your JWT audience is listed in EXTERNAL_OAUTH_ALLOWED_AUDIENCES

"Failed to load signing keys"

Test your JWKS URI directly: curl https://your-provider/.well-known/jwks.json
Check network connectivity from DataHub to your OAuth provider

Debugging

Enable debug logging to see detailed OAuth messages:

bash

# Set environment variable
DATAHUB_GMS_LOG_LEVEL=DEBUG

# Check logs
docker logs datahub-gms | grep -i oauth

Testing Your Setup

Decode your JWT token to verify the claims:

bash

# Replace with your actual token
echo "YOUR_JWT_TOKEN" | cut -d. -f2 | base64 -d | jq

Make sure the iss (issuer) and aud (audience) claims match your configuration.

Advanced Options

You can customize which JWT claim contains the user ID:

bash

# Use email claim instead of default 'sub'
EXTERNAL_OAUTH_USER_ID_CLAIM=email

OAuth users are automatically created as service accounts with usernames like __oauth_{issuer_domain}_{subject}.

External OAuth Authentication

External OAuth Authentication

Overview

Configuration

Docker Compose Example

Kubernetes Example

Multiple Providers

Discovery URI vs JWKS URI

Provider Examples

Okta

Auth0

Azure AD / Microsoft Entra

Google Cloud Identity

Keycloak

Using OAuth Tokens

Authenticating DataHub Clients (CLI, SDK, and Ingestion)

Declarative configuration (recipes and SDK)

Environment-based configuration (DATAHUB_AUTH_TYPE)

Precedence rules

Best Practices

Troubleshooting

Common Issues

Debugging

Testing Your Setup

Advanced Options

Environment-based configuration (`DATAHUB_AUTH_TYPE`)