docs/managed-datahub/release-notes/v_2_1_0.md
30-Jul-2026
ASYNC. Async emit() no longer raises on a rejected write and gives no read-after-write guarantee. To restore the previous behavior, set emit_mode: SYNC_PRIMARY (per-plugin config; [datahub] emit_mode = SYNC_PRIMARY in airflow.cfg). The Python SDK, CLI, and DataHub Actions defaults are unchanged.DataHubClassifier and its acryl-datahub-classify dependency have been removed from acryl-datahub. The classification framework is retained but no classifier is registered by default. Recipes that set classification.enabled: true without registering a replacement now fail fast at startup. Remove the classification block, register a custom classifier, or pin the last release that ships the built-in classifier.path_spec_list or file_partition_regexp) is configured. Recipes with neither configured pick up the auto-stripping, so dataset URNs for partitioned paths change and edges to the old URNs can orphan. Configure path trimming, or set spark.datahub.metadata.dataset.openLineageTrimmersEnabled=false to restore the pre-1.50 naming. Separately, file_partition_regexp now also applies to bare-namespace file/dbfs datasets.convert_column_urns_to_lowercase: true on an upstream SQL source as a workaround, remove it so both sides preserve casing.INFORMATION_SCHEMA.PARTITIONS instead of the undocumented __TABLES__. PARTITIONS covers base tables only, so empty tables, external tables, views, and snapshots lose their lastModified timestamp (and snapshots lose row/size custom properties). Set use_legacy_table_stats: true to restore the previous behavior.platform: documentdb (with hostingEnvironment: AWS_DOCUMENTDB) emits entities under a new documentdb data platform. The default is unchanged; enabling it changes dataset and container URNs, and stale mongodb entities must be cleaned up.convert_urns_to_lowercase: true (non-default), the connector now also lower-cases database-container URNs, not just datasets, so containers and their datasets match. Deployments on the default (false) are unaffected.WorkunitProcessor classes, and several processors were renamed. Custom code that called the helper functions directly must migrate to the processor-class API.ELASTICSEARCH_SEMANTIC_SEARCH_ENTITIES=document,dataset on GMS and the system-update job to keep dataset semantic search after upgrade.USAGE_AGGREGATION_ENABLED=true, datahub_request_count is emitted from the in-memory aggregation flush to Micrometer/Prometheus (not per-request) and the actor tag becomes actor_class. New byte and active-identity metrics are added. Aggregation is off by default; legacy JMX metrics are unaffected.DATAHUB_EXECUTOR_MONITORS_ENABLED and DATAHUB_EXECUTOR_TASKS_ENABLED are now hard opt-outs that skip subsystem wiring and heavy imports, not fetcher-only toggles. Use DATAHUB_EXECUTOR_INGESTION_PIPELINE_ENABLED to disable the Kafka / actions pipeline.notificationConfig (or send it without notificationSettings) when creating a subscription now leave it configured to use the actor's current notification defaults at delivery time, instead of creating a subscription with no sinks. Send notificationSettings.sinkTypes: [] explicitly for an intentional no-sink subscription.REQUEST_MINIMAL_SLACK_PERMISSIONS. Its role is now served by DATAHUB_SLACK_SERVER_SIDE_HISTORY_ENABLED, which marks the Slack history scopes as optional in the install screen so admins can deselect them per install. Remove the old variable if set.POST /openapi/v3/lineage/scroll now takes a single urns list (replacing the separate source/destination/edge filters); direction accepts UPSTREAM/DOWNSTREAM. No known callers, so no migration is expected.Metrics and Semantic Models — new first-class Metric and SemanticModel entities, with dedicated entity pages, a metrics home page, sidebar navigation, and autocomplete, so business metrics and the semantic models that define them are cataloged, searchable, and connected to lineage alongside the datasets they derive from.
Scoped MCP servers — create and manage purpose-built Model Context Protocol (MCP) servers from the UI, each exposing a curated set of tools, configurable instructions and optional data asset scoping via Views, to enable you to build domain-specific agents with access to specific data & tools. Available in Context Platform private beta by default.
Agent, Service, and Repository catalog entities — catalog AI agents, the skills they adopt and tools they invoke, the services those tools belong to (REST APIs, MCP servers), and the source-code repositories behind them as first-class entities that participate in lineage, with auto-registration available for LangChain and Google ADK agents. See the Agent Registry (API tutorial) and Service Catalog guides.
Bi-Directional Context Document Sync for GitHub - import context documents from one or more GitHub repositories, and sync changes made on DataHub back as GitHub PRs. Read more here.
Expanded internationalization — multi-language support can be enabled upon request. New locales: French (fr), Italian (it), Norwegian Bokmål (nb), Swedish (sv), and Hungarian (hu) (all Beta). When enabled, DataHub follows the browser locale where a translation exists.
New ingestion sources — Cube (semantic layer), AWS Kinesis Data Streams and Amazon Data Firehose, MicroStrategy, and Open Data Contract Standard (ODCS).
Data Product Lineage - Data products now appear directly in lineage, making it easier to understand how business-level data entities connect across your ecosystem.
All changes in https://github.com/datahub-project/datahub/releases/tag/v1.6.0
LOG_AGGREGATOR_ENDPOINT, which also enables support-bundle log collection without relying on the Kubernetes API.alert id that can be used for deduplicating alerts in your inbox or through external system integrations.DATAHUB_EXECUTOR_SNOWFLAKE_QUOTE_COLUMNS. Turn this flag on if your column names are case sensitive in Snowflake.DATAHUB_USE_OBSERVE_MODELS) and delta-space prediction bounds (DATAHUB_EXECUTOR_ENABLE_DELTA_BOUNDS) are now enabled by default, improving anomaly-detection accuracy out of the box.New ingestion sources:
Connector improvements:
profile_table_size_limit (default 5 GB, fail-open when size is unknown); lineage fetch batch size is configurable; view definitions can be offloaded to a disk-backed store; multi-row query reconstruction is capped to protect memory; permission errors during size checks are categorized; and zero-row lineage fetches surface warnings.system.query.history via the shared SQL parsing aggregator (window up to 365 days); zero-value usage stamped on idle windows; and fixed ML model ingestion controls (include_ml_models, an enforced ml_model_max_results cap, and MLModel URNs honoring the recipe env).queryUsageStatistics) on Query entities; optional column-level usage; and table_pattern now applied to SQL-parsing-derived lineage and usage. The synthetic per-user userEmail in usage stats is no longer populated (user attribution via CorpUser URN is unchanged).INFORMATION_SCHEMA.PARTITIONS (see Breaking Changes); usage.* window and formatting fields consolidated to top-level equivalents (see Deprecations).hive data platform (instead of hadoop), so Hive lineage parses correctly and aligns with the Hive connector.file_partition_regexp now applies to bare-namespace file/dbfs datasets.performance_schema.emit_folders_only.Ingestion infrastructure:
DataHubClassifier and its dependency are gone; the classification framework remains (see Breaking Changes).WorkunitProcessor classes (see Breaking Changes).DATAHUB_CLIENT_CERT_PATH (and optionally DATAHUB_CLIENT_KEY_PATH). Fully backwards compatible.DATAHUB_AUTH_TYPE selects OAuth for the CLI, default sink, and env-resolved emitters, backed by reusable OAuth token providers for the Python clients.DATAHUB_SQLGLOT_DISABLE_CExecutor:
DATAHUB_EXECUTOR_LOG_GC_ENABLED, default false).CALL statement (DATAHUB_EXECUTOR_ALLOW_CALL_STATEMENTS, default false).>=2.1.14) to address CVE-2026-8838.HTTP 422 - fieldPath is required.queryProperties aspects failing ingestion runs by measuring serialized size correctly; oversized statements are now truncated with a warning instead of failing the run.isEmbedded flag was unset.DATAHUB_AI_TOOL_APPROVAL_ENABLED (default true): gates the AI chat's human-in-the-loop approval of mutating tools on the web UI. Set false to auto-apply edits as before. Re-read per chat turn.CHAT_SUGGESTIONS_ENABLED (default true): gates generation of Ask DataHub follow-up suggestions.SQL_SKETCH_SYNTHESIS_MODEL (default: a Bedrock Sonnet profile): model for SQL sketch/advice synthesis; accepts bedrock/, openai/, or google_vertexai/ prefixes.I18N_ENABLED: enables multi-language translation. On DataHub Cloud, enable upon request; in OSS the default is true (set false for English-only).USAGE_AGGREGATION_ENABLED (default false): enables in-memory request/byte aggregation exported to Micrometer/Prometheus on flush.USAGE_AGGREGATION_INCLUDE_AGENT_NAME (default false): when usage aggregation is on, also roll up by client agent_name. Enable only when finer client attribution is required — values are client-controlled and can increase metric cardinality.ENTITY_COUNT_METRICS_ENABLED (default true): enables periodic Micrometer/Prometheus gauges for key-aspect entity counts (datahub.entity.count).ENTITY_COUNT_METRICS_UPDATE_INTERVAL_SECONDS (default 3600) / ENTITY_COUNT_METRICS_INITIAL_DELAY_SECONDS (default 60) / ENTITY_COUNT_METRICS_SKIP_CACHE (default false): refresh cadence and whether each refresh bypasses the key-aspect count cache.EBEAN_READ_POOL_ENABLED (default false) / EBEAN_READ_POOL_URL: optional Ebean read pool for entity-aspect reads (omit URL for same-DB split-pool; set URL to a replica for offload). See Primary storage read pool.CASSANDRA_READ_POOL_ENABLED (default false) / CASSANDRA_READ_POOL_HOSTS: optional Cassandra read session for entity-aspect reads.ENTITY_GRAPH_CACHE_ENABLED (default true on GMS): enables the GMS entity graph cache (Hazelcast snapshots). Set false when Hazelcast is unavailable or on non-GMS pods.ENTITY_GRAPH_CACHE_CONFIG_FILE_ENABLED / ENTITY_GRAPH_CACHE_CONFIG_FILE / ENTITY_GRAPH_CACHE_CONFIG_JSON: overlay graph definitions and bounds; see GMS Entity Graph Cache.DATAHUB_USE_INFERENCE_V2 (default false): routes smart-assertion training to the V2 trainer. Requires DATAHUB_USE_OBSERVE_MODELS=true.DATAHUB_USE_OBSERVE_MODELS (default true): gates whether observe-models may be loaded by the V1 preprocessor; prerequisite for DATAHUB_USE_INFERENCE_V2.DATAHUB_EXECUTOR_ENABLE_DELTA_BOUNDS (default true): gates whether the V1 trainer emits delta-space prediction bounds.DATAHUB_EXECUTOR_LOG_GC_ENABLED (default false): enables the in-process ingestion-log garbage collector (retention and size-cap tunables via the related DATAHUB_EXECUTOR_LOG_GC_* variables).DATAHUB_EXECUTOR_ALLOW_CALL_STATEMENTS (default false): allows a stored-procedure CALL statement in custom SQL assertions. Requires an executor restart.DATAHUB_CLIENT_CERT_PATH / DATAHUB_CLIENT_KEY_PATH: enable mTLS client authentication for CLI/SDK outbound HTTPS.DATAHUB_AUTH_TYPE: selects OAuth for the CLI, default sink, and env-resolved emitters.CORP_USER_SIBLINGS_ENABLED: enables the daily user-siblings de-duplication source.AUTH_GMS_SESSION_COOKIE_NAME (default SESSION): name of the GMS session cookie the frontend invalidates on OAuth2 logout.INGESTION_MAX_SERIALIZED_NAME_LENGTH (default 16 MB): tunable limit for JSON property-name length on metadata deserialization.NOTIFICATION_LOGO_URL / NOTIFICATION_FOOTER_TEXT: customize the logo and footer on SMTP notification emails.LOG_AGGREGATOR_ENDPOINT (opt-in): ships each service's logs to a separately-deployed log aggregator and enables the support bundle's log collector.usage.* window fields deprecated. usage.start_time, usage.end_time, usage.bucket_duration, and usage.max_query_duration are deprecated in favor of their top-level equivalents. Previously the usage-scoped fields were silently ignored; they now take effect (with a warning), which can widen the ingested window and increase query cost. Move them to the top level of the recipe; setting the same field in both places is now an error.