ContextQMD
Back to Datahub

v0.3.8

docs/managed-datahub/release-notes/v_0_3_8.md

1.5.0.36.5 KB
Original Source

v0.3.8

Release Availability Date

29-Jan-2025

0.15.0.1

Known Issues

  • Async APIs - DataHub's asynchronous APIs perform only basic schema validation when receiving MCP requests, similar to direct production to MCP Kafka topics. While requests must conform to the MCP schema to be accepted, actual processing happens later in the pipeline. Any processing failures that occur after the initial acceptance are captured in the Failed MCP topic, but these failures are not immediately surfaced to the API caller since they happen asynchronously.
  • Persistence Exception: No Rows Updated may occur if a transaction does not change any aspect's data.

Release Changelog

v0.3.8.2

  • Helm Chart Requirement: 1.5.0+
  • [Remote Executor]: Add support for file-based secret manager.
  • [UI]: Show user that manually triggered an ingestion.
  • [UI]: Add option to copy custom ownership type urn.
  • [Ingestion]: Add Restore Indicies as a DataHub Cloud ingestion source.
  • [Observability]: Fix ingestionSourceForEntity to only fetches sources that can execute assertions based on aspect source.
  • [SQL] Fix No rows updated exception.
  • [Search] Fix autocomplete duplicate field.
  • [Lineage] Fix lineage counts on search cards with ghost entities.
  • [UI] Show editable field info for fields based on exact fieldPath version.
  • [Metadata Tests] Add browse path v2 container as a selectable property.
  • [Modelling] Fix DashboardContainsDashboard relationship in DashboardInfo aspect.
  • [UI] Consider Dataset Profiles, Status and Browse Path v2 aspects when computing last ingested timestamp.
  • [Metadata Tests] Add ownership type as a filter for selecting assets.
  • [API] Disallow deleting system users.
  • [UI] Add 'Created At' filter.
  • [Search Ranking] Reset ranking on unused datasets.
  • [Integrations] Pre-install required duckdb extensions.
  • [ElasticSearch] Fix runId no document found exception.
  • [Modelling] Add UrnValidation PDL annotation.
  • [UI] Fix bug with nested selects and selectable parents
  • [UI] Downgrade V2 field paths in column stats table

v0.3.8

  • All changes in https://github.com/datahub-project/datahub/releases/tag/v0.14.1

  • Breaking Changes

    • Soft-deleted and SCIM suspended users are no longer able to access the UI/APIs with a pre-existing valid token

  • Bug Fixes

    • Ingestion page sorts across all sources rather than just the ones loaded on the page.
    • Data Products Preview cards now render correctly in scenarios where there are several fully filled out products in a list
    • Snowflake tag propagation looks for case-insensitive table match on tables with special characters
    • Lineage patch properly encodes special characters
    • Fixed multiple bugs around lineage visualization's handling of graph cycles
    • Fixed lineage filter node search on schema field level lineage
    • Search csv export properly includes data products
    • Properly show lineage to an upstream dbt source that has no further upstreams
    • Soft-deleted & suspended users with valid tokens are rejected
    • Fixes scroll behavior on quality tab with small screens
    • Add backwards compatibility to the UI for old policy filters in the UI
    • Fixes nesting logic in Properties tab for structured properties that prefix matched each other's qualified names
    • Show error when editing help link if there is an error
    • Fix UI bug on schema table when clicking hides part of table
    • Fix rendering schema field references as structured property values to show field name, not urn
    • Partially fix a bug where dbt sync back would generate invalid YAML. This won't happen anymore, but we might miss some changes because of it
    • Stability and error message improvements for AI docs generation
    • Various quality and performance improvements to AI classification
    • Fix searching for nested columns in the Columns tab of Dataset profile page
    • Remote Executor: make cron schedules in ingestions/assertions scheduler POSIX-compliant.
    • Remote Executor: limit the number of automated restarts of aborted tasks.
    • Remote Executor: always update task status synchronously.

  • Product

    • [BETA] Introducing the new Navigation Bar / App redesign! Starting with this release, you can try out a new look and feel to your DataHub app. To enable this BETA feature, reach out to your DataHub Cloud representative.
    • [BETA] Add ability to create tag propagation automations to downstream and sibling assets via the UI in the Automations tab
    • Support the Notes feature for all entity types with profile pages
    • Test notifications button is now easier to discover.
    • UX polishes to the Assertions Assertions tab
    • UX polishes to the Column sidebar
    • UX polishes to column and table deprecation, including deprecation visible in lineage graph
    • Add the ability to do AND for filters of a given type. For example, we now allow you to filter for assets that have tag1 AND tag2
    • Combine sibling entities on home page insights
    • Hardened structured properties feature by supporting broken references and adding better validation to the ID/qualified name
    • Hardened forms feature by supporting invalid / unresolvable references
    • Recurrent metadata sharing: Any assets shared between DataHub assets will be re-shared automatically once per day.
    • On entity page by hovering over subscribe button you can see list of subscribers of the entity.

  • Platform

    • Optional config to reduce Slack app permissions
    • Support batch processing for MCP consumption to increase throughput of the Kafka consumer significantly
    • Resumability for DataHub Upgrade jobs
    • Async actions for Metadata Tests corner case where some actions could be missed has been fixed
    • Structured Properties now have a stricter validation for no spaces in the qualified name
    • Urn Validation logic has been improved to reduce delayed error conditions
    • Database transaction and locking updates to fix conditions where retryable persistence exceptions would cause ingestion batches to fail
    • Fix for Metadata Tests to support primitive PDL Typeref fields like timestamps
    • Entity Versioning initial feature support, disabled by default and can be turned on via feature flag
    • OpenAPI fixes for some faulty parameters
    • Improved memory performance of the search ranking system ingestion source
    • Remote Executor: Added registration/discovery mechanism.
    • Remote Executor: Added weight-based task scheduling to allow variable concurrency for memory-greedy tasks.