ContextQMD
Back to Datahub

DataHub Great Expectations (GX) Plugin

metadata-ingestion-modules/gx-plugin/README.md

1.7.02.7 KB
Original Source
<!-- PyPI long description. Keep concise, feature-discovery-first. -->

DataHub Great Expectations (GX) Plugin

Send Great Expectations data quality results into DataHub — surface assertion outcomes alongside your dataset metadata so teams can see data health at a glance.

What you can do

  • Emit assertion results from GX Checkpoints directly into DataHub as data quality assertions
  • Link quality checks to datasets — results appear on the dataset's profile in DataHub
  • Track pass/fail history over time for every expectation suite
  • Works with any DataHub deployment — self-hosted or DataHub Cloud

Compatibility

GX versionModule
0.17.x / 0.18.xdatahub_gx_plugin.action.DataHubValidationAction
1.xdatahub_gx_plugin.action_v1.DataHubValidationAction

Both APIs are supported additively in the same package. Use the module that matches your installed GX major version.

Installation

bash
pip install acryl-datahub-gx-plugin

Quickstart (GX 0.17 / 0.18)

Add the DataHub action to your GX Checkpoint:

python
from datahub_gx_plugin.action import DataHubValidationAction
from great_expectations.checkpoint import Checkpoint

checkpoint = Checkpoint(
    name="my_checkpoint",
    data_context=context,
    action_list=[
        {
            "name": "datahub",
            "action": {
                "class_name": "DataHubValidationAction",
                "module_name": "datahub_gx_plugin.action",
                "server_url": "http://localhost:8080",
            },
        }
    ],
)

Quickstart (GX Core 1.x)

python
import great_expectations as gx
from datahub_gx_plugin.action_v1 import DataHubValidationAction

context = gx.get_context()
# ... create validation_definitions ...

checkpoint = context.checkpoints.add(
    gx.Checkpoint(
        name="my_checkpoint",
        validation_definitions=validation_definitions,
        actions=[
            DataHubValidationAction(
                name="datahub",
                server_url="http://localhost:8080",
                # Optional when batch_spec meta is incomplete:
                # platform="postgres",
                # dataset_name="public.my_table",
            )
        ],
    )
)
checkpoint.run()

Results from every Checkpoint run will appear in DataHub under the dataset's Validation tab.