import FeatureAvailability from '@site/src/components/FeatureAvailability';

AI Glossary Term Suggestions

<FeatureAvailability saasOnly />

:::info

This feature is currently in closed beta. Reach out to your DataHub Cloud representative to get access.

:::

The AI Glossary Term Suggestion automation uses LLMs to suggest Glossary Terms for tables and columns in your data.

This is useful for improving coverage of glossary terms across your organization, which is important for compliance and governance efforts.

This automation can:

• Automatically suggests glossary terms for tables and columns.
• Goes beyond a predefined set of terms and works with your business glossary.
• Generates proposals for owners to review, or can automatically add terms to tables/columns.
• Automatically adjusts to human-provided feedback and curation (coming soon).

Prerequisites

• A business glossary with terms defined. Additional metadata, like documentation and existing term assignments, will improve the accuracy of our suggestions.

Usage

1. Navigate to Automations: Click on 'Govern' > 'Automations' in the navigation bar.

<p align="center"> </p>

2. Create the Automation: Click on 'Create' and select 'AI Glossary Term Suggestions'.

<p align="center"> </p>

3. Configure the Automation: Fill in the required fields to configure the automation. The main fields to configure are (1) what terms to use for suggestions and (2) what entities to generate suggestions for.

<p align="center"> </p>

4. Once it's enabled, that's it! You'll start to see terms show up in the UI, either on assets or in the proposals page.

<p align="center"> </p>

Customize AI Classification Generation

As of v0.3.15, you can customize how classification recommendations are generated by providing custom instructions that are passed to the underlying AI model when generating classification decisions for any Table or Column. This is useful if you want AI-generated classification to follow specific guidelines or standards set by your organization.

To provide custom instructions for AI classification generation, simply provide them in the Custom Instruction input when creating the AI classification automation.

<p align="center"> </p>

Note that after updating instructions, it may take up to 5 minutes for the new instructions to take effect.

How it works

The automation will scan through all the datasets matched by the configured filters. For each one, it will generate suggestions. If new entities are added that match the configured filters, those will also be classified within 24 hours.

We take into account the following metadata when generating suggestions:

• Dataset name and description
• Column name, type, description, and sample values
• Glossary term name, documentation, and hierarchy
• Feedback loop: existing assignments and accepted/rejected proposals (coming soon)

Data privacy: Your metadata is not sent to any third-party LLMs. We use AWS Bedrock internally, which means all metadata remains within the DataHub Cloud AWS account. We do not fine-tune on customer data.

Limitations

• A single configured automation can classify at most 10k entities.
• We cannot do partial reclassification. If you add a new column to an existing table, we won't regenerate suggestions for that table.