dataplex-generate-data-insights

About

A dataplex-generate-data-insights tool triggers the creation and run of a Dataplex Data Insights scan on a BigQuery table.

Since the scan template creation is asynchronous, this tool returns a Long-Running Operation (LRO) resource name (format: projects/{project}/locations/{location}/operations/{operation_id}). To orchestrate this workflow, you must:

1. Capture the operation_id from this tool's response.
2. Poll the dataplex-get-operation tool with this ID until done is true.
3. Extract the created scan ID (scanId) from the completed operation's response.
4. Poll dataplex-get-run-status with the scanId until the job state is SUCCEEDED.
5. Call dataplex-get-data-insights with the scanId to fetch the final results.

Compatible Sources

{{< compatible-sources >}}

Requirements

IAM Permissions

Knowledge Catalog uses Identity and Access Management (IAM) to control user and group access to Knowledge Catalog resources. Toolbox will use your Application Default Credentials (ADC) to authorize and authenticate when interacting with [Knowledge Catalog][dataplex-docs].

In addition to setting the ADC for your server, you need to ensure the IAM identity has been given the correct IAM permissions for the tasks you intend to perform. See Knowledge Catalog IAM permissions and Knowledge Catalog IAM roles for more information on applying IAM permissions and roles to an identity.

Parameters

The dataplex-generate-data-insights tool accepts the following parameters:

Example

kind: tool
name: generate_data_insights
type: dataplex-generate-data-insights
source: my-dataplex-source
description: Trigger a new data insights scan.

Reference