ContextQMD
Back to Datahub

Configuration

docs/quick-ingestion-guides/snowflake/configuration.md

1.5.0.46.3 KB
Original Source

Configuring Your Snowflake Connector to DataHub

Now that you have created a DataHub-specific user with the relevant roles in Snowflake in the prior step, it's now time to set up a connection via the DataHub UI.

Configure Secrets

  1. Within DataHub, navigate to the Ingestion tab in the top, right corner of your screen
<p align="center"> </p>

:::note If you do not see the Ingestion tab, please contact your DataHub admin to grant you the correct permissions :::

  1. Navigate to the Secrets tab and click Create new secret
<p align="center"> </p>
  1. Create a Password secret

This will securely store your Snowflake password within DataHub

  • Enter a name like SNOWFLAKE_PASSWORD - we will use this later to refer to the secret
  • Enter the password configured for the DataHub user in the previous step
  • Optionally add a description
  • Click Create
<p align="center"> </p>

Configure Recipe

  1. Navigate to the Sources tab and click Create new source
<p align="center"> </p>
  1. Select Snowflake
<p align="center"> </p>
  1. Fill out the Snowflake Recipe

Enter the Snowflake Account Identifier as Account ID field. Account identifier is the part before .snowflakecomputing.com in your snowflake host URL:

<p align="center"> </p>

Learn more about Snowflake Account Identifiers here

Add the previously added Password secret to Password field:

  • Click on the Password input field
  • Select SNOWFLAKE_PASSWORD secret
<p align="center"> </p>

Populate the relevant fields using the same Username, Role, and Warehouse you created and/or specified in Snowflake Prerequisites.

<p align="center"> </p>
  1. Click Test Connection

This step will ensure you have configured your credentials accurately and confirm you have the required permissions to extract all relevant metadata.

<p align="center"> </p>

After you have successfully tested your connection, click Next.

Schedule Execution

Now it's time to schedule a recurring ingestion pipeline to regularly extract metadata from your Snowflake instance.

  1. Decide how regularly you want this ingestion to run-- day, month, year, hour, minute, etc. Select from the dropdown
<p align="center"> </p>
  1. Ensure you've configured your correct timezone
<p align="center"> </p>
  1. Click Next when you are done

Finish Up

  1. Name your ingestion source, then click Save and Run
<p align="center"> </p>

You will now find your new ingestion source running

<p align="center"> </p>

Validate Ingestion Runs

  1. View the latest status of ingestion runs on the Ingestion page
<p align="center"> </p>
  1. Click the plus sign to expand the full list of historical runs and outcomes; click Details to see the outcomes of a specific run
<p align="center"> </p>
  1. From the Ingestion Run Details page, pick View All to see which entities were ingested
<p align="center"> </p>
  1. Pick an entity from the list to manually validate if it contains the detail you expected
<p align="center"> </p>

Congratulations! You've successfully set up Snowflake as an ingestion source for DataHub!