Back to Datahub

Configuration

docs/quick-ingestion-guides/redshift/configuration.md

1.5.0.35.4 KB
Original Source

Configuring Your Redshift Connector to DataHub

Now that you have created a DataHub user in Redshift in the prior step, it's time to set up a connection via the DataHub UI.

Configure Secrets

  1. Within DataHub, navigate to the Ingestion tab in the top, right corner of your screen
<p align="center"> </p>

:::note If you do not see the Ingestion tab, please contact your DataHub admin to grant you the correct permissions :::

  1. Navigate to the Secrets tab and click Create new secret
<p align="center"> </p>
  1. Create a Redshift User's Password secret

This will securely store your Redshift User's password within DataHub

  • Click Create new secret again
  • Enter a name like REDSHIFT_PASSWORD - we will use this later to refer to the secret
  • Enter your datahub redshift user's password
  • Optionally add a description
  • Click Create
<p align="center"> </p>

Configure Recipe

  1. Navigate to the Sources tab and click Create new source
<p align="center"> </p>
  1. Select Redshift
<p align="center"> </p>
  1. Fill out the Redshift Recipe

Populate the Password field by selecting Redshift Password secrets you created in steps 3 and 4.

<p align="center"> </p> <!--- 7. Click **Test Connection** This step will ensure you have configured your credentials accurately and confirm you have the required permissions to extract all relevant metadata. <p align="center"> </p> After you have successfully tested your connection, click **Next**. -->

Schedule Execution

Now it's time to schedule a recurring ingestion pipeline to regularly extract metadata from your Redshift instance.

  1. Decide how regularly you want this ingestion to run-- day, month, year, hour, minute, etc. Select from the dropdown
<p align="center"> </p>
  1. Ensure you've configured your correct timezone
<p align="center"> </p>
  1. Click Next when you are done

Finish Up

  1. Name your ingestion source, then click Save and Run
<p align="center"> </p>

You will now find your new ingestion source running

<p align="center"> </p>

Validate Ingestion Runs

  1. View the latest status of ingestion runs on the Ingestion page
<p align="center"> </p>
  1. Click the plus sign to expand the full list of historical runs and outcomes; click Details to see the outcomes of a specific run
<p align="center"> </p>
  1. From the Ingestion Run Details page, pick View All to see which entities were ingested
<p align="center"> </p>
  1. Pick an entity from the list to manually validate if it contains the detail you expected
<p align="center"> </p>

Congratulations! You've successfully set up Redshift as an ingestion source for DataHub!