metadata-ingestion/docs/sources/nifi/nifi_pre.md
The nifi module ingests metadata from Nifi into DataHub. It is intended for production ingestion workflows and module-specific capabilities are documented below.
Before running ingestion, ensure network connectivity to the source, valid authentication credentials, and read permissions for metadata APIs required by this module.
This connector requires following access policies to be set in Nifi for ingestion user.
| Policy | Privilege | Resource | Action |
|---|---|---|---|
| view the UI | Allows users to view the UI | /flow | R |
| query provenance | Allows users to submit a Provenance Search and request Event Lineage | /provenance | R |
| Policy | Privilege | Resource | Action |
|---|---|---|---|
| view the component | Allows users to view component configuration details | /<component-type>/<component-UUID> | R |
| view the data | Allows users to view metadata and content for this component in flowfile queues in outbound connections and through provenance events | /data/<component-type>/<component-UUID> | R |
| view provenance | Allows users to view provenance events generated by this component | /provenance-data/<component-type>/<component-UUID> | R |
This connector supports following authentication mechanisms
auth: SINGLE_USER)Connector will pass this username and password as used on Nifi Login Page over /access/token REST endpoint. This mode also works when Kerberos login identity provider is set up for Nifi.
auth: CLIENT_CERT)Connector will use client_cert_file(required) and client_key_file(optional), client_key_password(optional) for mutual TLS authentication.
auth: Kerberos)If nifi has been configured to use Kerberos SPNEGO, connector will pass user’s Kerberos ticket to nifi over /access/kerberos REST endpoint. It is assumed that user's Kerberos ticket is already present on the machine on which ingestion runs. This is usually done by installing krb5-user and then running kinit for user.
sudo apt install krb5-user
kinit user@REALM
auth: BASIC_AUTH)Connector will use HTTPBasicAuth with username and password.
auth: NO_AUTH)This is useful for testing purposes.