prefect_dbt.cloud.runs

Module containing tasks and flows for interacting with dbt Cloud job runs

Functions

get_dbt_cloud_run_info ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-dbt/prefect_dbt/cloud/runs.py#L51" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

get_dbt_cloud_run_info(dbt_cloud_credentials: DbtCloudCredentials, run_id: int, include_related: Optional[List[Literal['trigger', 'job', 'debug_logs', 'run_steps']]] = None) -> Dict

A task to retrieve information about a dbt Cloud job run.

Args:

• dbt_cloud_credentials: Credentials for authenticating with dbt Cloud.
• run_id: The ID of the job to trigger.
• include_related: List of related fields to pull with the run. Valid values are "trigger", "job", "debug_logs", and "run_steps". If "debug_logs" is not provided in a request, then the included debug logs will be truncated to the last 1,000 lines of the debug log output file.

Returns:

• The run data returned by the dbt Cloud administrative API.

list_dbt_cloud_run_artifacts ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-dbt/prefect_dbt/cloud/runs.py#L108" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

list_dbt_cloud_run_artifacts(dbt_cloud_credentials: DbtCloudCredentials, run_id: int, step: Optional[int] = None) -> List[str]

A task to list the artifact files generated for a completed run.

Args:

• dbt_cloud_credentials: Credentials for authenticating with dbt Cloud.
• run_id: The ID of the run to list run artifacts for.
• step: The index of the step in the run to query for artifacts. The first step in the run has the index 1. If the step parameter is omitted, then this method will return the artifacts compiled for the last step in the run.

Returns:

• A list of paths to artifact files that can be used to retrieve the generated artifacts.

get_dbt_cloud_run_artifact ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-dbt/prefect_dbt/cloud/runs.py#L159" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

get_dbt_cloud_run_artifact(dbt_cloud_credentials: DbtCloudCredentials, run_id: int, path: str, step: Optional[int] = None) -> Union[Dict, str]

A task to get an artifact generated for a completed run. The requested artifact is saved to a file in the current working directory.

Args:

• dbt_cloud_credentials: Credentials for authenticating with dbt Cloud.
• run_id: The ID of the run to list run artifacts for.
• path: The relative path to the run artifact (e.g. manifest.json, catalog.json, run_results.json)
• step: The index of the step in the run to query for artifacts. The first step in the run has the index 1. If the step parameter is omitted, then this method will return the artifacts compiled for the last step in the run.

Returns:

• The contents of the requested manifest. Returns a Dict if the requested artifact is a JSON file and a str otherwise.

Examples:

Get an artifact of a dbt Cloud job run:

from prefect import flow

from prefect_dbt.cloud import DbtCloudCredentials
from prefect_dbt.cloud.runs import get_dbt_cloud_run_artifact

@flow
def get_artifact_flow():
    credentials = DbtCloudCredentials(api_key="my_api_key", account_id=123456789)

    return get_dbt_cloud_run_artifact(
        dbt_cloud_credentials=credentials,
        run_id=42,
        path="manifest.json"
    )

get_artifact_flow()

Get an artifact of a dbt Cloud job run and write it to a file:

import json

from prefect import flow

from prefect_dbt.cloud import DbtCloudCredentials
from prefect_dbt.cloud.jobs import get_dbt_cloud_run_artifact

@flow
def get_artifact_flow():
    credentials = DbtCloudCredentials(api_key="my_api_key", account_id=123456789)

    get_run_artifact_result = get_dbt_cloud_run_artifact(
        dbt_cloud_credentials=credentials,
        run_id=42,
        path="manifest.json"
    )

    with open("manifest.json", "w") as file:
        json.dump(get_run_artifact_result, file)

get_artifact_flow()

wait_for_dbt_cloud_job_run ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-dbt/prefect_dbt/cloud/runs.py#L250" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

wait_for_dbt_cloud_job_run(run_id: int, dbt_cloud_credentials: DbtCloudCredentials, max_wait_seconds: int = 900, poll_frequency_seconds: int = 10) -> Tuple[DbtCloudJobRunStatus, Dict]

Waits for the given dbt Cloud job run to finish running.

Args:

• run_id: The ID of the run to wait for.
• dbt_cloud_credentials: Credentials for authenticating with dbt Cloud.
• max_wait_seconds: Maximum number of seconds to wait for job to complete
• poll_frequency_seconds: Number of seconds to wait in between checks for run completion.

Raises:

• DbtCloudJobRunTimedOut: When the elapsed wait time exceeds max_wait_seconds.

Returns:

• An enum representing the final dbt Cloud job run status
• A dictionary containing information about the run after completion.

Example:

Classes

DbtCloudJobRunStatus ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-dbt/prefect_dbt/cloud/runs.py#L23" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

dbt Cloud Job statuses.

Methods:

is_terminal_status_code ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/integrations/prefect-dbt/prefect_dbt/cloud/runs.py#L34" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

is_terminal_status_code(cls, status_code: Any) -> bool

Returns True if a status code is terminal for a job run. Returns False otherwise.