prefect.runner.storage

Functions

create_storage_from_source ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L909" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

create_storage_from_source(source: str, pull_interval: Optional[int] = 60) -> RunnerStorage

Creates a storage object from a URL.

Args:

• url: The URL to create a storage object from. Supports git and fsspec URLs.
• pull_interval: The interval at which to pull contents from remote storage to local storage

Returns:

• A runner storage compatible object

Classes

RunnerStorage ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L34" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

A storage interface for a runner to use to retrieve remotely stored flow code.

Methods:

destination ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L56" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

destination(self) -> Path

The local file path to pull contents from remote storage to.

pull_code ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L62" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_code(self) -> None

Pulls contents from remote storage to the local filesystem.

pull_interval ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L48" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_interval(self) -> Optional[int]

The interval at which contents from remote storage should be pulled to local storage. If None, remote storage will perform a one-time sync.

set_base_path ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L40" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

set_base_path(self, path: Path) -> None

Sets the base path to use when pulling contents from remote storage to local storage.

to_pull_step ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L68" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

to_pull_step(self) -> dict[str, Any] | list[dict[str, Any]]

Returns a dictionary representation of the storage object that can be used as a deployment pull step.

GitCredentials ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L82" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

GitRepository ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L120" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

Pulls the contents of a git repository to the local filesystem.

Args:

• url: The URL of the git repository to pull from
• credentials: A dictionary of credentials to use when pulling from the repository. If a username is provided, an access token must also be provided.
• name: The name of the repository. If not provided, the name will be inferred from the repository URL.
• branch: The branch to pull from. Defaults to "main".
• pull_interval: The interval in seconds at which to pull contents from remote storage to local storage. If None, remote storage will perform a one-time sync.
• directories: The directories to pull from the Git repository (uses git sparse-checkout)

Examples:

Pull the contents of a private git repository to the local filesystem:

from prefect.runner.storage import GitRepository

storage = GitRepository(
    url="https://github.com/org/repo.git",
    credentials={"username": "oauth2", "access_token": "my-access-token"},
)

await storage.pull_code()

Methods:

destination ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L216" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

destination(self) -> Path

is_current_commit ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L336" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

is_current_commit(self) -> bool

Check if the current commit is the same as the commit SHA

is_shallow_clone ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L323" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

is_shallow_clone(self) -> bool

Check if the repository is a shallow clone

is_sparsely_checked_out ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L311" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

is_sparsely_checked_out(self) -> bool

Check if existing repo is sparsely checked out

pull_code ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L351" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_code(self) -> None

Pulls the contents of the configured repository to the local filesystem.

pull_interval ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L223" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_interval(self) -> Optional[int]

set_base_path ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L219" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

set_base_path(self, path: Path) -> None

to_pull_step ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L530" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

to_pull_step(self) -> dict[str, Any]

RemoteStorage ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L584" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

Pulls the contents of a remote storage location to the local filesystem.

Args:

• url: The URL of the remote storage location to pull from. Supports fsspec URLs. Some protocols may require an additional fsspec dependency to be installed. Refer to the fsspec docs for more details.
• pull_interval: The interval in seconds at which to pull contents from remote storage to local storage. If None, remote storage will perform a one-time sync.
• **settings: Any additional settings to pass the fsspec filesystem class.

Examples:

Pull the contents of a remote storage location to the local filesystem:

from prefect.runner.storage import RemoteStorage

storage = RemoteStorage(url="s3://my-bucket/my-folder")

await storage.pull_code()

Pull the contents of a remote storage location to the local filesystem with additional settings:

from prefect.runner.storage import RemoteStorage
from prefect.blocks.system import Secret

storage = RemoteStorage(
    url="s3://my-bucket/my-folder",
    # Use Secret blocks to keep credentials out of your code
    key=Secret.load("my-aws-access-key"),
    secret=Secret.load("my-aws-secret-key"),
)

await storage.pull_code()

Methods:

destination ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L690" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

destination(self) -> Path

The local file path to pull contents from remote storage to.

pull_code ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L704" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_code(self) -> None

Pulls contents from remote storage to the local filesystem.

pull_interval ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L682" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_interval(self) -> Optional[int]

The interval at which contents from remote storage should be pulled to local storage. If None, remote storage will perform a one-time sync.

set_base_path ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L678" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

set_base_path(self, path: Path) -> None

to_pull_step ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L738" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

to_pull_step(self) -> dict[str, Any]

Returns a dictionary representation of the storage object that can be used as a deployment pull step.

BlockStorageAdapter ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L779" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

A storage adapter for a storage block object to allow it to be used as a runner storage object.

Methods:

destination ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L810" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

destination(self) -> Path

pull_code ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L813" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_code(self) -> None

pull_interval ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L806" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_interval(self) -> Optional[int]

set_base_path ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L802" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

set_base_path(self, path: Path) -> None

to_pull_step ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L824" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

to_pull_step(self) -> dict[str, Any]

LocalStorage ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L847" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

Sets the working directory in the local filesystem. Parameters: Path: Local file path to set the working directory for the flow Examples: Sets the working directory for the local path to the flow: python from prefect.runner.storage import Localstorage storage = LocalStorage( path="/path/to/local/flow_directory", )

Methods:

destination ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L873" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

destination(self) -> Path

pull_code ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L883" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_code(self) -> None

pull_interval ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L880" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

pull_interval(self) -> Optional[int]

set_base_path ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L876" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

set_base_path(self, path: Path) -> None

to_pull_step ^{<a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/runner/storage.py#L888" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a>}

to_pull_step(self) -> dict[str, Any]

Returns a dictionary representation of the storage object that can be used as a deployment pull step.