Back to Prefect

utility

docs/v3/api-ref/python/prefect-deployments-steps-utility.mdx

3.6.30.dev35.0 KB
Original Source

prefect.deployments.steps.utility

Utility project steps that are useful for managing a project's deployment lifecycle.

Steps within this module can be used within a build, push, or pull deployment action.

Example: Use the run_shell_script setp to retrieve the short Git commit hash of the current repository and use it as a Docker image tag: yaml build: - prefect.deployments.steps.run_shell_script: id: get-commit-hash script: git rev-parse --short HEAD stream_output: false - prefect_docker.deployments.steps.build_docker_image: requires: prefect-docker image_name: my-image image_tag: "{{ get-commit-hash.stdout }}" dockerfile: auto

Functions

run_shell_script <sup><a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/deployments/steps/utility.py#L113" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

python
run_shell_script(script: str, directory: Optional[str] = None, env: Optional[Dict[str, str]] = None, stream_output: bool = True, expand_env_vars: bool = False, shell: bool = False) -> RunShellScriptResult

Runs one or more shell commands in a subprocess. Returns the standard output and standard error of the script.

Args:

  • script: The script to run
  • directory: The directory to run the script in. Defaults to the current working directory.
  • env: A dictionary of environment variables to set for the script
  • stream_output: Whether to stream the output of the script to stdout/stderr
  • expand_env_vars: Whether to expand environment variables in the script before running it
  • shell: Whether to run the command through the system shell. When True, shell operators like pipes (|), redirects (>), and logical operators (&&, ||) are supported. Only set this to True when you need shell features, as it has security implications similar to subprocess.run(shell=True).

Returns:

  • A dictionary with the keys stdout and stderr containing the output of the script

Examples:

Retrieve the short Git commit hash of the current repository to use as a Docker image tag:

yaml
build:
    - prefect.deployments.steps.run_shell_script:
        id: get-commit-hash
        script: git rev-parse --short HEAD
        stream_output: false
    - prefect_docker.deployments.steps.build_docker_image:
        requires: prefect-docker
        image_name: my-image
        image_tag: "{{ get-commit-hash.stdout }}"
        dockerfile: auto

Run a multi-line shell script:

yaml
build:
    - prefect.deployments.steps.run_shell_script:
        script: |
            echo "Hello"
            echo "World"

Run a shell script with environment variables:

yaml
build:
    - prefect.deployments.steps.run_shell_script:
        script: echo "Hello $NAME"
        env:
            NAME: World

Run a shell script with environment variables expanded from the current environment:

yaml
pull:
    - prefect.deployments.steps.run_shell_script:
        script: |
            echo "User: $USER"
            echo "Home Directory: $HOME"
        stream_output: true
        expand_env_vars: true

Run a shell script in a specific directory:

yaml
build:
    - prefect.deployments.steps.run_shell_script:
        script: echo "Hello"
        directory: /path/to/directory

Run a script stored in a file:

yaml
build:
    - prefect.deployments.steps.run_shell_script:
        script: "bash path/to/script.sh"

Run a command that uses shell operators like pipes:

yaml
push:
    - prefect.deployments.steps.run_shell_script:
        script: echo "hello world" | tr '[:lower:]' '[:upper:]'
        shell: true

pip_install_requirements <sup><a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/deployments/steps/utility.py#L278" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

python
pip_install_requirements(directory: Optional[str] = None, requirements_file: str = 'requirements.txt', stream_output: bool = True) -> dict[str, Any]

Installs dependencies from a requirements.txt file.

Args:

  • requirements_file: The requirements.txt to use for installation.
  • directory: The directory the requirements.txt file is in. Defaults to the current working directory.
  • stream_output: Whether to stream the output from pip install should be streamed to the console

Returns:

  • A dictionary with the keys stdout and stderr containing the output the pip install command

Raises:

  • subprocess.CalledProcessError: if the pip install command fails for any reason

Classes

RunShellScriptResult <sup><a href="https://github.com/PrefectHQ/prefect/blob/main/src/prefect/deployments/steps/utility.py#L100" target="_blank"><Icon icon="github" style="width: 14px; height: 14px;" /></a></sup>

The result of a run_shell_script step.

Attributes:

  • stdout: The captured standard output of the script.
  • stderr: The captured standard error of the script.