ContextQMD
Back to Prefect

prefect-gitlab

docs/integrations/prefect-gitlab/index.mdx

3.6.30.dev34.0 KB
Original Source

The prefect-gitlab library makes it easy to interact with GitLab repositories and credentials.

Getting started

Prerequisites

Install prefect-gitlab

The following command will install a version of prefect-gitlab compatible with your installed version of prefect. If you don't already have prefect installed, it will install the newest version of prefect as well.

bash
pip install "prefect[gitlab]"

Upgrade to the latest versions of prefect and prefect-gitlab:

bash
pip install -U "prefect[gitlab]"

Register newly installed block types

Register the block types in the prefect-gitlab module to make them available for use.

bash
prefect block register -m prefect_gitlab

Examples

In the examples below, you create blocks with Python code. Alternatively, blocks can be created through the Prefect UI.

Store deployment flow code in a private GitLab repository

To create a deployment where the flow code is stored in a private GitLab repository, you can use the GitLabCredentials block.

A deployment can use flow code stored in a GitLab repository without using this library in either of the following cases:

  • The repository is public
  • The deployment uses a Secret block to store the token

Code to create a GitLab Credentials block:

python
from prefect_gitlab import GitLabCredentials


gitlab_credentials_block = GitLabCredentials(token="my_token")
gitlab_credentials_block.save(name="my-gitlab-credentials-block")

Access flow code stored in a private GitLab repository in a deployment

Use the credentials block you created above to pass the GitLab access token during deployment creation. The code below assumes there's flow code in your private GitLab repository.

python
from prefect import flow
from prefect.runner.storage import GitRepository
from prefect_gitlab import GitLabCredentials


if __name__ == "__main__":

    source = GitRepository(
        url="https://gitlab.com/org/private-repo.git",
        credentials=GitLabCredentials.load("my-gitlab-credentials-block")
    )

    source = GitRepository(
    flow.from_source(
        source=source,
        entrypoint="my_file.py:my_flow",
    ).deploy(
        name="private-gitlab-deploy",
        work_pool_name="my_pool",
    )

Alternatively, if you use a prefect.yaml file to create the deployment, reference the GitLab Credentials block in the pull step:

yaml
pull:
    - prefect.deployments.steps.git_clone:
        repository: https://github.com/org/repo.git
        credentials: "{{ prefect.blocks.gitlab-credentials.my-gitlab-credentials-block }}"

Interact with a GitLab repository

The code below shows how to reference a particular branch or tag of a GitLab repository.

python
from prefect_gitlab import GitLabRepository


def save_private_gitlab_block():
    private_gitlab_block = GitLabRepository(
        repository="https://gitlab.com/testing/my-repository.git",
        access_token="YOUR_GITLAB_PERSONAL_ACCESS_TOKEN",
        reference="branch-or-tag-name",
    )

    private_gitlab_block.save("my-private-gitlab-block")


if __name__ == "__main__":
    save_private_gitlab_block()

Exclude the access_token field if the repository is public and exclude the reference field to use the default branch.

Use the newly created block to interact with the GitLab repository.

For example, download the repository contents with the .get_directory() method like this:

python
from prefect_gitlab.repositories import GitLabRepository


def fetch_repo():
    private_gitlab_block = GitLabRepository.load("my-gitlab-block")
    private_gitlab_block.get_directory()


if __name__ == "__main__":
    fetch_repo()

Resources

For assistance using GitLab, consult the GitLab documentation.

Refer to the prefect-gitlab SDK documentation to explore all the capabilities of the prefect-gitlab library.