doc/user/packages/pypi_repository/_index.md
{{< details >}}
{{< /details >}}
The Python Package Index (PyPI) is the official third-party software repository for Python. Use the GitLab PyPI package registry to publish and share Python packages in your GitLab projects, groups, and organizations. This integration enables you to manage your Python dependencies alongside your code, providing a seamless workflow for Python development within GitLab.
The package registry works with:
For documentation of the specific API endpoints that the pip and twine
clients use, see the PyPI API documentation.
Learn how to build a PyPI package.
When using the GitLab PyPI package registry, package requests that cannot be found in the GitLab registry are automatically forwarded to pypi.org. This behavior can result in packages being downloaded from pypi.org, even when using the --index-url flag.
For maximum security when working with private packages:
--index-url and --no-index flags when installing packages.Prerequisites:
To turn off package request forwarding:
For information on how to turn off package forwarding in the Admin area, see Control package forwarding.
Before you interact with the GitLab package registry, you must authenticate with it.
You can authenticate with:
api.read_package_registry, write_package_registry, or both.Do not use authentication methods other than the methods documented here. Undocumented authentication methods might be removed in the future.
To authenticate with a GitLab token:
TWINE_USERNAME and TWINE_PASSWORD environment variables.For example:
{{< tabs >}}
{{< tab title="With a personal access token" >}}
run:
image: python:latest
variables:
TWINE_USERNAME: <personal_access_token_name>
TWINE_PASSWORD: <personal_access_token>
script:
- pip install build twine
- python -m build
- python -m twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/*
{{< /tab >}}
{{< tab title="With a deploy token" >}}
run:
image: python:latest
variables:
TWINE_USERNAME: <deploy_token_username>
TWINE_PASSWORD: <deploy_token>
script:
- pip install build twine
- python -m build
- python -m twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/*
{{< /tab >}}
{{< tab title="With a CI/CD job token" >}}
run:
image: python:latest
variables:
TWINE_USERNAME: gitlab-ci-token
TWINE_PASSWORD: $CI_JOB_TOKEN
script:
- pip install build twine
- python -m build
- python -m twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/*
{{< /tab >}}
{{< /tabs >}}
To authenticate with the package registry for a group:
https://gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi
You can publish a package with twine.
Prerequisites:
description of 4000 characters or less. Longer description strings are truncated.400 Bad Request.PyPI packages are published using your project ID. If your project is in a group, PyPI packages published to the project registry are also available in the group registry. For more information, see Install from a group.
To publish a package:
Define your repository source, edit the ~/.pypirc file and add:
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
Upload your package with twine:
python3 -m twine upload --repository gitlab dist/*
When a package is published successfully, a message like this is displayed:
Uploading distributions to https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
Uploading mypypipackage-0.0.1-py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s]
Uploading mypypipackage-0.0.1.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s]
The package is published to the package registry, and is shown on the Packages and registries page.
If you didn't use a .pypirc file to define your repository source, you can
publish to the repository with inline authentication:
TWINE_PASSWORD=<personal_access_token, deploy_token, or $CI_JOB_TOKEN> \
TWINE_USERNAME=<username, deploy_token_username, or gitlab-ci-token> \
python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/*
You cannot publish a package if a package of the same name and version already exists.
You must delete the existing package first.
If you attempt to publish the same package
more than once, a 400 Bad Request error occurs.
By default, when a PyPI package is not found in the GitLab package registry, the request is forwarded to pypi.org. For more information about how to prevent request forwarding, see package request forwarding and security notice.
This behavior:
--index-url flagAdministrators can disable this behavior globally in the Continuous Integration settings. Group Owners can disable this behavior for specific groups in the Packages and registries section of the group settings.
[!note] When you use the
--index-urloption, do not specify the port if it is a default port.httpURLs default to 80, andhttpsURLs default to 443.
To install the latest version of a package, use the following command:
pip install --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name>
<package_name> is the package name.<personal_access_token_name> is a personal access token name with the read_api scope.<personal_access_token> is a personal access token with the read_api scope.<project_id> is either the project's URL-encoded
path (for example, group%2Fproject), or the project's ID (for example 42).In these commands, you can use --extra-index-url instead of --index-url. If you were following the guide and want to install the
MyPyPiPackage package, you can run:
pip install mypypipackage --no-deps --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
This message indicates that the package was installed successfully:
Looking in indexes: https://<personal_access_token_name>:****@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
Collecting mypypipackage
Downloading https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB)
Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.0.1
To install the latest version of a package from a group, use the following command:
pip install --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi/simple --no-deps <package_name>
In this command:
<package_name> is the package name.<personal_access_token_name> is a personal access token name with the read_api scope.<personal_access_token> is a personal access token with the read_api scope.<group_id> is the group ID.In these commands, you can use --extra-index-url instead of --index-url.
If you're following the guide and want to install the MyPyPiPackage package, you can run:
pip install mypypipackage --no-deps --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi/simple
GitLab looks for packages that use
Python normalized names (PEP-503).
The characters -, _, and . are all treated the same, and repeated
characters are removed.
A pip install request for my.package looks for packages that match any of
the three characters, such as my-package, my_package, and my....package.
The security implications of using --extra-index-url versus --index-url when installing PyPI
packages are significant and worth understanding in detail:
--index-url: This option replaces the default PyPI index URL with the specified URL. The GitLab package forwarding setting, which is turned on by default, might still download packages not found in the package registry from PyPI. To ensure packages are only installed from GitLab, either:
--index-url and --no-index flags together--extra-index-url: This option adds an additional index to search, alongside the default PyPI index.
It's less secure and more open to dependency confusion attacks, because it checks both the default PyPI
and the additional index for packages.When using private packages, keep in mind the following best practices:
--no-index and --index-url flags together when installing private packages.pip debug.Prerequisites:
Before you delete a package, make sure you understand the associated security risks.
To delete a package, you can either:
You can install private PyPI packages in CI/CD pipelines using a CI/CD job token. The job token authenticates automatically and can also access packages in other projects within the same top-level group.
Prerequisites:
gitlab-ci-token.Pass the CI_JOB_TOKEN directly in the pip install command:
{{< tabs >}}
{{< tab title="From a project" >}}
install:
image: python:latest
script:
- pip install <package_name> --index-url https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
{{< /tab >}}
{{< tab title="From a group" >}}
To install from a group index, which includes packages from all projects in the group:
install:
image: python:latest
script:
- pip install <package_name> --index-url https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi/simple
{{< /tab >}}
{{< /tabs >}}
.netrc file for CI/CD.netrc keeps your requirements.txt free of credentials,
so the same file works both locally and in CI/CD pipelines. pip reads the .netrc file
automatically to find credentials for a given hostname. For more information,
see .netrc support.
In your .gitlab-ci.yml, create the .netrc file in before_script:
install:
image: python:latest
before_script:
- |
echo "machine gitlab.example.com
login gitlab-ci-token
password ${CI_JOB_TOKEN}" > ~/.netrc
script:
- pip install -r requirements.txt
Your requirements.txt references the GitLab registry without credentials:
--extra-index-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
package-name==1.0.0
For local development, create a ~/.netrc file with a
personal access token that has the api scope:
machine gitlab.example.com
login <personal_access_token_name>
password <personal_access_token>
[!note] The
.netrcfile configures one set of credentials per hostname. If you must authenticate with packages on multiple GitLab instances, add a separatemachineentry for each hostname.
When building Docker images that require private PyPI packages, pass the
CI_JOB_TOKEN as a build argument:
build:
image: docker:latest
services:
- docker:dind
script:
- docker build --build-arg CI_JOB_TOKEN=$CI_JOB_TOKEN -t my-image .
In your Dockerfile, use the token to authenticate with the registry:
ARG CI_JOB_TOKEN
RUN pip install <package_name> --index-url https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
[!note] Build arguments might be visible in the image history. For production images, use multi-stage builds so the token is not present in the final image layer.
requirements.txtIf you want pip to access your public registry, add the --extra-index-url parameter along with the URL for your registry to your requirements.txt file.
--extra-index-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
package-name==1.0.0
If this is a private registry, you can authenticate in a couple of ways. For example:
Using your requirements.txt file with a personal access token:
--extra-index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
package-name==1.0.0
Using a ~/.netrc file with a personal access token:
machine gitlab.example.com
login <personal_access_token_name>
password <personal_access_token>
Proper versioning is important for managing PyPI packages effectively. Follow these best practices to ensure your packages are versioned correctly.
Adopt semantic versioning for your packages. The version number should be in the format MAJOR.MINOR.PATCH:
MAJOR version for incompatible API changes.MINOR version for backwards-compatible new features.PATCH version for backwards-compatible bug fixes.For example: 1.0.0, 1.1.0, 1.1.1.
For new projects, start with version 0.1.0. This indicates an initial development phase where the API is not yet stable.
Ensure your version string is valid according to PyPI standards. GitLab uses a specific regex to validate version strings:
\A(?:
v?
(?:([0-9]+)!)? (?# epoch)
([0-9]+(?:\.[0-9]+)*) (?# release segment)
([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)? (?# pre-release)
((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))? (?# post release)
([-_\.]?(dev)[-_\.]?([0-9]+)?)? (?# dev release)
(?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))? (?# local version)
)\z}xi
You can experiment with the regex and try your version strings by using this regular expression editor.
For more details about the regex, see the Python documentation.
The GitLab PyPI repository supports the following CLI commands:
twine upload: Upload a package to the registry.pip install: Install a PyPI package from the registry.To improve performance, the pip command caches files related to a package. Pip doesn't remove data by itself. The cache grows as new packages are installed. If you encounter issues, clear the cache with this command:
pip cache purge
index-url or extra-index-url parametersYou can define multiple index-url and extra-index-url parameters.
If you use the same domain name (such as gitlab.example.com) multiple times with different authentication
tokens, pip may not be able to find your packages. This problem is due to how pip
registers and stores your tokens during commands executions.
To workaround this issue, you can use a group deploy token with the
scope read_package_registry from a common parent group for all projects or groups targeted by the
index-url and extra-index-url values.
If packages are being installed from PyPI when you intended to use only the GitLab registry:
--no-index and --index-url flags together to prevent PyPI fallback.pip debug.