Back to Pixi

Pixi Pack

docs/deployment/pixi_pack.md

0.67.28.4 KB
Original Source
<!-- Keep in sync with https://github.com/quantco/pixi-pack/blob/main/README.md -->

pixi-pack is a simple tool that takes an environment and packs it into a compressed archive that can be shipped to the target machine. The corresponding pixi-unpack tool can be used to unpack the archive and recreate an environment.

Both tools can be installed via

bash
pixi global install pixi-pack pixi-unpack

Or by downloading our pre-built binaries from the releases page.

Instead of installing pixi-pack and pixi-unpack globally, you can also use pixi exec to run pixi-pack in a temporary environment:

bash
pixi exec pixi-pack
pixi exec pixi-unpack environment.tar

!!!note "" You can also write pixi pack (and pixi unpack) if you have pixi, and pixi-pack and pixi-unpack installed globally.

You can pack an environment with

bash
pixi-pack --environment prod --platform linux-64 pixi.toml

This will create an environment.tar file that contains all conda packages required to create the environment.

plain
# environment.tar
| pixi-pack.json
| environment.yml
| channel
|    ├── noarch
|    |    ├── tzdata-2024a-h0c530f3_0.conda
|    |    ├── ...
|    |    └── repodata.json
|    └── linux-64
|         ├── ca-certificates-2024.2.2-hbcca054_0.conda
|         ├── ...
|         └── repodata.json

pixi-unpack: Unpacking an environment

With pixi-unpack environment.tar, you can unpack the environment on your target system. This will create a new conda environment in ./env that contains all packages specified in your pixi.toml. It also creates an activate.sh (or activate.bat on Windows) file that lets you activate the environment without needing to have conda or micromamba installed.

bash
$ pixi-unpack environment.tar
$ ls
env/
activate.sh
environment.tar
$ cat activate.sh
export PATH="/home/user/project/env/bin:..."
export CONDA_PREFIX="/home/user/project/env"
. "/home/user/project/env/etc/conda/activate.d/activate_custom_package.sh"

Cross-platform Packs

Since pixi-pack just downloads the .conda and .tar.bz2 files from the conda repositories, you can trivially create packs for different platforms.

bash
pixi-pack --platform win-64

!!! note You can only unpack a pack on a system that has the same platform as the pack was created for.

Self-Extracting Binaries

You can create a self-extracting binary that contains the packed environment and a script that unpacks the environment. This can be useful if you want to distribute the environment to users that don't have pixi-unpack installed.

=== "Linux & macOS" bash $ pixi-pack --create-executable $ ls environment.sh $ ./environment.sh $ ls env/ activate.sh environment.sh

=== "Windows" powershell PS > pixi-pack --create-executable PS > ls environment.ps1 PS > .\environment.ps1 PS > ls env/ activate.sh environment.ps1

Custom pixi-unpack executable path

When creating a self-extracting binary, you can specify a custom path or URL to a pixi-unpack executable to avoid downloading it from the default location.

You can provide one of the following as the --pixi-unpack-source:

  • a URL to a pixi-unpack executable like https://my.mirror/pixi-pack/pixi-unpack-x86_64-unknown-linux-musl
  • a path to a pixi-unpack binary like ./pixi-unpack-x86_64-unknown-linux-musl
Example Usage

Using a URL:

bash
pixi-pack --create-executable --pixi-unpack-source https://my.mirror/pixi-pack/pixi-unpack-x86_64-unknown-linux-musl

Using a path:

bash
pixi-pack --create-executable --pixi-unpack-source ./pixi-unpack-x86_64-unknown-linux-musl

!!! note

The produced executable is a simple shell script that contains both the `pixi-unpack` binary as well as the packed environment.

Inject Additional Packages

You can inject additional packages into the environment that are not specified in pixi.lock by using the --inject flag:

bash
pixi-pack --inject local-package-1.0.0-hbefa133_0.conda pixi.toml

This can be particularly useful if you build the package itself and want to include the built package in the environment but still want to use pixi.lock from the workspace.

PyPi support

You can also pack PyPi wheel packages into your environment. pixi-pack only supports wheel packages and not source distributions. If you happen to use source distributions, you can ignore them by using the --ignore-pypi-non-wheel flag. This will skip the bundling of PyPi packages that are source distributions.

The --inject option also supports wheels.

bash
pixi-pack --ignore-pypi-non-wheel --inject my_webserver-0.1.0-py3-none-any.whl

!!! warning

In contrast to injecting from conda packages,
we cannot verify that injected wheels are compatible with the target environment. Please make sure the packages are compatible.

Mirror and S3 middleware

You can use mirror middleware by creating a configuration file as described in the pixi documentation and referencing it using --config.

toml
[mirrors]
"https://conda.anaconda.org/conda-forge" = ["https://my.artifactory/conda-forge"]

If you are using S3 in pixi, you can also add the appropriate S3 config in your config file and reference it.

toml
[s3-options.my-s3-bucket]
endpoint-url = "https://s3.eu-central-1.amazonaws.com"
region = "eu-central-1"
force-path-style = false

Setting maximum number of parallel downloads

toml
[concurrency]
downloads = 5

Use pixi-pack --config config.toml to use the custom configuration file. See pixi docs for more information.

Cache Downloaded Packages

You can cache downloaded packages to speed up subsequent pack operations by using the --use-cache flag:

bash
pixi-pack --use-cache ~/.pixi-pack/cache

This will store all downloaded packages in the specified directory and reuse them in future pack operations. The cache follows the same structure as conda channels, organizing packages by platform subdirectories (e.g., linux-64, win-64, etc.).

Using a cache is particularly useful when:

  • Creating multiple packs with overlapping dependencies
  • Working with large packages that take time to download
  • Operating on machines with limited bandwidth
  • Running CI/CD pipelines where package caching can significantly improve build times

Unpacking Without pixi-pack

If you don't have pixi-pack available on your target system, and do not want to use self-extracting binaries (see above), you can still install the environment if you have conda or micromamba available. Just unarchive the environment.tar, then you have a local channel on your system where all necessary packages are available. Next to this local channel, you will find an environment.yml file that contains the environment specification. You can then install the environment using conda or micromamba:

bash
tar -xvf environment.tar
micromamba create -p ./env --file environment.yml
# or
conda env create -p ./env --file environment.yml

!!! note

The `environment.yml` and `repodata.json` files are only for this use case, `pixi-unpack` does not use them.

!!! note

Both `conda` and `mamba` are always installing pip as a side effect when they install python, see [`conda`'s documentation](https://docs.conda.io/projects/conda/en/25.1.x/user-guide/configuration/settings.html#add-pip-as-python-dependency-add-pip-as-python-dependency).
This is different from how `pixi` works and can lead to solver errors when using `pixi-pack`'s compatibility mode since `pixi` doesn't include `pip` by default.
You can fix this issue in two ways:

- Add `pip` to your `pixi.lock` file using `pixi add pip`.
- Configuring `conda` (or `mamba`) to not install `pip` by default by running `conda config --set add_pip_as_python_dependency false` (or by adding `add_pip_as_python_dependency: False` to your `~/.condarc`)