docs/workspace/advanced_tasks.md
When building a package, you often have to do more than just run the code. Steps like formatting, linting, compiling, testing, benchmarking, etc. are often part of a workspace. With Pixi tasks, this should become much easier to do.
Here are some quick examples
[tasks]
# Commands as lists so you can also add documentation in between.
configure = { cmd = [
"cmake",
# Use the cross-platform Ninja generator
"-G",
"Ninja",
# The source is in the root directory
"-S",
".",
# We wanna build in the .build directory
"-B",
".build",
] }
# Add task descriptions, to be surfaced when the tasks are listed
say-hello = { cmd = ["echo", "hello world"], description = "Greet the world." }
# Depend on other tasks
build = { cmd = ["ninja", "-C", ".build"], depends-on = ["configure"] }
# Using environment variables
run = "python main.py $PIXI_PROJECT_ROOT"
set = "export VAR=hello && echo $VAR"
# Cross platform file operations
copy = "cp pixi.toml pixi_backup.toml"
clean = "rm pixi_backup.toml"
move = "mv pixi.toml backup.toml"
# Setting a default environment for the task
test = { cmd = "pytest", default-environment = "test" }
Just like packages can depend on other packages, our tasks can depend on other tasks. This allows for complete pipelines to be run with a single command.
An obvious example is compiling before running an application.
Checkout our cpp_sdl example for a running example.
In that package we have some tasks that depend on each other, so we can assure that when you run pixi run start everything is set up as expected.
pixi task add configure "cmake -G Ninja -S . -B .build"
pixi task add build "ninja -C .build" --depends-on configure
pixi task add start ".build/bin/sdl_example" --depends-on build
Results in the following lines added to the pixi.toml
[tasks]
# Configures CMake
configure = "cmake -G Ninja -S . -B .build"
# Build the executable but make sure CMake is configured first.
build = { cmd = "ninja -C .build", depends-on = ["configure"] }
# Start the built executable
start = { cmd = ".build/bin/sdl_example", depends-on = ["build"] }
The tasks will be executed after each other:
configure because it has no dependencies.build as it only depends on configure.start as all its dependencies are run.If one of the commands fails (exit with non-zero code.) it will stop and the next one will not be started.
With this logic, you can also create aliases as you don't have to specify any command in a task.
pixi task add fmt ruff
pixi task add lint pylint
--8<-- "docs/source_files/pixi_tomls/pixi_task_alias.toml:not-all"
!!! tip "Hiding Tasks"
Tasks can be hidden from user facing commands by naming them with an _ prefix.
Pixi supports a shorthand syntax for defining tasks that only depend on other tasks. Instead of using the more verbose depends-on field, you can define a task directly as an array of dependencies.
Executing:
pixi task alias style fmt lint
results in the following pixi.toml:
--8<-- "docs/source_files/pixi_tomls/pixi_task_alias.toml:all"
Now you can run both tools with one command.
pixi run style
You can specify the environment to use for a dependent task:
--8<-- "docs/source_files/pixi_tomls/tasks_depends_on.toml:tasks"
This allows you to run tasks in different environments as part of a single pipeline. When you run the main task, Pixi ensures each dependent task uses its specified environment:
pixi run test-all
The environment specified for a task dependency takes precedence over the environment specified via
the CLI --environment flag. This means even if you run pixi run test-all --environment py312,
the first dependency will still run in the py311 environment as specified in the TOML file.
In the example above, the test-all task runs the test task in both Python 3.11 and 3.12 environments,
allowing you to verify compatibility across different Python versions with a single command.
Pixi tasks support the definition of a working directory.
cwd stands for Current Working Directory.
The directory is relative to the Pixi workspace root, where the pixi.toml file is located.
By default, tasks are executed from the Pixi workspace root.
To change this, use the --cwd flag.
For example, consider a Pixi workspace structured as follows:
├── pixi.toml
└── scripts
└── bar.py
To add a task that runs the bar.py file from the scripts directory, use:
pixi task add bar "python bar.py" --cwd scripts
This will add the following line to manifest file:
[tasks]
bar = { cmd = "python bar.py", cwd = "scripts" }
You can set the default Pixi environment used by a task using the default-environment field:
--8<-- "docs/source_files/pixi_tomls/task_default_environment.toml:default-environment"
The default environment can be overridden as usual with the --environment argument:
pixi run -e other_environment test
Tasks can accept arguments that can be referenced in the command. This provides more flexibility and reusability for your tasks.
Task arguments make your tasks more versatile and maintainable:
For example, instead of creating separate build tasks for development and production modes, you can create a single parameterized task that handles both cases.
Arguments can be:
choicesDefine arguments in your task using the args field:
--8<-- "docs/source_files/pixi_tomls/task_arguments.toml:project_tasks"
!!! note "Argument naming restrictions"
Argument names cannot contain dashes (-) due to them being seen as a minus sign in MiniJinja. Use underscores (_) or camelCase instead.
When running a task, provide arguments in the order they are defined:
# Required argument
pixi run greet John
✨ Pixi task (greet in default): echo Hello, John!
# Default values are used when omitted
pixi run build
✨ Pixi task (build in default): echo Building my-app in development mode
# Override default values
pixi run build my-project production
✨ Pixi task (build in default): echo Building my-project in production mode
# Mixed argument types
pixi run deploy auth-service
✨ Pixi task (deploy in default): echo Deploying auth-service to staging
pixi run deploy auth-service production
✨ Pixi task (deploy in default): echo Deploying auth-service to production
--When a task defines typed args, all command-line values are matched against those definitions. If you need to pass additional flags or arguments directly to the underlying command on top of the typed args, use -- as a separator. Everything after -- is forwarded verbatim to the command, regardless of the task's args definition.
[tasks.test]
cmd = "pytest {{ target }} -v"
args = [{ arg = "target", default = "tests/unit" }]
# Without --, extra flags would cause an error:
# × task 'test' received more arguments than expected
# hint: use `--` to separate task arguments from extra passthrough arguments
pixi run test tests/integration --tb=short --maxfail=5
# With --, the typed arg is filled and the rest is forwarded:
pixi run test tests/integration -- --tb=short --maxfail=5
✨ Pixi task (test in default): pytest tests/integration -v --tb=short --maxfail=5
You can also use -- when relying on a default argument value:
# "target" uses its default, "--maxfail=5" is forwarded to the command
pixi run test -- --maxfail=5
✨ Pixi task (test in default): pytest tests/unit -v --maxfail=5
!!! note "Tasks without typed args"
For tasks that do not define args, -- is passed through to the underlying command unchanged. This preserves its meaning for programs that use -- themselves (e.g. git log -- somefile).
You can restrict the allowed values of an argument using the choices field. If a value is provided that is not in the list, Pixi will report an error instead of running the task.
--8<-- "docs/source_files/pixi_tomls/task_arguments.toml:project_tasks_choices"
# Providing a valid choice
pixi run compile debug
✨ Pixi task (compile): echo 'Compiling in debug mode'
Compiling in debug mode
# Default value must also be one of the choices
pixi run test
✨ Pixi task (test): echo 'Running unit tests'
Running unit tests
# Providing an invalid value results in an error
pixi run compile fast
× got 'fast' for argument 'mode' of task 'compile', choose from: debug, release
You can pass arguments to tasks that are dependencies of other tasks:
--8<-- "docs/source_files/pixi_tomls/task_arguments_dependent.toml:project_tasks"
When executing a dependent task, the arguments are passed to the dependency:
pixi run install-release
✨ Pixi task (install in default): echo Installing with manifest /path/to/manifest and flag --debug
pixi run deploy
✨ Pixi task (install in default): echo Installing with manifest /custom/path and flag --verbose
✨ Pixi task (deploy in default): echo Deploying
When a dependent task doesn't specify all arguments, the default values are used for the missing ones:
--8<-- "docs/source_files/pixi_tomls/task_arguments_partial.toml:project_tasks"
pixi run partial-override
✨ Pixi task (base-task in default): echo Base task with override1 and default2
For a dependent task to accept arguments to pass to the dependency, you can use the same syntax as passing arguments to the command:
--8<-- "docs/source_files/pixi_tomls/task_arguments_partial.toml:project_tasks_with_arg"
pixi run partial-override-with-arg
✨ Pixi task (base-task in default): echo Base task with override1 and new-default2
pixi run partial-override-with-arg cli-arg
✨ Pixi task (base-task in default): echo Base task with override1 and cli-arg
Task commands and env variables defined in the manifest support MiniJinja templating syntax for accessing and formatting argument values. This provides powerful flexibility when constructing commands.
!!! note "Templating and ad-hoc CLI commands"
MiniJinja templating is only applied to tasks defined in your manifest file (pixi.toml / pyproject.toml).
Ad-hoc commands passed directly to pixi run on the command line are not templated by default,
so commands like pixi run echo '{{ hello }}' are passed through as-is.
Use the --templated flag to opt in to template rendering for CLI commands:
```shell
pixi run --templated echo '{{ pixi.platform }}'
```
Basic syntax for using an argument in your command:
--8<-- "docs/source_files/pixi_tomls/task_minijinja_simple.toml:tasks"
You can also use filters to transform argument values:
--8<-- "docs/source_files/pixi_workspaces/minijinja/task_args/pixi.toml:tasks"
In addition to task arguments, Pixi automatically provides a pixi object in the MiniJinja context with system variables:
| Variable | Description | Example Value |
|---|---|---|
pixi.platform | The platform name for the environment in which the task will run | linux-64, osx-arm64, win-64 |
pixi.environment.name | The name of the current environment (when available) | default, prod, test |
pixi.manifest_path | Absolute path to the manifest file | /path/to/project/pixi.toml |
pixi.version | The version of pixi being used | 0.59.0 |
pixi.init_cwd | The current working directory when pixi was invoked | /path/to/cwd |
pixi.is_win | Boolean flag indicating if the platform is Windows | true or false |
pixi.is_unix | Boolean flag indicating if the platform is Unix-like | true or false |
pixi.is_linux | Boolean flag indicating if the platform is Linux | true or false |
pixi.is_osx | Boolean flag indicating if the platform is macOS | true or false |
These variables are particularly useful for creating platform-specific or environment-aware tasks:
[tasks]
# Platform-specific commands
build = { cmd = "cargo build --target {{ pixi.platform }}", args = [] }
download-binary = { cmd = "curl -O https://example.com/binary-{{ pixi.platform }}.tar.gz", args = [] }
# Conditional execution based on platform
install = { cmd = "{% if pixi.is_win %}install.bat{% else %}./install.sh{% endif %}", args = [] }
# Environment-aware tasks
deploy = { cmd = "deploy.sh --env {{ pixi.environment.name }}", args = [] }
# Using manifest path
validate = { cmd = "validator --manifest {{ pixi.manifest_path }}", args = [] }
# Using init_cwd in inputs/outputs for caching
mkdir_test = { cmd = "mkdir -p {{ pixi.init_cwd }}/test", outputs = ["{{ pixi.init_cwd }}/test"] }
The pixi variables can also be combined with task arguments:
[tasks]
deploy = {
cmd = "deploy.sh --platform {{ pixi.platform }} --env {{ environment }} --version {{ pixi.version }}",
args = [{ arg = "environment", default = "staging" }]
}
When running tasks with typed arguments, the platform will automatically reflect the best platform for the environment where the task executes.
For more information about available filters and template syntax, see the MiniJinja documentation.
A task name follows these rules:
_ at the start of the name will hide the task from the pixi task list command.Hiding tasks can be useful if your workspace defines many tasks but your users only need to use a subset of them.
--8<-- "docs/source_files/pixi_tomls/task_visibility.toml:project_tasks"
When you specify inputs and/or outputs to a task, Pixi will reuse the result of the task.
For the cache, Pixi checks that the following are true:
If all of these conditions are met, Pixi will not run the task again and instead use the existing result.
Inputs and outputs can be specified as globs, which will be expanded to all matching files. You can also use MiniJinja templates in your inputs and outputs fields to parameterize the paths, making tasks more reusable:
--8<-- "docs/source_files/pixi_tomls/tasks_minijinja_inputs_outputs.toml:tasks"
When using template variables in inputs/outputs, Pixi expands the templates using the provided arguments or environment variables, and uses the resolved paths for caching decisions. This allows you to create generic tasks that can handle different files without duplicating task configurations:
# First run processes the file and caches the result
pixi run process-file data1
# Second run with the same argument uses the cached result
pixi run process-file data1 # [cache hit]
# Run with a different argument processes a different file
pixi run process-file data2
Note: if you want to debug the globs you can use the --verbose flag to see which files are selected.
# shows info logs of all files that were selected by the globs
pixi run -v start
You can set environment variables directly for a task, as well as by other means. See the environment variable priority documentation for full details of ways to set environment variables, and how those ways interact with each other.
Notes on environment variables in tasks:
tasks.<name>.env are interpreted by deno_task_shell when the task runs. Shell-style expansions like env = { VAR = "$FOO" } therefore work the same on all operating systems.{ cmd="pytest", env={ BACKEND="{{ backend }}" }, args=[{arg="backend", default="numpy"}] }. The arg {{ backend }} value is interpreted by the backend values passed in.!!! warning
In older versions of Pixi, this priority was not well-defined, and there are a number of known
deviations from the current priority which exist in some older versions:
- `activation.scripts` used to take priority over `activation.env`
- activation scripts of dependencies used to take priority over `activation.env`
- outside environment variables used to override variables set in `task.env`
If you previously relied on a certain priority which no longer applies, you may need to change your
task definitions.
For the specific case of overriding `task.env` with outside environment variables, this behavior can
now be recreated using [task arguments](#task-arguments). For example, if you were previously using
a setup like:
```toml title="pixi.toml"
[tasks]
echo = { cmd = "echo $ARGUMENT", env = { ARGUMENT = "hello" } }
```
```shell
ARGUMENT=world pixi run echo
✨ Pixi task (echo in default): echo $ARGUMENT
world
```
you can now recreate this behavior like:
```toml title="pixi.toml"
[tasks]
echo = { cmd = "echo {{ ARGUMENT }}", args = [{"arg" = "ARGUMENT", "default" = "hello" }] }
```
```shell
pixi run echo world
✨ Pixi task (echo): echo world
world
```
You can make sure the environment of a task is "Pixi only".
Here Pixi will only include the minimal required environment variables for your platform to run the command in.
The environment will contain all variables set by the conda environment like "CONDA_PREFIX".
It will however include some default values from the shell, like:
"DISPLAY", "LC_ALL", "LC_TIME", "LC_NUMERIC", "LC_MEASUREMENT", "SHELL", "USER", "USERNAME", "LOGNAME", "HOME", "HOSTNAME","TMPDIR", "XPC_SERVICE_NAME", "XPC_FLAGS"
[tasks]
clean_command = { cmd = "python run_in_isolated_env.py", clean-env = true }
This setting can also be set from the command line with pixi run --clean-env TASK_NAME.
!!! warning "clean-env not supported on Windows"
On Windows it's hard to create a "clean environment" as conda-forge doesn't ship Windows compilers and Windows needs a lot of base variables.
Making this feature not worthy of implementing as the amount of edge cases will make it unusable.
To support the different OS's (Windows, OSX and Linux), Pixi integrates a shell that can run on all of them.
This is deno_task_shell.
The task shell is a limited implementation of a bourne-shell interface.
Task command lines and the values of tasks.<name>.env are parsed and expanded by this shell.
Next to running actual executable like ./myprogram, cmake or python the shell has some built-in commands.
cp: Copies files.mv: Moves files.rm: Remove files or directories.
Ex: rm -rf [FILE]... - Commonly used to recursively delete files or directories.mkdir: Makes directories.
Ex. mkdir -p DIRECTORY... - Commonly used to make a directory and all its parents with no error if it exists.pwd: Prints the name of the current/working directory.sleep: Delays for a specified amount of time.
Ex. sleep 1 to sleep for 1 second, sleep 0.5 to sleep for half a second, or sleep 1m to sleep a minuteecho: Displays a line of text.cat: Concatenates files and outputs them on stdout. When no arguments are provided, it reads and outputs stdin.exit: Causes the shell to exit.unset: Unsets environment variables.xargs: Builds arguments from stdin and executes a command.&& or || to separate two commands.
&&: if the command before && succeeds continue with the next command.||: if the command before || fails continue with the next command.; to run two commands without checking if the first command failed or succeeded.export ENV_VAR=value$ENV_VARunset ENV_VARVAR=valueVAR=value && echo $VAR|: echo Hello | python receiving_app.py|&: use this to also get the stderr as input.$() to use the output of a command as input for another command.
python main.py $(git rev-parse HEAD)! before any command will negate the exit code from 1 to 0 or visa-versa.> to redirect the stdout to a file.
echo hello > file.txt will put hello in file.txt and overwrite existing text.python main.py 2> file.txt will put the stderr output in file.txt.python main.py &> file.txt will put the stderr and stdout in file.txt.echo hello >> file.txt will append hello to the existing file.txt.* to expand all options.
echo *.py will echo all filenames that end with .pyecho **/*.py will echo all filenames that end with .py in this directory and all descendant directories.echo data[0-9].csv will echo all filenames that have a single number after data and before .csvMore info in deno_task_shell documentation.
Task can be very handy and may even replace simple Makefiles. There are a few principles that can make tasks more useful.
descriptionIt is a good habit to add a description to your task:
[tasks]
echo = { cmd = "echo Hello Pixi user", description = "Friendly greeting to a Pixi user" }
build = { cmd = "build", description = "Build everything" }
test = { cmd = "test", description = "Run all tests" }
Now, the command pixi task list will not only list all task names but also
the their descriptions.
pixi task list
Tasks that can run on this machine:
-----------------------------------
build, echo, test
Task Description
build Build everything
echo Friendly greeting to a Pixi user
test Run all tests
This list can be very helpful to quickly find the right tasks.
Tasks can get quite long:
echo-arg = { cmd = "echo {{ ARGUMENT }}", args = [{"arg" = "ARGUMENT", "default" = "hello"}], description = "Display the given argument" }
While it is possible to add line breaks inside the task in your pixi.toml to
make the task more readable:
echo-arg = {
cmd = "echo {{ ARGUMENT }}",
args = [{"arg" = "ARGUMENT", "default" = "hello"}],
description = "Display the given argument"
}
it will not work in pyproject.toml because it supports only TOML 1.0,
that doesn't allow line breaks inside the task specification.
Other tools can't parse the pyproject.toml anymore.
So if you use an editable dependency:
[tool.pixi.pypi-dependencies]
myproject = { path = ".", editable = true }
Pixi will use your build system and reports an error:
Error: × Failed to update PyPI packages for environment 'default'
├─▶ Failed to prepare distributions
├─▶ Failed to build `proj1 @ file:///.../proj1`
├─▶ The build backend returned an error
╰─▶ Call to `hatchling.build.build_editable` failed (exit status: 1)
There is an alternative syntax, adding the task name to the table header:
[tool.pixi.tasks.echo-arg]
cmd = "echo {{ ARGUMENT }}"
args = [{"arg" = "ARGUMENT", "default" = "hello"}]
description = "Display the given argument"
This is valid TOML 1.0 syntax and makes more complex task better readable.