docs/source/development/contributing/index.md
Thanks for taking the time to contribute! We appreciate all contributions, from reporting bugs to implementing new features. If you're unclear on how to proceed after reading this guide, please contact us on Discord.
We use GitHub issues to track bugs and suggested enhancements. You can report a bug by opening a new issue. Use the appropriate issue type for the language you are using (Rust / Python).
Before creating a bug report, please check that your bug has not already been reported, and that your bug exists on the latest version of Polars. If you find a closed issue that seems to report the same bug you're experiencing, open a new issue and include a link to the original issue in your issue description.
Please include as many details as possible in your bug report. The information helps the maintainers resolve the issue faster.
We use GitHub issues to track bugs and suggested enhancements. You can suggest an enhancement by opening a new feature request. Before creating an enhancement suggestion, please check that a similar issue does not already exist.
Please describe the behavior you want and why, and provide examples of how Polars would be used if your feature were added.
Pick an issue by going through the issue tracker and finding an issue you would like to work on. Feel free to pick any issue with an accepted label that is not already assigned. We use the help wanted label to indicate issues that are high on our wishlist.
If you are a first time contributor, you might want to look for issues labeled good first issue. The Polars code base is quite complex, so starting with a small issue will help you find your way around!
If you would like to take on an issue, please comment on the issue to let others know. You may use the issue to discuss possible solutions.
The Polars development flow relies on both Rust and Python, which means setting up your local development environment is not trivial. If you run into problems, please contact us on Discord.
!!! note
If you are a Windows user, the steps below might not work as expected.
Try developing using [WSL](https://learn.microsoft.com/en-us/windows/wsl/install).
Under native Windows, you may have to manually copy the contents of `toolchain.toml` to `py-polars/toolchain.toml`, as Git for Windows may not correctly handle symbolic links.
For contributing to Polars you need a free GitHub account and have
git installed on your machine. Start by
forking the Polars repository, then
clone your forked repository using git:
git clone https://github.com/<username>/polars.git
cd polars
Optionally set the upstream remote to be able to sync your fork with the Polars repository in the
future:
git remote add upstream https://github.com/pola-rs/polars.git
git fetch upstream
In order to work on Polars effectively, you will need Rust, Python, and dprint.
First, install Rust using rustup. After the initial installation, you will also need to install the nightly toolchain:
rustup toolchain install nightly --component miri
Next, install Python, for example using pyenv. We
recommend using the latest Python version (3.13). Make sure you deactivate any active virtual
environments (command: deactivate) or conda environments (command: conda deactivate), as the
steps below will create a new virtual environment
for Polars. You will need Python even if you intend to work on the Rust code only, as we rely on the
Python tests to verify all functionality.
Finally, install dprint. This is not strictly required, but it is recommended as we use it to autoformat certain file types.
You can now check that everything works correctly by going into the py-polars directory and
running the test suite (warning: this may be slow the first time you run it):
cd py-polars
make test
!!! note
You need to have [CMake](https://cmake.org/) installed for `make test` to work.
This will do a number of things:
.venv folder.!!! note
There are a small number of specialized dependencies that are not installed by default.
If you are running specific tests and encounter an error message about a missing dependency,
try running `make requirements-all` to install _all_ known dependencies).
Check if linting also works correctly by running:
make pre-commit
Note that we do not actually use the pre-commit tool. We use the Makefile to conveniently run the following formatting and linting tools:
If this all runs correctly, you're ready to start contributing to the Polars codebase!
Dependencies are updated regularly - at least once per month. If you do not keep your environment up-to-date, you may notice tests or CI checks failing, or you may not be able to build Polars at all.
To update your environment, first make sure your fork is in sync with the Polars repository:
git checkout main
git fetch upstream
git rebase upstream/main
git push origin main
Update all Python dependencies to their latest versions by running:
make requirements
If the Rust toolchain version has been updated, you should update your Rust toolchain. Follow it up
by running cargo clean to make sure your Cargo folder does not grow too large:
rustup update
cargo clean
Create a new git branch from the main branch in your local repository, and start coding!
The Rust code is located in the crates directory, while the Python codebase is located in the
py-polars directory. Both directories contain a Makefile with helpful commands. Most notably:
make test to run the test suite (see the test suite docs for more info)make pre-commit to run autoformatting and lintingNote that your work cannot be merged if these checks fail! Run make help to get a list of other
helpful commands.
Two other things to keep in mind:
If you have found a bug, it can be very helpful to identify the exact commit that introduced it.
This allows maintainers to understand the root cause more quickly. You can use
git bisect to do this automatically.
1. Create a minimal reproducible example (MRE)
First, reduce your bug to the smallest possible Python script that reproduces it. See this guide for guidance.
2. Turn your MRE into a bisect script
Save the following as bisect.py in the root of the repository, adding your MRE at the bottom. You
can add it to .git/info/exclude to ignore it locally without affecting the repository's
.gitignore:
import subprocess
import sys
import importlib
# Try to build Polars. Skip this commit if the build fails.
build = subprocess.run(["make", "build"])
if build.returncode != 0:
sys.exit(125) # Tell git bisect to skip this commit.
importlib.invalidate_caches() # Ensure we see newly built package.
import polars as pl
# Your MRE here.
# If Python exits with an unhandled exception, git bisect treats it as a bad commit.
# If it exits normally (giving exit code 0), it is treated as a good commit.
# Example:
# result = pl.Series([1, 2, 3]).some_method()
# assert result == expected, f"Got {result}"
The script must exit with:
0 if the bug is not present (good commit, normal Python exit)1 (or any non-zero code except 125) if the bug is present (bad commit, e.g. an assertion
error)125 to tell git bisect to skip the commit (e.g. it does not build)3. Mark a good and a bad commit
Find a commit hash or release tag where the bug did not
exist (e.g. py-1.32.0 for Polars 1.32.0) and another hash or tag which is confirmed to contain the
bug (e.g. HEAD if it still exists):
git bisect start
git bisect bad HEAD
git bisect good py-1.32.0
4. Run the bisect automatically
From the root of the repository, run:
git bisect run .venv/bin/python bisect.py
git bisect will binary-search through the commit history and print the first bad commit when it is
done.
5. Include the result in your issue
Copy the output (the commit hash and message) and add it to your bug report. This greatly speeds up the investigation for maintainers.
When you are done, reset your repository:
git bisect reset
When you have resolved your issue, open a pull request in the Polars repository. Please adhere to the following guidelines:
<!-- dprint-ignore-start -->rust and/or python, depending on your contribution: this tag determines which changelog(s) will include your change.
Omit the scope if your change affects both Rust and Python.fix(python): Fix `DataFrame.top_k` not handling nulls correctlymain branch.We unfortunately are overwhelmed by the amount of low-quality contributions created primarily using AI. These cost us a lot of time (and regularly simply don't work), while the author has barely spent any effort, so for first-time contributors there are some more rules:
make test), locally on
your machine (not the CI).After you have opened your pull request, a maintainer will review it and possibly leave some comments. Once all issues are resolved, the maintainer will merge your pull request, and your work will be part of the next Polars release!
Keep in mind that your work does not have to be perfect right away! If you are stuck or unsure about your solution, feel free to open a draft pull request and ask for help.
The most important components of Polars documentation are the user guide, the API references, and the database of questions on Stack Overflow for Python Polars and Rust Polars.
The user guide is maintained in the docs/source/user-guide folder. Before creating a PR first
raise an issue to discuss what you feel is missing or could be improved.
The user guide is built using MkDocs. You install the dependencies for
building the user guide by running make build in the root of the repo. Additionally, you need to
make sure the graphviz dot binary is on your path.
Activate the virtual environment and run mkdocs serve to build and serve the user guide, so you
can view it locally and see updates as you make changes.
Each user guide page is based on a .md markdown file. This file must be listed in mkdocs.yml.
To add a code block with code to be run in a shell with tabs for Python and Rust, use the following format:
=== ":fontawesome-brands-python: Python"
```shell
$ pip install fsspec
```
=== ":fontawesome-brands-rust: Rust"
```shell
$ cargo add aws_sdk_s3
```
The snippets for Python and Rust code blocks are in the docs/source/src/python/ and
docs/source/src/rust/ directories, respectively. To add a code snippet with Python or Rust code to
a .md page, use the following format:
{{code_block('user-guide/io/cloud-storage','read_parquet',['read_parquet','read_csv'])}}
docs/source/src/python/user-guide/io/cloud-storage.py and
docs/source/src/rust/user-guide/io/cloud-storage.rs..py or .rs
filedocs/source/_build/API_REFERENCE_LINKS.ymlIf the corresponding .py and .rs snippet files both exist then each snippet named in the second
argument to code_block above must exist or the build will fail. An empty snippet should be added
to the .py or .rs file if the snippet is not needed.
Each snippet is formatted as follows:
# --8<-- [start:read_parquet]
import polars as pl
df = pl.read_parquet("file.parquet")
# --8<-- [end:read_parquet]
The snippet is delimited by --8<-- [start:<snippet_name>] and --8<-- [end:<snippet_name>]. The
snippet name must match the name given in the second argument to code_block above.
In some cases, you may need to add links to different functions for the Python and Rust APIs. When
that is the case, you can use the two extra optional arguments that code_block accepts, that can
be used to pass Python-only and Rust-only links:
{{code_block('path', 'snippet_name', ['common_api_links'], ['python_only_links'], ['rust_only_links'])}}
Before committing, install dprint (see above) and run dprint fmt from the docs directory to
lint the markdown files.
Polars has separate API references for Rust and Python. These are generated directly from the codebase, so in order to contribute, you will have to follow the steps outlined in this section above.
Rust Polars uses cargo doc to build its documentation. Contributions to improve or clarify the API
reference are welcome.
For the Python API reference, we always welcome good docstring examples. There are still parts of the API that do not have any code examples. This is a great way to start contributing to Polars!
Note that we follow the numpydoc
convention. Docstring examples should also follow the Black
codestyle. From the py-polars directory, run make fmt to make sure your additions pass the
linter, and run make doctest to make sure your docstring examples are valid.
Polars uses Sphinx to build the API reference. This means docstrings in general should follow the
reST format. If you want
to build the API reference locally, go to the py-polars/docs directory and run make html. The
resulting HTML files will be in py-polars/docs/build/html.
New additions to the API should be added manually to the API reference by adding an entry to the
correct .rst file in the py-polars/docs/source/reference directory.
We use StackOverflow to create a database of high quality questions and answers that is searchable and remains up-to-date. There is a separate tag for each language:
Contributions in the form of well-formulated questions or answers are always welcome! If you add a new question, please notify us by adding a matching issue to our GitHub issue tracker.
This section is intended for Polars maintainers.
Polars releases Rust crates to crates.io and Python packages to PyPI.
New releases are marked by an official GitHub release and an associated git tag. We utilize Release Drafter to automatically draft GitHub releases with release notes.
The steps for releasing a new Rust or Python version are similar. The release process is mostly automated through GitHub Actions, but some manual steps are required. Follow the steps below to release a new version.
Start by bumping the version number in the source code:
Cargo.toml files in the polars directory and
subdirectories. You'll probably want to use some search/replace strategy, as there are quite a few
crates that need to be updated.py-polars/Cargo.toml
to match the version of the draft release.py-polars directory, run make build to generate a new Cargo.lock file.release(<language>): <Language> Polars <version-number>. For example:
release(python): Python Polars 0.16.1main branch of the main Polars repository.Directly after merging your pull request, release the new version:
It may happen that one or multiple release jobs fail. If so, you should first try to simply re-run the failed jobs from the GitHub Actions UI.
If that doesn't help, you will have to figure out what's wrong and commit a fix. Once your fix has
made it to the main branch, simply re-trigger the release workflow.
Any contributions you make to this project will fall under the MIT License that covers the Polars project.