ContextQMD
Back to Prql

Development

web/book/src/project/contributing/development.md

0.13.1221.5 KB
Original Source

Development

Setting up an initial dev environment

We can set up a local development environment sufficient for navigating, editing, and testing PRQL's compiler code in two minutes:

  • Install rustup & cargo.

  • [Optional but highly recommended] Install cargo-insta, our testing framework:

    sh
    cargo install cargo-insta

  • That's it! Running the unit tests for the prqlc crate after cloning the repo should complete successfully:

    sh
    cargo test --package prqlc --lib

    ...or, to run tests and update the test snapshots:

    sh
    cargo insta test --accept --package prqlc --lib

    There's more context on our tests in How we test below.

That's sufficient for making an initial contribution to the compiler.

Setting up a full dev environment

<!-- prettier-ignore -->

[!NOTE] We really care about this process being easy, both because the project benefits from more contributors like you, and to reciprocate your future contribution. If something isn't easy, please let us know in a GitHub Issue. We'll enthusiastically help you, and use your feedback to improve the scripts & instructions.

For more advanced development; for example compiling for wasm or previewing the website, we have two options:

Option 1: Use the project's task

<!-- prettier-ignore -->

[!NOTE] This is tested on macOS, should work on amd64 Linux, but won't work on others (include Windows), since it relies on brew.

  • Install Task.

  • Then run the setup-dev task. This runs commands from our Taskfile.yaml, installing dependencies with cargo, brew, npm & uv, and suggests some VS Code extensions.

    sh
    task setup-dev

Option 2: Install tools individually

  • We'll need cargo-insta, to update snapshot tests:

    sh
    cargo install cargo-insta

  • We'll need Python, which most systems will have already. The easiest way to check is to try running the full tests:

    sh
    cargo test

    ...and if that doesn't complete successfully, ensure we have Python >= 3.7, to compile prqlc-python.

  • For more involved contributions, such as building the website, playground, book, or some release artifacts, we'll need some additional tools. But we won't need those immediately, and the error messages on what's missing should be clear when we attempt those things. When we hit them, the Taskfile.yaml will be a good source to copy & paste instructions from.

Option 3: Use a Dev Container

This project has a devcontainer.json file and a pre-built dev container base Docker image. Learn more about Dev Containers at https://containers.dev/

Currently, the tools for Rust are already installed in the pre-built image, and, Node.js, Python and others are configured to be installed when build the container.

While there are a variety of tools that support Dev Containers, the focus here is on developing with VS Code in a container by GitHub Codespaces or VS Code Dev Containers extension.

To use a Dev Container on a local computer with VS Code, install the VS Code Dev Containers extension and its system requirements. Then refer to the links above to get started.

Option 4: Use nix development environment

<!-- prettier-ignore -->

[!NOTE] This is used by a member of the core team on Linux, but doesn't currently work on Mac. We're open to contributions to improve support.

A nix flake flake.nix provides 3 development environments:

  • default, for building the compiler
  • web, for the compiler and the website,
  • full, for the compiler, the website and the compiler bindings.

To load the shell:

  1. Install nix (the package manager). (only first time)

  2. Enable flakes, which are a (pretty stable) experimental feature of nix. (only first time)

    For non-NixOS users:

    sh
    mkdir -p ~/.config/nix/
tee 'experimental-features = nix-command flakes' >> ~/.config/nix/nix.conf

    For NixOS users, follow instructions here.

  3. Run:

    sh
    nix develop

    To use the "web" or "full" shell, run:

    sh
    nix develop .#web

Optionally, you can install direnv, to automatically load the shell when you enter this repo. The easiest way is to also install direnv-nix and configure your .envrc with:

sh
# .envrc
use flake .#full

Contribution workflow

We're similar to most projects on GitHub — open a Pull Request with a suggested change!

Commits

  • If a change is user-facing, please add a line in CHANGELOG.md, with {message}, ({@contributor, #X}) where X is the PR number.
    • If there's a missing entry, a follow-up PR containing just the changelog entry is welcome.
  • We're using Conventional Commits message format, enforced through action-semantic-pull-request.

Merges

  • We merge any code that makes PRQL better
  • A PR doesn't need to be perfect to be merged; it doesn't need to solve a big problem. It needs to:
    • be in the right direction,
    • make incremental progress,
    • be explicit on its current state, so others can continue the progress.
  • That said, there are a few instances when we need to ensure we have some consensus before merging code — for example non-trivial changes to the language, or large refactorings to the library.
  • If you have merge permissions, and are reasonably confident that a PR is suitable to merge (whether or not you're the author), feel free to merge.
    • If you don't have merge permissions and have authored a few PRs, ask and ye shall receive.
  • The primary way we ratchet the code quality is through automated tests.
    • This means PRs almost always need a test to demonstrate incremental progress.
    • If a change breaks functionality without breaking tests, our tests were probably insufficient.
    • If a change breaks existing tests (for example, changing an external API), that indicates we should be careful about merging a change, including soliciting others' views.
  • We use PR reviews to give general context, offer specific assistance, and collaborate on larger decisions.
    • Reviews around 'nits' like code formatting / idioms / etc are very welcome. But the norm is for them to be received as helpful advice, rather than as mandatory tasks to complete. Adding automated tests & lints to automate these suggestions is welcome.
    • If you have merge permissions and would like a PR to be reviewed before it merges, that's great — ask or assign a reviewer.
    • If a PR hasn't received attention after a day, please feel free to ping the pull request.
  • People may review a PR after it's merged. As part of the understanding that we can merge quickly, contributors are expected to incorporate substantive feedback into a future PR.
  • We should revert quickly if the impact of a PR turns out not to be consistent with our expectations, or there isn't as much consensus on a decision as we had hoped. It's very easy to revert code and then re-revert when we've resolved the issue; it's a sign of moving quickly. Other options which resolve issues immediately are also fine, such as commenting out an incorrect test or adding a quick fix for the underlying issue.

Docs

We're very keen on contributions to improve our documentation.

This includes our docs in the book, on the website, in our code, or in a Readme. We also appreciate issues pointing out that our documentation was confusing, incorrect, or stale — if it's confusing for you, it's probably confusing for others.

Some principles for ensuring our docs remain maintainable:

  • Docs should be as close as possible to the code. Doctests are ideal on this dimension — they're literally very close to the code and they can't drift apart since they're tested on every commit. Or, for example, it's better to add text to a --help message, rather than write a paragraph in the Readme explaining the CLI.
  • We should have some visualization of how to maintain docs when we add them. Docs have a habit of falling out of date — the folks reading them are often different from those writing them, they're sparse from the code, generally not possible to test, and are rarely the by-product of other contributions. Docs that are concise & specific are easier to maintain.
  • Docs should be specifically relevant to PRQL; anything else we can instead link to.

If something doesn't fit into one of these categories, there are still lots of ways of getting the word out there — a blog post / gist / etc. Let us know and we're happy to link to it / tweet it.

How we test

We use a pyramid of tests — we have fast, focused tests at the bottom of the pyramid, which give us low latency feedback when developing, and then slower, broader tests which ensure that we don't miss anything as PRQL develops{{footnote: Our approach is very consistent with @matklad's advice, in his excellent blog post How to Test.}}.

<!-- markdownlint-disable MD053 --> <!-- prettier-ignore -->

[!NOTE] If you're making your first contribution, you don't need to engage with all this — it's fine to just make a change and push the results; the tests that run in GitHub will point you towards any errors, which can be then be run locally if needed. We're always around to help out.

Our tests, from the bottom of the pyramid to the top:

  • Static checks — we run a few static checks to ensure the code stays healthy and consistent. They're defined in .pre-commit-config.yaml, using pre-commit. They can be run locally with

    sh
    task lint
# or
pre-commit run -a

    The tests fix most of the issues they find themselves. Most of them also run on GitHub on every commit; any changes they make are added onto the branch automatically in an additional commit.

    • Checking by MegaLinter, which includes more Linters, is also done automatically on GitHub. (experimental)

  • Unit tests & inline insta snapshots — we rely on unit tests to rapidly check that our code basically works. We extensively use Insta, a snapshot testing tool which writes out the values generated by our code, making it fast & simple to write and modify tests{{footnote: Here's an example of an insta test — note that only the initial line of each test is written by us; the remainder is filled in by insta.}}

    These are the fastest tests which run our code; they're designed to run on every save while you're developing. We include a task which does this:

    sh
    task prqlc:test
# or
cargo insta test --accept --package prqlc --lib
<!-- This is the previous doc. It has the advantage that it explains what it's doing, and is easy to change (e.g. to run all packages). But because of https://github.com/watchexec/watchexec/issues/371, the ignore behavior is unfortunately quite inconsistent in watchexec. Let's revert back if it gets solved. [^2]: For example, this is a command I frequently run: ```sh RUST_BACKTRACE=1 watchexec -e rs,toml,md -cr --ignore='target/**' -- cargo -q insta test --accept -p prqlc --lib ``` Breaking this down: - `RUST_BACKTRACE=1` will print a full backtrace, including where an error value was created, for Rust tests which return `Result`s. - `watchexec -e rs,toml,md -cr --ignore='target/**' --` will run the subsequent command on any change to files with extensions which we are generally editing. - `cargo insta test --accept --` runs tests with `insta`, a snapshot library, and writes any results immediately. I rely on git to track changes, so I run with `--accept`, but YMMV. - `-p prqlc --lib` is passed to cargo by `insta`; `-p prqlc` tells it to only run the tests for `prqlc` rather than the other crates, and `--lib` to only run the unit tests rather than the integration tests, which are slower. - Note that we don't want to re-run on _any_ file changing, because we can get into a loop of writing snapshot files, triggering a change, writing a snapshot file, etc. -->

  • Documentation — we compile all examples from our documentation in the Website, README, and PRQL Book, to test that they produce the SQL we expect, and that changes to our code don't cause any unexpected regressions. These are included in:

    sh
    cargo insta test --accept

  • Database integration tests — we run tests with example queries against databases with actual data to ensure we're producing correct SQL across our supported dialects. The in-process tests can be run locally with:

    sh
    task prqlc:test-all
# or
cargo insta test --accept --features=default,test-dbs

    More details on running with external databases are in the Readme.

<!-- prettier-ignore -->

[!NOTE] Integration tests use DuckDB, and so require a clang compiler to compile duckdb-rs. Most development systems will have one, but if the test command fails, install a clang compiler with:

  • On macOS, install xcode with xcode-select --install
  • On Debian Linux, apt-get update && apt-get install clang
  • On Windows, duckdb-rs isn't supported, so these tests are excluded

  • GitHub Actions on every commit — we run tests relevant to a PR's changes in CI — for example changes to docs will attempt to build docs, changes to a binding will run that binding's tests. The vast majority of changes trigger tests which run in less than five minutes, and we should be reassessing their scope if they take longer than that. Once these pass, a pull request can be merged.

  • GitHub Actions on merge — we run a wider set tests on every merge to main. This includes testing across OSs, all our language bindings, a measure of test code coverage, and some performance benchmarks.

    If these tests fail after merging, we should revert the commit before fixing the test and then re-reverting.

    Most of these will run locally with:

    sh
    task test-all

  • GitHub Actions nightly — every night, we run tests that take longer, are less likely to fail, or are unrelated to code changes — such as security checks, bindings' tests on multiple OSs, or expensive timing benchmarks.

    We can run these tests before a merge by adding a label pr-nightly to the PR.

The goal of our tests is to allow us to make changes quickly. If they're making it more difficult to make changes, or there are missing tests that would offer the confidence to make changes faster, please raise an issue.

Website

The website is published together with the book and the playground, and is automatically built and released on any push to the web branch.

The web branch points to the latest release plus any website-specific fixes. That way, the compiler behavior in the playground matches the latest release while allowing us to fix mistakes in the docs with a tighter loop than every release.

Fixes to the playground, book, or website should have a pr-backport-web label added to their PR — a bot will then open & merge another PR onto the web branch once the initial branch merges.

The website components will run locally with:

sh
# Run the main website
task web:run-website
# Run the PRQL online book
task web:run-book
# Run the PRQL playground
task web:run-playground

Bindings

We have a number of language bindings, as documented at https://prql-lang.org/book/project/bindings/index.html. Some of these are within our monorepo, some are in separate repos. Here's a provisional framework for when we use the main prql repo vs separate repos for bindings:

FactorRationaleExample
Does someone want to sign up to maintain a repo?A different repo is harder for the core team to maintaintree-sitter-prql is well maintained
Can it change independently from the compiler?If it's in a different repo, it can't be changed in lockstep with the compilerprql-vscode is fine to change "behind" the language
Would a separate repo invite new contributors?A monorepo with all the rust code can be less inviting for those familiar with other langsprql-vscode had some JS-only contributors
Is there an convention for a stand-alone repo?A small number of ecosystems require a separate repohomebrew-prql needs to be named that way for a Homebrew tap

Releasing

Currently we release in a semi-automated way:

  1. PR & merge an updated Changelog. GitHub will produce a draft version at https://github.com/PRQL/prql/releases/new, including "New Contributors".

    Use this script to generate a line introducing the enumerated changes:

    sh
    echo "It has $(git rev-list --count $(git rev-list --tags --max-count=1)..) commits from $(git shortlog --summary $(git rev-list --tags --max-count=1).. | wc -l | tr -d '[:space:]') contributors. Selected changes:"

    When a fix closes an issue reported by someone other than the PR author, thank them in the changelog entry, e.g. (@pr-author, #5639; reported by @issue-reporter).

  2. If the current version is correct, then skip ahead. But if the version needs to be changed — for example, we had planned on a patch release, but instead require a minor release — then run cargo release version $version -x && cargo release replace -x && task prqlc:test-all to bump the version, and PR the resulting commit.

  3. Ensure all changes intended for the release are merged to main. Then create the release (which creates the tag on the latest commit on main):

    Web UI: Go to Draft a new release{{footnote: Only maintainers have access to this page.}}, copy the changelog entry into the release description{{footnote: Unfortunately GitHub's markdown parser interprets linebreaks as newlines. I haven't found a better way of editing the markdown to look reasonable than manually editing the text or asking LLM to help.}}, enter the tag to be created, and hit "Publish".

    CLI:

    sh
    gh release create $version --title "$version" --notes "$(cat <<'EOF'
<paste changelog entry here>
EOF
)"

  4. From there, both the tag and release is created and all packages are published automatically based on our release workflow.

  5. Run cargo release patch --no-publish --no-push --execute --no-verify --no-confirm --no-tag && task prqlc:test-all to bump the versions and add a new Changelog section; then PR the resulting commit. Note this currently contains task prqlc:test-all to update snapshot tests which contain the version.

<!-- Note we used to have `cargo release version patch -x --no-confirm && cargo release replace -x --no-confirm && task prqlc:test-all`, which was simpler, but in order for `prev_version` to work, we can't separate the `patch` and `replace`, and we need `prev_version` for the prqlc version constraint (search for `prev_version` if unclear). If we moved back to upgrading the tags at the time of release rather than after, we could go back to that. -->

  1. Check whether there are milestones that need to be pushed out.

  2. Review the Current Status on the README.md to ensure it reflects the project state.

We may make this more automated in future; e.g. automatic changelog creation.