docs/lang/articles/contribution/contributor_guide.md
Thank you for your interest in contributing to Taichi. Taichi was born as an academic research project. Though we are working hard to improve its code quality, Taichi has a long way to go to become a mature, large-scale engineering project. This is also why we decided to open source Taichi from the very beginning: We rely on our community to help Taichi evolve and thrive. From document updates, bug fix, to feature implementation, wherever you spot an issue, you are very welcome to file a PR (pull request) with us!:-)
Centered around the common process of taking on an issue, testing, and making a corresponding PR, this document provides guidelines, tips, and major considerations for Taichi's contributors. We highly recommend that you spend some time familiarizing yourself with this contribution guide before contributing to Taichi.
This section provides some general guidelines for the Taichi community and tips that we find practically useful.
Always use straightforward (sometimes even brute-force) solutions: Complicated code usually suggests a lack of design or over-engineering.
- "There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult." — C.A.R. Hoare
- "Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away." — Antoine de Saint-Exupéry
When it comes to making a design decision, weigh up its pros and cons. A design is good to go so long as its advantages outweigh its disadvantages.
Our ultimate goal is to build a sustainable, prosperous Taichi community, and effective communication is the cornerstone of that goal. Following are tips that may contribute to effective communication:
Concise:
Professional:
Constructive and courteous: Base your feedback and discussions on facts, NOT on personal feelings.
We welcome all kinds of contributions, including but not limited to:
If you would like to propose a new feature, or if you spot a potential issue, you can file an issue with Taichi.
:::note
When you try to report potential bugs in an issue, please consider running ti diagnose and offer its output as an attachment. This helps the maintainers to learn more about the context and the system information of your environment to make the debugging process more efficient and solve your issue more easily.
:::
:::caution When filing your issue, review it once again to ensure that no sensitive information about your data or yourself creeps in. :::
Except for minor updates, most PRs start from a developer taking over an issue. This section provides some corresponding tips and best practices.
| Issue Tag | Description | Target developer |
|---|---|---|
| good first issue | Issues that are easy to start with | Developers new to Taichi |
| welcome contribution | Issues slightly more challenging | Developers who wish to dive deeper into Taichi |
As part of the effort to increase visibility of the community and to improve developer experience, we highly recommend including documentation updates in your PR if applicable. Here are some of the documentation-specific references and tips:
If your PR is to implement a new feature, we recommend that you write your own test cases to cover corner cases for your codes before filing a PR.
We highly recommend that you complete code style checks and integration tests on your local computer before filing a PR.
Taichi enforces code style via pre-commit hooks, which includes the following checks:
clang-format-10.black v23.3.0.pylint.You will need to install pre-commit first:
pip install pre-commit
and run the code checkers:
pre-commit run -a
With this command, black will format your Python codes automatically.
You can install it as a pre-commit hook so that it is run before you commit the changes to git:
pre-commit install
No problem, the CI bot will run the code checkers and format your codes automatically when you submit a PR.
</details> <!-- Todo: Make this a reusable fragment. -->For more style information for your C++ code, see our C++ style.
We generally follow Google C++ Style Guide. One major exception is the naming convention of functions: Taichi adopts the snake case for function naming, as opposed to the camel case suggested in Google's style, e.g. this_is_a_taichi_function().
Taichi uses clang-tidy-10 (install via sudo apt install clang-tidy-10) to automatically check C++ code for stype violations, programming errors and enforce coding style best practices. You can find a list of enabled checks in .clang-tidy.
Taichi's clang-tidy integration test is enabled per PR basis and you can run it locally via:
python ./scripts/run_clang_tidy.py $PWD/taichi -clang-tidy-binary clang-tidy-10 -header-filter=$PWD/taichi -j4
clang-tidy also provides an easy way to automatically apply suggested fixes by passing -fix argument:
python ./scripts/run_clang_tidy.py $PWD/taichi -clang-tidy-binary clang-tidy-10 -clang-apply-replacements-binary clang-apply-replacements-10 -header-filter=$PWD/taichi -j4 -fix
To run all the C++ and Python tests:
python tests/run_tests.py
Example 1:
python tests/run_tests.py -v -t3 -a cpu,metal -s
-v: Verbose output.-t <threads>: Set a custom number of threads for parallel testing.-a <arch(s)>: Test only the specified backends (separated by comma).-s: Original output from the tests.Example 2:
python tests/run_tests.py numpy_io
<filename(s)>: Run test cases in specified files only (separated by comma).Example 3:
python tests/run_tests.py linalg -k "cross or diag"
-k <key>: Run only the tests that match the specified keys (supports expression in a key string).test_cross() and test_diag() in tests/python/test_linalg.py.To show all available options
python tests/run_tests.py -h
We have both Python and C++ test cases, but C++ test cases are disabled by default. To enable C++ test cases:
- Build Taichi from source using the
python setup.py developcommand.- Set
TAICHI_CMAKE_ARGS="-DTI_BUILD_TESTS:BOOL=ON".
Now you get to the point where you need to get your hands dirty with your PRs. This section provides the following:
When implementing a complex feature:
When creating a PR:
When describing your PR:
fixes #<issue_number>.If you create a PR still in progress:
Your PR will make it into the commit history in the the master branch or even Taichi's release notes, therefore it is important to keep your PR title self-explanatory. This section describes our PR naming conventions:
[tag1] [tag2]...[tagN] Your PR title must be short but carry necessary info
^----^ ^----^...^----^ ^--------------------------------------------------^
| | | |
| | | +---> Capitalize the initial of your title.
| | +---> Adjacent tags are separated with precisely one space.
| +---> Frequently used tags: [cuda], [lang], [ci], [ir], [refactor].
+---> Prepend at least one tag to your PR title.
Tag naming conventions:
[lang], to your PR title.[Metal], [Vulkan], [IR], [Lang], or [CUDA]. Ensure that your PR title has AT MOST one tag dealt this way.[metal], [vulkan], [ir], [lang], or [cuda].:::danger INCORRECT
[Lang][refactor] (sans space)
:::
:::tip CORRECT
[Lang] [refactor]
:::
:::danger INCORRECT
[GUI] [Mac] Support modifier keys (both tags have their initial capitalized)
:::
:::tip CORRECT
[gui] [Mac] Support modifier keys (only one tag has its initial capitalized)
:::
Title naming conventions:
:::danger INCORRECT
[Doc] improve documentation (the initial of the title is not capitalized)
:::
:::tip CORRECT
[Doc] Improve documentation
:::
:::note
Following are some frequently used tags:
[cuda]: Backend-specific changes.[lang]: Frontend language features, including syntax sugars.[ir]: Intermediate representation-specific changes.[refactor]: Code refactoring changes.[ci]: CI/CD workflow-specific changes.[Doc]: Documentation updates.When introducing a new tag, ensure that you add it to misc/prtags.json so that others can follow.
:::
Follow this checklist during PR review or merging:
Ensure that your PR title follows our naming conventions.
Ensure that Taichi's master branch has a linear history. See Linear vs Non-Linear History for more information.
Ensure that your PR passes all Continuous Integration (CI) tests before merging it.
CI is triggered each time you push a commit to an open PR. It builds and tests all commits in your PR in multiple environments. Keep an eye on the CI test results:
Here, we do not want to repeat some best practices summarized in the following Google blog articles. But please spare a couple of minutes reading them if your PR is being reviewed or if you are reviewing a PR. They have our recommendation!
<a name="ghstack-workflow"></a>
The standard GitHub PR workflow can become unwieldy when dealing with large changesets. With the help of ghstack, you can break down a large changeset into multiple PRs, each building upon the previous one. Each PR will undergo its own review process and CI/CD checks.
For details on how to use ghstack, please consult its documentation, with the exception of the landing step.
To land ghstack commits, simply comment /land on the PR. A landing bot will then verify if the corresponding PR and all its predecessors have been approved and passed the CI/CD checks.
If all checks have been cleared, the bot will execute ghstack land for you.
Taichi implements warning-free code by turning on -Werror by default. This means that Taichi takes warnings as errors, and we highly recommend that you resolve a warning as soon as it occurs.
In the following section, we provide several practical tips for handling some of the common scenarios that you may encounter during CI compilation.
There is little we can do to third-party warnings other than turning them off. To turn off or mute warnings from specific third-party header files, use the SYSTEM option when configuring include_directories in your CMake files. Then, the included header files are treated as system headers. See the following two examples taken from cmake/TaichiCore.cmake:
# Treat files under "external/Vulkan-Headers/include" as system headers and mute warnings from them.
include_directories(SYSTEM external/Vulkan-Headers/include)
# Treat files under "external/VulkanMemoryAllocator/include" as system headers for target "${CORE_LIBRARY_NAME}"
target_include_directories(${CORE_LIBRARY_NAME} SYSTEM PRIVATE external/VulkanMemoryAllocator/include)
Ideally, third-party libraries or targets ought to be built completely independent of your Taichi project. In practice, because of the design of the CMake system, CMake variables from the Taichi and third-party submodules are sometimes messed up. Therefore, we recommend that you disable warnings from a third-party library or target:
CMAKE_CXX_FLAGS from the same variable defined in Taichi.-Wall option from the submodule's CMAKE_CXX_FLAGS variables.You can find details about how to mute certain warning types from the Clang Compiler User Manual; it usually starts with -Wno-. Please explain what the warning is about and why we should ignore it in the comments.
The following examples can be found in cmake/TaichiCXXFlags.cmake:
# [Global] Clang warns if a C++ pointer's nullability was not explicitly marked (__nonnull, nullable, ...).
# Nullability seems to be a clang-specific feature; thus we disable this warning.
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-nullability-completeness ")
# [Global] By evaluating "constexpr", compiler throws a warning for functions known to be dead at compile time.
# However, some of these "constexpr" specifiers are debug flags and will be manually enabled upon debugging.
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-unneeded-internal-declaration ")
:::caution The approach presented here is not recommended and considered your last approach, because it is not reliable. :::
In rare situations where you can neither fix nor mute the warnings from specific code blocks via conventional approaches, your last approach is to mute them by decorating your code block using the #pragma clang diagnostic macros. Beware that #pragmas are not defined in the C++ standard and that their implementations depend heavily on the compiler. That is to say, this solution is neither stable nor elegant.
To ignore all warnings from a specific code block, wrap it up with the following two groups of macros. Further, you can even replace -Wall with a group of warning types for finer control. See the following example:
#if defined(__clang__)
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wall"
#endif
{Your Code Goes Here}
#if defined(__clang__)
#pragma clang diagnostic pop
#endif
Taichi's CI system is implemented using the Github Actions, the entrance of which lies in testing.yaml. Depending on the CI pipeline, testing.yml will execute one of the corresponding test scripts under this directory
There are a few CI pipelines that work slightly different from the standard CI pipeline:
Build Andriod Demos builds both taichi-repo with your PR applied and an external taichi-aot-demo repo. After that, it executes the demos from taichi-aot-demo with the just-compiled Taichi program and libraries.
If your PR to taichi-repo contains changes to some public interface, you may need to adjust the codes in taichi-aot-demo to avoid breaking the demos. To achieve that, please follow these steps:
taichi-repo. If this PR changes the public interface, then it probably breaks the demos thus fail the Build Android Demos CI pipeline - Don't panic, this is expected.taichi-aot-demo to make it work with the above mentioned PR, then file a separate PR to taichi-aot-demo repo and have it merged.taichi-repo, update the commit id for taichi-aot-demo in aot-demo.sh. This time your PR is expected to pass Build Android Demos.If you encounter any issue that is not covered here, feel free to ask us on GitHub discussions or open an issue on GitHub with all the details attached. We are always there to help!
Finally, thanks again for your interest in contributing to Taichi. We look forward to seeing your contributions!