docs/speed/perf_waterfall.md
The chrome.perf waterfall continuously builds and runs our performance tests on real Android, Windows, Mac, and Linux hardware; see list of platforms. Results are reported to the Performance Dashboard for analysis. The waterfall exists to uphold Chrome's Core Principles of speed:
"If you make a change that regresses measured performance, you will be required to fix it or revert".
The perf waterfall is split into three stages - builders, testers, and processors.
For each commit to the Chromium repo, a Builder is invoked. The builder builds Google Chrome, acquires test assets, and bundles everything up to be passed along to Testers. We execute a builder for each platform we support. To ensure we can keep up with the rapid flow of commits, we have a set of builders per platform, each building a different commit.
For each platform, we have a single tester job running continuously. Each run of a tester covers the set of commits from the end commit of the previous invocation to the latest built commit. Each tester has one or more subdevices (shards). The tester delegates a subset of tests to each shard and aggregates high-level results (pass/fail, runtimes).
Processors analyze the raw data generated by each Tester and convert it into a form that can be utilized by the Chrome Performance Dashboard. For some jobs, this work is executed by the tester instead.