Testing

The following outlines the specific requirements we have for testing the Authelia code contributions.

• While we aim for 100% coverage on changes and additions, we do not enforce this where it doesn't make practical sense:
  • A test which just marks a line as tested is not necessarily an effectual test
  • Sometimes there is limited ways in which tests can be performed and the limitation makes the test ineffectual
• Tests should be named to reflect what they testing for and which part of the code they are testing
• It's required for bug fixes that contributors create a test that fails prior to and passes subsequent to the fix being applied, this test must be included in the contribution, excluding this test will likely result in the fix being rejected unless explicitly agreed and advised otherwise by the core team
• It's strongly encouraged for features that contributors create have as much testing as is reasonable i.e. any line that can be tested should be tested, if the line can't be tested generally this is an indication a refactor may be required

Testing Methodology and Frequency

We run tests across multiple frameworks and platforms, and employ both SAST and DAST tooling to detect known security issues in the code. The rationale for this approach is that, while using multiple tools may increase noise, it improves confidence by providing more data on which to base our judgment.

Tool	Purpose	Notes
Go Test	Coverage, Static and Dynamic Code Analysis	Analysis of Go Code, Executed with go test -cover, go test -race, and go test -fuzz before and on every commit to master
React Testing Library	Coverage, Static and Dynamic Code Analysis	Analysis of React Code before and on every commit to master
SonarQube	Static Code Analysis	Analysis of All Code before and on every commit to master
CodeQL	Static Code Analysis	Analysis of All Code before and on every commit to master, and on a schedule
Codecov	Coverage Statistics	Produces Statistics for Go and TypeScript before and on every commit to master
Grype	Vulnerability Management	SBOM Scanning Only before and on every commit to master
Renovate	Vulnerability and Dependency Management	On a schedule
golangci-Lint	Static Code Analysis	Analysis of Go Code before and on every commit to master
GitGuardian	Secrets Management	Analysis of Secrets before and on every commit to master
Code Rabbit	Quality and Security Assessment	Analysis of General Pull Requests before every commit to master
OpenSSF Scorecard	Security Practices Assessment	Automated on every new commit to master
OpenSSF Best Practices	Security Practices Assessment	Manual Assessment for Security Practice Posture Improvements
StepSecurity Harden-Runner	CI Agent Security	As Part of any Job Running in GitHub CI Job Runners
zizmor	GitHub Action Static Code Analysis	Prevents Security Issues with GitHub Actions

Linting

In addition to the above SAST and DAST tools we also implement several linters which ensure code quality and consistency. These linters generally run via lefthook which is installed as a git hook.

Tool	Area	Purpose
golangci-Lint	Go	Code Quality and Consistency of Go Code
goimports-reviser	Go	Import Order Consistency
ESLint	JavaScript and TypeScript	Code Quality and Consistency of JS/TS Code
ShellCheck	Shell Files	Code Quality and Consistency
yamllint	YAML Files	Consistent YAML Formatting
commitlint	Git	Ensure Conformant Commit Messages
TruffleHog	All	Preventing Secret Commit Accidents
typos	All	Preventing Spelling and General Typos