docs/contributing/testing-guide.md
This document outlines the testing practices and guidelines for the Thunderbird for Android project.
Key Testing Principles:
testSubjectTests in this project should follow the Arrange-Act-Assert (AAA) pattern:
Example:
@Test
fun `example test using AAA pattern`() {
// Arrange
val input = "test input"
val expectedOutput = "expected result"
val testSubject = SystemUnderTest()
// Act
val result = testSubject.processInput(input)
// Assert
assertThat(result).isEqualTo(expectedOutput)
}
Use comments to clearly separate these sections in your tests:
// Arrange
// Act
// Assert
Use descriptive test names that clearly indicate what is being tested. For JVM tests, use backticks:
@Test
fun `method should return expected result when given valid input`() {
// Test implementation
}
Note: Android instrumentation tests do not support backticks in test names. For these tests, use camelCase instead:
@Test
fun methodShouldReturnExpectedResultWhenGivenValidInput() {
// Test implementation
}
In this project, we prefer using fake implementations over mocks:
Fakes provide better test reliability and are more maintainable in the long run. They also make tests more readable and less prone to breaking when implementation details change.
Mocks can lead to brittle tests that are tightly coupled to the implementation details, making them harder to maintain. They also negatively impact test performance, particularly during test initialization. Which can quickly become overwhelming when an excessive number of tests includes mock implementations.
Example of a fake implementation:
// Interface
interface DataRepository {
fun getData(): List<String>
}
// Fake implementation for testing
class FakeDataRepository(
// Allow passing initial data during construction
initialData: List<String> = emptyList()
) : DataRepository {
// Mutable property to allow changing data between tests
var dataToReturn = initialData
override fun getData(): List<String> {
return dataToReturn
}
}
// In test
@Test
fun `processor should transform data correctly`() {
// Arrange
val fakeRepo = FakeDataRepository(listOf("item1", "item2"))
val testSubject = DataProcessor(fakeRepo)
// Act
val result = testSubject.process()
// Assert
assertThat(result).containsExactly("ITEM1", "ITEM2")
}
When writing tests, use the following naming conventions:
testSubject (not "sut" or other abbreviations)FakeDataRepository)Use AssertK for assertions in tests:
@Test
fun `example test`() {
// Arrange
val list = listOf("apple", "banana")
// Act
val result = list.contains("apple")
// Assert
assertThat(result).isTrue()
assertThat(list).contains("banana")
}
Note: You'll need to import the appropriate AssertK assertions:
assertk.assertThat for the base assertion functionassertk.assertions namespace for specific assertion types (e.g., import assertk.assertions.isEqualTo, import assertk.assertions.contains, import assertk.assertions.isTrue, etc.)This section describes the different types of tests we use in the project. Each type serves a specific purpose in our testing strategy, and together they help ensure the quality and reliability of our codebase.
Unit tests verify that individual components work correctly in isolation.
What to Test:
Key Characteristics:
Frameworks:
Location:
src/test directory or src/{platformTarget}Test for Kotlin MultiplatformContributor Expectations:
Integration tests verify that components work correctly together.
What to Test:
Key Characteristics:
Frameworks:
src/test)src/test)src/androidTest)Location:
src/test or src/commonTest, src/{platformTarget}Test for Kotlin Multiplatformsrc/androidTest, when there's a specific need for Android dependenciesWhy prefer test over androidTest:
When to use androidTest:
Contributor Expectations:
UI tests verify the application from a user's perspective.
What to Test:
Key Characteristics:
Frameworks:
Location:
src/test directory for Compose UI testssrc/androidTest directory for Espresso testsContributor Expectations:
โ ๏ธ Work in Progress โ ๏ธ
Screenshot tests verify the visual appearance of UI components.
What to Test:
Key Characteristics:
golden images)Frameworks:
Location:
src/test directoryContributor Expectations:
This section helps contributors understand our testing strategy and future plans.
End-to-End Tests โจ
Performance Tests โก
Accessibility Tests โฟ
Localization Tests ๐
Manual Test Scripts ๐
Quick commands to run tests in the project.
Run all tests:
./gradlew test
Run tests for a specific module:
./gradlew :module-name:test
Run Android instrumentation tests:
./gradlew connectedAndroidTest
Run tests with coverage verification:
./gradlew koverVerifyDebug
Generate an HTML coverage report:
./gradlew koverHtmlReportDebug
We use the Kover code coverage plugin with project defaults enforced via our Gradle build. Coverage is verified as part of CI and will fail if requirements are not met.
Minimum requirements:
How to check locally:
./gradlew koverVerifyDebug
For a specific module:
./gradlew :module-name:koverVerifyDebug
Generate a human-readable HTML report:
./gradlew koverHtmlReportDebug
Temporarily disable coverage globally (e.g., for local runs or exceptional CI cases):
Using Gradle property:
./gradlew check -PcodeCoverageDisabled=true
Using environment variable:
CODE_COVERAGE_DISABLED=true ./gradlew check
Where to find reports:
Scope:
Exemptions:
If you believe an exception is warranted (e.g., trivial or generated code), discuss it with maintainers in the PR before requesting an exemption.
[!NOTE] High coverage does not guarantee high-quality tests. Focus on meaningful tests that validate behavior and edge cases while meeting the thresholds above.