docs/decision_records/engineering/ENG-003-test-infrastructure.md
Proposal
E2E tests ensure the functional correctness in an e2e environment in order to make sure Dapr works with the user code deployments. The tests will be run before / after PR is merged or by a scheduler.
Dapr E2E tests require the test infrastructure in order to not only test Dapr functionalities, but also show these test results in a consistent way. This document will decide how to bring up the test cluster, run the test, and report the test results.
Although Dapr is designed for multi cloud environments, e2e tests will be run under Kubernetes environments for now. We will support two different options to run e2e tests with local machine and CI on the pre-built Kubernetes cluster.
Local machine. contributors or developers will use Minikube to validate their changes and run new tests before creating Pull Request.
Continuous Integration. E2E tests will be run in the pre-built Azure Kubernetes Service before/after PR is merged or by a scheduler. Even if we will use Azure Kubernetes Service in our test infrastructure, contributors should run e2e tests in any RBAC-enabled Kubernetes clusters.
We will provide the manual instruction or simple script to bring up test infrastructure unlike the other Kubernetes projects using kubetest. Dapr E2E tests will clean up and revert all configurations in the cluster once the test is done. Without kubetest, we can create e2e tests simpler without the dependency of the 3rd party test frameworks, such as ginkgo, gomega.
Many Kubernetes-related projects use Prow, and Testgrid for Test CI, PR, and test result management. However, we will not use them to run Dapr E2E tests and share the test result since we need to self-host them on Google cloud platform.
Instead, Dapr will use Azure Pipeline to run e2e tests and its test report feature without self-hosted CI and test report services. Even contributors can get their own azure pipelines accounts for free without self-hosting them.