qa/README.md
The acceptance test framework for Logstash is intended to test the functionality of packages (.deb, .rpm)
on various supported platforms.
In this small README we describe its features and the steps necessary for executing it and adding new tests.
In summary this test framework is composed of:
The tests are expected to be executed on the target environment e.g. an Ubuntu 22.04 vm.
To run the tests from a fresh Logstash checkout, you need to:
./gradlew clean boostrap
2a. Build the necessary package artifacts e.g. rake artifact:deb
OR
2b. Supply a directory where pregenerated package artifacts exit via the LS_ARTIFACTS_PATH environment variable (relative and absolute paths are supported).cd qabundle installNow you are ready to kick off the tests:
rake qa:acceptance:all.Steps 1, 2b, 3, 4, 5 are executed by the ci/acceptance_tests.sh script.
acceptance/: all the specs definitions.rspec: all framework parts necessary to get the test
running. Includes the commands, the rspec matchers and a
collection of useful helpers.To add a test you basically should start by the acceptance directory, here you will find an already created tests, most important locations here are:
lib here is where the tests are living. If a test is not going
to be reused it should be created here.shared_examples inside that directory should be living all tests
that could be reused in different scenarios, like you can see the CLI
ones.but we want to write tests, here is an example of how do they look like, including the different moving parts we encounter in the framework.
logstash = ServiceTester::Artifact.new()
## your test code goes here.
# example:
it_behaves_like "installable_with_jdk", logstash
it_behaves_like "updated", logstash, from_release_branch="7.17"
Inside the rspec directory you will find a
collection of commands, organized per operating system, which
will let you operate and get your tests done.
You'll probably find enough supporting classes for different platforms, but if not, feel free to add more.
An example of an install command on debian looks like:
def installed?(package)
stdout = ""
cmd = sudo_exec!("dpkg -s #{package}")
stdout = cmd.stdout
stdout.match(/^Package: #{package}$/)
stdout.match(/^Status: install ok installed$/)
end
end
this is how we run operations and wrap them as ruby code.
There is also the possibility to run your tests with more granularity by
using the rspec command, this will let you for example run a single
tests, a collection of them using filtering, etc.
Check https://relishapp.com/rspec/rspec-core/v/3-4/docs/command-line for more details, but here is a quick cheat sheet to run them: