docs/docsite/rst/dev_guide/testing_running_locally.rst
:orphan:
.. _testing_running_locally:
Testing Ansible and Collections
This document describes how to run tests using ansible-test.
.. contents:: :local:
Before running ansible-test, set up your environment for :ref:testing_an_ansible_collection or
:ref:testing_ansible_core, depending on which scenario applies to you.
.. warning::
If you use git for version control, make sure the files you are working with are not ignored by git.
If they are, ansible-test will ignore them as well.
.. _testing_an_ansible_collection:
If you are testing an Ansible Collection, you need a copy of the collection, preferably a Git clone.
For example, to work with the community.windows collection, follow these steps:
Clone the collection you want to test into a valid collection root:
.. code-block:: shell
git clone https://github.com/ansible-collections/community.windows ~/dev/ansible_collections/community/windows
.. important::
The path must end with /ansible_collections/{collection_namespace}/{collection_name} where
{collection_namespace} is the namespace of the collection and {collection_name} is the collection name.
Clone any collections on which the collection depends:
.. code-block:: shell
git clone https://github.com/ansible-collections/ansible.windows ~/dev/ansible_collections/ansible/windows
.. important::
If your collection has any dependencies on other collections, they must be in the same collection root, since
ansible-test will not use your configured collection roots (or other Ansible configuration).
.. note::
See the collection's galaxy.yml for a list of possible dependencies.
Switch to the directory where the collection to test resides:
.. code-block:: shell
cd ~/dev/ansible_collections/community/windows
.. _testing_ansible_core:
ansible-coreIf you are testing ansible-core itself, you need a copy of the ansible-core source code, preferably a Git clone.
Having an installed copy of ansible-core is not sufficient or required.
For example, to work with the ansible-core source cloned from GitHub, follow these steps:
Clone the ansible-core repository:
.. code-block:: shell
git clone https://github.com/ansible/ansible ~/dev/ansible
Switch to the directory where the ansible-core source resides:
.. code-block:: shell
cd ~/dev/ansible
Add ansible-core programs to your PATH:
.. code-block:: shell
source hacking/env-setup
.. note::
You can skip this step if you only need to run ansible-test, and not other ansible-core programs.
In that case, simply run bin/ansible-test from the root of the ansible-core source.
.. caution::
If you have an installed version of ansible-core and are trying to run ansible-test from your PATH,
make sure the program found by your shell is the one from the ansible-core source:
.. code-block:: shell
which ansible-test
The most commonly used test commands are:
ansible-test sanity - Run sanity tests (mostly linters and static analysis).ansible-test integration - Run integration tests.ansible-test units - Run unit tests.Run ansible-test --help to see a complete list of available commands.
.. note::
For detailed help on a specific command, add the --help option after the command.
Most ansible-test commands support running in one or more isolated test environments to simplify testing.
Containers are recommended for running sanity, unit and integration tests, since they provide consistent environments. Unit tests will be run with network isolation, which avoids unintentional dependencies on network resources.
The --docker option runs tests in a container using either Docker or Podman.
.. note::
If both Docker and Podman are installed, Docker will be used.
To override this, set the environment variable ANSIBLE_TEST_PREFER_PODMAN to any non-empty value.
Choosing a container ^^^^^^^^^^^^^^^^^^^^
Without an additional argument, the --docker option uses the default container.
To use another container, specify it immediately after the --docker option.
.. note::
The default container is recommended for all sanity and unit tests.
To see the list of supported containers, use the --help option with the ansible-test command you want to use.
.. note::
The list of available containers is dependent on the ansible-test command you are using.
You can also specify your own container.
When doing so, you will need to indicate the Python version in the container with the --python option.
Custom containers """""""""""""""""
When building custom containers, keep in mind the following requirements:
USER should be root.init process, such as systemd.sshd and accept connections on the default port of 22.sh shell which can be found on PATH.sleep utility which runs as a subprocess.VOLUME statement.Docker and SELinux ^^^^^^^^^^^^^^^^^^
Using Docker on a host with SELinux may require setting the system in permissive mode. Consider using Podman instead.
Docker Desktop with WSL2 ^^^^^^^^^^^^^^^^^^^^^^^^
These instructions explain how to use ansible-test with WSL2 and Docker Desktop without systemd support.
.. note::
If your WSL2 environment includes systemd support, these steps are not required.
.. _configuration_requirements:
Configuration requirements """"""""""""""""""""""""""
Open Docker Desktop and go to the Settings screen.
On the the General tab:
a. Uncheck the Start Docker Desktop when you log in checkbox. b. Check the Use the WSL 2 based engine checkbox.
On the Resources tab under the WSL Integration section:
a. Enable distros you want to use under the Enable integration with additional distros section.
Click Apply and restart if changes were made.
.. _setup_instructions:
Setup instructions """"""""""""""""""
.. note::
If all WSL instances have been stopped, these changes will need to be re-applied.
Verify Docker Desktop is properly configured (see :ref:configuration_requirements).
Quit Docker Desktop if it is running:
a. Right click the Docker Desktop taskbar icon. b. Click the Quit Docker Desktop option.
Stop any running WSL instances with the command:
.. code-block:: shell
wsl --shutdown
Verify all WSL instances have stopped with the command:
.. code-block:: shell
wsl -l -v
Start a WSL instance and perform the following steps as root:
a. Verify the systemd subsystem is not registered:
a. Check for the systemd cgroup hierarchy with the following command:
.. code-block:: shell
grep systemd /proc/self/cgroup
b. If any matches are found, re-check the :ref:configuration_requirements and follow the
:ref:setup_instructions again.
b. Mount the systemd cgroup hierarchy with the following commands:
.. code-block:: shell
mkdir /sys/fs/cgroup/systemd mount cgroup -t cgroup /sys/fs/cgroup/systemd -o none,name=systemd,xattr
Start Docker Desktop.
You should now be able to use ansible-test with the --docker option.
.. _linux_cgroup_configuration:
Linux cgroup configuration ^^^^^^^^^^^^^^^^^^^^^^^^^^
.. note::
These changes will need to be re-applied each time the container host is booted.
For certain container hosts and container combinations, additional setup on the container host may be required.
In these situations ansible-test will report an error and provide additional instructions to run as root:
.. code-block:: shell
mkdir /sys/fs/cgroup/systemd mount cgroup -t cgroup /sys/fs/cgroup/systemd -o none,name=systemd,xattr
If you are using rootless Podman, an additional command must be run, also as root.
Make sure to substitute your user and group for {user} and {group} respectively:
.. code-block:: shell
chown -R {user}:{group} /sys/fs/cgroup/systemd
Podman """"""
When using Podman, you may need to stop existing Podman processes after following the :ref:linux_cgroup_configuration
instructions. Otherwise Podman may be unable to see the new mount point.
You can check to see if Podman is running by looking for podman and catatonit processes.
Remote virtual machines are recommended for running integration tests not suitable for execution in containers.
The --remote option runs tests in a cloud hosted ephemeral virtual machine.
.. note::
An API key is required to use this feature, unless running under an approved Azure Pipelines organization.
To see the list of supported systems, use the --help option with the ansible-test command you want to use.
.. note::
The list of available systems is dependent on the ansible-test command you are using.
Python virtual environments provide a simple way to achieve isolation from the system and user Python environments.
They are recommended for unit and integration tests when the --docker and --remote options cannot be used.
The --venv option runs tests in a virtual environment managed by ansible-test.
Requirements are automatically installed before tests are run.
The environment arguments covered in this document are sufficient for most use cases. However, some scenarios may require the additional flexibility offered by composite environment arguments.
The --controller and --target options are alternatives to the --docker, --remote and --venv options.
.. note::
When using the shell command, the --target option is replaced by three platform specific options.
Add the --help option to your ansible-test command to learn more about the composite environment arguments.
Some ansible-test commands have additional requirements.
You can use the --requirements option to automatically install them.
.. note::
When using a test environment managed by ansible-test the --requirements option is usually unnecessary.
When using environment variables to manipulate tests there some limitations to keep in mind. Environment variables are:
Not propagated from the host to the test environment when using the --docker or --remote options.
Not exposed to the test environment unless enabled in test/lib/ansible_test/_internal/util.py in the common_environment function.
Example: ANSIBLE_KEEP_REMOTE_FILES=1 can be set when running ansible-test integration --venv. However, using the --docker option would
require running ansible-test shell to gain access to the Docker environment. Once at the shell prompt, the environment variable could be set
and the tests executed. This is useful for debugging tests inside a container by following the
:ref:debugging_modules instructions.
Use the ansible-test shell command to get an interactive shell in the same environment used to run tests. Examples:
ansible-test shell --docker - Open a shell in the default docker container.ansible-test shell --venv --python 3.13 - Open a shell in a Python 3.13 virtual environment.Code coverage reports make it easy to identify untested code for which more tests should
be written. Online reports are available but only cover the devel branch (see
:ref:developing_testing). For new code local reports are needed.
Add the --coverage option to any test command to collect code coverage data. If you
aren't using the --venv or --docker options which create an isolated python
environment then you may have to use the --requirements option to ensure that the
correct version of the coverage module is installed:
.. code-block:: shell
ansible-test coverage erase ansible-test units --coverage apt ansible-test integration --coverage aws_lambda ansible-test coverage html
Reports can be generated in several different formats:
ansible-test coverage report - Console report.ansible-test coverage html - HTML report.ansible-test coverage xml - XML report.To clear data between test runs, use the ansible-test coverage erase command.