docs/docsite/rst/dev_guide/developing_collections_distributing.rst
.. _distributing_collections:
Distributing collections
A collection is a distribution format for Ansible content. A typical collection contains modules and other plugins that address a set of related use cases. For example, a collection might automate administering a particular database. A collection can also contain roles and playbooks.
.. note::
Before distributing your collection, ensure you have updated the galaxy.yml file.
See :ref:collection_structure for details.
To distribute your collection and allow others to use it, you can publish your collection on one or more :term:distribution server. Distribution servers include:
================================= ===================================================================
Distribution server Collections accepted
================================= ===================================================================
Ansible Galaxy All collections
:term:Pulp 3 Galaxy All collections, supports signed collections
Red Hat Automation Hub Only collections certified by Red Hat, supports signed collections
Privately hosted Automation Hub Collections authorized by the owners
================================= ===================================================================
Distributing collections involves four major steps:
#. Initial configuration of your distribution server or servers #. Building your collection tarball #. Preparing to publish your collection #. Publishing your collection
.. contents:: :local: :depth: 2
.. _config_distribution_server:
Configure a connection to one or more distribution servers so you can publish collections there. You only need to configure each distribution server once. You must repeat the other steps (building your collection tarball, preparing to publish, and publishing your collection) every time you publish a new collection or a new version of an existing collection.
.. _get_namespace:
You must upload your collection into a namespace on each distribution server. If you have a login for Ansible Galaxy, your Ansible Galaxy username is usually also an Ansible Galaxy namespace.
.. warning::
Namespaces on Ansible Galaxy cannot include hyphens. If you have a login for Ansible Galaxy that includes a hyphen, your Galaxy username is not also a Galaxy namespace. For example, awesome-user is a valid username for Ansible Galaxy, but it is not a valid namespace.
You can create additional namespaces on Ansible Galaxy if you choose. For Red Hat Automation Hub and private Automation Hub you must create a namespace before you can upload your collection. To create a namespace:
Galaxy namespaces <https://galaxy.ansible.com/docs/contributing/namespaces.html#galaxy-namespaces>_ on the Galaxy docsite for details.Red Hat automation content documentation <https://docs.redhat.com/en/documentation/red_hat_ansible_automation_platform/>_.Specify the namespace in the :file:galaxy.yml file for each collection. For more information on the :file:galaxy.yml file, see :ref:collections_galaxy_meta.
.. _galaxy_get_token:
An API token authenticates your connection to each distribution server. You need a separate API token for each distribution server. Use the correct API token to connect to each distribution server securely and protect your content.
To get your API token:
the Galaxy documentation <https://galaxy.ansible.com/ui/token/>_.Red Hat automation content documentation <https://docs.redhat.com/en/documentation/red_hat_ansible_automation_platform/>_... _galaxy_specify_token:
Each time you publish a collection, you must specify the API token and the distribution server to create a secure connection. You have two options for specifying the token and distribution server:
galaxy_server_list entry in your :file:ansible.cfg file. Using configuration is the most secure option.ansible-galaxy command. If you pass the token at the command line, you can specify the server at the command line, by using the default setting, or by setting the server in configuration. Passing the token at the command line is insecure, because typing secrets at the command line may expose them to other users on the system... _galaxy_token_ansible_cfg:
Specifying the token and distribution server in configuration ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
By default, Ansible Galaxy is configured as the only distribution server. You can add other distribution servers and specify your API token or tokens in configuration by editing the galaxy_server_list section of your :file:ansible.cfg file. This is the most secure way to manage authentication for distribution servers. Specify a URL and token for each server. For example:
.. code-block:: ini
[galaxy] server_list = release_galaxy
[galaxy_server.release_galaxy] url=https://galaxy.ansible.com/ token=abcdefghijklmnopqrtuvwxyz
You cannot use apt-key with any servers defined in your :ref:galaxy_server_list <galaxy_server_config>. See :ref:galaxy_server_config for complete details.
.. _galaxy_use_token_arg:
Specifying the token at the command line ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You can specify the API token at the command line using the --token argument of the :ref:ansible-galaxy command. There are three ways to specify the distribution server when passing the token at the command line:
--server argument of the :ref:ansible-galaxy commandGALAXY_SERVER setting in your :file:ansible.cfg fileFor example:
.. code-block:: bash
ansible-galaxy collection publish path/to/my_namespace-my_collection-1.0.0.tar.gz --token abcdefghijklmnopqrtuvwxyz
.. warning::
Using the --token argument is insecure. Passing secrets at the command line may expose them to others on the system.
.. _building_collections:
After configuring one or more distribution servers, build a collection tarball. The collection tarball is the published artifact, the object that you upload and other users download to install your collection. To build a collection tarball:
#. Review all settings in your :file:galaxy.yml file. See :ref:collections_galaxy_meta for details. Ensure you have updated the version number. Each time you publish your collection, it must have a new version number. You cannot make changes to existing versions of your collection on a distribution server. If you try to upload the same collection version more than once, the distribution server returns the error Code: conflict.collection_exists. Collections follow semantic versioning rules. For more information on versions, see :ref:collection_versions. For more information on the :file:galaxy.yml file, see :ref:collections_galaxy_meta.
#. Run ansible-galaxy collection build from inside the top-level directory of the collection. For example:
.. code-block:: bash
collection_dir#> ansible-galaxy collection build
This command builds a tarball of the collection in the current directory, which you can upload to your selected distribution server:
.. code-block:: shell
my_collection/
├── galaxy.yml
├── ...
├── my_namespace-my_collection-1.0.0.tar.gz
└── ...
.. note::
ignoring_files_and_folders_collections if your collection directory contains other files you want to exclude.You can upload your tarball to one or more distribution servers. You can also distribute your collection locally by copying the tarball to install your collection directly on target systems.
.. _ignoring_files_and_folders_collections:
You can exclude files from your collection with either :ref:build_ignore <build_ignore> or :ref:manifest_directives. For more information on the :file:galaxy.yml file, see :ref:collections_galaxy_meta.
.. _build_ignore:
Include all, with explicit ignores ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
By default, the build step includes all the files in the collection directory in the tarball except for the following:
galaxy.yml*.pyc*.retrytests/output.git/To exclude other files and folders from your collection tarball, set a list of file glob-like patterns in the build_ignore key in the collection's galaxy.yml file. These patterns use the following special characters for wildcard matching:
*: Matches everything?: Matches any single character[seq]: Matches any character in sequence[!seq]:Matches any character not in sequenceFor example, to exclude the :file:sensitive folder within the playbooks folder as well any .tar.gz archives, set the following in your galaxy.yml file:
.. code-block:: yaml
build_ignore:
- playbooks/sensitive
- '*.tar.gz'
.. note::
The build_ignore feature is only supported with ansible-galaxy collection build in Ansible 2.10 or newer.
.. _manifest_directives:
Manifest Directives ^^^^^^^^^^^^^^^^^^^
.. versionadded:: 2.14
The :file:galaxy.yml file supports manifest directives that are historically used in Python packaging, as described in MANIFEST.in commands <https://packaging.python.org/en/latest/guides/using-manifest-in/#manifest-in-commands>_.
.. note::
The use of manifest requires installing the optional distlib Python dependency.
.. note::
The manifest feature is only supported with ansible-galaxy collection build in ansible-core 2.14 or newer, and is mutually exclusive with build_ignore.
For example, to exclude the :file:sensitive folder within the playbooks folder as well as any .tar.gz archives, set the following in your :file:galaxy.yml file:
.. code-block:: yaml
manifest: directives: - recursive-exclude playbooks/sensitive ** - global-exclude *.tar.gz
By default, the MANIFEST.in style directives would exclude all files by default, but there are default directives in place. Those default directives are described below. To see the directives in use during build, pass -vvv with the ansible-galaxy collection build command.
.. code-block:: text
include meta/*.yml include *.txt *.md *.rst COPYING LICENSE recursive-include tests ** recursive-include docs **.rst **.yml **.yaml **.json **.j2 **.txt recursive-include roles **.yml **.yaml **.json **.j2 recursive-include playbooks **.yml **.yaml **.json recursive-include changelogs **.yml .yaml recursive-include plugins */.py recursive-include plugins/become **.yml **.yaml recursive-include plugins/cache **.yml **.yaml recursive-include plugins/callback **.yml **.yaml recursive-include plugins/cliconf **.yml **.yaml recursive-include plugins/connection **.yml **.yaml recursive-include plugins/filter **.yml **.yaml recursive-include plugins/httpapi **.yml **.yaml recursive-include plugins/inventory **.yml **.yaml recursive-include plugins/lookup **.yml **.yaml recursive-include plugins/netconf **.yml **.yaml recursive-include plugins/shell **.yml **.yaml recursive-include plugins/strategy **.yml **.yaml recursive-include plugins/test **.yml **.yaml recursive-include plugins/vars **.yml **.yaml recursive-include plugins/modules **.ps1 **.yml **.yaml recursive-include plugins/module_utils **.ps1 **.psm1 **.cs
exclude galaxy.yml galaxy.yaml MANIFEST.json FILES.json <namespace>-<name>-.tar.gz recursive-exclude tests/output ** global-exclude /. /pycache
.. note::
<namespace>-<name>-*.tar.gz is expanded with the actual namespace and name.
The manifest.directives supplied in :file:galaxy.yml are inserted after the default includes and before the default excludes.
To enable the use of manifest directives without supplying your own, insert either manifest: {} or manifest: null in the :file:galaxy.yml file and remove any use of build_ignore.
If the default manifest directives do not meet your needs, you can set manifest.omit_default_directives to a value of true in :file:galaxy.yml. You then must specify a full compliment of manifest directives in :file:galaxy.yml. The defaults documented above are a good starting point.
Below is an example where the default directives are not included.
.. code-block:: yaml
manifest: directives: - include meta/runtime.yml - include README.md LICENSE - recursive-include plugins /**.py - exclude galaxy.yml MANIFEST.json FILES.json <namespace>-<name>-.tar.gz - recursive-exclude tests/output ** omit_default_directives: true
.. _signing_collections:
You can include a GnuPG signature with your collection on a :term:Pulp 3 Galaxy server. See Enabling collection signing <https://galaxyng.netlify.app/config/collection_signing/>_ for details.
You can manually generate detached signatures for a collection using the gpg CLI using the following step. This step assume you have generated a GPG private key, but do not cover this process.
.. code-block:: bash
ansible-galaxy collection build tar -Oxzf namespace-name-1.0.0.tar.gz MANIFEST.json | gpg --output namespace-name-1.0.0.asc --detach-sign --armor --local-user [email protected] -
.. _trying_collection_locally:
Each time you publish your collection, you must create a :ref:new version <collection_versions> on the distribution server. After you publish a version of a collection, you cannot delete or modify that version. To avoid unnecessary extra versions, check your collection for bugs, typos, and other issues locally before publishing:
#. Install the collection locally. #. Review the locally installed collection before publishing a new version.
You have two options for installing your collection locally:
Installing your collection locally from the tarball ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To install your collection locally from the tarball, run ansible-galaxy collection install and specify the collection tarball. You can optionally specify a location using the -p flag. For example:
.. code-block:: bash
collection_dir#> ansible-galaxy collection install my_namespace-my_collection-1.0.0.tar.gz -p ./collections
Install the tarball into a directory configured in :ref:COLLECTIONS_PATHS so Ansible can easily find and load the collection. If you do not specify a path value, ansible-galaxy collection install installs the collection in the first path defined in :ref:COLLECTIONS_PATHS.
.. _collections_scm_install:
Installing your collection locally from a Git repository ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To install your collection locally from a Git repository, specify the repository and the branch you want to install:
.. code-block:: bash
collection_dir#> ansible-galaxy collection install git+https://github.com/org/repo.git,devel
.. include:: ../shared_snippets/installing_collections_git_repo.txt
Review the collection:
Using collections <using_collections>... _collection_versions:
The only way to change a collection is to release a new version. The latest version of a collection (by highest version number) is the version displayed everywhere in Galaxy and Automation Hub. Users can still download older versions.
Follow semantic versioning when setting the version for your collection. In summary:
x of x.y.z, for an incompatible API change.y of x.y.z, for new functionality in a backwards compatible manner (for example new modules/plugins, parameters, return values).z of x.y.z, for backwards compatible bug fixes.Read the official Semantic Versioning <https://semver.org/>_ documentation for details and examples.
.. _publish_collection:
The last step in distributing your collection is publishing the tarball to Ansible Galaxy, Red Hat Automation Hub, or a privately hosted Automation Hub instance. You can publish your collection in two ways:
ansible-galaxy collection publish command.. _upload_collection_ansible_galaxy: .. _publish_collection_galaxy_cmd:
To upload the collection tarball from the command line using ansible-galaxy:
.. code-block:: bash
ansible-galaxy collection publish path/to/my_namespace-my_collection-1.0.0.tar.gz
.. note::
This ansible-galaxy command assumes you have retrieved and stored your API token in configuration. See :ref:`galaxy_specify_token` for details.
The ansible-galaxy collection publish command triggers an import process, just as if you uploaded the collection through the Galaxy website. The command waits until the import process completes before reporting the status back. If you want to continue without waiting for the import result, use the --no-wait argument and manually look at the import progress in your My Imports <https://galaxy.ansible.com/my-imports/>_ page.
.. _upload_collection_galaxy:
See the Galaxy documentation <https://ansible.readthedocs.io/projects/galaxy-ng/en/latest/usage_guide/collections/#via-the-ui>_ to learn how to publish your collection directly on the Galaxy website.
.. seealso::
:ref:collections
Learn how to install and use collections.
:ref:collections_galaxy_meta
Table of fields used in the :file:galaxy.yml file
:ref:Communication<communication>
Got questions? Need help? Want to share your ideas? Visit the Ansible communication guide