ContextQMD
Back to Etcd

Release

Documentation/contributor-guide/release.md

3.7.09.5 KB
Original Source

Release

The guide talks about how to release a new version of etcd.

The procedure includes some manual steps for sanity checking, but it can probably be further scripted. Please keep this document up-to-date if making changes to the release process.

Release management

Under the leadership of James Blair @jmhbnz and Ivan Valdes Castillo @ivanvc, the following pool of release candidates manages the release of each etcd major/minor version as well as manages patches to each stable release branch. They are responsible for communicating the timelines and status of each release and for ensuring the stability of the release branch.

All release version numbers follow the format of semantic versioning 2.0.0.

Major, minor version release, or its pre-release

Patch version release

  • To request a backport, developers submit cherry-pick PRs targeting the release branch. The commits should not include merge commits. The commits should be restricted to bug fixes and security patches.
  • The cherrypick PRs should target the appropriate release branch (base:release-<major>-<minor>). The k8s infra cherry pick robot /cherrypick <branch> PR chatops command may be used to automatically generate cherrypick PRs.
  • The release patch manager reviews the cherrypick PRs. Please discuss carefully what is backported to the patch release. Each patch release should be strictly better than its predecessor.
  • The release patch manager will cherry-pick these commits starting from the oldest one into stable branch.

Write a release note

  • Write an introduction for the new release. For example, what major bug we fix, what new features we introduce, or what performance improvement we make.
  • Put [GH XXXX] at the head of the change line to reference the Pull Request that introduces the change. Moreover, add a link on it to jump to the Pull Request.
  • Find PRs with the release-note label and explain them in the NEWS file, as a straightforward summary of changes for end-users.

Patch release criteria

The etcd project aims to release a new patch version if any of the following conditions are met:

  • Fixed one or more major CVEs (>=7.5).
  • Fixed one or more critical bugs.
  • Fixed three or more major bugs.
  • Fixed five or more minor bugs.

Release guide

Prerequisites

There are some prerequisites, which should be done before the release process. These are one-time operations, which don't need to be executed before releasing each version.

  1. Generate a GPG key and add it to your GitHub account. Refer to the links on settings.
  2. Ensure you have a Linux machine, on which the git, Golang, and docker have been installed.
    • Ensure the Golang version matches the version defined in .go-version file.
    • Ensure non-privileged users can run docker commands, refer to the Linux postinstall.
    • Ensure there is at least 5GB of free space on your Linux machine.
  3. Install gsutil, refer to gsutil_install. When asked about cloud project to use, pick etcd-development.
  4. Authenticate the image registry, refer to Authentication methods.
    • gcloud auth login
    • gcloud auth configure-docker
  5. Install gh, refer to GitHub's documentation. Ensure that running gh auth login succeeds for the GitHub account you use to contribute to etcd, and that gh auth status has a clean exit and doesn't show any issues.

Release steps

At least one day before the release:

  1. Raise an issue to publish the release plan, e.g. issues/17350.
  2. Raise a kubernetes/org pull request (example PR) to ensure members of the release team are added to the release github team.

On the day of the release:

  1. Verify you can pass the authentication to the image registries,

    • docker login gcr.io
    • docker login quay.io
      • If the release person doesn't have access to 1password, one of the owners (@ahrtr, @ivanvc, @jmhbnz, @serathius) needs to share the password with them per this guide. See rough steps below,
        • Sign in to your account on 1password.com.
        • Click Your Vault Items on the right side.
        • Select Password of quay.io.
        • Click Share on the top right, and set expiration as 1 hour and only available to the release person using his/her email.
        • Click Copy Link then send the link to the release person via slack or email.

  2. Clone the etcd repository and checkout the target branch,

  3. Run the release script under the repository's root directory, replacing ${VERSION} with a value without the v prefix, i.e. 3.5.13.

    • DRY_RUN=false ./scripts/release.sh ${VERSION}

    • NOTE: When doing a pre-release (i.e., a version from the main branch, 3.6.0-alpha.2), you will need to explicitly set the branch to main:

      bash
      DRY_RUN=false BRANCH=main ./scripts/release.sh ${VERSION}

    It generates all release binaries under the directory /tmp/etcd-release-${VERSION}/etcd/release/ and images. Binaries are pushed to the Google Cloud bucket under project etcd-development, and images are pushed to quay.io and gcr.io.

    • It is advisable to do a dry run before the actual release. This will create a /tmp directory. Do NOT forget to remove this directory before the actual release.

      bash
      DRY_RUN=true BRANCH=${BRANCH} ./scripts/release.sh ${VERSION}

  4. Publish the release page on GitHub

    • Open the draft release URL shown by the release script
    • Click the pen button at the top right to edit the release
    • Review that it looks correct, reviewing that the bottom checkboxes are checked depending on the release version (v3.4 & v3.5 no checkboxes, v3.6 has the set as latest release checkbox checked, v3.7 has the set as pre-release checkbox checked)
    • Then, publish the release

  5. Announce to the etcd-dev googlegroup

    Follow the format of previous release emails sent to [email protected], see an example below. After sending out the email, ask one of the mailing list maintainers to approve the email from the pending list. Additionally, label the release email as Release.

    text
    Hello,

etcd v3.4.30 is now public!

https://github.com/etcd-io/etcd/releases/tag/v3.4.30

Thanks to everyone who contributed to the release!

etcd team

  6. Update the changelog to reflect the correct release date.

  7. Paste the release link to the issue raised in Step 1 and close the issue.

  8. Raise a follow-up kubernetes/org pull request to return the GitHub release team to empty, least privilege state.

  9. Crease a new stable branch through git push origin release-${VERSION_MAJOR}.${VERSION_MINOR} if this is a new major or minor stable release.

  10. Re-generate a new password for quay.io if needed (e.g. shared to a contributor who isn't in the release team, and we should rotate the password at least once every 3 months).

  11. Bump the new etcd release in Kubernetes, refer to Bump etcd Version in Kubernetes.

  • For etcd 3.6 patches, bump it to Kubernetes 1.34 and all newer minor versions (including master branch)
  • For etcd 3.5 patches, bump it to Kubernetes 1.33 and all older supported versions

Release known issues

  1. Timeouts pushing binaries - If binaries fail to fully upload to Google Cloud storage, the script must be re-run using the same command. Any artifacts that are already pushed will be overwritten to ensure they are correct. The storage bucket does not use object versioning so incorrect files cannot remain.

  2. Timeouts pushing images - It is rare, although possible for connection timeouts to occur when publishing etcd release images to quay.io or gcr.io. If this occurs, it is known to be safe to rerun the release script command appending the --no-upload flag, and image uploads will gracefully resume.

  3. GPG vs SSH signing - The release scripts assume that git tags will be signed with a GPG key. Since 2022 GitHub has supported signing commits and tags using ssh. Until further release script updates are completed you will need to disable this feature in your ~/.gitconfig and revert to signing via GPG to perform etcd releases.