Release Troubleshooting & Recovery

This page provides guidance for troubleshooting and recovering from issues that may arise during the WooCommerce release process. It covers common scenarios, recommended actions, and best practices to help ensure releases are handled smoothly and any problems are resolved efficiently.

Scenarios / FAQ

A workflow failed while building the release

1. Open the workflow run details in GitHub (under the Actions tab) to see exactly where and why the failure occurred. Most of the time, the workflow will display a clear error message.
2. Read the error message carefully. Sometimes the problem is as simple as a missing workflow configuration or skipped step.
3. If you're unsure what the error means or how to proceed, don't hesitate to ask for help in the release Slack channel. It's better to get a second opinion than to guess.

⚠️ Do not re-run any workflows until you understand the cause of the failure. Re-running without fixing the root issue can make things more complicated.

CI is failing on a release-related PR

During the release process, you may encounter CI test failures on release-related PRs. These failures sometimes occur because test fixes were merged to trunk but not backported to the release branch before it was cut.

1. Identify the cause: Check if the failing tests pass on trunk. If they do, the fix likely needs to be backported.
2. Backport test fixes: If possible, backport the relevant test fixes from trunk to the release branch, then re-run the CI workflow.
3. Handle complex cases: If backporting isn't possible due to dependencies or the cause isn't clear, document what you've found and ask for help in the release Slack channel. The "Heart of Gold - Flux" team can assist with resolving CI issues that block release work.

<h3 id="can-i-start-over-id">Something looks wrong in the final release ZIP. Can I start over?</h3>

If, after downloading and unzipping the generated artifact, something seems off (e.g., missing files, incorrect changelog, or version mismatch), this usually means:

• A required workflow didn't run or failed (for example, the changelog step was skipped).
• An auto-generated PR from a workflow wasn't merged into the release branch before building the ZIP.

Before you try to build the version again:

1. Delete any GitHub draft release or tag for the incorrect release:
   • Go to Code > Releases and delete the draft release.
   • In Code > Tags, delete the tag for the incorrect version. If you skip this, the final release may point to the wrong commit in history.
2. Check the status of the release/X.Y branch (either in the GitHub UI or locally after pulling the latest changes).
3. Figure out which step failed. For example, if the plugin header version is correct but the changelog is missing, only the changelog step needs to be re-run.
4. Review any auto-generated PRs: if there are open PRs that weren't merged and are no longer needed, close them and delete their branches.

Once you know which step failed, re-run only that step as described in the Building & Publishing guide. Make sure to run skipped workflows in the correct order and double-check all configuration (version number, release type, etc.) before proceeding.

<h3 id="deploy-serious-bug">A serious bug was detected during internal checks / monitoring</h3>

For RC and stable releases, deploying to our staging environment and monitoring for errors is required before the release is made publicly available. If a serious bug is detected during this monitoring period, follow these steps:

1. Request a revert of the deploy in the staging environment.
2. Pause the release process immediately and do not continue with any remaining steps in the tracking issue.
3. Update the tracking issue to reflect that the release is blocked, including details about the bug.
4. Do not publish any of the draft GitHub releases that were created, but also do not delete them. They will be published later along with the version that passes validation.
5. Coordinate with the relevant engineering team(s) to develop a fix.
6. Involve Developer Advocacy if the release schedule needs to be adjusted or communicated publicly (read more on delays below).

How to proceed once the bug fix is merged into the release branch?

1. Create a new tracking issue for the new version (e.g., -rc.2 if the bug was detected during -rc.1, or x.y.2 if detected while monitoring x.y.1) by running the Release: Create Tracking Issue workflow. Do not reuse the existing tracking issue.
2. Follow the release procedure as normal for the new version.
3. Publish all draft releases for the affected version series. Even if the prior version wasn't made publicly available, it must be published along with the valid version. Each version will have its own changelog section.

A critical bug surfaced after the release was marked stable on WordPress.org

If a severe regression or bug is discovered (e.g., checkout failure or unrecoverable data loss):

1. Immediately notify the relevant engineering team(s).
2. Prepare to do a Point Release.
3. Temporarily move the stable tag on WordPress.org back to the previous known-good version:
   • Identify the correct previous version and note its exact number.
   • Use the Release: Update stable tag workflow, making sure to check the Revert option to allow downgrading.
   • Merge any auto-generated PRs right away.

<h3 id="release-delay">The release needs to be delayed. What should we do?</h3>

1. Create an internal Slack thread to communicate with the engineering teams as well as Dev Advocacy. This also provides an opportunity for teams to share any additional context and verify or challenge schedule changes.
2. Ask Dev Advocacy to communicate the delay publicly.
3. If there's a clear ETA on the patch release with a fix, update the release calendar with the new dates.

Remember to not plan the patch release too close to the weekend.

<h3 id="release-delay-weekend-id">The release was delayed. Can we still release after Tuesday?</h3>

In general, avoid releasing after Tuesday, especially close to a weekend.

Even if a patch is ready and seems to fix the problem, it's hard to be sure there aren't other hidden issues and a rushed release late in the week means most of the team won't be available to monitor or respond to problems.

As a rule of thumb, when in doubt, consider to delay the release by a week for confidence.