content/operate/kubernetes/upgrade/openshift-cli.md
Redis implements rolling updates for software upgrades in Kubernetes deployments. The upgrade process includes updating three components:
The following steps ensure you have the minimum versions of all components necessary to upgrade to 7.8.2-6 or later. Without these minimum versions, the upgrade will freeze and require manual recovery.
See the troubleshooting section for details on recovering a failed upgrade.
Check [Supported Kubernetes distributions]({{<relref "/operate/kubernetes/reference/supported_k8s_distributions" >}}) to make sure your Kubernetes distribution is supported. If not, upgrade your Kubernetes distribution before upgrading the Redis operator.
Your Redis Enterprise clusters must be running version 7.4.2-2 or later before upgrading to 7.8.2-6 or later. See the 7.4 upgrade for detailed steps.
Your Redis databases must be running version 7.2 or later before upgrading your cluster version. See upgrade databases for detailed steps. You can find your database version in the [REDB spec.redisVersion field]({{<relref "/operate/kubernetes/reference/api/redis_enterprise_database_api#redisversion" >}}).
If your databases use user-defined modules (custom non-bundled modules):
autoUpgradeRedisEnterprise: false in the REC custom resource before upgrading the operator.redisEnterpriseImageSpec for more details.For more information about user-defined modules, see [User-defined modules]({{< relref "/operate/kubernetes/re-databases/modules#user-defined-modules" >}}).
Upgrading to Redis operator versions 7.8.2-6 and later involves migrating your Redis Enterprise nodes to RHEL9 from either Ubuntu 18 or RHEL8. If your databases use modules, you need to manually install modules compatible with RHEL9.
To see which modules you have installed, run:
curl -k -u <rec_username>:<rec_password> -X GET https://localhost:9443/v1/modules | jq -r 'map([.module_name, .semantic_version, (.platforms | keys)]) | .[] | .[0] as $name | .[1] as $version | .[2][] | $name + "-" + $version + "-" + .' | sort
To see which modules are currently in use, run:
curl -k -u <rec_username>:<rec_password> -X GET https://localhost:9443/v1/bdbs | jq -r '.[].module_list | map(.module_name + "-" + .semantic_version) | .[]'
See [Upgrade modules]({{<relref "/operate/oss_and_stack/stack-with-enterprise/install/upgrade-module">}}) for details on how to upgrade modules with the rladmin tool.
Use kubectl get rec and verify the LICENSE STATE is valid on your REC before you start the upgrade process.
If you installed the Redis Enterprise operator using Helm charts on OpenShift, you can upgrade using Helm commands. This method automatically handles the operator upgrade and Custom Resource Definition (CRD) updates.
To upgrade using Helm on OpenShift:
helm upgrade <release-name> <repo-name>/redis-enterprise-operator --version <chart-version> \
--set openshift.mode=true
For example:
helm upgrade my-redis-enterprise <repo-name>/redis-enterprise-operator --version 7.8.2-2 \
--set openshift.mode=true
After the Helm upgrade completes, continue with upgrading the Redis Enterprise cluster and upgrading databases.
For detailed Helm upgrade instructions, see [Upgrade the chart]({{<relref "/operate/kubernetes/deployment/helm#upgrade-the-chart">}}).
{{<warning>}}If your databases use user-defined modules, set autoUpgradeRedisEnterprise: false in the REC custom resource before upgrading the operator.{{</warning>}}
Make sure you pull the correct version of the bundle. You can find the version tags by checking the operator releases on GitHub or by using the GitHub API.
For OpenShift environments, the name of the bundle is openshift.bundle.yaml, and so the curl command to run is:
curl --silent -O https://raw.githubusercontent.com/RedisLabs/redis-enterprise-k8s-docs/$VERSION/openshift.bundle.yaml
If you need a different release, replace VERSION in the above with a specific release tag.
Apply the bundle to deploy the new operator binary. This will also apply any changes in the new release to custom resource definitions, roles, role binding, or operator service accounts.
{{< note >}} If you are not pulling images from Docker Hub, update the operator image spec to point to your private repository. If you have made changes to the role, role binding, RBAC, or custom resource definition (CRD) in the previous version, merge them with the updated declarations in the new version files. {{< /note >}}
If you are using OpenShift, run this instead:
oc apply -f openshift.bundle.yaml
After running this command, you should see a result similar to this:
role.rbac.authorization.k8s.io/redis-enterprise-operator configured
serviceaccount/redis-enterprise-operator configured
rolebinding.rbac.authorization.k8s.io/redis-enterprise-operator configured
customresourcedefinition.apiextensions.k8s.io/redisenterpriseclusters.app.redislabs.com configured
customresourcedefinition.apiextensions.k8s.io/redisenterprisedatabases.app.redislabs.com configured
deployment.apps/redis-enterprise-operator configured
If you have the admission controller enabled, you need to manually reapply the ValidatingWebhookConfiguration.
{{<note>}} {{< embed-md "k8s-642-redb-admission-webhook-name-change.md" >}} {{</note>}}
{{< embed-md "k8s-admission-webhook-cert.md" >}}
You can check your deployment to verify the operator is running in your namespace.
oc get deployment/redis-enterprise-operator
You should see a result similar to this:
NAME READY UP-TO-DATE AVAILABLE AGE
redis-enterprise-operator 1/1 1 1 0m36s
{{< warning >}} We recommend upgrading the REC as soon as possible after updating the operator. After the operator upgrade completes, the operator suspends the management of the REC and its associated REDBs, until the REC upgrade completes. {{< /warning >}}
Upgrades to versions 7.22.0-6 and later run in unprivileged mode without any additional permissions or capabilities. If you don't specifally require additional capabilities, we recommend you maintain the default unprivileged mode, as its more secure. After upgrading, remove the existing redis-enterprise-scc-v2 SCC and unbind it from the REC service account.
To enable automatic resource adjustment, see [Allow automatic resource adjustment > OpenShift upgrades]({{< relref "/operate/kubernetes/security/allow-resource-adjustment#openshift-upgrades" >}}).
{{<warning>}} Verify your license is valid before upgrading. Invalid licenses will cause the upgrade to fail.
Use oc get rec and verify the LICENSE STATE is valid on your REC before you start the upgrade process.
{{</warning>}}
The Redis Enterprise cluster (REC) can be updated automatically or manually. To trigger automatic upgrade of the REC after the operator upgrade completes, specify autoUpgradeRedisEnterprise: true in your REC spec. If you don't have automatic upgrade enabled, follow the below steps for the manual upgrade.
Before beginning the upgrade of the Redis Enterprise cluster, check the K8s operator [release notes]({{<relref "/operate/kubernetes/release-notes">}}) to find the Redis Enterprise image tag.
After the operator upgrade is complete, you can upgrade Redis Enterprise cluster (REC).
redisEnterpriseImageSpecEdit the REC custom resource YAML file.
oc edit rec <your-rec.yaml>
Replace the versionTag: declaration under redisEnterpriseImageSpec with the new version tag.
spec:
redisEnterpriseImageSpec:
imagePullPolicy: IfNotPresent
repository: redislabs/redis
versionTag: <new-version-tag>
Define any user-defined modules used by databases in the cluster.
spec:
userDefinedModules:
- name: "custom-module"
source:
https:
url: "https://modules.company.com/search-v2.1.zip"
credentialsSecret: "module-repo-creds"
The name field must match the display_name or module_name that appears in the module manifest (for example, "redisgears"). This enables the operator to run validation on the user-defined module. If these names don't match, the operator can't run validation on the user-defined module and preventable errors may occur.
{{< note >}}
Adding or modifying the userDefinedModules list triggers a rolling restart of the Redis Enterprise cluster pods in addition to the rolling upgrade for the version change.
{{< /note >}}
If your operator is monitoring multiple namespaces, you'll need to [reapply your role and role bindings]({{< relref "/operate/kubernetes/re-clusters/multi-namespace#create-role-and-role-binding-for-managed-namespaces" >}}) for each managed namespace. See [Manage databases in multiple namespaces]({{< relref "/operate/kubernetes/re-clusters/multi-namespace" >}}) for more details.
You can view the state of the REC with oc get rec.
During the upgrade, the state should be Upgrade.
When the upgrade is complete and the cluster is ready to use, the state will change to Running.
If the state is InvalidUpgrade, there is an error (usually relating to configuration) in the upgrade.
$ oc get rec
NAME NODES VERSION STATE SPEC STATUS LICENSE STATE SHARDS LIMIT LICENSE EXPIRATION DATE AGE
rec 3 6.2.10-107 Upgrade Valid Valid 4 2022-07-16T13:59:00Z 92m
To see the status of the current rolling upgrade, run:
oc rollout status sts <REC_name>
After the cluster is upgraded, you can upgrade your databases.
To upgrade your REDB, specify your new database version in the spec.redisVersion field in the REDB or REAADB custom resources. Supported database versions for operator versions include "7.2", "7.4", "8.0", and "8.2" (note this value is a string).
For Active-Active databases, the redis.Version change only needs to be applied on one participating cluster and will automatically propagate to all other participating clusters. All participating clusters must be running operator version 8.0.2-2 or later.
If your REAADB uses supported modules, keep the existing moduleList version numbers unchanged when upgrading redisVersion. The database will automatically use the module versions that are bundled with the new Redis version, regardless of what versions are specified in moduleList. After the upgrade is complete, you can optionally change the old version numbers from moduleList, but this change has no functional impact.
If a user-defined module is used by any database in the cluster, the module must be defined in the REC custom resource before upgrading the database. See [Add user-defined modules to the REC]({{< relref "/operate/kubernetes/re-databases/modules#add-user-defined-modules-to-the-rec" >}}) for detailed instructions.
Note that if your cluster [redisUpgradePolicy]({{<relref "/operate/kubernetes/reference/api/redis_enterprise_cluster_api#redisupgradepolicy" >}}) or your database [redisVersion]({{< relref "/operate/kubernetes/reference/api/redis_enterprise_database_api#redisversion" >}}) are set to major, you won't be able to upgrade those databases to minor versions. See [Redis upgrade policy]({{< relref "/operate/rs/installing-upgrading/upgrading#redis-upgrade-policy" >}}) for more details.
If you start an upgrade without meeting the prerequisites, the operator will freeze the upgrade. Check the operator logs for the source of the error. The REDB reconsilliation doesn't work during an upgrade, so you need to apply a manual fix with the Redis Software API (examples below). The updates will also need to be added to the REDB custom resource.
If the operator logs show an event related to an unsupported module, download the updated module locally, and install it using the v2/modules API endpoint.
curl -sfk -u <rec_username>:<rec_password> -X POST -F 'module=@<full path to your module>' https://localhost:9443/v2/modules
After updating the modules with the Redis Software API, update the REDB custom resource to reflect the change.
If the operator logs show an event related to an incompatible database version, upgrade the database using the Redis Software API.
curl -sfk -u <rec_username>:<rec_password> -X POST -H "Content-Type: application/json" -d '{"redis_version": <target redis version>}' https://localhost:9443/v1/bdbs/<BDB UID>/upgrade
After updating the database with the Redis Software API, update the REDB custom resource to reflect the change.