Back to Charts

⚠️ Repo Archive Notice

stable/redis-ha/README.md

latest37.3 KB
Original Source

⚠️ Repo Archive Notice

As of Nov 13, 2020, charts in this repo will no longer be updated. For more information, see the Helm Charts Deprecation and Archive Notice, and Update.

Redis


Deprecation Warning

As part of the deprecation timeline. We will move this to an official repository here

Please make PRs / Issues here from now on

We will keep the changes in sync as best we can, but we will be notifying people to submit PRs here from now on instead. If you have any questions, feel free to get in touch with either of the maintainers.

Redis is an advanced key-value cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps and hyperloglogs.

TL;DR;

bash
$ helm install stable/redis-ha

By default this chart install 3 pods total:

  • one pod containing a redis master and sentinel container (optional prometheus metrics exporter sidecar available)
  • two pods each containing a redis slave and sentinel containers (optional prometheus metrics exporter sidecars available)

Introduction

This chart bootstraps a Redis highly available master/slave statefulset in a Kubernetes cluster using the Helm package manager.

Prerequisites

  • Kubernetes 1.8+ with Beta APIs enabled
  • PV provisioner support in the underlying infrastructure

Upgrading the Chart

Please note that there have been a number of changes simplifying the redis management strategy (for better failover and elections) in the 3.x version of this chart. These changes allow the use of official redis images that do not require special RBAC or ServiceAccount roles. As a result when upgrading from version >=2.0.1 to >=3.0.0 of this chart, Role, RoleBinding, and ServiceAccount resources should be deleted manually.

Upgrading the chart from 3.x to 4.x

Starting from version 4.x HAProxy sidecar prometheus-exporter removed and replaced by the embedded HAProxy metrics endpoint, as a result when upgrading from version 3.x to 4.x section haproxy.exporter should be removed and the haproxy.metrics need to be configured for fit your needs.

Installing the Chart

To install the chart

bash
$ helm install stable/redis-ha

The command deploys Redis on the Kubernetes cluster in the default configuration. By default this chart install one master pod containing redis master container and sentinel container along with 2 redis slave pods each containing their own sentinel sidecars. The configuration section lists the parameters that can be configured during installation.

Tip: List all releases using helm list

Uninstalling the Chart

To uninstall/delete the deployment:

bash
$ helm delete <chart-name>

The command removes all the Kubernetes components associated with the chart and deletes the release.

Configuration

The following table lists the configurable parameters of the Redis chart and their default values.

ParameterDescriptionDefault
imageRedis imageredis
imagePullSecretsReference to one or more secrets to be used when pulling redis images[]
tagRedis tag5.0.6-alpine
replicasNumber of redis master/slave pods3
serviceAccount.createSpecifies whether a ServiceAccount should be createdtrue
serviceAccount.nameThe name of the ServiceAccount to createGenerated using the redis-ha.fullname template
rbac.createCreate and use RBAC resourcestrue
redis.portPort to access the redis service6379
redis.masterGroupNameRedis convention for naming the cluster group: must match ^[\\w-\\.]+$ and can be templatedmymaster
redis.configAny valid redis config options in this section will be applied to each server (see below)see values.yaml
redis.customConfigAllows for custom redis.conf files to be applied. If this is used then redis.config is ignored``
redis.resourcesCPU/Memory for master/slave nodes resource requests/limits{}
sentinel.portPort to access the sentinel service26379
sentinel.quorumMinimum number of servers necessary to maintain quorum2
sentinel.configValid sentinel config options in this section will be applied as config options to each sentinel (see below)see values.yaml
sentinel.customConfigAllows for custom sentinel.conf files to be applied. If this is used then sentinel.config is ignored``
sentinel.resourcesCPU/Memory for sentinel node resource requests/limits{}
init.resourcesCPU/Memory for init Container node resource requests/limits{}
authEnables or disables redis AUTH (Requires redisPassword to be set)false
redisPasswordA password that configures a requirepass and masterauth in the conf parameters (Requires auth: enabled)``
authKeyThe key holding the redis password in an existing secret.auth
existingSecretAn existing secret containing a key defined by authKey that configures requirepass and masterauth in the conf parameters (Requires auth: enabled, cannot be used in conjunction with .Values.redisPassword)``
nodeSelectorNode labels for pod assignment{}
tolerationsToleration labels for pod assignment[]
hardAntiAffinityWhether the Redis server pods should be forced to run on separate nodes.true
additionalAffinitiesAdditional affinities to add to the Redis server pods.{}
securityContextSecurity context to be added to the Redis server pods.{runAsUser: 1000, fsGroup: 1000, runAsNonRoot: true}
affinityOverride all other affinity settings with a string.""
persistentVolume.sizeSize for the volume10Gi
persistentVolume.annotationsAnnotations for the volume{}
persistentVolume.reclaimPolicyMethod used to reclaim an obsoleted volume. Delete or Retain""
emptyDirConfiguration of emptyDir, used only if persistentVolume is disabled and no hostPath specified{}
exporter.enabledIf true, the prometheus exporter sidecar is enabledfalse
exporter.imageExporter imageoliver006/redis_exporter
exporter.tagExporter tagv0.31.0
exporter.portExporter port9121
exporter.annotationsPrometheus scrape annotations{prometheus.io/path: /metrics, prometheus.io/port: "9121", prometheus.io/scrape: "true"}
exporter.extraArgsAdditional args for the exporter{}
exporter.scriptA custom custom Lua script that will be mounted to exporter for collection of custom metrics. Creates a ConfigMap and sets env var REDIS_EXPORTER_SCRIPT.
exporter.serviceMonitor.enabledUse servicemonitor from prometheus operatorfalse
exporter.serviceMonitor.namespaceNamespace the service monitor is created indefault
exporter.serviceMonitor.intervalScrape interval, If not set, the Prometheus default scrape interval is usednil
exporter.serviceMonitor.telemetryPathPath to redis-exporter telemetry-path/metrics
exporter.serviceMonitor.labelsLabels for the servicemonitor passed to Prometheus Operator{}
exporter.serviceMonitor.timeoutHow long until a scrape request times out. If not set, the Prometheus default scape timeout is usednil
haproxy.enabledEnabled HAProxy LoadBalancing/Proxyfalse
haproxy.replicasNumber of HAProxy instances3
haproxy.image.repositoryHAProxy Image Repositoryhaproxy
haproxy.image.tagHAProxy Image Tag2.0.1
haproxy.image.pullPolicyHAProxy Image PullPolicyIfNotPresent
haproxy.imagePullSecretsReference to one or more secrets to be used when pulling haproxy images[]
haproxy.annotationsHAProxy template annotations{}
haproxy.customConfigAllows for custom config-haproxy.cfg file to be applied. If this is used then default config will be overwriten``
haproxy.extraConfigAllows to place any additional configuration section to add to the default config-haproxy.cfg``
haproxy.resourcesHAProxy resources{}
haproxy.emptyDirConfiguration of emptyDir{}
haproxy.service.typeHAProxy service type "ClusterIP", "LoadBalancer" or "NodePort"ClusterIP
haproxy.service.nodePortHAProxy service nodePort value (haproxy.service.type must be NodePort)not set
haproxy.service.annotationsHAProxy service annotations{}
haproxy.stickyBalancingHAProxy sticky load balancing to Redis nodes. Helps with connections shutdown.false
haproxy.hapreadport.enableEnable a read only port for redis slavesfalse
haproxy.hapreadport.portHaproxy port for read only redis slaves6380
haproxy.metrics.enabledHAProxy enable prometheus metric scrapingfalse
haproxy.metrics.portHAProxy prometheus metrics scraping port9101
haproxy.metrics.portNameHAProxy metrics scraping port nameexporter-port
haproxy.metrics.scrapePathHAProxy prometheus metrics scraping port/metrics
haproxy.metrics.serviceMonitor.enabledUse servicemonitor from prometheus operator for HAProxy metricsfalse
haproxy.metrics.serviceMonitor.namespaceNamespace the service monitor for HAProxy metrics is created indefault
haproxy.metrics.serviceMonitor.intervalScrape interval, If not set, the Prometheus default scrape interval is usednil
haproxy.metrics.serviceMonitor.telemetryPathPath to HAProxy metrics telemetry-path/metrics
haproxy.metrics.serviceMonitor.labelsLabels for the HAProxy metrics servicemonitor passed to Prometheus Operator{}
haproxy.metrics.serviceMonitor.timeoutHow long until a scrape request times out. If not set, the Prometheus default scape timeout is usednil
haproxy.init.resourcesExtra init resources{}
haproxy.timeout.connecthaproxy.cfg timeout connect setting4s
haproxy.timeout.serverhaproxy.cfg timeout server setting30s
haproxy.timeout.clienthaproxy.cfg timeout client setting30s
haproxy.timeout.checkhaproxy.cfg timeout check setting2s
haproxy.priorityClassNamepriorityClassName for haproxy deploymentnot set
haproxy.securityContextSecurity context to be added to the HAProxy deployment.{runAsUser: 1000, fsGroup: 1000, runAsNonRoot: true}
haproxy.hardAntiAffinityWhether the haproxy pods should be forced to run on separate nodes.true
haproxy.affinityOverride all other haproxy affinity settings with a string.""
haproxy.additionalAffinitiesAdditional affinities to add to the haproxy server pods.{}
podDisruptionBudgetPod Disruption Budget rules{}
priorityClassNamepriorityClassName for redis-ha-statefulsetnot set
hostPath.pathUse this path on the host for data storagenot set
hostPath.chownRun an init-container as root to set ownership on the hostPathtrue
sysctlImage.enabledEnable an init container to modify Kernel settingsfalse
sysctlImage.commandsysctlImage command to execute[]
sysctlImage.registrysysctlImage Init container registrydocker.io
sysctlImage.repositorysysctlImage Init container namebusybox
sysctlImage.tagsysctlImage Init container tag1.31.1
sysctlImage.pullPolicysysctlImage Init container pull policyAlways
sysctlImage.mountHostSysMount the host /sys folder to /host-sysfalse
sysctlImage.resourcessysctlImage resources{}
schedulerNameAlternate scheduler namenil

Specify each parameter using the --set key=value[,key=value] argument to helm install. For example,

bash
$ helm install \
  --set image=redis \
  --set tag=5.0.5-alpine \
    stable/redis-ha

The above command sets the Redis server within default namespace.

Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,

bash
$ helm install -f values.yaml stable/redis-ha

Tip: You can use the default values.yaml

Custom Redis and Sentinel config options

This chart allows for most redis or sentinel config options to be passed as a key value pair through the values.yaml under redis.config and sentinel.config. See links below for all available options.

Example redis.conf Example sentinel.conf

For example repl-timeout 60 would be added to the redis.config section of the values.yaml as:

yml
    repl-timeout: "60"

Note:

  1. Some config options should be renamed by redis version,e.g.:

    # In redis 5.x,see https://raw.githubusercontent.com/antirez/redis/5.0/redis.conf
    min-replicas-to-write: 1
    min-replicas-max-lag: 5
    
    # In redis 4.x and redis 3.x,see https://raw.githubusercontent.com/antirez/redis/4.0/redis.conf and https://raw.githubusercontent.com/antirez/redis/3.0/redis.conf
    min-slaves-to-write 1
    min-slaves-max-lag 5
    

Sentinel options supported must be in the the sentinel <option> <master-group-name> <value> format. For example, sentinel down-after-milliseconds 30000 would be added to the sentinel.config section of the values.yaml as:

yml
    down-after-milliseconds: 30000

If more control is needed from either the redis or sentinel config then an entire config can be defined under redis.customConfig or sentinel.customConfig. Please note that these values will override any configuration options under their respective section. For example, if you define sentinel.customConfig then the sentinel.config is ignored.

Host Kernel Settings

Redis may require some changes in the kernel of the host machine to work as expected, in particular increasing the somaxconn value and disabling transparent huge pages. To do so, you can set up a privileged initContainer with the sysctlImage config values, for example:

sysctlImage:
  enabled: true
  mountHostSys: true
  command:
    - /bin/sh
    - -xc
    - |-
      sysctl -w net.core.somaxconn=10000
      echo never > /host-sys/kernel/mm/transparent_hugepage/enabled

HAProxy startup

When HAProxy is enabled, it will attempt to connect to each announce-service of each redis replica instance in its init container before starting. It will fail if announce-service IP is not available fast enough (10 seconds max by announce-service). A such case could happen if the orchestator is pending the nomination of redis pods. Risk is limited because announce-service is using publishNotReadyAddresses: true, although, in such case, HAProxy pod will be rescheduled afterward by the orchestrator.