Elasticsearch Helm Chart

This chart uses a standard Docker image of Elasticsearch (docker.elastic.co/elasticsearch/elasticsearch-oss) and uses a service pointing to the master's transport port for service discovery. Elasticsearch does not communicate with the Kubernetes API, hence no need for RBAC permissions.

This Helm chart is deprecated

As mentioned in #10543 this chart has been deprecated in favour of the official Elastic Helm Chart. We have made steps towards that goal by producing a migration guide to help people switch the management of their clusters over to the new Charts. The Elastic Helm Chart supports version 6 and 7 of Elasticsearch and it was decided it would be easier for people to upgrade after migrating to the Elastic Helm Chart because it's upgrade process works better. During deprecation process we want to make sure that Chart will do what people are using this chart to do. Please look at the Elastic Helm Charts and if you see anything missing from please open an issue to let us know what you need. The Elastic Chart repo is also in Helm Hub.

Warning for previous users

If you are currently using an earlier version of this Chart you will need to redeploy your Elasticsearch clusters. The discovery method used here is incompatible with using RBAC. If you are upgrading to Elasticsearch 6 from the 5.5 version used in this chart before, please note that your cluster needs to do a full cluster restart. The simplest way to do that is to delete the installation (keep the PVs) and install this chart again with the new version. If you want to avoid doing that upgrade to Elasticsearch 5.6 first before moving on to Elasticsearch 6.0.

Prerequisites Details

• Kubernetes 1.10+
• PV dynamic provisioning support on the underlying infrastructure

StatefulSets Details

• https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/

StatefulSets Caveats

• https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#limitations

Todo

• Implement TLS/Auth/Security
• Smarter upscaling/downscaling
• Solution for memory locking

Chart Details

This chart will do the following:

• Implemented a dynamically scalable elasticsearch cluster using Kubernetes StatefulSets/Deployments
• Multi-role deployment: master, client (coordinating) and data nodes
• Statefulset Supports scaling down without degrading the cluster

Installing the Chart

To install the chart with the release name my-release:

$ helm install --name my-release stable/elasticsearch

Deleting the Charts

Delete the Helm deployment as normal

$ helm delete my-release

Deletion of the StatefulSet doesn't cascade to deleting associated PVCs. To delete them:

$ kubectl delete pvc -l release=my-release,component=data

Configuration

The following table lists the configurable parameters of the elasticsearch chart and their default values.

Specify each parameter using the --set key=value[,key=value] argument to helm install.

In terms of Memory resources you should make sure that you follow that equation:

• ${role}HeapSize < ${role}MemoryRequests < ${role}MemoryLimits

The YAML value of cluster.config is appended to elasticsearch.yml file for additional customization ("script.inline: on" for example to allow inline scripting)

Deep dive

Application Version

This chart aims to support Elasticsearch v2 to v6 deployments by specifying the values.yaml parameter appVersion.

Version Specific Features

• Memory Locking (variable renamed)
• Ingest Node (v5)
• X-Pack Plugin (v5)

Upgrade paths & more info: https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html

Mlocking

This is a limitation in kubernetes right now. There is no way to raise the limits of lockable memory, so that these memory areas won't be swapped. This would degrade performance heavily. The issue is tracked in kubernetes/#3595.

[WARN ][bootstrap] Unable to lock JVM Memory: error=12,reason=Cannot allocate memory
[WARN ][bootstrap] This can result in part of the JVM being swapped out.
[WARN ][bootstrap] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536

Minimum Master Nodes

The minimum_master_nodes setting is extremely important to the stability of your cluster. This setting helps prevent split brains, the existence of two masters in a single cluster.

When you have a split brain, your cluster is at danger of losing data. Because the master is considered the supreme ruler of the cluster, it decides when new indices can be created, how shards are moved, and so forth. If you have two masters, data integrity becomes perilous, since you have two nodes that think they are in charge.

This setting tells Elasticsearch to not elect a master unless there are enough master-eligible nodes available. Only then will an election take place.

This setting should always be configured to a quorum (majority) of your master-eligible nodes. A quorum is (number of master-eligible nodes / 2) + 1

More info: https://www.elastic.co/guide/en/elasticsearch/guide/1.x/_important_configuration_changes.html#_minimum_master_nodes

Client and Coordinating Nodes

Elasticsearch v5 terminology has updated, and now refers to a Client Node as a Coordinating Node.

More info: https://www.elastic.co/guide/en/elasticsearch/reference/5.5/modules-node.html#coordinating-node

Enabling elasticsearch internal monitoring

Requires version 6.3+ and standard non oss repository defined. Starting with 6.3 Xpack is partially free and enabled by default. You need to set a new config to enable the collection of these internal metrics. (https://www.elastic.co/guide/en/elasticsearch/reference/6.3/monitoring-settings.html)

To do this through this helm chart override with the three following changes:

image.repository: docker.elastic.co/elasticsearch/elasticsearch
cluster.xpackEnable: true
cluster.env.XPACK_MONITORING_ENABLED: true

Note: to see these changes you will need to update your kibana repo to image.repository: docker.elastic.co/kibana/kibana instead of the oss version

Select right storage class for SSD volumes

GCE + Kubernetes 1.5

Create StorageClass for SSD-PD

$ kubectl create -f - <<EOF
kind: StorageClass
apiVersion: extensions/v1beta1
metadata:
  name: ssd
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-ssd
EOF

Create cluster with Storage class ssd on Kubernetes 1.5+

$ helm install stable/elasticsearch --name my-release --set data.persistence.storageClass=ssd,data.storage=100Gi

Usage of the tpl Function

The tpl function allows us to pass string values from values.yaml through the templating engine. It is used for the following values:

• extraInitContainers

It is important that these values be configured as strings. Otherwise, installation will fail.