@kbn/es

A command line utility for running elasticsearch from snapshot, source, archive, docker, serverless or even building snapshot artifacts.

Getting started

To run, go to the Kibana root and run node scripts/es --help to get the latest command line options.

The script attempts to preserve the existing interfaces used by Elasticsearch CLI. This includes passing through options with the -E argument and the ES_JAVA_OPTS environment variable for Java options.

If running elasticsearch from source, elasticsearch needs to be cloned to a sibling directory of Kibana.

Serverless & Docker Prerequisites

If running elasticsearch serverless or a docker container, there is some required initial setup:

1. Install Docker. Instructions can be found here.
2. Authentication with Elastic's Docker registry here.
3. Increase OS virtual memory limits. More info in the ES docs.

Examples

Run a snapshot install with a trial license

node scripts/es snapshot --license=trial

Run from source with a configured data directory

node scripts/es source --Epath.data=/home/me/es_data

Run serverless with a specific image tag

node scripts/es serverless --tag git-fec36430fba2-x86_64

Run an official Docker release

node scripts/es docker --tag 8.8.2

API

run

Start a cluster

var es = require('@kbn/es');
es.run({
  license: 'basic',
  version: 7.0,
})
.catch(function (e) {
  console.error(e);
  process.exitCode = 1;
});

Options

options.license

Type: String

License type, one of: basic, trial

options.version

Type: String

Desired elasticsearch version

options['source-path']

Type: String

Cloned location of elasticsearch repository, used when running from source

options['base-path']

Type: String

Location where snapshots are cached

Snapshot Updates

Snapshots are built and tested daily. If tests pass, the snapshot is promoted and will automatically be used when started from the CLI.

CI pipelines supporting this can be found at:

• https://buildkite.com/elastic/kibana-elasticsearch-snapshot-build
• https://buildkite.com/elastic/kibana-elasticsearch-snapshot-verify
• https://buildkite.com/elastic/kibana-elasticsearch-snapshot-promote

Snapshot Pinning

Sometimes we need to pin snapshots for a specific version. We'd really like to get this automated, but until that is completed here are the steps to take to build, upload, and switch to pinned snapshots for a branch.

To use these steps you'll need to setup the google-cloud-sdk, which can be installed on macOS with brew cask install google-cloud-sdk. Login with the CLI and you'll have access to the gsutil to do efficient/parallel uploads to GCS from the command line.

1. Clone the elasticsearch repo somewhere

2. Checkout the branch you want to build

3. Build the new artifacts

   Copy

   node scripts/es build_snapshots --output=~/Downloads/tmp-artifacts --source-path=/path/to/es/repo
   

4. Check that the files in ~/Downloads/tmp-artifacts look reasonable

5. Upload the files to GCS

   Copy

   gsutil -m rsync . gs://kibana-ci-tmp-artifacts/
   

6. Once the artifacts are uploaded, modify src/platform/packages/shared/kbn-es/src/custom_snapshots.js in a PR to use a URL formatted like:

   Copy

   // force use of manually created snapshots until ReindexPutMappings fix
   if (!process.env.KBN_ES_SNAPSHOT_URL && !process.argv.some(isVersionFlag)) {
     // return undefined;
     return 'https://storage.googleapis.com/kibana-ci-tmp-artifacts/{name}-{version}-{os}-x86_64.{ext}';
   }
   

   For 6.8, the format of the url should look like:

   Copy

   'https://storage.googleapis.com/kibana-ci-tmp-artifacts/{name}-{version}.{ext}';