Substrate/Polkadot node Helm chart

Overview

The Polkadot Helm Chart provides a convenient way to deploy and manage a Polkadot blockchain node in a Kubernetes cluster. This chart is designed to be highly configurable, supporting various configurations and features of the Polkadot node, including persistent storage, resource management, and custom networking.

Features:

• Compatible with all Substrate-based relaychains, including Polkadot, Kusama, Paseo, Westend, Rococo, and more.
• Compatible with all Substrate-based parachains, including Asset-hub, Bridge-hub, Coretime, People, Acala, Astar, Moonbase, and others.
• Deploy RPC, collators, validators, or full nodes in a Kubernetes cluster.
• Use snapshots to speed up the deployment process.
• Supports session key and node-key (ID) injection.

Maintainers

Installing the chart

helm repo add parity https://paritytech.github.io/helm-charts/
helm install polkadot-node parity/node

This will deploy a single Polkadot node with the default configuration.

Public snapshots

You can use the following public URLs to download chain snapshots:

• https://snapshots.polkadot.io/polkadot-paritydb-prune
• https://snapshots.polkadot.io/polkadot-rocksdb-prune
• https://snapshots.polkadot.io/polkadot-rocksdb-archive
• https://snapshots.polkadot.io/kusama-paritydb-prune
• https://snapshots.polkadot.io/kusama-rocksdb-prune
• https://snapshots.polkadot.io/kusama-rocksdb-archive
• https://snapshots.polkadot.io/westend-paritydb-archive
• https://snapshots.polkadot.io/westend-paritydb-prune
• https://snapshots.polkadot.io/westend-rocksdb-prune
• https://snapshots.polkadot.io/westend-rocksdb-archive
• https://snapshots.polkadot.io/westend-collectives-rocksdb-archive

For example, to restore Polkadot pruned snapshot running ParityDB, configure chart values like the following:

node:
  chain: polkadot
  role: full
  chainData:
    chainSnapshot:
      enabled: true
      method: http-filelist
      url: https://snapshots.polkadot.io/polkadot-paritydb-prune
    pruning: 256

Polkadot and Kusama backups are pruned at 256 blocks. Westend backups are pruned at 1000 blocks.

Resizing the node disk

To resize the node persistent volume, perform the following steps:

1. Patch the PVC storage size, eg. to 1000Gi:

kubectl patch pvc chain-data-polkadot-node-0  -p '{"spec":{"resources":{"requests":{"storage":"1000Gi"}}}}}'

2. Delete the StatefulSet object with cascade=orphan (ie. without removing the attached pods):

kubectl delete sts polkadot-node --cascade=orphan

3. Update the node.chainData.volumeSize to the new value (eg. 1000Gi) and upgrade the helm release.

Note that for a Kubernetes Persistent Volume Claims to be resizable, its StorageClass must have specific characteristics. More information on this topic is available in the Expanding Persistent Volumes Claims section of the Kubernetes documentation.

Optional Vault Integration

To integrate this chart with vault:

• Vault agent injector installed on the cluster
• Kubernetes auth enabled in your vault instance
• Secrets for either the keys or the nodeKey created in your vault using kv-2
• A policy for each of your secrets configured in your vault
• an authentication role crated in your vault instance (should match the serviceAccount name for this chart) with policies for accessing your keys attached

node:
  vault:
    keys:
      - name: aura
        type: aura
        scheme: sr25519
        vaultPath: kv/secret/polkadot-node # path at which the secret is located in Vault
        vaultKey: aura # key under which the secret value is stored in Vault
        extraDerivation: "//${HOSTNAME}//aura" # allows to have unique derived keys for each pod of the statefulset
    nodeKey:
      name: nodekey
      vaultPath: kv/secret/polkadot-node
      vaultKey: nodekey

Setting Up Node Key for Bootnodes and Validators

For both bootnodes and validators (refer to paritytech/polkadot-sdk#3852), it is necessary to set up a network key.

Steps to Set Up a Node Key

1. Generate a Custom Node Key

   You can generate a custom node key using the following command:

   shCopy

   polkadot key generate-node-key
   

2. Add the Generated Node Key

   To add the generated node key, use the following configuration:

   yamlCopy

   node:
     customNodeKey: "<your-generated-node-key>"
   

3. Point to an Existing Node Key K8s Secret

   If you have an existing Kubernetes secret for the node key, point to it using:

   yamlCopy

   node:
     existingSecrets:
       nodeKey: "<your-existing-node-key-secret>"
   

4. Retrieve Node Key from vault

   see Optional Vault Integration

5. Automatically Generate and Persist Node Key

   Alternatively, you can set the following to automatically generate a node key on startup and store it to the volume:

   yamlCopy

   node:
     persistGeneratedNodeKey: true
   

Upgrade

From v5.5.x to v5.5.2

• Fix Bug from v5.5.0: --pruning is alias for --state-pruning not --blocks-pruning.

From v5.x.x to v5.5.0 (⚠️ breaking changes)

• The pruning flag is now using --blocks-pruning which starts from polkadot version v0.9.28
• The flag --pruning is now allowed in both .Values.node.flags and .Values.node.collatorRelayChain.flags. When using --pruning, ensure that the values of .Values.node.chainData.pruning and .Values.node.collatorRelayChain.chainData.pruning are explicitly set to false to maintain previous behavior.

From v5.x.x to v5.3.0 (⚠️ breaking changes)

• The following flags have changed:
  • externalRelayChain.* -> replaced with collatorExternalRelayChain.* to match to new naming convention of different modes;

From v4.x.x to v5.0.0 (⚠️ breaking changes)

• Chain backup upload functionality has been removed. I.e., the node.enableChainBackupGcs flag is no longer available. Backup upload was implemented in the form of init container. Since backup init container starts before the main container runs, the node does not have a chance to sync to the latest block. Instead, backup container syncs the DB chunks from the time the node was last online which most of the times would be a stale data. Additionally, after backup is completed the node will continue to run which is not always necessary as you probably just wanted to make a backup and exit the script. A more complete solution for making node backups will be published in the future releases of the chart;
• Chain backup download scripts have been updated to use rclone. Multiple flags associated with this functionality have changed. Chain backup and relay chain backup restoration are now controlled by node.chainData.chainSnapshot.* and node.collatorRelayChain.chainData.chainSnapshot.* blocks of flags accordingly.
• Chain backup restoration now supports a new method: downloading DB files by direct HTTP links using a list of files as a reference. I.e., a restoration process would first download a file containing a list of DB files that need to be downloaded. rclone will then use this file to generate HTTP links to the DB files and download it in parallel.
• The following flags have changed:
  • initContainer.* -> replaced with initContainers.* to enable individual configuration of each init container;
  • kubectl.* -> merged into initContainers.*;
  • googleCloudSdk.* -> replaced with node.chainData.chainSnapshot.* and node.collatorRelayChain.chainData.chainSnapshot.*
  • node.chainData.snapshotUrl -> replaced with node.chainData.chainSnapshot.url
  • node.chainData.snapshotFormat -> replaced with node.chainData.chainSnapshot.method
  • node.chainData.GCSBucketUrl -> replaced with node.chainData.chainSnapshot.url
  • node.collatorRelayChain.chainData.snapshotUrl -> replaced with node.collatorRelayChain.chainData.chainSnapshot.url
  • node.collatorRelayChain.chainData.snapshotFormat -> replaced with node.collatorRelayChain.chainData.chainSnapshot.method
  • node.collatorRelayChain.chainData.GCSBucketUrl -> replaced with node.collatorRelayChain.chainData.chainSnapshot.url

v4.6.0 (⚠️ breaking change)

Substrate changed the default rpc flags: https://github.com/paritytech/substrate/pull/13384
The dual RPC ports; --rpc-port=9933 (HTTP) ,--ws-port=9944 (WS) was replaced by a combined port --rpc-port=9944. Flags replaced:

--rpc-max--payload (replaced by --rpc--max-request-size and --rpc-max-response-size)
--ws-max-out-buffer-capacity (removed)
--ws-external (replaced by --rpc-external)
--unsafe-ws--external (replaced by --unsafe-rpc-external)
--ipc-path (removed)
--ws-port (replaced by --rpc-port)
--ws-max-connections (replaced by --rpc-max-connections)
--rpc-http (replaced by --rpc-addr)
--rpc-ws (replaced by --rpc-addr)

New value was added to support this change:

• node.legacyRpcFlags

If your node is still using the old RPC flags, please set node.legacyRpcFlags=true

v4.5.0 (⚠️ small change)

The storage classes are now set to "" by default instead of "default". Make sure that the following values are set to the storage classes you are using if not already set (before 4.5.0, those were set explicitly to default) :

• node.chainData.storageClass
• node.chainKeystore.storageClass
• node.collatorRelayChain.chainData.storageClass
• node.collatorRelayChain.chainKeystore.storageClass

From v3.x.x to v4.0.0 (⚠️ breaking changes)

The following chart parameters have been renamed or rearranged:

• node.pruning -> node.chainData.pruning
• node.database -> node.chainData.database
• node.collator.isParachain -> node.isParachain
• node.collator.relayChain -> node.collatorRelayChain.chain
• node.collator.relayChainCustomChainspecPath -> node.collatorRelayChain.customChainspecPath
• node.collator.relayChainCustomChainspecUrl -> node.collatorRelayChain.customChainspecUrl
• node.collator.relayChainFlags -> node.collatorRelayChain.flags
• node.collator.relayChainData.* -> node.collatorRelayChain.chainData.*
• node.collator.relayChainPruning -> node.collatorRelayChain.chainData.pruning
• node.collator.relayChainDatabase -> node.collatorRelayChain.chainData.database
• node.collator.relayChainKeystore.* -> node.collatorRelayChain.chainKeystore.*
• node.collator.relayChainPrometheus.* -> node.collatorRelayChain.prometheus.*

The following flags are now invalid if they were previously set in node.flags or node.collator.relayChainFlags. An error will be thrown if any of those flags are set directly.

• --name
• --base-path
• --chain
• --validator
• --collator
• --light
• --database
• --pruning
• --prometheus-external
• --prometheus-port
• --node-key
• --wasm-runtime-overrides
• --jaeger-agent
• --rpc-methods
• --rpc-external
• --unsafe-rpc-external
• --ws-external
• --unsafe-ws-external
• --rpc-cors
• --rpc-port
• --ws-port

From v2.x.x to v3.0.0 (⚠️ breaking changes)

There are now separate volumes for:

• relaychain data
• relaychain keystore
• parachain data
• parachain keystore

Some chart parameters have been grouped together and renamed. There are now separate sections for the following values:

• node.chainData
• node.chainKeystore
• node.collator

Common storageClass parameter has been moved to the corresponding separate groups mentioned above.

As both the chain data and keystore can now be stored on up to 4 different volumes you may need to manually relocate the existing data to the newly created volumes.

If you're running a non-collator node:

• Move chain files from the /data/chains/<chain_name> in the chain-data volume to /chain-data in the chain-data volume.
• Move keystore files from the /data/chains/<chain_name>/keystore in the chain-data volume to /keystore in the chain-keystore volume.

If you're running a collator node:

• Move chain files from the /data/chains/<chain_name> in the chain-data volume to /chain-data in the chain-data volume.
• Move keystore files from the /data/chains/<chain_name>/keystore in the chain-data volume to /keystore in the chain-keystore volume.
• Move relaychain files from the /data/relay/polkadot in the chain-data volume to /relaychain-data/polkadot in the relaychain-data volume.
• Move relaychain keystore from /data/relay/polkadot/chains/<relay_chain_name>/keystore in the chain-data volume to /relaychain-keystore in the relaychain-keystore volume

Values

Autogenerated from chart metadata using helm-docs v1.12.0