ContextQMD
Back to Pulsar

PIP-135: Include MetadataStore backend for Etcd

pip/pip-135.md

4.2.1954 B
Original Source

PIP-135: Include MetadataStore backend for Etcd

Since all the pieces that composed the proposal in PIP-45 were finally merged and are currently ready for 2.10 release, it is now possible to add other metadata backends that can be used to support a BookKeeper + Pulsar cluster.

One of the popular systems that is most commonly used in alternative of ZooKeeper is Etcd, thus it makes sense to have this as the first non-zookeeper implementation.

Goal

Provide a Etcd implementation for the MetadataStore API. This will allow users to deploy Pulsar clusters using Etcd service for the metadata and it will not require the presence of ZooKeeper.

Implementation

  • Use the existing JEtcd Java client library for Etcd
  • Extends the AbstractBatchedMetadataStore class, in order to reuse the transparent batching logic that will be shared with the ZookKeeper implementation.

Work in progress: https://github.com/apache/pulsar/pull/13225