pip/pip-462.md
Apache Pulsar supports multiple metadata store backends through the pluggable MetadataStore interface introduced in PIP-45. One of these backends is Etcd, implemented via the jetcd client library. While the Etcd backend has been available since the metadata store abstraction was introduced, it has seen very limited adoption in production environments.
Maintaining Etcd support imposes several costs on the project:
Dependency burden: The jetcd-core library pulls in a large transitive dependency tree (gRPC, Netty, Vert.x, and others) that conflicts with dependencies used elsewhere in Pulsar. This required creating a dedicated shading module (jetcd-core-shaded) with relocations for io.vertx and io.netty packages.
Distribution size: The shaded jetcd JAR and its transitive dependencies increase both the Pulsar tarball and Docker image size significantly, even for users who never use Etcd.
Library maintenance concerns: The jetcd library is not actively maintained, making it increasingly risky to depend on for a production metadata backend. Security patches and compatibility updates may lag behind.
Maintenance overhead: The EtcdMetadataStore implementation, EtcdSessionWatcher, and associated test infrastructure must be maintained, tested, and considered in every metadata store interface change.
Strategic direction: Both ZooKeeper and Oxia are fully supported and battle-tested metadata store backends. Oxia is a purpose-built metadata store designed for Pulsar with superior operational characteristics, and the community's future efforts are focused there. Maintaining a third, unused backend dilutes this focus.
Given that Etcd is not used in production deployments, adds significant distribution size, relies on a poorly maintained client library, and competes for attention with the strategically preferred Oxia backend, removing it simplifies the project and reduces the maintenance surface area.
EtcdMetadataStore and EtcdSessionWatcher implementation classes from pulsar-metadataEtcdMetadataStoreProvider registration from MetadataStoreFactoryImpljetcd-core-shaded shading module entirelyjetcd-core and jetcd-test dependency declarations from the root pom.xmlEtcdMetadataStoreTest)etcd: metadata store URL, guiding them to use ZooKeeper or Oxia insteadMetadataStore interface or plugin systemThe removal is a straightforward deletion of code and dependencies:
Implementation removal: Delete EtcdMetadataStore.java and EtcdSessionWatcher.java from pulsar-metadata.
Provider deregistration: Remove the Etcd provider from MetadataStoreFactoryImpl. Add a check that produces a clear error message when an etcd: URL is configured, informing operators that Etcd is no longer supported and recommending ZooKeeper or Oxia.
Module removal: Delete the jetcd-core-shaded module and remove it from the root pom.xml modules list.
Dependency cleanup: Remove jetcd-core, jetcd-test, and jetcd-core-shaded dependency declarations from the root pom.xml dependency management section. Remove these dependencies from pulsar-metadata/pom.xml.
Test removal: Delete EtcdMetadataStoreTest.java.
The following source files will be deleted:
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/EtcdMetadataStore.java (includes inner EtcdMetadataStoreProvider class)pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/EtcdSessionWatcher.javaIn MetadataStoreFactoryImpl, the Etcd provider registration will be removed from loadProviders(). A runtime check will be added so that if a user configures metadataStoreUrl=etcd:..., the broker fails fast with a clear error:
Etcd metadata store backend has been removed in Pulsar 5.0.
Please use ZooKeeper (zk:) or Oxia (oxia:) as your metadata store.
See PIP-462 for details.
The jetcd-core-shaded/ directory will be deleted entirely, and the <module>jetcd-core-shaded</module> entry will be removed from the root pom.xml.
The following entries will be removed from the root pom.xml:
From <dependencyManagement>:
io.etcd:jetcd-coreio.etcd:jetcd-testorg.apache.pulsar:jetcd-core-shadedFrom pulsar-metadata/pom.xml:
org.apache.pulsar:jetcd-core-shaded (compile dependency)io.etcd:jetcd-test (test dependency)The following test file will be deleted:
pulsar-metadata/src/test/java/org/apache/pulsar/metadata/impl/EtcdMetadataStoreTest.javaThe metadataStoreUrl and configurationMetadataStoreUrl configuration keys will no longer accept etcd: URLs. Attempting to start a broker, proxy, or other Pulsar component with an Etcd metadata URL will result in a clear error message at startup.
No changes.
No changes. CLI tools that accept metadata store URLs (e.g., pulsar initialize-cluster-metadata) will reject etcd: URLs with a descriptive error.
No changes.
The MetadataStore plugin system (PIP-45) remains fully intact. If there is community interest in continuing Etcd support, it can be maintained as an external plugin outside the core Pulsar repository, without any changes to the Pulsar codebase.