Meta Service

<!-- toc -->

Background

RisingWave provides both real-time analytical query as well as high-concurrent access to Materialized Views. Therefore, both the frontend and compute nodes are designed to be scalable, and they may share the same set of host machines or not, depending on cluster size and whether the user cares about resource isolation.

Meanwhile, components such as metadata provider, scheduler, monitoring are more suitable for a centralized design. For example, a typical on-premise deployment may look like below, where the dotted boxes represent minimal unit of deployment (VM or container).

Meta Store

Metadata should be regarded as transactional data. Especially, we need guarantees like atomicity, consistency (e.g. read-after-write) and transaction support. Besides, it's not necessary to scale out it to multiple nodes.

We choose PostgreSQL for production deployments, which is a robust, open-source OLTP database system, providing reliable and persistent data storage with strong support for data integrity and complex queries.

To fit into the key-value data model, metadata entries are serialized with Protobuf as the value and the key is the ID or version of them, with respect to the kind of metadata.

Types of Metadata

Catalog

Catalog is the metadata of relational tables in databases.

• Database & Schema: Namespace of database objects like in PostgreSQL.
• Table & Materialized Views: Definition of tables & materialized views along with the columns on them.
• Source: User-defined external data sources.

To execute a DDL statement like CREATE or DROP TABLE, the frontend sends an RPC to meta node and waits the updated catalog to take effect.

Storage

Hummock, an LSM-Tree-based storage engine, stores the mapping from version to the set of SSTable files in Meta Service. See more details in the overview of State Store.

Push on Updates

There are 2 choices on how to distribute information across multiple nodes.

• Push: When metadata changes, the meta node tells all nodes to update, and master node must wait for others to acknowledge before continuing.
• Pull: When data changes, the master node does nothing. Other nodes may not have the latest information, so they need to ask the master node every time.

Currently, for simplicity, we choose the push-style approach for all kinds of metadata. This is implemented as NotificationService on meta service and ObserverManager on frontend and compute nodes.

ObserverManager will register itself to meta service on bootstrap and subscribe metadata it needs. Afterwards, once metadata changed, the meta node streams the changes to it, expecting all subscribers to acknowledge.