modules/transport-grpc/README.md
An auxiliary transport which runs in parallel to the REST API.
The transport-grpc module initializes a new client/server transport implementing a gRPC protocol on Netty4.
Note: As a module, transport-grpc is included by default with all OpenSearch installations. However, it remains opt-in and must be explicitly enabled via configuration settings.
This section is for contributors who want to add new gRPC functionality directly to OpenSearch core (not as an external plugin). This includes adding new query converters, extending gRPC services, or building custom interceptors that will be part of the core distribution.
Contributing to core gRPC may involve working with up to four repositories:
Note: If you're only building interceptors (without protobuf changes), you can skip Steps 1-2 and start directly at Step 3.
┌─────────────────────────────────────────────────────────────┐
│ 1. opensearch-protobufs: Define/modify .proto files │
│ - Add new message types │
│ - Add new service endpoints │
│ - Generate local protobuf JAR │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ 2. OpenSearch: Implement Java code │
│ - Import local protobuf JAR (if protos changed) │
│ - Implement query converters / services / interceptors │
│ - Test your implementation │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ 3. Submit PRs (coordinated merge) │
│ - PR to opensearch-protobufs (merged first) │
│ - PR to OpenSearch (update version, then merge) │
│ - PR to documentation-website (can merge independently) │
└─────────────────────────────────────────────────────────────┘
Before generating protobufs, if you discover errors while working with the API:
toXContent/fromXContent methods for REST APIs) is the source of truth. If there's a discrepancy between the API spec and server code, the server is correct.Skip this step if you're only building interceptors without protobuf changes.
Clone the opensearch-protobufs repository and follow the Protobuf Local Convert Guide to generate protobufs from the OpenSearch API specification.
git clone https://github.com/opensearch-project/opensearch-protobufs.git
cd opensearch-protobufs
Skip this step if you're only building interceptors without protobuf changes.
Follow the complete guide: Generate Java Code and Packaging as a Maven/Gradle Dependency
Summary:
./tools/java/package_proto_jar.sh to create and install the JAR to your local Maven repositoryversion.properties (e.g., 1.1.0-SNAPSHOT)The JAR will be installed to ~/.m2/repository/org/opensearch/protobufs/opensearch-protobufs/
Clone OpenSearch repository (if not already cloned):
git clone https://github.com/opensearch-project/OpenSearch.git
cd OpenSearch
Configure to use local protobuf JAR (if you made protobuf changes):
Edit build.gradle in the OpenSearch root:
allprojects {
repositories {
mavenLocal() // Add this to use your local Maven repository
// ... other repositories ...
}
}
Then update the version in gradle/libs.versions.toml:
opensearchprotobufs = "1.1.0-SNAPSHOT"
Verify the setup:
./gradlew :modules:transport-grpc:build
Now you can implement your gRPC functionality in the OpenSearch codebase. Choose the appropriate section based on what you're building.
Requires protobuf changes (Steps 1-2).
Query converters transform protobuf query messages into OpenSearch QueryBuilder objects.
Implementation details: See SPI documentation for complete examples.
Requires protobuf changes (Steps 1-2).
gRPC services expose new API endpoints for client applications.
Implementation details: See SPI documentation for complete examples.
Does NOT require protobuf changes - you can skip Steps 1-2 and start here!
Interceptors process all incoming gRPC requests for cross-cutting concerns like authentication, logging, and metrics.
Implementation details: See SPI documentation for complete examples.
See the Testing section in the Development Guide below for complete instructions on:
Important: PRs must be coordinated and merged in a specific order.
Submit PR to opensearch-protobufs (if you made protobuf changes):
.proto filesAfter protobufs PR is merged, update OpenSearch:
gradle/libs.versions.toml:
opensearchprotobufs = "1.1.0" # Use the newly published version
mavenLocal() from build.gradle if you added itSubmit PR to OpenSearch:
gradle/libs.versions.toml (if protobuf version changed)Submit PR to documentation-website:
0. (If needed) opensearch-api-specification PR → MERGE FIRST (if API spec fixes are required)
↓ Regenerate protobufs after spec fix is merged
↓
1. opensearch-protobufs PR → MERGE FIRST
↓
2. Protobufs published to Maven Central
↓
3. Update OpenSearch gradle/libs.versions.toml with new version
↓
4. OpenSearch PR → MERGE SECOND
↓
5. documentation-website PR → MERGE (can be independent)
Enable this transport with:
setting 'aux.transport.types', '[transport-grpc]'
setting 'aux.transport.transport-grpc.port', '9400-9500' //optional
For the secure transport:
setting 'aux.transport.types', '[secure-transport-grpc]'
setting 'aux.transport.secure-transport-grpc.port', '9400-9500' //optional
| Setting Name | Description | Example Value | Default Value |
|---|---|---|---|
| grpc.publish_port | The external port number that this node uses to publish itself to peers for gRPC transport. | 9400 | -1 (disabled) |
| grpc.host | List of addresses the gRPC server will bind to. | ["0.0.0.0"] | [] |
| grpc.bind_host | List of addresses to bind the gRPC server to. Can be distinct from publish hosts. | ["0.0.0.0", "::"] | Value of grpc.host |
| grpc.publish_host | List of hostnames or IPs published to peers for client connections. | ["thisnode.example.com"] | Value of grpc.host |
| grpc.netty.worker_count | Number of Netty worker threads for the gRPC server. Controls network I/O concurrency. | 2 | Number of processors |
| grpc.netty.executor_count | Number of threads in the ForkJoinPool for processing gRPC service calls. Controls request processing parallelism. | 32 | 2 × Number of processors |
| grpc.netty.max_concurrent_connection_calls | Maximum number of simultaneous in-flight requests allowed per client connection. | 200 | 100 |
| grpc.netty.max_connection_age | Maximum age a connection is allowed before being gracefully closed. Supports time units like ms, s, m. | 500ms | Not set (no limit) |
| grpc.netty.max_connection_idle | Maximum duration a connection can be idle before being closed. Supports time units like ms, s, m. | 2m | Not set (no limit) |
| grpc.netty.keepalive_timeout | Time to wait for keepalive ping acknowledgment before closing the connection. Supports time units. | 1s | Not set |
| grpc.netty.max_msg_size | Maximum inbound message size for gRPC requests. Supports units like b, kb, mb, gb. | 10mb or 10485760 | 10mb |
max_connection_age), you can use units such as ms (milliseconds), s (seconds), m (minutes), etc.max_msg_size), you can use units such as b (bytes), kb, mb, gb, etc.setting 'grpc.publish_port', '9400'
setting 'grpc.host', '["0.0.0.0"]'
setting 'grpc.bind_host', '["0.0.0.0", "::", "10.0.0.1"]'
setting 'grpc.publish_host', '["thisnode.example.com"]'
setting 'grpc.netty.worker_count', '2'
setting 'grpc.netty.executor_count', '32'
setting 'grpc.netty.max_concurrent_connection_calls', '200'
setting 'grpc.netty.max_connection_age', '500ms'
setting 'grpc.netty.max_connection_idle', '2m'
setting 'grpc.netty.max_msg_size', '10mb'
setting 'grpc.netty.keepalive_timeout', '1s'
The dedicated thread pool used for gRPC request processing is registered as a standard OpenSearch thread pool named grpc, controlled by the grpc.netty.executor_count setting.
The gRPC thread pool stats can be monitored using:
curl -X GET "localhost:9200/_nodes/stats/thread_pool?filter_path=nodes.*.thread_pool.grpc"
./gradlew :modules:transport-grpc:test
./gradlew :modules:transport-grpc:internalClusterTest
To run OpenSearch with the gRPC transport enabled:
./gradlew run -Dtests.opensearch.aux.transport.types="[transport-grpc]"