istio/src/main/resources/proto/mcp/Readme.md
// Copyright 2018 Istio Authors // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License.
This folder contains the proto buffers for the Mesh Configuration Protocol (MCP). MCP is based on XDS and maintains conceptual alignment with it, despite the specific service and proto definitions being different.
MCP is a subscription-based configuration distribution API. The configuration consumer (i.e. sink) requests updates for collections of resources from a configuration producer (i.e. source). The source pushes resource updates to the sink when resources are added, updated, or deleted. The sink positively ACK's the resource update, if it was accepted, and NACK's if it was rejected, e.g. because a resource was invalid. The source may push additional update once the previous update was ACK/NACK'd. The source should only have one outstanding update (per-collection) in flight at a time.
MCP is a pair of bidirectional streaming gRPC API services
(ResourceSource and ResourceSink).
The ResourceSource service is used when the resource source is the
server and the sink is a client. By default, Galley implements the
ResourceSource service and Pilot/Mixer connect as clients.
The ResourceSink service is used when the resource source is a
client and the sink is the server. Galley can be configured to
optionally "dial-out" to a remote configuration sink, e.g. Pilot is in
another cluster where it cannot, as a client, initiate
connection to Galley. In this scenario, Pilot would implement the
ResourceSink service and Galley would connect as a client.
ResourceSource and ResourceSink are semantically equivalent with regards to the message exchange. The only meaningful difference is who initiates the connection and opens the grpc stream.
MCP is the mechanism of transport whereby Pilot and Mixer can be configured by a manager component. MCP defines a common per-resource metadata format and resource specific contents is defined elsewhere (e.g. https://github.com/istio/api/tree/master/networking/v1alpha3).
Resources of the same type are organized into named
collections. Istio API collection names are of the form
istio/<area>/<version>/<api> where <area>, <version>, and <api>
are defined by the API style guidelines. For
example, the collection name for VirtualService is
istio/networking/v1alpha3/virtualservices.
ResourceSource service - The client is the resource sink. The
client dials the server and establishes a new gRPC stream. The client
sends RequestResources and receive Resources messages.ResourceSink service - The client is the resource source. The
client dials the server and establishes a new gRPC stream. The server
sends RequestResources and receive Resources messages.The following overview applies to both ResourceSink and ResourceSource services, regardless of client/server roles.
The resource update protocol is derived from Incremental xDS. The protocol exchange is mostly the same except that resource hints have been removed. Most of the text and diagrams below are copied from the Incremental xDS docs and adjusted accordingly.
In MCP, resources are first organized by collection. Within each collection, resources are uniquely identifiable by their metadata name. Individual resources are versioned to differentiate newer versions of the same named resource.
A RequestResource message can be sent in two situations:
Initial message in an MCP bidirectional change stream
As an ACK or NACK response to a previous Resources message. In
this case the response_nonce is set to the nonce value from the
Resources message. ACK/NACK is determined by the presence of
error_detail in the subsequent request.
The initial RequestResources messages includes the collection
corresponding to the subscribed set of resources
(e.g. VirtualService), the node sink identifier, and nonce fields, and
initial_resource_version (more on that later). The source send a
Resources message when the requested resources are available. After
processing the Resources message , the sink sends a new
RequestResources message on the stream, specifying the last version
successfully applied and the nonce provided by the source.
The nonce field is used to pair RequestResources and Resources
messages per collection. The source should only send one outstanding
Resource message at a time (per-collection) and wait for the sink to
ACK/NACK. Upon receiving an update, the sink is expected to send an
ACK/NACK relatively quickly after decoding, validating, and persisting
the update to its internal configuration store.
The source should ignore requests with stale and unknown nonces that
do not match the nonce in the most recently sent Resource message.
The following example shows the sink receiving a sequence of changes which are successfully ACK'd.
The following example shows the same desired resource delivered with
incremental updates. This example assumes incremental is supported by
the source. When source does not support incremental updates, the
pushed Resources will always have incremental set to false,
regardles of whether the sink requested an incremental update. At any
time, the source can decide to push a full-state update, ignoring the
sink's request. Both sides must negotiate (i.e. agree) to use
incremental on a per request/response basis for an update to be sent
incrementally.
The following example shows what happens when a change cannot be applied.
The sink should only NACK in exceptional cases. For example, if a set of resources was invalid, malformed, or could not be decoded. NACK'd updates should raise an alarm for subsequent investigation by a human. The source should not resend the same set of resources that were previously NACK'd. Canary pushes to dedicated sinks may also be used to verify correctness (non-NACK) before pushing to a larger fleet of resource sinks.
The nonce in MCP is used to match RequestResources and Resources. On reconnect, the sinks may attempt to resume a session with the same source by specifying the known resources version with initial_resource_version for each collection.