ContextQMD
Back to Envoy

Dynamic modules

docs/root/intro/arch_overview/advanced/dynamic_modules.rst

1.39.09.3 KB
Original Source

.. _arch_overview_dynamic_modules:

Dynamic modules

.. attention::

The dynamic modules feature is currently under active development. Capabilities will be expanded over time and it still lacks some features that are available in other extension mechanisms. We are looking for feedback from the community to improve the feature.

Envoy has support for loading shared libraries at runtime to extend its functionality. In Envoy, these are known as "dynamic modules." More specifically, dynamic modules are shared libraries that implement the :repo:ABI <source/extensions/dynamic_modules/abi/abi.h> written in a pure C header file. The ABI defines a set of functions that the dynamic module must implement to be loaded by Envoy. Also, it specifies the functions implemented by Envoy that the dynamic module can call to interact with Envoy.

Implementing the ABI from scratch requires an extensive understanding of the Envoy internals. For users, we provide an official SDK that abstracts these details and provides a high-level API to implement dynamic modules. The SDK is currently available in C++, Go, and Rust. In theory, any language that can produce a shared library can be used to implement dynamic modules. Future development may include support for other languages.

Currently, dynamic modules are supported at the following extension points:

  • As a :ref:bootstrap extension <envoy_v3_api_msg_extensions.bootstrap.dynamic_modules.v3.DynamicModuleBootstrapExtension>.
  • As a :ref:cluster <envoy_v3_api_msg_extensions.clusters.dynamic_modules.v3.ClusterConfig>.
  • As a :ref:listener filter <envoy_v3_api_msg_extensions.filters.listener.dynamic_modules.v3.DynamicModuleListenerFilter>.
  • As a :ref:UDP listener filter <envoy_v3_api_msg_extensions.filters.udp.dynamic_modules.v3.DynamicModuleUdpListenerFilter>.
  • As an :ref:access logger <envoy_v3_api_msg_extensions.access_loggers.dynamic_modules.v3.DynamicModuleAccessLog>.
  • As a :ref:formatter <envoy_v3_api_msg_extensions.formatter.dynamic_modules.v3.DynamicModuleFormatter>.
  • As a :ref:stats sink <envoy_v3_api_msg_extensions.stat_sinks.dynamic_modules.v3.DynamicModuleStatsSink>.
  • As a :ref:network filter <envoy_v3_api_msg_extensions.filters.network.dynamic_modules.v3.DynamicModuleNetworkFilter>.
  • As an :ref:HTTP filter <envoy_v3_api_msg_extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter>.
  • As an :ref:HTTP matching data input <envoy_v3_api_msg_extensions.matching.http.dynamic_modules.v3.HttpDynamicModuleMatchInput>.
  • As an :ref:input matcher <envoy_v3_api_msg_extensions.matching.input_matchers.dynamic_modules.v3.DynamicModuleMatcher>.
  • As a :ref:TLS certificate validator <envoy_v3_api_msg_extensions.transport_sockets.tls.cert_validator.dynamic_modules.v3.DynamicModuleCertValidatorConfig>.
  • As a :ref:transport socket <envoy_v3_api_msg_extensions.transport_sockets.dynamic_modules.v3.DynamicModuleTransportSocket>.
  • As a :ref:load balancing policy <envoy_v3_api_msg_extensions.load_balancing_policies.dynamic_modules.v3.DynamicModulesLoadBalancerConfig>.
  • As an :ref:upstream HTTP TCP bridge <envoy_v3_api_msg_extensions.upstreams.http.dynamic_modules.v3.Config>.
  • As a :ref:tracer <envoy_v3_api_msg_extensions.tracers.dynamic_modules.v3.DynamicModuleTracer>.
  • As a :ref:health checker <envoy_v3_api_msg_extensions.health_checkers.dynamic_modules.v3.DynamicModuleHealthCheck>.

There are a few design goals for the dynamic modules:

  1. Performance: The dynamic modules should have minimal overhead compared to the built-in C++ extensions. For example, the dynamic modules are able to access HTTP headers as well as body without copying them unlike any other extension mechanisms.
  2. Ease of Use: The SDK should provide a high-level API that abstracts the details of the Envoy internals.
  3. Flexibility: The dynamic modules should be able to implement any functionality that can be implemented by the built-in C++ extensions without performance penalty. This is work in progress and many features are not yet available.

Compatibility

Since a dynamic module is loaded at runtime, it must be ABI-compatible with the Envoy binary that loads it.

Envoy's dynamic modules have stricter compatibility requirements than Envoy's other extension mechanisms, such as Lua, Wasm or External Processor. Stabilizing the ABI is challenging due to the way the ABI needs to be tightly coupled to Envoy's internals.

Currently, we guarantee forward compatibility within one version: a dynamic module built with the SDK for Envoy version X.Y will work with Envoy versions X.Y and X.(Y+1). Breaking changes to the ABI may occur in later versions.

To ensure compatibility, it is recommended to rebuild your dynamic modules with the SDK matching your target Envoy version in a timely manner.

Module discovery

A dynamic module is referenced by its name as in the :ref:configuration API <envoy_v3_api_msg_extensions.dynamic_modules.v3.DynamicModuleConfig>. The name is used to search for the shared library file in the search path. The search path is configured by the environment variable ENVOY_DYNAMIC_MODULES_SEARCH_PATH. The actual search path is ${ENVOY_DYNAMIC_MODULES_SEARCH_PATH}/lib${name}.so. If the environment variable is not set, the current working directory is used instead. After searching in the specified search path, the standard library paths such as LD_LIBRARY_PATH and /usr/lib are searched as well following the behavior of dlopen(3).

For example, when the name my_module is referenced in the configuration and ENVOY_DYNAMIC_MODULES_SEARCH_PATH is set to /path/to/modules, Envoy will first look for /path/to/modules/libmy_module.so, then $LD_LIBRARY_PATH/libmy_module.so, and finally /usr/lib/libmy_module.so, etc.

Safety

The dynamic modules should be used under the assumption that all modules are fully trusted and have the same privilege level as the main Envoy program. Since these modules run in the same process as Envoy, they can access all memory and resources available to the main process. This makes it unfeasible to enforce security boundaries between Envoy and the modules, as they share the same address space and permissions. It is essential that any dynamic module undergo thorough testing and validation before deployment just like any other application code.

Error handling (Rust SDK)

The Rust SDK provides an optional CatchUnwind wrapper that can be used to wrap filter implementations. When a wrapped callback panics, the SDK logs the panic payload and returns a fail-closed default:

  • HTTP request-path callbacks send a 500 response and return StopIteration.
  • HTTP response-path callbacks reset the stream and return StopIteration.
  • Network filter callbacks close the connection and return StopIteration.
  • Listener filter callbacks close the socket and return StopIteration.

The SDK always guards each callback at the FFI boundary so a panic can never unwind into Envoy and corrupt the process, regardless of whether CatchUnwind is used. The wrapper adds graceful filter-level teardown on top of that guard. The affected request or connection is terminated. Other traffic is unaffected.

Getting started

We have a dedicated repository for the dynamic module examples to help you get started. The repository is available at envoyproxy/dynamic-modules-examples <https://github.com/envoyproxy/dynamic-modules-examples>_

Statistics

All dynamic-module extension types emit the following statistics in the shared dynamic_modules. namespace. These stats track failures encountered while loading the extension's configuration. Each one is tagged with config_name, set to the configured name of the dynamic-module extension instance — for example the :ref:filter_name <envoy_v3_api_field_extensions.filters.http.dynamic_modules.v3.DynamicModuleFilter.filter_name> for the HTTP filter, transport_socket_name for the transport socket, lb_policy_name for the load-balancing policy, tracer_name for the tracer or cluster_name for the cluster (default if the extension has no per-instance name, as for the UDP listener filter).

.. csv-table:: :header: Name, Type, Description :widths: 1, 1, 2

module_load_error, Counter, "Total dynamic modules that could not be loaded (missing or invalid module source, dlopen failure, by-name lookup miss, or a required ABI symbol could not be resolved)." config_init_error, Counter, "Total configurations that failed to initialize after the module loaded successfully (the module rejected or failed to parse the supplied configuration)." remote_fetch_error, Counter, "Total failures fetching or loading a remote module source, including rejected cache misses when nack_on_cache_miss is set. Only the HTTP filter supports remote module sources." per_route_config_error, Counter, "Total per-route configurations that failed to load or initialize. Only emitted by the HTTP filter."

In addition to the counters above, a module may define its own custom metrics. These are emitted under the configurable :ref:metrics_namespace <envoy_v3_api_field_extensions.dynamic_modules.v3.DynamicModuleConfig.metrics_namespace> (dynamicmodulescustom by default), separately from the dynamic_modules. namespace above.