release-notes/4.1.0.md
RabbitMQ 4.1.0 is a new feature release.
See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.
Some key improvements in this release are listed below.
Quorum queue log reads are now offloaded to channels (sessions, connections).
In practical terms this means improved consumer throughput, lower interference of publishers on queue delivery rate to consumers, and improved CPU core utilization by each quorum queue (assuming there are enough cores available to the node).
Support for the properties and application-properties filters of AMQP Filter Expressions Version 1.0 Working Draft 09.
As described in the AMQP 1.0 Filter Expressions blog post, this feature enables multiple concurrent clients each consuming only a subset of messages from a stream while maintaining message order.
Graduated (mandatory) feature flags several minors ago has proven that they could use some user experience improvements. For example, certain required feature flags will now be enabled on node boot when all nodes in the cluster support them.
See core server changes below as well as the GitHub project dedicated to feature flags improvements for the complete list of related changes.
rabbitmqadmin v2 is a major revision of the
original CLI client for the RabbitMQ HTTP API.
It supports a much broader set of operations, including health checks, operations
on federation upstreams, shovels, transformations of exported definitions,
(some) Tanzu RabbitMQ HTTP API endpoints, --long-option and subcommand inference in interactive mode,
and more.
Before a client connection can negotiate a maximum frame size (frame_max), it must authenticate
successfully. Before the authenticated phase, a special lower frame_max value
is used.
With this release, the value was increased from the original 4096 bytes to 8192 to accommodate larger JWT tokens.
Clients that do override frame_max now must use values of 8192 bytes or greater.
We recommend using the default server value of 131072: do not override the frame_max
key in rabbitmq.conf and do not set it in the application code.
amqplib Must Be Upgradedamqplib is a popular client library that has been using
a low frame_max default of 4096. Its users must upgrade to a compatible version
(starting with 0.10.7) or explicitly use a higher frame_max.
amqplib versions older than 0.10.7 will not be able to connect to
RabbitMQ 4.1.0 and later versions due to the initial AMQP 0-9-1 maximum frame size
increase covered above.
The default MQTT Maximum Packet Size changed from 256 MiB to 16 MiB.
This default can be overridden by configuring mqtt.max_packet_size_authenticated.
Note that this value must not be greater than max_message_size (which also defaults to 16 MiB).
The following rabbitmq.conf settings are unsupported:
cluster_formation.etcd.ssl_options.fail_if_no_peer_certcluster_formation.etcd.ssl_options.dhcluster_formation.etcd.ssl_options.dhfileThis release requires Erlang 26.2 and supports Erlang 27.x.
Provisioning Latest Erlang Releases explains what package repositories and tools can be used to provision latest patch versions of Erlang 26.x and 27.x.
Artifacts are distributed via GitHub releases.
Community RabbitMQ image has a tag, 4.1, that is
periodically updated to the latest patch in these series.
See the Upgrading guide for documentation on upgrades and GitHub releases for release notes of individual releases.
This release series supports upgrades from 4.0.x and 3.13.x.
Blue/Green Deployment-style upgrades are avaialble for migrations
from RabbitMQ 3.12.x series.
None. The required feature flag set is the same as in 4.0.x.
RabbitMQ 4.1.0 nodes can run alongside 4.0.x nodes. 4.1.x-specific features can only be made available when all nodes in the cluster
upgrade to 4.1.0 or a later patch release in the new series.
While operating in mixed version mode, some aspects of the system may not behave as expected. Once all nodes are upgraded to 4.1.0, these irregularities will go away.
Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended periods of time (no more than a few hours).
This version does not require any additional post-upgrade procedures compared to other versions.
This section can be incomplete and will be expanded as 4.1 approaches its release candidate stage.
Quorum queue log reads are now offloaded to channels (sessions, connections).
In practical terms this means improved consumer throughput, lower interference of publishers on queue delivery rate to consumers, and improved CPU core utilization by each quorum queue (assuming there are enough cores available to the node).
GitHub issue: #12713
Feature flag quality of live improvements.
Certain required feature flags will now be automatically required on node boot and do not have to be explicitly enabled before an upgrade. This does not apply to all feature flags, however.
GitHub project: #4.
properties and appliation-properties filters of AMQP Filter Expressions Version 1.0 Working Draft 09
when consuming from a stream via AMQP 1.0. String prefix and suffix matching is also supported.
This feature adds the ability to RabbitMQ to have multiple concurrent clients each consuming only a subset of messages while maintaining message order. It also reduces network traffic between RabbitMQ and clients by only dispatching those messages that the clients are actually interested in.
GitHub issue: #12415
Larger (up to 8192 bytes) JWT tokens now can be used by AMQP 0-9-1 clients.
Before a client connection can negotiate a maximum frame size (frame_max), it must authenticate
successfully. Before the authenticated phase, a special lower frame_max value
is used.
Clients that do override frame_max now must use values of 8192 bytes or greater.
We recommend using the default server value of 131072: do not override the frame_max
key in rabbitmq.conf and do not set it in the application code.
amqplib is a popular client library that has been using
a low frame_max default of 4096. Its users must upgrade to a compatible version
(starting with 0.10.7) or explicitly use a higher frame_max.
GitHub issue: #13541
AMQP 1.0 connections that use OAuth 2.0 now can renew their JWT tokens This allows clients to set a new token proactively before the current one expires, ensuring uninterrupted connectivity. If a client does not set a new token before the existing one expires, RabbitMQ will automatically close the AMQP 1.0 connection.
GitHub issue: #12599
AMQP 1.0 filters now have capped complexity: filtering on more than 16 properties won't be possible. This is a protection mechanism recommended in the AMQP 1.0 spec.
GitHub issue: #13196
Support for Multiple Routing Keys in AMQP 1.0 via x-cc Message Annotation.
AMQP 1.0 publishers now can set multiple routing keys by using the x-cc message annotation.
This annotation allows publishers to specify a list
of routing keys (strings) for more flexible message distribution,
similar to the CC header in AMQP 0.9.1.
GitHub issue: #12559
Support field dynamic of AMQP 1.0 source and target.
This allows AMQP clients to dynamically create exclusive queues, which can be useful for RPC workloads.
GitHub issue: #13231
Quorum queue checkpoint algorithm was tweaked to take checkpoints more frequently, thus clearing older segment files more aggressively.
Workloads that use larger messages should continue following the documented recommendations to avoid large disk space footprint of segment files.
GitHub issue: #13622
Previously a node that was a cluster member but then was reset could not rejoin the cluster if the schema data store was Mnesia.
Now the reset node will try to leave the cluster and retry rejoining again. This was already the case for Khepri.
Contributed by @SimonUnge.
GitHub issue: #13643
Nodes will now fall back to system CA certificate list (if available) when no CA certificate is explicitly configured.
Contributed by @LoisSotoLopez.
AMQP 1.0 and AMQP 0-9-1 connections now dynamically adjust their TCP socket buffers.
GitHub issue: #13363
Peer discovery resilience improvements.
Quorum queue's continuous membership reconciliation mechanism (CMR) efficiency and resilience improvements.
GitHub issue: #13703
AMQP 1.0 and AMQP 0-9-1 connections now produce more specific error messages when an incorrect data is sent by the client during connection negotiation.
For example, when a TLS-enabled client connects to a non-TLS port, or an HTTP GET request is sent to the AMQP port.
GitHub issue: #13559
AMQP 0-9-1 and AMQP 1.0 connections now use a higher pre-authentication maximum allowed frame limit size by default. This means that larger JWT tokens can be accepted without any configuration.
GitHub issue: #13542
Plugins now can mark queues and streams as protected from deletion by applications.
GitHub issue: #13525
Internal API changes needed by a future version of the message deduplication plugin.
Contributed by @noxdafox.
GitHub issue: #13374
Quorum queue leadership transfer could take several seconds longer than necessary to complete.
GitHub issue: #13190
Quorum queue follow replica's last index could lag behind that of the leader.
GitHub issue: #13328
AMQP 0-9-1 channel exception generator could not handle entity names (say, queue or stream names) that contained non-ASCII characters.
This affected applications that use passive queue declarations, such as the Shovel plugin.
Contributed by @bpint.
GitHub issue: #12888
Reintroduced transient flow control between classic queue replicas and AMQP 0-9-1 channels, MQTT connections.
Flow control between these specific parts of the core were unintentionally
removed in 4.0.0 together with classic queue mirroring.
Contributed by @gomoripeti.
GitHub issue: #12907
AMQP 1.0 connections with a higher consumption rate could set the incoming window field on the flow frame to a negative value, which resulted in an exception that affected the consumer.
GitHub issues: #12816
In rare cases quorum queue could end up without an elected leader because chosen candidate replica was not verified for aliveness.
Contributed by @Ayanda-D.
When a new replica is added to a quorum queue, the node that handles this request will now wait
the operation to complete. Previously an early return could result in confusing cluster_change_not_permitted
errors for subsequent operations, for example, an addition of another replica.
GitHub issue: #12837
In a mixed version 4.0/3.13 cluster, dead lettering a message could fail.
GitHub issue: #12933
In very rare cases, RabbitMQ could fail to notify stream consumers connected to follower replicas about newly committed offsets as quickly as it usually happens for consumers connected to the stream leader.
GitHub issue: #12785
Default cluster formation timeout with Khepri now matches that of Mnesia (5 minutes by default).
Discovered and reported by @evolvedlight.
GitHub issue: #13195
Quorum queues could return a list of members (replicas) with duplicates in some cases.
GitHub issue: #13168
Classic queues with priorities could run into an exception.
GitHub issue: #13088
The default MQTT Maximum Packet Size changed from 256 MiB to 16 MiB.
This default can be overridden by configuring mqtt.max_packet_size_authenticated.
Note that this value must not be greater than max_message_size (which also defaults to 16 MiB).
rabbitmqctl force_reset is deprecated as it is incompatible with Khepri,
which will become the default metadata store in RabbitMQ by 4.2.0.
GitHub issue: #13217
New major version of rabbitmqadmin, a CLI tool that targets RabbitMQ's HTTP API, has matured.
Unlike its predecessor, the tool is distirbuted via GitHub as as a standalone native binary.
There are minor command line interface changes and a slightly different configuration file
format (TOML instead of ini).
GitHub repository: rabbitmq/rabbitmqadmin-ng
New health check for detecting quorum queues without an elected leader.
# across all virtual host
rabbitmq-diagnostics check_for_quorum_queues_without_an_elected_leader --across-all-vhosts "^name.pattern.*"
# in a specific virtual host
rabbitmq-diagnostics check_for_quorum_queues_without_an_elected_leader --vhost "vhost-1" "^name.pattern.*"
Contributed by @Ayanda-D.
GitHub issue: #13487
rabbitmq-diagnostics check_if_any_deprecated_features_are_used implementation is now more complete
(checks for a more deprecated features).
Fixes a false positive that incorrectly reported deprecated feature use, specifically the use of non-exclusive transient classic queues.
GitHub issue: #12840
RabbitMQ nodes now provide a Prometheus histogram for message sizes published by applications.
This feature allows operators to gain insights into the message sizes being published to RabbitMQ, such as average message size, number of messages per pre-defined bucket (which can both be computed accurately), and percentiles (which will be approximated). Each metric is labelled by protocol (AMQP 1.0, AMQP 0.9.1, MQTT 5.0, MQTT 3.1.1, and MQTT 3.1).
GitHub issue: #12342
Two new stream metrics for streams.
Contributed by @gomoripeti and @markus812498.
GitHub issue: #12765
New metric: queue_identity_info.
queue_identity_info is a new metric whose labels describe the type
of the queue and its membership status (leader/follower) on the scraped node.
GitHub issue: #13583
rabbitmq_identity_info now includes a new label, rabbitmq_endpoint.
The label allows operators differntiate between metrics scraped from different RabbitMQ endpoints (the per-object one and the aggregated one).
Since some metric names are identical regardless of whether they were
scraped from /metrics and /metrics/per-object, scraping both endpoints could lead to
duplicates and confusion.
GitHub issue: #13218
Grafana 11.3.x compatibility.
Contributed by @anhanhnguyen.
GitHub issue: #12720
The original HTTP API One True Health Check™ is now a no-op. A comparable "mega health check"
has long been deprecated in CLI tools and was made a no-op in 4.0.0.
This endpoint was using a deprecated feature: a classic non-exclusive transient (non-durable) queue.
See Health Checks for modern focused alternatives.
GitHub issue: #13047
Static assets served by the management UI now have a control-cache header set
to make sure that browsers reload them between upgrades.
UI code now tries to track version changes to perform a forced reload of the entire page.
GitHub issue: #12749
Connection pages now display detailed AMQP 1.0 session and link information:
The management UI now shows if a feature flag has a migration function (in other words, it may take time to be enabled), if it is experimental and whether it is supported or not. To enable an experimental feature flag, a user must to tick checkboxes to confirm they know what they are doing.
GitHub issue: #12643
Feature flags are now enabled using asynchronous requests in the management UI. This means that feature flags that perform data migrations (which can take some time) won't block the browser tab.
GitHub issue: #12643
The HTTP API and management UI now can use a separate chain of authentication and authorization backends.
This means that a separate list of backends can now be used for the messaging protocol clients and the HTTP API access.
Contributed by @aaron-seo.
GitHub issue: #13465
The UI now provides a dark theme.
Contributed by @efimov90.
New health check commands help detect quorum queues without an elected leader.
# verifies all quorum queues across all virtual hosts
GET /health/checks/quorum-queues-without-elected-leaders/all-vhosts/
# verifies all quorum queues in the given virtual host
GET /health/checks/quorum-queues-without-elected-leaders/vhost/{vhost}
# verifies a subset of quorum queue that match the pattern across all virtual hosts
GET /health/checks/quorum-queues-without-elected-leaders/all-vhosts/pattern/{pattern}
# verifies a subset of quorum queue that match the pattern in the given virtual host
GET /health/checks/quorum-queues-without-elected-leaders/vhost/{vhost}/pattern/{pattern}
Note that the values in the path must be percent-encoded, including the pattern.
GitHub issue: #13571
Web app tab title now changes depending on the selected top-level tab.
GitHub issue: #13512
Fixes a false positive that incorrectly reported deprecated feature use, specifically the use of non-exclusive transient classic queues.
GitHub issue: #12840
Nodes will fail to boot if rabbit.credit_flow_default_credit (an advanced.config-only, rarely used setting)
is set to an incompatible pair of values.
Contributed by @JimmyWang6.
GitHub issue: #13046
When a logged in user's JWT token was refreshed, the user identity displayed in the UI was changed.
GitHub issue: #12818
Stream replication connections now can be configured to use IPv6 using advanced.config:
[
{osiris, [
{replica_ip_address_family, inet6}
]}
].
When a connection of one or more consumers in a Single Active Consumer group failed, the group could try to activate (promote) one of the consumers are are no longer online. In practical terms this means that other consumers were not getting any deliveries.
GitHub issue: #13657
A TCP connection to the stream protocol port that sent no data (e.g. a TCP load balancer check) produced a harmless but scary looking exception in the log.
GitHub issue: #13701
The OAuth 2 plugin now requires several values to be explicitly configured by the user and will not provide any defaults.
The affected identity providers are Azure Entra (né Azure AD) and auth0.
GitHub issue: #12235
Scope aliases now can be configured in rabbitmq.conf.
auth_oauth2.scope_aliases.admin = tag:administrator configure:*/*
auth_oauth2.scope_aliases.developer = tag:management configure:*/* read:*/* write:*/*
auth_oauth2.scope_aliases.1.alias = api://administrator
auth_oauth2.scope_aliases.1.scope = tag:administrator configure:*/*
auth_oauth2.scope_aliases.2.alias = api://developer
auth_oauth2.scope_aliases.2.scope = tag:management configure:*/* read:*/* write:*/*
GitHub issue: #12210
Support for more complex JWT token structures, such as those sometimes used by Keycloak.
OpenID Discovery Endpoint now can be configured. This is particularly relevant for Azure Entra (né Azure AD) users.
GitHub issue: #12211
JWT token refreshes will no longer affect the identity shown in the management UI.
GitHub issue: #12598
The in_group_nested query now uses case-insensitive matching, which is more typical of the LDAP tooling.
GitHub issue: #13629
Exchange federation now can be used with MQTTv5 consumers.
GitHub issue: #13115
Queue federation could cause a deadlock in a quorum queue replica process.
GitHub issue: #12713
Federation status command and HTTP API endpoint could run into an exception.
GitHub issue: #13701
New Shovel metric: the number of forwarded messages.
GitHub issue: #13626
AMQP 0-9-1 channel exception generator could not handle entity names (say, queue or stream names) that contained non-ASCII characters.
This affected applications that use passive queue declarations, such as the Shovel plugin.
Contributed by @bpint.
GitHub issue: #12888
The shutdown sequence of Shovel connections and AMQP 1.0 sessions is now safer.
GitHub issue: #2596
The rabbitmq_event_exchange plugin now can be configured to internally publish AMQP 1.0 instead of AMQP 0.9.1 messages to the amq.rabbitmq.event topic exchange.
This allows AMQP 1.0 consumers to receive event properties containing complex types such as lists
or maps, for example queue arguments for the queue.created
event or client provided properties for the connection.created event.
GitHub issue: #12714
Avoids an exception during automatic removal of cluster members that are no longer returned by peer discovery (an opt-in feature).
GitHub issue: #12809
This plugin was significantly reworked to further reduce the probability of two (or more) clusters being formed in a small percentage of cases when the entire cluster was started for the first time.
The plugin no longer relies on the Kubernetes API and instead will try to join
the first (indexed at 0) node as the seed node.
This change is backwards compatible.
GitHub issue: #13050
Avoids an exception during automatic removal of cluster members that are no longer returned by peer discovery (an opt-in feature).
GitHub issue: #12809
cluster_formation.registration.enabled is a new configuration setting that allows the backend to skip registration.
This is useful when Consul is used for peer discovery but a different tool such as Nomad is used to keep track of the services and their registration, unregistration.
Contributed by @frederikbosch.
GitHub issue: #13201
Avoids an exception during automatic removal of cluster members that are no longer returned by peer discovery (an opt-in feature).
GitHub issue: #12809
Avoids an exception during automatic removal of cluster members that are no longer returned by peer discovery (an opt-in feature).
GitHub issue: #12809
The client now only can declare durable entities in preparation for a Khepri-only future version of RabbitMQ.
GitHub issue: #12947
ra was upgraded to 2.16.8osiris was upgraded to 1.8.7khepri was upgraded to 0.16.0khepri_mnesia_migration was upgraded to 0.7.2observer_cli was upgraded to 1.8.2eetcd was upgraded to 0.5.0gun was upgraded to 2.1.0credentials_obfuscation was upgraded to 3.5.0To obtain source code of the entire distribution, please download the archive named rabbitmq-server-4.1.0.tar.xz
instead of the source tarball produced by GitHub.