doc/dev/msgr2.rst
.. _msgr2-protocol:
This is a revision of the legacy Ceph on-wire protocol that was implemented by the SimpleMessenger. It addresses performance and security issues.
This protocol revision has several goals relative to the original protocol:
A connection has four distinct phases:
#. banner #. authentication frame exchange #. message flow handshake frame exchange #. message frame exchange
Both the client and server, upon connecting, send a banner::
"ceph v2\n" __le16 banner payload length banner payload
A banner payload has the form::
__le64 peer_supported_features __le64 peer_required_features
This is a new, distinct feature bit namespace (CEPH_MSGR2_*). Currently, only CEPH_MSGR2_FEATURE_REVISION_1 is defined. It is supported but not required, so that msgr2.0 and msgr2.1 peers can talk to each other.
If the remote party advertises required features we don't support, we can disconnect.
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| send banner |
|----+ +----|
| | | |
| +-------+--->|
| send banner| |
|<-----------+ |
| |
After the banners are exchanged, all further communication happens in frames. The exact format of the frame depends on the connection mode (msgr2.0-crc, msgr2.0-secure, msgr2.1-crc or msgr2.1-secure). All connections start in crc mode (either msgr2.0-crc or msgr2.1-crc, depending on peer_supported_features from the banner).
Each frame has a 32-byte preamble::
__u8 tag __u8 number of segments { __le32 segment length __le16 segment alignment } * 4 __u8 flags reserved (1 byte) __le32 preamble crc
An empty frame has one empty segment. A non-empty frame can have between one and four segments, all segments except the last may be empty.
If there are less than four segments, unused (trailing) segment length and segment alignment fields are zeroed.
Currently supported flags
1. FRAME_EARLY_DATA_COMPRESSED (see :ref:`msgr-post-compression`)
The reserved bytes are zeroed.
The preamble checksum is CRC32-C (see :ref:`crc`). It covers
everything up to itself (28 bytes) and is calculated and verified
irrespective of the connection mode (i.e. even if the frame is
encrypted).
msgr2.0-crc mode
~~~~~~~~~~~~~~~~
A msgr2.0-crc frame has the form::
preamble (32 bytes)
{
segment payload
} * number of segments
epilogue (17 bytes)
where epilogue is::
__u8 late_flags
{
__le32 segment crc
} * 4
late_flags is used for frame abortion. After transmitting the
preamble and the first segment, the sender can fill the remaining
segments with zeros and set a flag to indicate that the receiver must
drop the frame. This allows the sender to avoid extra buffering
when a frame that is being put on the wire is revoked (i.e. yanked
out of the messenger): payload buffers can be unpinned and handed
back to the user immediately, without making a copy or blocking
until the whole frame is transmitted. Currently this is used only
by the kernel client, see ceph_msg_revoke().
The segment checksum is CRC32-C (see :ref:`crc`). For used
segments, the initial value is ``-1``. For unused (trailing)
segments, it is zeroed.
The checksums are calculated just to protect against bit errors.
No authenticity guarantees are provided, unlike in msgr1 which
attempted to provide some authenticity guarantee by optionally
signing segment lengths and checksums with the session key.
Issues
^^^^^^
1. As part of introducing a structure for a generic frame with
variable number of segments suitable for both control and
message frames, msgr2.0 moved the crc of the first segment of
the message frame (ceph_msg_header2) into the epilogue.
As a result, ceph_msg_header2 can no longer be safely
interpreted before the whole frame is read off the wire.
This is a regression from msgr1, because in order to scatter
the payload directly into user-provided buffers and thus avoid
extra buffering and copying when receiving message frames,
ceph_msg_header2 must be available in advance -- it stores
the transaction id which the user buffers are keyed on.
The implementation has to choose between forgoing this
optimization or acting on an unverified segment.
2. late_flags is not covered by any crc. Since it stores the
abort flag, a single bit flip can result in a completed frame
being dropped (causing the sender to hang waiting for a reply)
or, worse, in an aborted frame with garbage segment payloads
being dispatched.
This was the case with msgr1 and got carried over to msgr2.0.
msgr2.1-crc mode
~~~~~~~~~~~~~~~~
Differences from msgr2.0-crc:
1. The crc of the first segment is stored at the end of the
first segment, not in the epilogue. The epilogue stores up to
three checksums, not up to four.
If the first segment is empty, its CRC value is not written to
the frame.
2. The epilogue is generated only if the frame has more than one
segment (i.e. at least one of second to fourth segments is not
empty). Rationale: If the frame has only one segment, it cannot
be aborted and there are no checksums to store in the epilogue.
3. Unchecksummed ``late_flags`` is replaced with ``late_status``.
A completed frame is indicated by a ``late_flags`` value of
``FRAME_LATE_STATUS_COMPLETE``, while an aborted frame is indicated
by a value of ``FRAME_LATE_STATUS_ABORTED``. This adjusted field
has better bit error detection by using a 4-bit nibble per flag
and two code words that are Hamming Distance = 4 apart (and not
all zeros or ones). This comes at the expense of having only
one reserved flag, of course.
Some example frames:
* A 0+0+0+0 frame (empty, no epilogue)::
preamble (32 bytes)
* A 20+0+0+0 frame (no epilogue)::
preamble (32 bytes)
segment1 payload (20 bytes)
__le32 segment1 crc
* A 0+70+0+0 frame::
preamble (32 bytes)
segment2 payload (70 bytes)
epilogue (13 bytes)
* A 20+70+0+350 frame::
preamble (32 bytes)
segment1 payload (20 bytes)
__le32 segment1 crc
segment2 payload (70 bytes)
segment4 payload (350 bytes)
epilogue (13 bytes)
where epilogue is::
__u8 late_status
{
__le32 segment crc
} * 3
Hello
-----
* TAG_HELLO: client->server and server->client::
__u8 entity_type
entity_addr_t peer_socket_address
- We immediately share our entity type and the address of the peer (which can be useful
for detecting our effective IP address, especially in the presence of NAT).
Authentication
--------------
* TAG_AUTH_REQUEST: client->server::
__le32 method; // CEPH_AUTH_{NONE, CEPHX, ...}
__le32 num_preferred_modes;
list<__le32> mode // CEPH_CON_MODE_*
method specific payload
* TAG_AUTH_BAD_METHOD server -> client: reject client-selected auth method::
__le32 method
__le32 negative error result code
__le32 num_methods
list<__le32> allowed_methods // CEPH_AUTH_{NONE, CEPHX, ...}
__le32 num_modes
list<__le32> allowed_modes // CEPH_CON_MODE_*
- Returns the attempted auth method, and error code (-EOPNOTSUPP if
the method is unsupported), and the list of allowed authentication
methods.
* TAG_AUTH_REPLY_MORE: server->client::
__le32 len;
method specific payload
* TAG_AUTH_REQUEST_MORE: client->server::
__le32 len;
method specific payload
* TAG_AUTH_DONE: (server->client)::
__le64 global_id
__le32 connection mode // CEPH_CON_MODE_*
method specific payload
- The server is the one to decide authentication has completed and what
the final connection mode will be.
Example of authentication phase interaction when the client uses an
allowed authentication method:
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| auth request |
|---------------->|
|<----------------|
| auth more|
| |
|auth more |
|---------------->|
|<----------------|
| auth done|
Example of authentication phase interaction when the client uses a forbidden
authentication method as the first attempt:
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| auth request |
|---------------->|
|<----------------|
| bad method |
| |
| auth request |
|---------------->|
|<----------------|
| auth more|
| |
| auth more |
|---------------->|
|<----------------|
| auth done|
Post-auth frame format
----------------------
Depending on the negotiated connection mode from TAG_AUTH_DONE, the
connection either stays in crc mode or switches to the corresponding
secure mode (msgr2.0-secure or msgr2.1-secure).
msgr2.0-secure mode
~~~~~~~~~~~~~~~~~~~
A msgr2.0-secure frame has the form::
{
preamble (32 bytes)
{
segment payload
zero padding (out to 16 bytes)
} * number of segments
epilogue (16 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
where epilogue is::
__u8 late_flags
zero padding (15 bytes)
late_flags has the same meaning as in msgr2.0-crc mode.
Each segment and the epilogue are zero padded out to 16 bytes.
Technically, GCM doesn't require any padding because Counter mode
(the C in GCM) essentially turns a block cipher into a stream cipher.
But, if the overall input length is not a multiple of 16 bytes, some
implicit zero padding would occur internally because GHASH function
used by GCM for generating auth tags only works on 16-byte blocks.
Issues
^^^^^^
1. The sender encrypts the whole frame using a single nonce
and generating a single auth tag. Because segment lengths are
stored in the preamble, the receiver has no choice but to decrypt
and interpret the preamble without verifying the auth tag -- it
can't even tell how much to read off the wire to get the auth tag
otherwise! This creates a decryption oracle, which, in conjunction
with Counter mode malleability, could lead to recovery of sensitive
information.
This issue extends to the first segment of the message frame as
well. As in msgr2.0-crc mode, ceph_msg_header2 cannot be safely
interpreted before the whole frame is read off the wire.
2. Deterministic nonce construction with a 4-byte counter field
followed by an 8-byte fixed field is used. The initial values are
taken from the connection secret -- a random byte string generated
during the authentication phase. Because the counter field is
only four bytes long, it can wrap and then repeat in under a day,
leading to GCM nonce reuse and therefore a potential complete
loss of both authenticity and confidentiality for the connection.
This was addressed by disconnecting before the counter repeats
(CVE-2020-1759).
msgr2.1-secure mode
~~~~~~~~~~~~~~~~~~~
Differences from msgr2.0-secure:
1. The preamble, the first segment and the rest of the frame are
encrypted separately, using separate nonces and generating
separate auth tags. This gets rid of unverified plaintext use
and keeps msgr2.1-secure mode close to msgr2.1-crc mode, allowing
the implementation to receive message frames in a similar fashion
(little to no buffering, same scatter/gather logic, etc).
In order to reduce the number of en/decryption operations per
frame, the preamble is grown by a fixed size inline buffer (48
bytes) that the first segment is inlined into, either fully or
partially. The preamble auth tag covers both the preamble and the
inline buffer, so if the first segment is small enough to be fully
inlined, it becomes available after a single decryption operation.
2. As in msgr2.1-crc mode, the epilogue is generated only if the
frame has more than one segment. The rationale is even stronger,
as it would require an extra en/decryption operation.
3. For consistency with msgr2.1-crc mode, late_flags is replaced
with late_status (the built-in bit error detection isn't really
needed in secure mode).
4. In accordance with `NIST Recommendation for GCM`_, deterministic
nonce construction with a 4-byte fixed field followed by an 8-byte
counter field is used. An 8-byte counter field should never repeat
but the nonce reuse protection put in place for msgr2.0-secure mode
is still there.
The initial values are the same as in msgr2.0-secure mode.
.. _`NIST Recommendation for GCM`: https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-38d.pdf
As in msgr2.0-secure mode, each segment is zero padded out to
16 bytes. If the first segment is fully inlined, its padding goes
to the inline buffer. Otherwise, the padding is on the remainder.
The corollary to this is that the inline buffer is consumed in
16-byte chunks.
The unused portion of the inline buffer is zeroed.
Some example frames:
* A 0+0+0+0 frame (empty, nothing to inline, no epilogue)::
{
preamble (32 bytes)
zero padding (48 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
* A 20+0+0+0 frame (first segment fully inlined, no epilogue)::
{
preamble (32 bytes)
segment1 payload (20 bytes)
zero padding (28 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
* A 0+70+0+0 frame (nothing to inline)::
{
preamble (32 bytes)
zero padding (48 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
{
segment2 payload (70 bytes)
zero padding (10 bytes)
epilogue (16 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
* A 20+70+0+350 frame (first segment fully inlined)::
{
preamble (32 bytes)
segment1 payload (20 bytes)
zero padding (28 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
{
segment2 payload (70 bytes)
zero padding (10 bytes)
segment4 payload (350 bytes)
zero padding (2 bytes)
epilogue (16 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
* A 105+0+0+0 frame (first segment partially inlined, no epilogue)::
{
preamble (32 bytes)
segment1 payload (48 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
{
segment1 payload remainder (57 bytes)
zero padding (7 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
* A 105+70+0+350 frame (first segment partially inlined)::
{
preamble (32 bytes)
segment1 payload (48 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
{
segment1 payload remainder (57 bytes)
zero padding (7 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
{
segment2 payload (70 bytes)
zero padding (10 bytes)
segment4 payload (350 bytes)
zero padding (2 bytes)
epilogue (16 bytes)
} ^ AES-128-GCM cipher
auth tag (16 bytes)
where epilogue is::
__u8 late_status
zero padding (15 bytes)
late_status has the same meaning as in msgr2.1-crc mode.
Compression
-----------
Compression handshake is implemented using msgr2 feature-based handshaking.
In this phase, the client will indicate the server if on-wire-compression can be used for message transmitting,
in addition to the list of supported compression methods. If on-wire-compression is enabled for both client and server,
the server will choose a compression method based on client's request and its' own preferences.
Once the handshake is completed, both peers have setup their compression handlers (if desired).
* TAG_COMPRESSION_REQUEST (client->server): declares compression capabilities and requirements::
bool is_compress
std::vector<uint32_t> preferred_methods
- if the client identifies that both peers support compression feature, it initiates the handshake.
- is_compress flag indicates whether the client's configuration is to use compression.
- preferred_methods is a list of compression algorithms that are supported by the client.
* TAG_COMPRESSION_DONE (server->client) : determines on compression settings::
bool is_compress
uint32_t method
- the server determines whether compression is possible according to the configuration.
- if it is possible, it will pick the most prioritized compression method that is also supported by the client.
- if none exists, it will determine that session between the peers will be handled without compression.
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| compression request |
|----------------------->|
|<-----------------------|
| compression done |
msgr2.x-secure mode
~~~~~~~~~~~~~~~~~~~
Combining compression with encryption introduces security implications.
Compression will not be possible when using secure mode, unless configured specifically by an admin.
.. _msgr-post-compression:
Post-compression frame format
-----------------------------
Depending on the negotiated connection mode from TAG_COMPRESSION_DONE, the connection is able to accept/send compressed frames or process all frames as decompressed.
msgr2.x-force mode
~~~~~~~~~~~~~~~~~~
All subsequent frames that will be sent via the connection will be compressed if compression requirements are met (e.g, the frames size).
For compressed frames, the sending peer will enable the FRAME_EARLY_DATA_COMPRESSED flag, thus allowing the accepting peer to detect it and decompress the frame.
msgr2.x-none mode
~~~~~~~~~~~~~~~~~
FRAME_EARLY_DATA_COMPRESSED flag will be disabled in preamble.
Message flow handshake
----------------------
In this phase the peers identify each other and (if desired) reconnect to
an established session.
* TAG_CLIENT_IDENT (client->server): identify ourselves::
__le32 num_addrs
entity_addrvec_t*num_addrs entity addrs
entity_addr_t target entity addr
__le64 gid (numeric part of osd.0, client.123456, ...)
__le64 global_seq
__le64 features supported (CEPH_FEATURE_* bitmask)
__le64 features required (CEPH_FEATURE_* bitmask)
__le64 flags (CEPH_MSG_CONNECT_* bitmask)
__le64 cookie
- client will send first, server will reply with same. if this is a
new session, the client and server can proceed to the message exchange.
- the target addr is who the client is trying to connect *to*, so
that the server side can close the connection if the client is
talking to the wrong daemon.
- type.gid (entity_name_t) is set here, by combining the type shared in the hello
frame with the gid here. this means we don't need it
in the header of every message. it also means that we can't send
messages "from" other entity_name_t's. the current
implementations set this at the top of _send_message etc so this
shouldn't break any existing functionality. implementation will
likely want to mask this against what the authenticated credential
allows.
- cookie is the client cookie used to identify a session, and can be used
to reconnect to an existing session.
- we've dropped the 'protocol_version' field from msgr1
* TAG_IDENT_MISSING_FEATURES (server->client): complain about a TAG_IDENT
with too few features::
__le64 features we require that the peer didn't advertise
* TAG_SERVER_IDENT (server->client): accept client ident and identify server::
__le32 num_addrs
entity_addrvec_t*num_addrs entity addrs
__le64 gid (numeric part of osd.0, client.123456, ...)
__le64 global_seq
__le64 features supported (CEPH_FEATURE_* bitmask)
__le64 features required (CEPH_FEATURE_* bitmask)
__le64 flags (CEPH_MSG_CONNECT_* bitmask)
__le64 cookie
- The server cookie can be used by the client if it is later disconnected
and wants to reconnect and resume the session.
* TAG_RECONNECT (client->server): reconnect to an established session::
__le32 num_addrs
entity_addr_t * num_addrs
__le64 client_cookie
__le64 server_cookie
__le64 global_seq
__le64 connect_seq
__le64 msg_seq (the last msg seq received)
* TAG_RECONNECT_OK (server->client): acknowledge a reconnect attempt::
__le64 msg_seq (last msg seq received)
- once the client receives this, the client can proceed to message exchange.
- once the server sends this, the server can proceed to message exchange.
* TAG_RECONNECT_RETRY_SESSION (server only): fail reconnect due to stale connect_seq
* TAG_RECONNECT_RETRY_GLOBAL (server only): fail reconnect due to stale global_seq
* TAG_RECONNECT_WAIT (server only): fail reconnect due to connect race.
- Indicates that the server is already connecting to the client, and
that direction should win the race. The client should wait for that
connection to complete.
* TAG_RESET_SESSION (server only): ask client to reset session::
__u8 full
- full flag indicates whether peer should do a full reset, i.e., drop
message queue.
Example of failure scenarios:
* First client's client_ident message is lost, and then client reconnects.
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| |
c_cookie(a) | client_ident(a) |
|-------------X |
| |
| client_ident(a) |
|-------------------->|
|<--------------------|
| server_ident(b) | s_cookie(b)
| |
| session established |
| |
* Server's server_ident message is lost, and then client reconnects.
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| |
c_cookie(a) | client_ident(a) |
|-------------------->|
| X------------|
| server_ident(b) | s_cookie(b)
| |
| |
| client_ident(a) |
|-------------------->|
|<--------------------|
| server_ident(c) | s_cookie(c)
| |
| session established |
| |
* Server's server_ident message is lost, and then server reconnects.
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| |
c_cookie(a) | client_ident(a) |
|-------------------->|
| X------------|
| server_ident(b) | s_cookie(b)
| |
| |
| reconnect(a, b) |
|<--------------------|
|-------------------->|
| reset_session(F) |
| |
| client_ident(a) | c_cookie(a)
|<--------------------|
|-------------------->|
s_cookie(c) | server_ident(c) |
| |
* Connection failure after session is established, and then client reconnects.
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| |
c_cookie(a) | session established | s_cookie(b)
|<------------------->|
| X------------|
| |
| reconnect(a, b) |
|-------------------->|
|<--------------------|
| reconnect_ok |
| |
* Connection failure after session is established because server reset,
and then client reconnects.
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| |
c_cookie(a) | session established | s_cookie(b)
|<------------------->|
| X------------| reset
| |
| reconnect(a, b) |
|-------------------->|
|<--------------------|
| reset_session(RC*) |
| |
c_cookie(c) | client_ident(c) |
|-------------------->|
|<--------------------|
| server_ident(d) | s_cookie(d)
| |
RC* means that the reset session full flag depends on the policy.resetcheck
of the connection.
* Connection failure after session is established because client reset,
and then client reconnects.
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| |
c_cookie(a) | session established | s_cookie(b)
|<------------------->|
reset | X------------|
| |
c_cookie(c) | client_ident(c) |
|-------------------->|
|<--------------------| reset if policy.resetcheck
| server_ident(d) | s_cookie(d)
| |
Message exchange
----------------
Once a session is established, we can exchange messages.
* TAG_MSG: a message::
ceph_msg_header2
front
middle
data_pre_padding
data
- The ceph_msg_header2 is modified from ceph_msg_header:
* include an ack_seq. This avoids the need for a TAG_ACK
message most of the time.
* remove the src field, which we now get from the message flow
handshake (TAG_IDENT).
* specifies the data_pre_padding length, which can be used to
adjust the alignment of the data payload. (NOTE: is this is
useful?)
* TAG_ACK: acknowledge receipt of message(s)::
__le64 seq
- This is only used for stateful sessions.
* TAG_KEEPALIVE2: check for connection liveness::
ceph_timespec stamp
- Time stamp is local to sender.
* TAG_KEEPALIVE2_ACK: reply to a keepalive2::
ceph_timestamp stamp
- Time stamp is from the TAG_KEEPALIVE2 we are responding to.
* TAG_CLOSE: terminate a connection
Indicates that a connection should be terminated. This is equivalent
to a hangup or reset (i.e., should trigger ms_handle_reset). It
isn't strictly necessary or useful as we could just disconnect the
TCP connection.
Example of protocol interaction (WIP)
.. ditaa::
+---------+ +--------+
| Client | | Server |
+---------+ +--------+
| send banner |
|----+ +------|
| | | |
| +-------+----->|
| send banner| |
|<-----------+ |
| |
| send new stream |
|------------------>|
| auth request |
|------------------>|
|<------------------|
| bad method |
| |
| auth request |
|------------------>|
|<------------------|
| auth more |
| |
| auth more |
|------------------>|
|<------------------|
| auth done |
| |
.. graphviz:: :caption: client side state machine
digraph lossy_client { node [shape = doublecircle]; "start_connect" "closed"; node [shape = oval]; start_connect -> banner_connecting [label = "<connected>"]; subgraph hello_banner { banner_connecting -> hello_connecting [label = "banner exchange"]; hello_connecting -> banner_connecting [label = "hello exchange"]; label = "hello banner exchange"; color = blue; } banner_connecting -> auth_connecting [label = "<exchange done>"]; auth_connecting -> auth_connecting [label = "auth reply more"]; auth_connecting -> auth_connecting [label = "auth bad method"]; auth_connecting -> auth_connecting_sign [label = "auth done"]; auth_connecting_sign -> session_connecting [label = "auth signature"]; session_connecting -> wait [label = "wait"]; wait -> start_connect [label = "<backoff>"]; session_connecting -> closed [label = "ident missing features"]; session_connecting -> ready [label = "server ident", tooltip = "set peer_name, peer_addr and connection features"]; ready -> ready [label = "keep alive"]; }
.. graphviz:: :caption: server side state machine
digraph lossy_server { node [shape = doublecircle]; "start_accept" "closed"; node [shape = oval]; start_accept -> banner_accepting [label = "<accepted>"]; subgraph hello_banner { banner_accepting -> hello_accepting [label = "banner exchange"]; hello_accepting -> banner_accepting [label = "hello exchange"]; label = "hello banner exchange"; color = blue; }; banner_accepting -> auth_accepting [label = "<exchange done>"]; auth_accepting -> auth_accepting_more [label = "auth_request => 0"]; auth_accepting -> auth_accepting_sign [label = "auth_request => 1"]; auth_accepting_more -> auth_accepting_more [label = "auth_request => 0"]; auth_accepting_more -> auth_accepting_sign [label = "auth_request => 1"]; auth_accepting_more -> standby [label = "auth_request => EBUSY"]; auth_accepting_more -> auth_accepting_more [label = "auth_request => *"]; auth_accepting -> standby [label = "auth_request => EBUSY"]; auth_accepting -> auth_accepting [label = "send <auth bad method>"]; auth_accepting_sign -> session_accepting [label = "auth signature"]; session_accepting -> session_accepting [label = "reconnect"]; session_accepting -> closed [label = "ident missing features"]; session_accepting -> ready [label = "client ident", tooltip = "set connection features"]; ready -> ready [label = "keep alive"]; }
.. _crc:
The CRC algorithm used in this section has the following parameters:
0x1EDC6F410 (for preamble CRC), -1 = 0xFFFFFFFF (for segment data CRC)truetrue000