developers.diem.com/docs/transactions/basics-life-of-txn.md
To get a deeper understanding of the lifecycle of a Diem transaction (from an operational perspective), we will follow a transaction on its journey from being submitted to a Diem node to being committed to the Diem Blockchain. We will then “zoom-in” on the logical components of Diem nodes and take a look at its interactions with other components.
For the purpose of this doc, we will assume that:
A Diem client constructs a raw transaction (let's call it T<sub>5</sub>raw) to transfer 10 Diem Coins from Alice’s account to Bob’s account. The Diem client signs the transaction with Alice's private key. The signed transaction T<sub>5</sub> includes the following:
The raw transaction includes the following fields:
| Fields | Description |
|---|---|
| account address | Alice's account address |
| Move Module | A module (or program) that indicates the actions to be performed on Alice's behalf. In this case, it contains: |
In this section, we will describe the lifecycle of transaction T<sub>5</sub>, from when the client submits it to when it is committed to the Diem Blockchain.
For the relevant steps, we've included a link to the corresponding inter-component interactions of the validator node. After you are familiar with all the steps in the lifecycle of the transaction, you may want to refer to the information on the corresponding inter-component interactions for each step. <small className="figure">Figure 1.0 Lifecycle of a transaction</small>
<BlockQuote type="warning"> The arrows in all the visuals in this article originate on the component initiating an interaction/action and terminate on the component on which the action is being performed. The arrows do not represent data read, written, or returned. </BlockQuote>The lifecycle of a transaction has five stages:
We've described what happens in each stage below, along with links to the corresponding Diem node component interactions.
| Description | Diem Node Component Interactions |
|---|---|
| 1. Client → JSON-RPC service: The client submits transaction T<sub>5</sub> to the JSON-RPC service of a Diem FullNode. FullNodes use the JSON-RPC service to forward the transaction to their own mempools which forward the transaction to upstream mempool services running on validator FullNodes which in turn forward to the validator node V<sub>1</sub>. | 1. JSON-RPC |
| 2. JSON-RPC service → Mempool: The FullNode's JSON-RPC service transmits transaction T<sub>5</sub> to validator V<sub>1</sub>'s mempool. | 2. JSON-RPC, 1. Mempool |
| 3. Mempool → Virtual Machine: Mempool will use the virtual machine (VM) component to perform validation checks, such as signature verification, checking that Alice's account has sufficient balance, checking that transaction T<sub>5</sub> is not being replayed using the sequence number, and so on. | 4. Mempool, 3. Virtual Machine |
| Description | Diem Node Component Interactions |
|---|---|
| 4. Mempool: The mempool will hold T<sub>5</sub> in an in-memory buffer. Mempool may already contain multiple transactions sent from Alice's address. | Mempool |
| 5. Mempool → Other Validators: Using the shared-mempool protocol, V<sub>1</sub> will share the transactions (including T<sub>5</sub>) in its mempool with other validator nodes and place transactions received from them into its own (V<sub>1</sub>) mempool. | 2. Mempool |
| Description | Diem Node Component Interactions |
|---|---|
| 6. Consensus → Mempool: — As validator V<sub>1</sub> is a proposer/leader for this transaction, it will pull a block of transactions from its mempool and replicate this block as a proposal to other validator nodes via its consensus component. | 1. Consensus, 3. Mempool |
| 7. Consensus → Other Validators: The consensus component of V<sub>1</sub> is responsible for coordinating agreement among all validators on the order of transactions in the proposed block. Refer to our technical paper State Machine Replication in the Diem Blockchain for details of our proposed consensus protocol DiemBFT. | 2. Consensus |
| Description | Diem Node Component Interactions |
|---|---|
| 8. Consensus → Execution: As part of reaching agreement, the block of transactions (containing T<sub>5</sub>) is shared with the execution component. | 3. Consensus, 1. Execution |
| 9. Execution → Virtual Machine: The execution component manages the execution of transactions in the virtual machine (VM). Note that this execution happens speculatively before the transactions in the block have been agreed upon. | 2. Execution, 3. Virtual Machine |
| 10. Consensus → Execution: After executing the transactions in the block, the execution component appends the transactions in the block (including T<sub>5</sub>) to the Merkle accumulator (of the ledger history). This is an in-memory/temporary version of the Merkle accumulator. The necessary part of the proposed/speculative result of executing these transactions is returned to the consensus component to agree on. The arrow from "consensus" to "execution" indicates that the request to execute transactions was made by the consensus component. (For consistent use of arrows throughout this article, the arrows do not represent the flow of data). | 3. Consensus, 1. Execution |
| 11. Consensus → Other Validators: V<sub>1</sub> (the consensus leader) attempts to reach consensus on the proposed block's execution result with the other validator nodes participating in consensus. | 3. Consensus |
| Description | Diem Node Component Interactions |
|---|---|
| 12. Consensus → Execution, **Execution → Storage **:If the proposed block's execution result is agreed upon and signed by a set of validators that have the quorum of votes, validator V<sub>1</sub>'s execution component reads the full result of the proposed block execution from the speculative execution cache and commits all the transactions in the proposed block to persistent storage with their results. | 4. Consensus, 3. Execution, 4. Execution, 3. Storage |
Alice's account will now have 100 Diem Coins, and its sequence number will be 6. If T<sub>5</sub> is replayed by Bob, it will be rejected as the sequence number of Alice's account (6) is greater than the sequence number of the replayed transaction (5).
In the previous section, we described the typical lifecycle of a sample transaction from being submitted to being committed to the Diem Blockchain's distributed database. Now let's look in more depth at the inter-component interactions of Diem nodes as the blockchain processes transactions and responds to queries. This information will be most useful to those who:
You can learn more about the different types of Diem nodes here:
For our narrative, we will assume that a client submits a transaction T<sub>N</sub> to a validator V<sub>X</sub>. For each validator component, we will describe each of its inter-component interactions in subsections under the respective component's section. Note that subsections describing the inter-component interactions are not listed strictly in the order in which they are performed. Most of the interactions are relevant to the processing of a transaction, and some are relevant to clients querying the blockchain (queries for existing information on the blockchain).
The following are the core logical components of a Diem node used in the lifecycle of a transaction:
FullNode
Validator node
<small className="figure">Figure 1.1 JSON-RPC Service</small>
Any request made by a client goes to the JSON-RPC Service of a FullNode first. Then, the submitted transaction is forwarded to the validator FullNode, which then sends it to the validator node V<sub>X</sub>.
A client submits a transaction to the JSON-RPC service of a Diem FullNode.
The JSON-RPC service forwards the transaction to validator FullNode, which then sends it to validator node V<sub>X</sub>'s mempool. The mempool will accept the transaction T<sub>N</sub> only if the sequence number of T<sub>N</sub> is greater than or equal to the current sequence number of the sender's account (note that the transaction will not be passed to consensus until the sequence number matches the sequence number of the sender’s account).
When a client performs a read query on the Diem Blockchain (for example, to get the balance of Alice's account), the JSON-RPC service interacts with the storage component directly to obtain the requested information.
<small className="figure">Figure 1.2 Virtual machine</small>
The Move virtual machine (VM) verifies and executes transaction scripts written in Move bytecode.
When mempool requests the VM to validate a transaction via VM::ValidateTransaction(), the VM loads the transaction sender's account from storage and performs verifications, some of which have been described in the list below. View the entire list of verifications here.
The execution component utilizes the VM to execute a transaction via VM::ExecuteTransaction().
It is important to understand that executing a transaction is different from updating the state of the ledger and persisting the results in storage. A transaction T<sub>N</sub> is first executed as part of an attempt to reach agreement on blocks during consensus. If agreement is reached with the other validators on the ordering of transactions and their execution results, the results are persisted in storage and the state of the ledger is updated.
When mempool receives a transaction from other validators via shared mempool or from the JSON-RPC service, mempool invokes VM::ValidateTransaction() on the VM to validate the transaction.
For implementation details refer to the Virtual Machine README.
<small className="figure">Figure 1.3 Mempool</small>
Mempool is a shared buffer that holds the transactions that are “waiting” to be executed. When a new transaction is added to the mempool, the mempool shares this transaction with other validator nodes in the system. To reduce network consumption in the “shared mempool,” each validator is responsible for delivering its own transactions to other validators. When a validator receives a transaction from the mempool of another validator, the transaction is added to the mempool of the recipient validator.
When mempool receives a transaction from other validators, mempool invokes <code>VM::ValidateTransaction()</code> on the VM to validate the transaction.
For implementation details refer to the Mempool README.
<small className="figure">Figure 1.4 Consensus</small>
The consensus component (consensus) is responsible for ordering blocks of transactions and agreeing on the results of execution by participating in the consensus protocol with other validators in the network.
When validator V<sub>X</sub> is a leader/proposer, the consensus component of V<sub>X</sub> pulls a block of transactions from its mempool via: Mempool::GetBlock(), and forms a proposed block of transactions.
If V<sub>X</sub> is a proposer/leader, its consensus component replicates the proposed block of transactions to other validators.
Execution:ExecuteBlock() (Refer to Consensus → executionIf enough validators vote for the same execution result, the consensus component of V<sub>X</sub> informs execution via Execution::CommitBlock() that this block is ready to be committed.
For implementation details refer to the Consensus README.
<small className="figure">Figure 1.5 Execution</small>
The execution component coordinates the execution of a block of transactions and maintains a transient state that can be voted upon by consensus. If these transactions are successful, they are committed to storage.
Execution::ExecuteBlock().When consensus requests execution to execute a block of transactions via Execution::ExecuteBlock(), execution uses the VM to determine the results of executing the block of transactions.
If a quorum of validators agrees on the block execution results, the consensus component of each validator informs its execution component via Execution::CommitBlock() that this block is ready to be committed. This call to the execution component will include the signatures of the agreeing validators to provide proof of their agreement.
Execution takes the values from its “scratchpad” and sends them to storage for persistence via Storage::SaveTransactions(). Execution then prunes the old values from the “scratchpad” that are no longer needed (for example, parallel blocks that cannot be committed).
For implementation details refer to the Execution README.
<small className="figure">Figure 1.6 Storage</small>
The storage component persists agreed upon blocks of transactions and their execution results to the Diem Blockchain. A block of transactions (which includes transaction T<sub>N</sub>) will be saved via storage when:
Refer to Merkle accumulator for information on how a transaction is appended to the data structure representing the Diem Blockchain.
When AC or mempool invokes VM::ValidateTransaction() to validate a transaction, VM::ValidateTransaction() loads the sender's account from storage and performs the read-only validity checks on the transaction.
When the consensus component calls Execution::ExecuteBlock(), execution reads the current state from storage combined with the in-memory “scratchpad” data to determine the execution results.
Storage::SaveTransactions() to save the block of transactions and permanently record them. This will also store the signatures from the validator nodes that agreed on this block of transactions.For client queries that read information from the blockchain, the JSON-RPC service directly interacts with storage to read the requested information.
For implementation details refer to the Storage README.