Review of other codebases

This document contains some of our research on how other codebases designed various parts of their stack.

P2P

• Sentry, a pluggable p2p node following the Erigon gRPC architecture:

  • vorot93 first started by implementing a rust devp2p stack in devp2p
  • vorot93 then started work on sentry, using devp2p, to satisfy the erigon architecture of modular components connected with gRPC.
  • The code from rust-ethereum/devp2p was merged into sentry, and rust-ethereum/devp2p was archived
  • vorot93 was working concurrently on akula, which used sentry to connect to the network.
  • The code from sentry was merged into akula, and sentry was deleted (hence the 404 on the sentry link)

• Ranger, an ethereum p2p client capable of interacting with peers without a full node:

  • Rjected built Ranger for p2p network research
  • It required extracting devp2p-rs from Akula's sentry directory to a separate repository (keeping the GPL License), so that it could be used in Ranger (also GPL Licensed)
  • It also required creating ethp2p, a clean room implementation of the eth wire protocol, for use in ranger.

Database

• Erigon's DB walkthrough contains an overview. They made the most noticeable improvements in storage reduction.
• Gio's erigon-db table macros + Akula's macros.

Header Downloaders

• Erigon Header Downloader:

  • A header downloader algorithm was introduced in erigon#1016 and finished in erigon#1145. At a high level, the downloader concurrently requested headers by hash, then sorted, validated and fused the responses into chain segments. Smaller segments were fused into larger as the gaps between them were filled. The downloader is also used to maintain hardcoded hashes (later renamed to preverified) to bootstrap the sync.
  • The downloader was refactored multiple times: erigon#1471, erigon#1559 and erigon#2035.
  • With PoS transition in erigon#3075 terminal td was introduced to the algorithm to stop forward syncing. For the downward sync (post merge), the downloader was now delegated to EthBackendServer
  • Proper reverse PoS downloader was introduced in erigon#3092 which downloads the header batches from tip until local head is reached. Refactored later in erigon#3340 and erigon#3717.

• Akula Headers & Stage Downloader:

  • What seems to be the first working version was wired together in akula#89. The headers stage invoked the downloader which created a staged stream for header download. The rough description of the process would be request -> receive response -> retry if necessary -> verify response -> verify attachment -> save -> refill (flush?). Same as erigon, it used to rely on preverified hashes to bootstrap the download process.
  • A Concurrent downloader was introduced in akula#bbde8d. It dispatched multiple requests, collected & verified responses and afterwards inserted the headers. The same logic was refactored in akula#38381e.
  • Watching chain tip changes was introduced in akula#cdc083.
  • Proper consensus engine together with reverse download was introduced in akula#fcc1a08. The majority of code from this commit is akula's headers stage as we know it today.