vyper/venom/README.md
Venom serves as the next-gen intermediate representation language specifically tailored for use with the Vyper smart contract compiler. Drawing inspiration from LLVM IR, Venom has been adapted to be simpler, and to be architected towards emitting code for stack-based virtual machines. Designed with a Single Static Assignment (SSA) form, Venom allows for sophisticated analysis and optimizations, while accommodating the idiosyncrasies of the EVM architecture.
In Venom, values are denoted as strings commencing with the '%' character, referred to as variables. Variables can only be assigned to at declaration (they remain immutable post-assignment). Constants are represented as decimal numbers (hexadecimal may be added in the future).
Reserved words include all the instruction opcodes and 'IRFunction', 'param', 'dbname' and 'db'.
Any content following the ';' character until the line end is treated as a comment.
For instance, an example of incrementing a variable by one is as follows:
%sum = add %x, 1 ; Add one to x
Each instruction is identified by its opcode and a list of input operands. In cases where an instruction produces a result, it is stored in a new variable, as indicated on the left side of the assignment character.
Code is organized into non-branching instruction blocks, known as "Basic Blocks". Each basic block is defined by a label and contains its set of instructions. The final instruction of a basic block should either be a terminating instruction or a jump (conditional or unconditional) to other block(s).
Basic blocks are grouped into functions that are named and dictate the first block to execute.
A function header may carry an optional bracketed annotation list:
function my_func [fmp_lowered] { ... }
function my_producer [fmp_lowered, fmp_publishes] { ... }
The annotation is the explicit carrier of the FMP (free-memory pointer)
calling-convention facts that are not opcodes (see "Dynamic memory" below):
fmp_lowered declares that the function's FMP convention has been
materialized (its fmp_signature is frozen), and fmp_publishes declares
that every ret carries a hidden adopted-FMP value before the return PC.
Whether the function has a hidden FMP param is not annotated: it is
carried syntactically by the fmp_param opcode. A function containing
lowered FMP artifacts (fmp_param, bump, initial_fmp) without the
annotation is rejected by the input validator; raw IR needs no annotation.
The printer emits the annotation for every function with a frozen
signature, so lowered IR round-trips through the parser.
Venom employs two scopes: global and function level.
function global {
global:
%1 = calldataload 0
%2 = shr 224, %1
jmp @selector_bucket_0
selector_bucket_0:
%3 = xor %2, 1579456981
%4 = iszero %3
jnz %4, @true, @false
false:
jmp @fallback
true:
%5 = callvalue
%6 = calldatasize
%7 = lt 164, %6
%8 = or %7, %5
%9 = iszero %8
assert %9
stop
fallback:
revert 0, 0
}
data readonly {}
To see a definition of grammar see the venom parser
Vyper ships with a venom compiler which compiles venom code to bytecode directly. It can be run by running venom, which is installed as a standalone binary when vyper is installed via pip.
In the current implementation the compiler was extended to incorporate a new pass responsible for translating the original s-expr based IR into Venom. Subsequently, the generated Venom code undergoes processing by the actual Venom compiler, ultimately converting it to assembly code. That final assembly code is then passed to the original assembler of Vyper to produce the executable bytecode.
Currently there is no implementation of the text format (that is, there is no front-end), although this is planned. At this time, Venom IR can only be constructed programmatically.
The Venom implementation is composed of several distinct passes that iteratively transform and optimize the Venom IR code until it reaches the assembly emitter, which produces the stack-based EVM assembly. The compiler is designed to be more-or-less pluggable, so passes can be written without too much knowledge of or dependency on other passes.
These passes encompass generic transformations that streamline the code (such as dead code elimination and normalization), as well as those generating supplementary information about the code, like liveness analysis and control-flow graph (CFG) construction. Some passes may rely on the output of others, requiring a specific execution order. For instance, the code emitter expects the execution of a normalization pass preceding it, and this normalization pass, in turn, requires the augmentation of the Venom IR with code flow information.
The primary categorization of pass types are:
The Venom compiler currently implements the following passes.
The compiler generates a fundamental data structure known as the Control Flow Graph (CFG). This graph illustrates the interconnections between basic blocks, serving as a foundational data structure upon which many subsequent passes depend.
To enable the compiler to analyze the movement of data through the code during execution, a specialized graph, the Dataflow Graph (DFG), is generated. The compiler inspects the code, determining where each variable is defined (in one location) and all the places where it is utilized.
This pass depends on the DFG construction, and reorders variable declarations to try to reduce stack traffic during instruction selection.
This pass conducts a dataflow analysis, utilizing information from previous passes to identify variables that are live at each instruction in the Venom IR code. A variable is deemed live at a particular instruction if it holds a value necessary for future operations. Variables only alive for their assignment instructions are identified here and then eliminated by the dead code elimination pass.
This pass eliminates all basic blocks that are not reachable from any other basic block, leveraging the CFG.
A Venom program may feature basic blocks with multiple CFG inputs and outputs. This currently can occur when multiple blocks conditionally direct control to the same target basic block. We define a Venom IR as "normalized" when it contains no basic blocks that have multiple inputs and outputs. The normalization pass is responsible for converting any Venom IR program to its normalized form. EVM assembly emission operates solely on normalized Venom programs, because the stack layout is not well defined for non-normalized basic blocks.
This final pass of the compiler aims to emit EVM assembly recognized by Vyper's assembler. It calculates the desired stack layout for every basic block, schedules items on the stack and selects instructions. It ensures that deploy code, runtime code, and data segments are arranged according to the assembler's expectations.
A number of passes that are planned to be implemented, or are implemented for immediately after the initial PR merge are below.
An IRContext consists of multiple IRFunctions, with one designated as the main entry point of the program.
Additionally, the IRContext maintains its own representation of the data segment.
An IRFunction is composed of a name and multiple IRBasicBlocks, with one marked as the entry point to the function.
An IRBasicBlock contains a label and a sequence of IRInstructions.
Each IRBasicBlock has a single entry point and exit point.
The exit point must be one of the following terminator instructions:
jmpdjmpjnzretdretretfmpreturnstopexitNormalized basic blocks cannot have multiple predecessors and successors. It has either one (or zero) predecessors and potentially multiple successors or vice versa.
An IRInstruction consists of an opcode, a list of operands, and an optional return value.
An operand can be a label, a variable, or a literal.
By convention, variables have a %- prefix, e.g. %1 is a valid variable. However, the prefix is not required.
To enable Venom IR in Vyper, use the --experimental-codegen CLI flag or the corresponding pragma statements (e.g. #pragma experimental-codegen). To view the Venom IR output, use -f ir_runtime for the runtime code, or -f ir to see the deploy code. To get a dot file (for use e.g. with xdot -), use -f cfg or -f cfg_runtime.
Assembly can be inspected with -f asm, whereas an opcode view of the final bytecode can be seen with -f opcodes or -f opcodes_runtime, respectively.
dalloca
%ptr = dalloca %size
ceil32(size) bytes and returns its base pointer.
The producer does not receive a restore token and should not emit a release instruction:
the reclaim contract is leak-until-ret. Plain ret is callee-save (the FMP implicitly
reverts to its value at function entry), so any allocation that cannot be proven dead is
simply left allocated until return -- always sound, never required to be reclaimed.
FmpLoweringPass runs after SSA, threads the free-memory pointer explicitly,
and may synthesize conservative LIFO rewinds when the allocation and all aliases are
provably dead and unescaped (escaped pointers pin their allocation, fail-closed).
It is the single owner of the hidden-FMP calling convention: it materializes the hidden
fmp_param (normalizing a discovered plain return-PC param to retpc_param), seeds the
entry function's FMP root with an explicit initial_fmp instruction, appends hidden
invoke operands (assert-and-set) and freezes the resulting shape as the function's
fmp_signature (printed as the [fmp_lowered(, fmp_publishes)?] function-header
annotation). The deletion-only FmpPrunePass runs after the optimization tail and
deletes a hidden FMP param whose use chain died, resealing the signature before any
caller is lowered (the pass driver is callee-first).dret
dret dyn_count, <ordinary returns...>, src0, size0, ..., return_pc
2 * dyn_count operands before return_pc
are (src, size) pairs. DretDesugarPass runs before inlining and locally desugars the
terminator into FMP virtual-register IR: pack destinations computed off getfmp, the pack
copies, setfmp (advancing the register over the packed data) and a retfmp terminator.
FmpLoweringPass later threads the register and materializes the physical convention
(hidden FMP param/operand, ret with the hidden adopted-FMP value).DretDesugarPass and FmpLoweringPass)
%v = getfmp reads the FMP virtual register; setfmp %v writes it.retfmp <ordinary returns...>, <packed dst ptrs...>, return_pc
ret is callee-save (FMP implicitly reverts
to its value at entry), so a function whose terminators are all ret does not publish even
if its body contains setfmp (e.g. after inlining a publishing callee).
FmpLoweringPass lowers retfmp to ret <values...>, <adopted FMP>, return_pc; the
lowered publish fact is then carried by the fmp_publishes annotation token, and the
caller's invoke binds the adopted FMP as a hidden extra output.invoke
invoke offset, @label
label.offset address.JUMP, and marks the call site as a valid return destination (for callee to jump back to) by JUMPDEST.alloca
%out = alloca size
size.ConcretizeMemLocPass).alloca instruction is meant to simplify that.iload
%out = iload offset
offset into out variable.MLOAD on an immutable section of memory. So, for example
%op = 12
%out = iload %op
PUSH1 12 _mem_deploy_end ADD MLOAD.offset is a literal the location is computed statically during compilation from assembly to bytecode.istore
istore offset value
iload, the offset operand can be a literal.MSTORE on an immutable section of memory. For example,
%op = 12
istore 24 %op
PUSH1 12 PUSH1 24 _mem_deploy_end ADD MSTORE.phi
%out = phi @label_a, %var_a, @label_b, %var_b
phi instructions. They are are magic instructions, used in basic blocks where the control flow path merges.%out variable is set to %var_a if the program entered the current block from @label_a or to %var_b when it went through @label_b. Note that %var_a% must be defined in the @label_a block and %var_b must be defined in the @label_b block.offset
%ret = offset @label, op
mstore, mload and such.@label + op.asm output could show something like _OFST _sym_<op> label.param
%out = param
param instruction is used to represent function arguments passed by the stack.param instruction is used to ensure we represent the argument by the out variable.param instructions are exactly the user params; the hidden calling-convention
slots have dedicated opcodes (below). Callee params are laid out as
[user params..., fmp_param?, retpc_param?], matching invoke operands
[target, user args..., hidden_fmp?] plus the return PC pushed by invoke itself.retpc_param
%out = retpc_param
ret are self-describing. In hand-written raw IR a plain
param may serve as the return PC instead; there it is defined as the unique param the
last operand of ret/dret/retfmp aliases (ret-anchored discovery -- the raw-level
definition, not a heuristic). Lowered ([fmp_lowered]-annotated) functions must use the
dedicated opcode. Assembles identically to param (no code emitted).fmp_param
%out = fmp_param
FmpLoweringPass; its presence is the has_fmp_param fact of the function's
fmp_signature. Only legal in [fmp_lowered]-annotated functions, at most once, in the
entry block, after all plain params. Assembles identically to param.bump
%ptr, %fmp_out = bump %size, %fmp_in
%ptr is the pre-bump FMP (the allocation's
base pointer and reclaim mark) and %fmp_out = %fmp_in + %size is the advanced FMP.
Created only by FmpLoweringPass (from dalloca, after ceil32-aligning the size);
%fmp_in must be FMP-rooted (validated post-lowering). Pure stack arithmetic
(assembles to DUP2 ADD), but never idempotent: each bump is a distinct allocation,
so two bumps must not be CSE-merged even with identical operands.initial_fmp
%out = initial_fmp
fmp_param instead. The concrete value is only
known once spill analysis completes, so it assembles to a PUSH of an assembler-level
CONST resolved at assembly time. Pure: repeated initial_fmp instructions may CSE
together.store
%out = op
out variable.dbname
dbname label
label in the data segment so it can be referenced.db
db data
data into data segment.dloadbytes
codecopy for legacy reasons. May be removed in future versions.CODECOPY.ret
ret op
op.op is a label it can effectively translate into op JUMP.exit
exit
stop, but used for constructor exit. The assembler is expected to jump to a special initcode sequence which returns the runtime code._sym__ctor_exit JUMP.assert
assert op
op is zero. If it is not, revert.%op = 13
assert %op
PUSH1 13 ISZERO _sym___revert JUMPI.assert_unreachable
assert_unreachable op
op is zero. If it is not, terminate with 0xFE ("INVALID" opcode).op reachable JUMPI INVALID reachable JUMPDEST.%op = 13
assert_unreachable %op
PUSH1 13 _sym_reachable1 JUMPI
INVALID
_sym_reachable1 JUMPDEST
log
log offset, size, [topic] * topic_count , topic_count
LOGX instruction in EVM.topic_count value (which can be only from 0 to 4) translates to LOG0 ... LOG4.LOGX instructions.log %53, 32, 64, %56, 2
%56, 64, 32, %53 LOG2
nop
nop
offset
%2 = offset %1 @label1
add, but takes a label as the second argument. If the first argument is a literal, the addition will get optimized at assembly time.jmp
jmp @label
label.label JUMP.jnz
jnz op, @label1, @label2
op.label2 when op is not zero, otherwise jumps to label1.%op = 15
jnz %op, @label1, @label2
PUSH1 15 label2 JUMPI label1 JUMP.djmp
djmp %var, @label1, @label2, ..., @labeln
JUMP.The following instructions map one-to-one with EVM instructions. Operands correspond to stack inputs in the same order. Stack outputs are the instruction's output. Instructions have the same effects.
returnrevertcoinbasecalldatasizecalldatacopymcopycalldataloadgasgaspricegaslimitchainidaddressoriginnumberextcodesizeextcodehashextcodecopyreturndatasizereturndatacopycallvalueselfbalancesloadsstoremloadmstoretloadtstoretimestampcallerblockhashselfdestructsignextendstopshrshlsarandxororaddsubmuldivsmulsdivmodsmodexpaddmodmulmodeqiszeronotltgtsltsgtcreatecreate2balancecallstaticcalldelegatecallcodesizebasefeeblobhashblobbasefeeprevrandaodifficultyinvalidsha3