src/content/docs/internals/architecture.mdx
This document covers some of the internals of Biome, and how they are used inside the project.
Biome has a scanner that is responsible for crawling the file system to extract important metadata about projects. Specifically, there are three ways in which the scanner is used:
biome.json/biome.jsonc files in monorepos..gitignore files if the vcs.useIgnoreFile setting is enabled.package.json manifests as well as source files in a project if any
rules from the project domain are enabled.If project rules are not enabled, the scanner automatically targets only the folders that are relevant for a given session.
This means that if you have a large monorepo, and you run biome check from
inside the packages/foo/ folder, that folder will be "targeted". This means
the following folders get scanned for nested configuration files and/or nested ignore files:
packages/ folder.packages/foo/ folder.packages/foo/, except node_modules/ or
those that are excluded by your configuration (see
below).Other folders that may be adjacent to either packages/ or packages/foo/ will
be automatically skipped.
Similarly, if you run biome format packages/bar/src/index.ts from the root
of the repository, the scanner will target the packages/bar/src/ folder.
If project rules are enabled, these optimisations don't apply.
The scanner can be configured through the
files.includes setting.
The architecture of the parser is bumped by an internal fork of rowan, a library that implements the Green and Red tree pattern.
The CST (Concrete Syntax Tree) is a data structure very similar to an AST (Abstract Syntax Tree) that keeps track of all the information of a program, trivia included.
Trivia is represented by all that information that is important to a program to run:
Trivia is attached to a node. A node can have leading trivia and trailing trivia. If you read code from left to right, leading trivia appears before a keyword, and trialing trivia appears after a keyword.
Leading trivia and trailing trivia are categorized as follows:
Given the following JavaScript snippet, // comment 1 is a trailing trivia of the token ;, and // comment 2 is a leading trivia to the keyword const. Below is a minimized version of the CST represented by Biome:
const a = "foo"; // comment 1
// comment 2
const b = "bar";
0: [email protected]
...
1: [email protected] ";" [] [Whitespace(" "), Comments("// comment 1")]
1: [email protected]
...
1: [email protected] "const" [Newline("\n"), Comments("// comment 2"), Newline("\n")] [Whitespace(" ")]
3: [email protected] "" [] []
The CST is never directly accessible by design; a developer can read its information using the Red tree, using a number of APIs that are autogenerated from the grammar of the language.
In order to construct a CST, a parser needs to be error-resilient and recoverable:
The recoverable part of the parser is not a science, and no rules are set in stone. This means that depending on what the parser was parsing and where an error occurred, the parser might be able to recover itself in an expected way.
The parser also uses' Bogus' nodes to protect the consumers from consuming incorrect syntax. These nodes are used to decorate the broken code caused by a syntax error.
In the following example, the parentheses in the while are missing, although the parser can recover itself in a good manner and can represent the code with a decent CST. The parenthesis and condition of the loop are marked as missing, and the code block is correctly parsed:
while {}
JsModule {
interpreter_token: missing (optional),
directives: JsDirectiveList [],
items: JsModuleItemList [
JsWhileStatement {
while_token: [email protected] "while" [] [Whitespace(" ")],
l_paren_token: missing (required),
test: missing (required),
r_paren_token: missing (required),
body: JsBlockStatement {
l_curly_token: [email protected] "{" [] [],
statements: JsStatementList [],
r_curly_token: [email protected] "}" [] [],
},
},
],
eof_token: [email protected] "" [] [],
}
This is an error emitted during parsing:
main.tsx:1:7 parse ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✖ expected `(` but instead found `{`
> 1 │ while {}
│ ^
ℹ Remove {
The same can't be said for the following snippet. The parser can't properly understand the syntax during the recovery phase, so it needs to rely on the bogus nodes to mark some syntax as erroneous. Notice the JsBogusStatement:
function}
JsModule {
interpreter_token: missing (optional),
directives: JsDirectiveList [],
items: JsModuleItemList [
TsDeclareFunctionDeclaration {
async_token: missing (optional),
function_token: [email protected] "function" [] [],
id: missing (required),
type_parameters: missing (optional),
parameters: missing (required),
return_type_annotation: missing (optional),
semicolon_token: missing (optional),
},
JsBogusStatement {
items: [
[email protected] "}" [] [],
],
},
],
eof_token: [email protected] "" [] [],
}
This is the error we get from the parsing phase:
main.tsx:1:9 parse ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✖ expected a name for the function in a function declaration, but found none
> 1 │ function}
│ ^
:::note Work in progress :::
:::note Work in progress :::
:::note Work in progress :::
Biome uses a server-client architecture to run its tasks.
A daemon is a long-running server that Biome spawns in the background and uses to process requests from the editor and CLI.