ContextQMD
Back to Chromium

Web IDL compiler (web_idl package)

third_party/blink/renderer/bindings/scripts/web_idl/README.md

149.0.7827.29.0 KB
Original Source

Web IDL compiler (web_idl package)

[TOC]

What's web_idl?

Python package web_idl is the core part of Web IDL compiler of Blink. build_web_idl_database.py is the driver script, which takes a set of ASTs of *.idl files as inputs and produces a database(-ish) pickle file that contains all information about spec-author defined types (IDL interface, IDL dictionary, etc.). The database file will be used as the input of Blink-V8 bindings code generator.

Quick walk in web_idl_database.pickle

The following snippet of shell commands and Python scripts demonstrates how you can build and use the database file (web_idl_database.pickle) and what you can do with the database file just as an example.

shell
# Build the database file `web_idl_database.pickle`.
# output: out/Default/gen/third_party/blink/renderer/bindings/web_idl_database.pickle
$ autoninja -C out/Default web_idl_database

# Play with the produced database.
$ PYTHONPATH=third_party/blink/renderer/bindings/scripts:$PYTHONPATH python3
>>> import web_idl
>>> web_idl_database_path = 'out/Default/gen/third_party/blink/renderer/bindings/web_idl_database.pickle'
>>> web_idl_database = web_idl.Database.read_from_file(web_idl_database_path)

# Print all IDL attributes whose type is boolean (but not nullable boolean).
>>> for interface in web_idl_database.interfaces:
...   for attribute in interface.attributes:
...     if attribute.idl_type.is_boolean:
...       print("{}.{}".format(interface.identifier, attribute.identifier))

# Print all IDL dictionary members which are required members.
>>> for dictionary in web_idl_database.dictionaries:
...   for member in dictionary.own_members:
...     if member.is_required:
...       print("{}.{}".format(dictionary.identifier, member.identifier))

# Print API references of IDL dictionary class.
>>> help(web_idl.Dictionary)

# Print API references of a certain object (Window interface in this case).
>>> window = web_idl_database.find('Window')
>>> help(window)

Preprocessing of *.idl files

*.idl files are preprocessed by collect_idl_files.py before the main IDL compiler starts processing. There are two major purposes.

  1. To transform the text files (*.idl) into ASTs (abstract syntax trees). The *.idl files are parsed by //tools/idl_parser/, which is a Blink-independent IDL parser. The resulting ASTs are represented as trees of idl_parser.idl_node.IDLNode.

  2. To group the ASTs by component. The resulting ASTs are grouped by component (core/, modules/, etc.) into web_idl.AstGroup.

The web_idl.AstGroups are saved in intermediate *.pickle files. See also collect_idl_files rules in BUILD.gn.

Overview of compilation flow of *.idl files

The goal of Web IDL compilation is to produce Python objects that represent Web IDL definitions and to save them into a single *.pickle file (web_idl_database.pickle). These Python objects are designed to be immutable as much as possible (they have no setter methods, and their getter methods return tuples rather than lists, for example).

Very roughly speaking, the compilation flow of Web IDL files are as below.

AST nodes ==> IRs (mutable) ==> public (final immutable) objects

AST nodes are represented in idl_parser.idl_node.IDLNode, IR objects are represented in web_idl.Interface.IR, web_idl.Dictionary.IR, etc., and public objects are represented in web_idl.Interface, web_idl.Dictionary, etc.

Step 1. Read AST nodes and build IRs

_IRBuilder.build_top_level_def takes an AST node of a top-level definition and returns an IR. The structure of AST nodes is very specific to each Web IDL definition, so each _IRBuilder._build_xxxx is also specific to Web IDL definition.

In _IRBuilder, two factory functions are used: _create_ref_to_idl_def and _idl_type_factory.

_create_ref_to_idl_def is a function that creates a web_idl.RefById, which is a placeholder of a (final immutable) public object. In the middle of compilation, we don't have any public object yet, so we need a placeholder which will be later resolved to a reference to a public object. For example, when obj.attr is a RefById to "my_object", then obj.attr.foo behaves the same as x.foo where x is a public object with identifier "my_object".

_idl_type_factory is a factory object that creates a (subclass of) web_idl.IdlType. Unlike IRs, web_idl.IdlType is designed to be (almost) immutable (although there are some exceptions) so there is no web_idl.IdlType.IR.

These two factories are used in order to track all the instances of web_idl.RefById and web_idl.IdlType respectively (each factory has a for_each method). For example, the IDL compiler replaces placeholders with (final immutable) public objects. The IDL compiler also creates web_idl.Union objects, which represent Blink's IDL union objects, based on all instances of web_idl.IdlType.

Step 2. Apply changes on IRs in each compilation phase

The IDL compiler has multiple "compilation phases" in order to process the IDL definitions step by step. For example, (1) apply a partial interface's [RuntimeEnabled] to each member, and then (2) copy the partial interface's members to the primary interface definition; these two steps make interface members have appropriate extended attributes propagated from partial interface definitions. This is just an example of two compilation phases; there actually exist many compilation phases.

webidl
interface MyInterface {
  attribute DOMString name;
};

[RuntimeEnabled=Foo] partial interface MyInterface {
  attribute DOMString nickname;
};

// The IDL compiler turns the above definitions into the following in two
// compilation phases.
interface MyInterface {
  attribute DOMString name;
  [RuntimeEnabled=Foo] attribute DOMString nickname;
};

At each compilation phase, a set of new IRs is produced from the current IRs. The new IRs are saved in an IRMap associated with the compilation phase number. By using the IRMap, a compilation phase runs in the following way.

  1. Reads IRs from the IRMap and the current compilation phase number. These IRs are called old_ir in IdlCompiler.
  2. Increments the compilation phase number.
  3. Creates copies of the old IRs (so that we don't overwrite the existing IRs). The copies are called new_ir in IdlCompiler.
  4. Makes changes to the new IRs. The new IRs have a new state.
  5. Saves the new IRs in the IRMap associated with the (incremented) compilation phase number.

A compilation phase consists of these steps, and the IDL compiler runs as many compilation phases as needed. You can see the list of compilation phases at IdlCompiler.build_database.

The initial set of IRs is constructed by _IRBuilder.build_top_level_def (as explained above) and registered in the IRMap.

Step 3. Create the public objects

At the very last compilation phase, the public objects (which are final, immutable, and exposed to users of this web_idl module) are constructed from the last IRs, and registered to web_idl.Database. The Database is saved as a pickle file, whose name by default is out/Default/gen/third_party/blink/renderer/bindings/web_idl_database.pickle. It can be read via web_idl.Database.read_from_file.