Semantic highlighting

LSP:    [textDocument/semanticTokens/*][] requests
Legacy: `analysis.highlights` notification

Semantic highlighting makes it easier for the user to understand the code by making certain syntactic and semantic aspects of the code more visible. The client typically uses this information to apply text color or other highlighting to the code being shown in the editor.

The design of the feature is largely dictated by the protocol, but there are some areas where we have some flexibility. This document will cover those areas.

Syntactic highlighting

Syntactic highlighting covers highlighting that is based purely on the syntax of the language. For example, the highlighting of string literals or comments is part of syntactic highlighting.

Handling keywords

There is one question related to syntactic highlighting: the handling of keywords. Dart has three categories of identifiers with special semantics, and we needed to decide whether to explicitly represent these three categories of identifiers or whether to ignore the distinctions. It could be argued that ignoring the distinction would violate the principle of language fidelity. It could also be argued that making the distinction might be confusing for the user.

In the end, we decided that the better interpretation of the language spec is that there are two important categories: identifiers that are functioning as a keyword and identifiers that are not. As a result, we treat all of these as keywords whenever they serve the function of a keyword, and treat them as identifiers when they don't.

The `new` keyword in constructor declarations

For the case where the new is introducing a constructor, it's acting as a keyword, similar to class, that indicates that what follows is a constructor declaration.

For the case where .new follows a class name, it's not acting as a keyword, but as an alternative to omitting the name. In those cases it should be highlighted as an identifier.