UPGRADE-5.0.md
PHP-Parser now requires PHP 7.4 or newer to run. It is however still possible to parse code for older versions, while running on a newer version.
The dedicated parser for PHP 5 has been removed. The PHP 7 parser now accepts a PhpVersion argument, which can be used to improve compatibility with older PHP versions.
In particular, if an older PhpVersion is specified, then:
$foo =& new Bar() assignments are allowed without error.089 are allowed without error.\u{123} in strings are not parsed.Name or as a built-in Identifier depending on PHP
version, for example int is treated as a class name on PHP 5.6 and as a built-in on PHP 7.0.However, some aspects of PHP 5 parsing are no longer supported:
$$foo[0] are valid in both PHP 5 and PHP 7, but have different interpretation. In that case, the PHP 7 AST will always be constructed (($$foo)[0] rather than ${$foo[0]}).global $$var[0] are not supported in PHP 7 and will cause a parse error. In error recovery mode, it is possible to continue parsing after such declarations.The following symbols are affected by this removal:
PhpParser\Parser\Php5 class has been removed.PhpParser\Parser\Multiple class has been removed. While not strictly related to PHP 5 support, this functionality is no longer useful without it.PhpParser\ParserFactory::ONLY_PHP5 and PREFER_PHP5 options have been removed.The ParserFactory::create() method has been removed in favor of three new methods that provide more fine-grained control over the PHP version being targeted:
createForNewestSupportedVersion(): Use this if you don't know the PHP version of the code you're parsing. It's better to assume a too new version than a too old one.createForHostVersion(): Use this if you're parsing code for the PHP version you're running on.createForVersion(): Use this if you know the PHP version of the code you want to parse.The createForNewestSupportedVersion() and createForHostVersion() are available since PHP-Parser 4.18.0, to allow libraries to support PHP-Parser 4 and 5 at the same time more easily.
In all cases, the PHP version is a fairly weak hint that is only used on a best-effort basis. The parser will usually accept code for newer versions if it does not have any backwards-compatibility implications.
For example, if you specify version "8.0", then class ReadOnly {} is treated as a valid class declaration, while using public readonly int $prop will lead to a parse error. However, final public const X = Y; will be accepted in both cases.
use PhpParser\ParserFactory;
use PhpParser\PhpVersion;
$factory = new ParserFactory();
# Before
$parser = $factory->create(ParserFactory::PREFER_PHP7);
# After (this is roughly equivalent to PREFER_PHP7 behavior)
$parser = $factory->createForNewestSupportedVersion();
# Or
$parser = $factory->createForHostVersion();
# Before
$parser = $factory->create(ParserFactory::ONLY_PHP5);
# After (supported on a best-effort basis)
$parser = $factory->createForVersion(PhpVersion::fromString("5.6"));
Previously, throw statements like throw $e; were represented using the Stmt\Throw_ class,
while uses inside other expressions (such as $x ?? throw $e) used the Expr\Throw_ class.
Now, throw $e; is represented as a Stmt\Expression that contains an Expr\Throw_. The
Stmt\Throw_ class has been removed.
# Code
throw $e;
# Before
Stmt_Throw(
expr: Expr_Variable(
name: e
)
)
# After
Stmt_Expression(
expr: Expr_Throw(
expr: Expr_Variable(
name: e
)
)
)
Previously, the list($x) = $y destructuring syntax was represented using a Node\Expr\List_
node, while [$x] = $y used a Node\Expr\Array_ node, the same used for the creation (rather than
destructuring) of arrays.
Now, destructuring is always represented using Node\Expr\List_. The kind attribute with value
Node\Expr\List_::KIND_LIST or Node\Expr\List_::KIND_ARRAY specifies which syntax was actually
used.
# Code
[$x] = $y;
# Before
Expr_Assign(
var: Expr_Array(
items: array(
0: Expr_ArrayItem(
key: null
value: Expr_Variable(
name: x
)
byRef: false
unpack: false
)
)
)
expr: Expr_Variable(
name: y
)
)
# After
Expr_Assign(
var: Expr_List(
items: array(
0: ArrayItem(
key: null
value: Expr_Variable(
name: x
)
byRef: false
unpack: false
)
)
)
expr: Expr_Variable(
name: y
)
)
Previously, Name nodes had a parts subnode, which stores an array of name parts, split by
namespace separators. Now, Name nodes instead have a name subnode, which stores a plain string.
For example, the name Foo\Bar was previously represented by Name(parts: ['Foo', 'Bar']) and is
now represented by Name(name: 'Foo\Bar') instead.
It is possible to convert the name to the previous representation using $name->getParts(). The
Name constructor continues to accept both the string and the array representation.
The Name::getParts() method is available since PHP-Parser 4.16.0, to allow libraries to support
PHP-Parser 4 and 5 at the same time more easily.
Previously, code blocks { ... } were always flattened into their parent statement list. For
example while ($x) { $a; { $b; } $c; } would produce the same node structure as
if ($x) { $a; $b; $c; }, namely a Stmt\While_ node whose stmts subnode is an array of three
statements.
Now, the nested { $b; } block is represented using an explicit Stmt\Block node. However, the
outer { $a; { $b; } $c; } block is still represented using a simple array in the stmts subnode.
# Code
while ($x) { $a; { $b; } $c; }
# Before
Stmt_While(
cond: Expr_Variable(
name: x
)
stmts: array(
0: Stmt_Expression(
expr: Expr_Variable(
name: a
)
)
1: Stmt_Expression(
expr: Expr_Variable(
name: b
)
)
2: Stmt_Expression(
expr: Expr_Variable(
name: c
)
)
)
)
# After
Stmt_While(
cond: Expr_Variable(
name: x
)
stmts: array(
0: Stmt_Expression(
expr: Expr_Variable(
name: a
)
)
1: Stmt_Block(
stmts: array(
0: Stmt_Expression(
expr: Expr_Variable(
name: b
)
)
)
)
2: Stmt_Expression(
expr: Expr_Variable(
name: c
)
)
)
)
Previously, comments were assigned to all nodes starting at the same position. Now they will be assigned to the outermost node only.
# Code
// Comment
$a + $b;
# Before
Stmt_Expression(
expr: Expr_BinaryOp_Plus(
left: Expr_Variable(
name: a
comments: array(
0: // Comment
)
)
right: Expr_Variable(
name: b
)
comments: array(
0: // Comment
)
)
comments: array(
0: // Comment
)
)
# After
Stmt_Expression(
expr: Expr_BinaryOp_Plus(
left: Expr_Variable(
name: a
)
right: Expr_Variable(
name: b
)
)
comments: array(
0: // Comment
)
)
A number of AST nodes have been renamed or moved in the AST hierarchy:
Node\Scalar\LNumber is now Node\Scalar\Int_.Node\Scalar\DNumber is now Node\Scalar\Float_.Node\Scalar\Encapsed is now Node\Scalar\InterpolatedString.Node\Scalar\EncapsedStringPart is now Node\InterpolatedStringPart and no longer extends
Node\Scalar or Node\Expr.Node\Expr\ArrayItem is now Node\ArrayItem and no longer extends Node\Expr.Node\Expr\ClosureUse is now Node\ClosureUse and no longer extends Node\Expr.Node\Stmt\DeclareDeclare is now Node\DeclareItem and no longer extends Node\Stmt.Node\Stmt\PropertyProperty is now Node\PropertyItem and no longer extends Node\Stmt.Node\Stmt\StaticVar is now Node\StaticVar and no longer extends Node\Stmt.Node\Stmt\UseUse is now Node\UseItem and no longer extends Node\Stmt.The old class names have been retained as aliases for backwards compatibility. However, the Node::getType() method will now always return the new name (e.g. ClosureUse instead of Expr_ClosureUse).
Modifier flags (as used by the $flags subnode of Class_, ClassMethod, Property, etc.) are now available as class constants on a separate PhpParser\Modifiers class, instead of being part of PhpParser\Node\Stmt\Class_, to make it clearer that these are used by many different nodes. The old constants are deprecated, but are still available.
PhpParser\Node\Stmt\Class_::MODIFIER_PUBLIC -> PhpParser\Modifiers::PUBLIC
PhpParser\Node\Stmt\Class_::MODIFIER_PROTECTED -> PhpParser\Modifiers::PROTECTED
PhpParser\Node\Stmt\Class_::MODIFIER_PRIVATE -> PhpParser\Modifiers::PRIVATE
PhpParser\Node\Stmt\Class_::MODIFIER_STATIC -> PhpParser\Modifiers::STATIC
PhpParser\Node\Stmt\Class_::MODIFIER_ABSTRACT -> PhpParser\Modifiers::ABSTRACT
PhpParser\Node\Stmt\Class_::MODIFIER_FINAL -> PhpParser\Modifiers::FINAL
PhpParser\Node\Stmt\Class_::MODIFIER_READONLY -> PhpParser\Modifiers::READONLY
PhpParser\Node\Stmt\Class_::VISIBILITY_MODIFIER_MASK -> PhpParser\Modifiers::VISIBILITY_MASK
Node constructor arguments accepting types no longer accept plain strings. Either an Identifier or Name (or ComplexType) should be passed instead. This affects the following constructor arguments:
'returnType' key of $subNodes argument of Node\Expr\ArrowFunction.'returnType' key of $subNodes argument of Node\Expr\Closure.'returnType' key of $subNodes argument of Node\Stmt\ClassMethod.'returnType' key of $subNodes argument of Node\Stmt\Function_.$type argument of Node\NullableType.$type argument of Node\Param.$type argument of Node\Stmt\Property.$type argument of Node\ClassConst.To follow the previous behavior, an Identifier should be passed, which indicates a built-in type.
A number of changes to the standard pretty printer have been made, to make it match contemporary coding style conventions (and in particular PSR-12). Options to restore the previous behavior are not provided, but it is possible to override the formatting methods (such as pStmt_ClassMethod) with your preferred formatting.
Return types are now formatted without a space before the ::
# Before
function test() : Type
{
}
# After
function test(): Type
{
}
abstract and final are now printed before visibility modifiers:
# Before
public abstract function test();
# After
abstract public function test();
A space is now printed between use and the following ( for closures:
# Before
function () use($var) {
};
# After
function () use ($var) {
};
Backslashes in single-quoted strings are now only printed if they are necessary:
# Before
'Foo\\Bar';
'\\\\';
# After
'Foo\Bar';
'\\\\';
else if structures will now omit redundant parentheses:
# Before
else {
if ($x) {
// ...
}
}
# After
else if ($x) {
// ...
}
The pretty printer now accepts a phpVersion option, which accepts a PhpVersion object and defaults to PHP 7.4. The pretty printer will make formatting choices to make the code valid for that version. It currently controls the following behavior:
[] will be used by default. This does not affect nodes that specify an explicit array syntax using the kind attribute.yield expressions will only be printed when necessary. Previously, parentheses were always printed, even if yield was used as a statement.[] will be used for destructuring by default (instead of list()). This does not affect nodes that specify an explicit syntax using the kind attribute.The pretty printer now more accurately models operator precedence. Especially for unary operators, less unnecessary parentheses will be printed. Conversely, many bugs where semantically meaningful parentheses were omitted have been fixed.
To support these changes, precedence is now handled differently in the pretty printer. The internal p() method, which is used to recursively print nodes, now has the following signature:
protected function p(
Node $node, int $precedence = self::MAX_PRECEDENCE, int $lhsPrecedence = self::MAX_PRECEDENCE,
bool $parentFormatPreserved = false
): string;
The $precedence is the precedence of the direct parent operator (if any), while $lhsPrecedence is that precedence of the nearest binary operator on whose left-hand-side the node occurs. For unary operators, only the $lhsPrecedence is relevant.
Recursive calls in pretty-printer methods should generally continue calling p() without additional parameters. However, pretty-printer methods for operators that participate in precedence resolution need to be adjusted. For example, typical implementations for operators look as follows now:
protected function pExpr_BinaryOp_Plus(
BinaryOp\Plus $node, int $precedence, int $lhsPrecedence
): string {
return $this->pInfixOp(
BinaryOp\Plus::class, $node->left, ' + ', $node->right, $precedence, $lhsPrecedence);
}
protected function pExpr_UnaryPlus(
Expr\UnaryPlus $node, int $precedence, int $lhsPrecedence
): string {
return $this->pPrefixOp(Expr\UnaryPlus::class, '+', $node->expr, $precedence, $lhsPrecedence);
}
The new $precedence and $lhsPrecedence arguments need to be passed down to the pInfixOp(), pPrefixOp() and pPostfixOp() methods.
If there are multiple visitors, the node traverser will now call leaveNode() and afterTraverse() methods in the reverse order of the corresponding enterNode() and beforeTraverse() calls:
# Before
$visitor1->enterNode($node);
$visitor2->enterNode($node);
$visitor1->leaveNode($node);
$visitor2->leaveNode($node);
# After
$visitor1->enterNode($node);
$visitor2->enterNode($node);
$visitor2->leaveNode($node);
$visitor1->leaveNode($node);
Additionally, the special NodeVisitor return values have been moved from NodeTraverser to NodeVisitor. The old names are deprecated, but still available.
PhpParser\NodeTraverser::REMOVE_NODE -> PhpParser\NodeVisitor::REMOVE_NODE
PhpParser\NodeTraverser::DONT_TRAVERSE_CHILDREN -> PhpParser\NodeVisitor::DONT_TRAVERSE_CHILDREN
PhpParser\NodeTraverser::DONT_TRAVERSE_CURRENT_AND_CHILDREN -> PhpParser\NodeVisitor::DONT_TRAVERSE_CURRENT_AND_CHILDREN
PhpParser\NodeTraverser::STOP_TRAVERSAL -> PhpParser\NodeVisitor::STOP_TRAVERSAL
Visitors can now also be passed directly to the NodeTraverser constructor:
# Before (and still supported)
$traverser = new NodeTraverser();
$traverser->addVisitor(new NameResolver());
# After
$traverser = new NodeTraverser(new NameResolver());
Tokens are now internally represented using the PhpParser\Token class, which exposes the same base interface as
the PhpToken class introduced in PHP 8.0. On PHP 8.0 or newer, PhpParser\Token extends from PhpToken, otherwise
it extends from a polyfill implementation. The most important parts of the interface may be summarized as follows:
class Token {
public int $id;
public string $text;
public int $line;
public int $pos;
public function is(int|string|array $kind): bool;
}
The token array is now an array of Tokens, rather than an array of arrays and strings.
Additionally, the token array is now terminated by a sentinel token with ID 0.
The lexer API is reduced to a single Lexer::tokenize() method, which returns an array of tokens. The startLexing() and getNextToken() methods have been removed.
Responsibility for determining start and end attributes for nodes has been moved from the lexer to the parser. The lexer no longer accepts an options array. The usedAttributes option has been removed without replacement, and the parser will now unconditionally add the comments, startLine, endLine, startFilePos, endFilePos, startTokenPos and endTokenPos attributes.
There should no longer be a need to directly interact with the Lexer for end users, as the ParserFactory will create an appropriate instance, and no additional configuration of the lexer is necessary. To use formatting-preserving pretty printing, the setup boilerplate changes as follows:
# Before
$lexer = new Lexer\Emulative([
'usedAttributes' => [
'comments',
'startLine', 'endLine',
'startTokenPos', 'endTokenPos',
],
]);
$parser = new Parser\Php7($lexer);
$oldStmts = $parser->parse($code);
$oldTokens = $lexer->getTokens();
$traverser = new NodeTraverser();
$traverser->addVisitor(new NodeVisitor\CloningVisitor());
$newStmts = $traverser->traverse($oldStmts);
# After
$parser = (new ParserFactory())->createForNewestSupportedVersion();
$oldStmts = $parser->parse($code);
$oldTokens = $parser->getTokens();
$traverser = new NodeTraverser(new NodeVisitor\CloningVisitor());
$newStmts = $traverser->traverse($oldStmts);
Builder\Param::setTypeHint() method has been removed in favor of Builder\Param::setType().Error constructor taking a start line has been removed. Pass ['startLine' => $startLine] attributes instead.Comment::getLine(), Comment::getTokenPos() and Comment::getFilePos() methods have been removed. Use Comment::getStartLine(), Comment::getStartTokenPos() and Comment::getStartFilePos() instead.Comment::getReformattedText() now normalizes CRLF newlines to LF newlines.Node::getLine() method has been deprecated. Use Node::getStartLine() instead.