mediawiki-extensions-AbuseF.../includes/Parser/AFPTreeNode.php

154 lines
5.3 KiB
PHP
Raw Normal View History

<?php
namespace MediaWiki\Extension\AbuseFilter\Parser;
use MediaWiki\Extension\AbuseFilter\Parser\Exception\InternalException;
/**
* Represents a node of a parser tree.
*/
class AFPTreeNode {
// Each of the constants below represents a node corresponding to a level
// of the parser, from the top of the tree to the bottom.
// ENTRY is always one-element and thus does not have its own node.
// SEMICOLON is a many-children node, denoting that the nodes have to be
// evaluated in order and the last value has to be returned.
public const SEMICOLON = 'SEMICOLON';
// ASSIGNMENT (formerly known as SET) is a node which is responsible for
// assigning values to variables. ASSIGNMENT is a (variable name [string],
// value [tree node]) tuple, INDEX_ASSIGNMENT (which is used to assign
// values at array offsets) is a (variable name [string], index [tree node],
// value [tree node]) tuple, and ARRAY_APPEND has the form of (variable name
// [string], value [tree node]).
public const ASSIGNMENT = 'ASSIGNMENT';
public const INDEX_ASSIGNMENT = 'INDEX_ASSIGNMENT';
public const ARRAY_APPEND = 'ARRAY_APPEND';
// CONDITIONAL represents both a ternary operator and an if-then-else-end
// construct. The format is (condition, evaluated-if-true, evaluated-in-false).
// The first two are tree nodes, the last one can be a node, or null if there's no else.
public const CONDITIONAL = 'CONDITIONAL';
// LOGIC is a logic operator accepted by AFPData::boolOp. The format is
// (operation, left operand, right operand).
public const LOGIC = 'LOGIC';
// COMPARE is a comparison operator accepted by AFPData::boolOp. The format is
// (operation, left operand, right operand).
public const COMPARE = 'COMPARE';
// SUM_REL is either '+' or '-'. The format is (operation, left operand,
// right operand).
public const SUM_REL = 'SUM_REL';
// MUL_REL is a multiplication-related operation accepted by AFPData::mulRel.
// The format is (operation, left operand, right operand).
public const MUL_REL = 'MUL_REL';
// POW is an exponentiation operator. The format is (base, exponent).
public const POW = 'POW';
// BOOL_INVERT is a boolean inversion operator. The format is (operand).
public const BOOL_INVERT = 'BOOL_INVERT';
// KEYWORD_OPERATOR is one of the binary keyword operators supported by the
// filter language. The format is (keyword, left operand, right operand).
public const KEYWORD_OPERATOR = 'KEYWORD_OPERATOR';
// UNARY is either unary minus or unary plus. The format is (operator, operand).
public const UNARY = 'UNARY';
// ARRAY_INDEX is an operation of accessing an array by an offset. The format
// is (array, offset).
public const ARRAY_INDEX = 'ARRAY_INDEX';
// Since parenthesis only manipulate precedence of the operators, they are
// not explicitly represented in the tree.
// FUNCTION_CALL is an invocation of built-in function. The format is a
// tuple where the first element is a function name, and all subsequent
// elements are the arguments.
public const FUNCTION_CALL = 'FUNCTION_CALL';
// ARRAY_DEFINITION is an array literal. The $children field contains tree
// nodes for the values of each of the array element used.
public const ARRAY_DEFINITION = 'ARRAY_DEFINITION';
// ATOM is a node representing a literal. The only element of $children is a
// token corresponding to the literal.
public const ATOM = 'ATOM';
Add a static analyzer for the filter language This commit adds a class AFPSyntaxChecker which can statically analyze a filter code to detect the following errors: - unbound variables (which comes in two modes: conservative and liberal, default to conservative) - unused variables (disabled by default for compatibilty) - assignment on built-in identifiers - function application's arity mismatch - function application's invalid function name - non-string literal in the first argument of set / set_var The existing parser and evaluator are modified as follows: - The new (caching) evaluator no longer needs to perform variable hoisting at runtime. - Note that for array assignment, this changes the semantics. - The new parser is more lenient, reducing parsing errors. The static analyzer will catch these errors instead, allowing us to give a much better error message and reduces the complexity of the parser. * The parser now allows function name to be any identifier. * The parser now allows arity mismatch to occur. * The parser now allows the first argument of set to be any expression. Concretely, obvious changes that users will see are: 1. a := [1]; false & (a[] := 2); a[0] === 1 would evaluate to true, while it used to evaluate to the undefined value due to hoisting 2. f(1) will now error with 'f is not a valid function' as opposed to 'Unexpected "T_BRACE"' 3. length will now error with 'Illegal use of built-in identifier "length"' as opposed to 'Expected a (' Appendix: conservative and liberal mode The conservative mode is completely compatible with the current evaluator. That is, false & (a := 1); a will not deem `a` as unbound, though this is actually undesirable because `a` would then be bound to the troublesome undefined value. The liberal mode rejects the above pattern by deeming `a` as unbound. However, it also rejects true & (a := 1); a even though (a := 1) is always executed. Since there are several filters in Wikimedia projects that rely on this behavior, we default the mode to conservative for now. Note that even the liberal mode doesn't really respect lexical scope appeared in some other programming languages (see also T234690). For instance: (if true then (a := 1) else (a := 2) end); a would be accepted by the liberal checker, even though under lexical scope, `a` would be unbound. However, it is unlikely that lexical scope will be suitable for the filter language, as most filters in Wikimedia projects that have user-defined variable do violate lexical scope. Bug: T260903 Bug: T238709 Bug: T237610 Bug: T234690 Bug: T231536 Change-Id: Ic6d030503e554933f8d220c6f87b680505918ae2
2019-11-20 07:09:41 +00:00
// BINOP is a combination of LOGIC (^), COMPARE (<=, <, etc.),
// SUM_REL (+, -), MUL_REL (*, /, %), POW (**),
// KEYWORD_OPERATOR (like, rlike, etc.), and ARRAY_INDEX ([]).
// The format is (operator, operand, operand).
// Currently, it's only used in SyntaxChecker
// & and | which is in LOGIC is not in BINOP because it affects
// control flow.
public const BINOP = 'BINOP';
/** @var string Type of the node, one of the constants above */
public $type;
/**
* Parameters of the value. Typically it is an array of children nodes,
* which might be either strings (for parametrization of the node) or another
* node. In case of ATOM it's a parser token.
* @var AFPTreeNode[]|string[]|AFPToken
*/
public $children;
/** @var int Position used for error reporting. */
public $position;
/**
* @param string $type
* @param (AFPTreeNode|null)[]|string[]|AFPToken $children
* @param int $position
*/
public function __construct( $type, $children, $position ) {
$this->type = $type;
$this->children = $children;
$this->position = $position;
}
/**
* @return string
* @codeCoverageIgnore
*/
public function toDebugString() {
return implode( "\n", $this->toDebugStringInner() );
}
/**
* @return array
* @codeCoverageIgnore
*/
private function toDebugStringInner() {
if ( $this->type === self::ATOM ) {
return [ "ATOM({$this->children->type} {$this->children->value})" ];
}
$align = static function ( $line ) {
return ' ' . $line;
};
$lines = [ $this->type ];
// @phan-suppress-next-line PhanTypeSuspiciousNonTraversableForeach children is array here
foreach ( $this->children as $subnode ) {
if ( $subnode instanceof AFPTreeNode ) {
$sublines = array_map( $align, $subnode->toDebugStringInner() );
} elseif ( is_string( $subnode ) ) {
$sublines = [ " {$subnode}" ];
} else {
throw new InternalException( "Each node parameter has to be either a node or a string" );
}
$lines = array_merge( $lines, $sublines );
}
return $lines;
}
}