Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
<?php
|
|
|
|
|
|
|
|
use MediaWiki\Block\DatabaseBlock;
|
|
|
|
use MediaWiki\Logger\LoggerFactory;
|
|
|
|
use MediaWiki\MediaWikiServices;
|
|
|
|
use MediaWiki\Session\SessionManager;
|
2020-01-24 17:26:03 +00:00
|
|
|
use Wikimedia\IPUtils;
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
use Wikimedia\Rdbms\IDatabase;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* This class contains the logic for executing abuse filters and their actions. The entry points are
|
|
|
|
* run() and runForStash(). Note that run() can only be executed once on a given instance.
|
|
|
|
* @todo In a perfect world, every time this class gets constructed we should have a context
|
|
|
|
* source at hand. Unfortunately, this currently isn't true, as the hooks used for filtering
|
|
|
|
* don't pass a full context. If they did, this class would just extend ContextSource and use
|
|
|
|
* that to retrieve user, title, globals etc.
|
|
|
|
*/
|
|
|
|
class AbuseFilterRunner {
|
|
|
|
/**
|
|
|
|
* @var User The user who performed the action being filtered
|
|
|
|
*/
|
|
|
|
protected $user;
|
|
|
|
/**
|
|
|
|
* @var Title The title where the action being filtered was performed
|
|
|
|
*/
|
|
|
|
protected $title;
|
|
|
|
/**
|
|
|
|
* @var AbuseFilterVariableHolder The variables for the current action
|
|
|
|
*/
|
|
|
|
protected $vars;
|
|
|
|
/**
|
|
|
|
* @var string The group of filters to check (as defined in $wgAbuseFilterValidGroups)
|
|
|
|
*/
|
|
|
|
protected $group;
|
2019-01-30 19:30:59 +00:00
|
|
|
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
/**
|
2019-01-30 19:30:59 +00:00
|
|
|
* @var array Data from per-filter profiling. Shape:
|
2019-10-09 10:37:38 +00:00
|
|
|
* [ filterName => [ 'time' => float, 'conds' => int, 'result' => bool ] ]
|
|
|
|
* @phan-var array<string,array{time:float,conds:int,result:bool}>
|
2019-01-30 19:30:59 +00:00
|
|
|
*
|
|
|
|
* Where 'timeTaken' is in seconds, 'result' is a boolean indicating whether the filter matched
|
|
|
|
* the action, and 'filterID' is "{prefix}-{ID}" ; Prefix should be empty for local
|
|
|
|
* filters. In stash mode this member is saved in cache, while in execute mode it's used to
|
|
|
|
* update profiling after checking all filters.
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
*/
|
2019-01-30 19:30:59 +00:00
|
|
|
protected $profilingData;
|
|
|
|
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
/**
|
|
|
|
* @var AbuseFilterParser The parser instance to use to check all filters
|
|
|
|
* @protected Public for back-compat only, will be made protected. self::init already handles
|
|
|
|
* building a parser object.
|
|
|
|
*/
|
|
|
|
public $parser;
|
|
|
|
/**
|
|
|
|
* @var bool Whether a run() was already performed. Used to avoid multiple executions with the
|
|
|
|
* same members.
|
|
|
|
*/
|
|
|
|
private $executed = false;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* @param User $user The user who performed the action being filtered
|
|
|
|
* @param Title $title The title where the action being filtered was performed
|
|
|
|
* @param AbuseFilterVariableHolder $vars The variables for the current action
|
|
|
|
* @param string $group The group of filters to check. It must be defined as so in
|
|
|
|
* $wgAbuseFilterValidGroups, or this will throw.
|
|
|
|
* @throws InvalidArgumentException
|
|
|
|
*/
|
|
|
|
public function __construct( User $user, Title $title, AbuseFilterVariableHolder $vars, $group ) {
|
|
|
|
global $wgAbuseFilterValidGroups;
|
|
|
|
if ( !in_array( $group, $wgAbuseFilterValidGroups ) ) {
|
|
|
|
throw new InvalidArgumentException( '$group must be defined in $wgAbuseFilterValidGroups' );
|
|
|
|
}
|
|
|
|
$this->user = $user;
|
|
|
|
$this->title = $title;
|
|
|
|
$this->vars = $vars;
|
2019-08-11 13:11:20 +00:00
|
|
|
$this->vars->setLogger( LoggerFactory::getInstance( 'AbuseFilter' ) );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$this->group = $group;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Inits variables and parser right before running
|
|
|
|
*/
|
|
|
|
private function init() {
|
|
|
|
// Add vars from extensions
|
|
|
|
Hooks::run( 'AbuseFilter-filterAction', [ &$this->vars, $this->title ] );
|
|
|
|
Hooks::run( 'AbuseFilterAlterVariables', [ &$this->vars, $this->title, $this->user ] );
|
|
|
|
$this->vars->addHolders( AbuseFilter::generateStaticVars() );
|
|
|
|
|
|
|
|
$this->vars->forFilter = true;
|
|
|
|
$this->vars->setVar( 'timestamp', (int)wfTimestamp( TS_UNIX ) );
|
|
|
|
$this->parser = $this->getParser();
|
2019-09-19 17:06:14 +00:00
|
|
|
$this->parser->setStatsd( MediaWikiServices::getInstance()->getStatsdDataFactory() );
|
2019-01-30 19:30:59 +00:00
|
|
|
$this->profilingData = [];
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
2019-08-21 10:04:10 +00:00
|
|
|
* Shortcut method, so that it can be overridden in mocks.
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
* @return AbuseFilterParser
|
|
|
|
*/
|
|
|
|
protected function getParser() : AbuseFilterParser {
|
2019-08-21 10:04:10 +00:00
|
|
|
return AbuseFilter::getDefaultParser( $this->vars );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* The main entry point of this class. This method runs all filters and takes their consequences.
|
|
|
|
*
|
|
|
|
* @param bool $allowStash Whether we are allowed to check the cache to see if there's a cached
|
|
|
|
* result of a previous execution for the same edit.
|
|
|
|
* @throws BadMethodCallException If run() was already called on this instance
|
|
|
|
* @return Status Good if no action has been taken, a fatal otherwise.
|
|
|
|
*/
|
|
|
|
public function run( $allowStash = true ) : Status {
|
2019-08-26 13:01:09 +00:00
|
|
|
global $wgAbuseFilterActions;
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
if ( $this->executed ) {
|
|
|
|
throw new BadMethodCallException( 'run() was already called on this instance.' );
|
|
|
|
}
|
|
|
|
$this->executed = true;
|
|
|
|
$this->init();
|
|
|
|
|
2019-07-29 16:56:21 +00:00
|
|
|
$action = $this->vars->getVar( 'action' )->toString();
|
|
|
|
|
|
|
|
$skipReasons = [];
|
|
|
|
$shouldFilter = Hooks::run(
|
|
|
|
'AbuseFilterShouldFilterAction',
|
|
|
|
[ $this->vars, $this->title, $this->user, &$skipReasons ]
|
|
|
|
);
|
|
|
|
if ( !$shouldFilter ) {
|
|
|
|
$logger = LoggerFactory::getInstance( 'AbuseFilter' );
|
|
|
|
$logger->info( "Skipping action $action. Reasons provided: " . implode( ', ', $skipReasons ) );
|
|
|
|
return Status::newGood();
|
|
|
|
}
|
|
|
|
|
|
|
|
$useStash = $allowStash && $action === 'edit';
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
$fromCache = false;
|
|
|
|
$result = [];
|
|
|
|
if ( $useStash ) {
|
|
|
|
$cacheData = $this->seekCache();
|
|
|
|
if ( $cacheData !== false ) {
|
2019-08-26 13:01:09 +00:00
|
|
|
if ( isset( $wgAbuseFilterActions['tag'] ) && $wgAbuseFilterActions['tag'] ) {
|
|
|
|
// Merge in any tags to apply to recent changes entries
|
|
|
|
AbuseFilter::bufferTagsToSetByAction( $cacheData['tags'] );
|
|
|
|
}
|
2019-01-30 19:30:59 +00:00
|
|
|
// Use cached vars (T176291) and profiling data (T191430)
|
|
|
|
$this->vars = AbuseFilterVariableHolder::newFromArray( $cacheData['vars'] );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$result = [
|
|
|
|
'matches' => $cacheData['matches'],
|
|
|
|
'runtime' => $cacheData['runtime'],
|
|
|
|
'condCount' => $cacheData['condCount'],
|
2019-01-30 19:30:59 +00:00
|
|
|
'profiling' => $cacheData['profiling']
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
];
|
|
|
|
$fromCache = true;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if ( !$fromCache ) {
|
2019-01-30 19:30:59 +00:00
|
|
|
$startTime = microtime( true );
|
2019-03-25 19:19:17 +00:00
|
|
|
// Ensure there's no extra time leftover
|
|
|
|
AFComputedVariable::$profilingExtraTime = 0;
|
|
|
|
|
2019-01-30 19:30:59 +00:00
|
|
|
// This also updates $this->profilingData and $this->parser->mCondCount used later
|
|
|
|
$matches = $this->checkAllFilters();
|
2019-03-25 19:19:17 +00:00
|
|
|
$timeTaken = ( microtime( true ) - $startTime - AFComputedVariable::$profilingExtraTime ) * 1000;
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$result = [
|
2019-01-30 19:30:59 +00:00
|
|
|
'matches' => $matches,
|
|
|
|
'runtime' => $timeTaken,
|
|
|
|
'condCount' => $this->parser->getCondCount(),
|
|
|
|
'profiling' => $this->profilingData
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
];
|
|
|
|
}
|
2019-12-07 17:20:10 +00:00
|
|
|
'@phan-var array{matches:array,runtime:int,condCount:int,profiling:array} $result';
|
2019-03-24 18:01:35 +00:00
|
|
|
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$matchedFilters = array_keys( array_filter( $result['matches'] ) );
|
2019-03-24 18:01:35 +00:00
|
|
|
$allFilters = array_keys( $result['matches'] );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
2019-03-24 18:01:35 +00:00
|
|
|
$this->profileExecution( $result, $matchedFilters, $allFilters );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
if ( count( $matchedFilters ) === 0 ) {
|
|
|
|
return Status::newGood();
|
|
|
|
}
|
|
|
|
|
|
|
|
$status = $this->executeFilterActions( $matchedFilters );
|
|
|
|
$actionsTaken = $status->getValue();
|
|
|
|
|
2019-01-30 19:30:59 +00:00
|
|
|
$this->addLogEntries( $actionsTaken );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
return $status;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Similar to run(), but runs in "stash" mode, which means filters are executed, no actions are
|
|
|
|
* taken, and the result is saved in cache to be later reused. This can only be used for edits,
|
|
|
|
* and not doing so will throw.
|
|
|
|
*
|
|
|
|
* @throws InvalidArgumentException
|
|
|
|
* @return Status Always a good status, since we're only saving data.
|
|
|
|
*/
|
|
|
|
public function runForStash() : Status {
|
|
|
|
$action = $this->vars->getVar( 'action' )->toString();
|
|
|
|
if ( $action !== 'edit' ) {
|
|
|
|
throw new InvalidArgumentException(
|
|
|
|
__METHOD__ . " can only be called for edits, called for action $action."
|
|
|
|
);
|
|
|
|
}
|
|
|
|
|
|
|
|
$this->init();
|
|
|
|
|
2019-07-29 16:56:21 +00:00
|
|
|
$skipReasons = [];
|
|
|
|
$shouldFilter = Hooks::run(
|
|
|
|
'AbuseFilterShouldFilterAction',
|
|
|
|
[ $this->vars, $this->title, $this->user, &$skipReasons ]
|
|
|
|
);
|
|
|
|
if ( !$shouldFilter ) {
|
|
|
|
// Don't log it yet
|
|
|
|
return Status::newGood();
|
|
|
|
}
|
|
|
|
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$cache = ObjectCache::getLocalClusterInstance();
|
|
|
|
$stashKey = $this->getStashKey( $cache );
|
|
|
|
|
|
|
|
$startTime = microtime( true );
|
2019-03-25 19:19:17 +00:00
|
|
|
// Ensure there's no extra time leftover
|
|
|
|
AFComputedVariable::$profilingExtraTime = 0;
|
|
|
|
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$matchedFilters = $this->checkAllFilters();
|
|
|
|
// Save the filter stash result and do nothing further
|
|
|
|
$cacheData = [
|
|
|
|
'matches' => $matchedFilters,
|
|
|
|
'tags' => AbuseFilter::$tagsToSet,
|
|
|
|
'condCount' => $this->parser->getCondCount(),
|
2019-03-25 19:19:17 +00:00
|
|
|
'runtime' => ( microtime( true ) - $startTime - AFComputedVariable::$profilingExtraTime ) * 1000,
|
2019-01-30 19:30:59 +00:00
|
|
|
'vars' => $this->vars->dumpAllVars(),
|
|
|
|
'profiling' => $this->profilingData
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
];
|
|
|
|
|
|
|
|
$cache->set( $stashKey, $cacheData, $cache::TTL_MINUTE );
|
|
|
|
$this->logCache( 'store', $stashKey );
|
|
|
|
|
|
|
|
return Status::newGood();
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Search the cache to find data for a previous execution done for the current edit.
|
|
|
|
*
|
|
|
|
* @return false|array False on failure, the array with data otherwise
|
|
|
|
*/
|
|
|
|
protected function seekCache() {
|
|
|
|
$cache = ObjectCache::getLocalClusterInstance();
|
|
|
|
$stashKey = $this->getStashKey( $cache );
|
|
|
|
|
|
|
|
$ret = $cache->get( $stashKey );
|
|
|
|
$status = $ret !== false ? 'hit' : 'miss';
|
|
|
|
$this->logCache( $status, $stashKey );
|
|
|
|
|
|
|
|
return $ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Get the stash key for the current variables
|
|
|
|
*
|
|
|
|
* @param BagOStuff $cache
|
|
|
|
* @return string
|
|
|
|
*/
|
|
|
|
protected function getStashKey( BagOStuff $cache ) {
|
|
|
|
$inputVars = $this->vars->exportNonLazyVars();
|
|
|
|
// Exclude noisy fields that have superficial changes
|
|
|
|
$excludedVars = [
|
|
|
|
'old_html' => true,
|
|
|
|
'new_html' => true,
|
|
|
|
'user_age' => true,
|
|
|
|
'timestamp' => true,
|
|
|
|
'page_age' => true,
|
|
|
|
'moved_from_age' => true,
|
|
|
|
'moved_to_age' => true
|
|
|
|
];
|
|
|
|
|
|
|
|
$inputVars = array_diff_key( $inputVars, $excludedVars );
|
|
|
|
ksort( $inputVars );
|
|
|
|
$hash = md5( serialize( $inputVars ) );
|
|
|
|
|
|
|
|
return $cache->makeKey(
|
|
|
|
'abusefilter',
|
|
|
|
'check-stash',
|
|
|
|
$this->group,
|
|
|
|
$hash,
|
2019-08-04 16:07:54 +00:00
|
|
|
'v2'
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
);
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Log cache operations related to stashed edits, i.e. store, hit and miss
|
|
|
|
*
|
|
|
|
* @param string $type Either 'store', 'hit' or 'miss'
|
|
|
|
* @param string $key The cache key used
|
|
|
|
* @throws InvalidArgumentException
|
|
|
|
*/
|
|
|
|
protected function logCache( $type, $key ) {
|
|
|
|
if ( !in_array( $type, [ 'store', 'hit', 'miss' ] ) ) {
|
|
|
|
throw new InvalidArgumentException( '$type must be either "store", "hit" or "miss"' );
|
|
|
|
}
|
|
|
|
$logger = LoggerFactory::getInstance( 'StashEdit' );
|
|
|
|
// Bots do not use edit stashing, so avoid distorting the stats
|
|
|
|
$statsd = $this->user->isBot()
|
|
|
|
? new NullStatsdDataFactory()
|
|
|
|
: MediaWikiServices::getInstance()->getStatsdDataFactory();
|
|
|
|
|
|
|
|
$logger->debug( __METHOD__ . ": cache $type for '{$this->title}' (key $key)." );
|
|
|
|
$statsd->increment( "abusefilter.check-stash.$type" );
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Returns an associative array of filters which were tripped
|
|
|
|
*
|
|
|
|
* @protected Public for back compat only; this will actually be made protected in the future.
|
|
|
|
* You should either rely on $this->run() or subclass this class.
|
|
|
|
* @todo This method should simply return an array with IDs of matched filters as values,
|
|
|
|
* since we always end up filtering it after calling this method.
|
|
|
|
* @return bool[] Map of (integer filter ID => bool)
|
|
|
|
*/
|
|
|
|
public function checkAllFilters() : array {
|
|
|
|
global $wgAbuseFilterCentralDB, $wgAbuseFilterIsCentral, $wgAbuseFilterConditionLimit;
|
|
|
|
|
|
|
|
// Ensure that we start fresh, see T193374
|
|
|
|
$this->parser->resetCondCount();
|
|
|
|
|
|
|
|
// Fetch filters to check from the database.
|
|
|
|
$matchedFilters = [];
|
|
|
|
|
|
|
|
$dbr = wfGetDB( DB_REPLICA );
|
|
|
|
$res = $dbr->select(
|
|
|
|
'abuse_filter',
|
2019-10-03 12:13:26 +00:00
|
|
|
AbuseFilter::ALL_ABUSE_FILTER_FIELDS,
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
[
|
|
|
|
'af_enabled' => 1,
|
|
|
|
'af_deleted' => 0,
|
|
|
|
'af_group' => $this->group,
|
|
|
|
],
|
|
|
|
__METHOD__
|
|
|
|
);
|
|
|
|
|
|
|
|
foreach ( $res as $row ) {
|
2019-01-30 19:30:59 +00:00
|
|
|
$matchedFilters[$row->af_id] = $this->checkFilter( $row );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if ( $wgAbuseFilterCentralDB && !$wgAbuseFilterIsCentral ) {
|
|
|
|
// Global filters
|
|
|
|
$globalRulesKey = AbuseFilter::getGlobalRulesKey( $this->group );
|
|
|
|
|
|
|
|
$fname = __METHOD__;
|
|
|
|
$res = MediaWikiServices::getInstance()->getMainWANObjectCache()->getWithSetCallback(
|
|
|
|
$globalRulesKey,
|
|
|
|
WANObjectCache::TTL_WEEK,
|
|
|
|
function () use ( $fname ) {
|
|
|
|
$fdb = AbuseFilter::getCentralDB( DB_REPLICA );
|
|
|
|
|
|
|
|
return iterator_to_array( $fdb->select(
|
|
|
|
'abuse_filter',
|
2019-10-03 12:13:26 +00:00
|
|
|
AbuseFilter::ALL_ABUSE_FILTER_FIELDS,
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
[
|
|
|
|
'af_enabled' => 1,
|
|
|
|
'af_deleted' => 0,
|
|
|
|
'af_global' => 1,
|
|
|
|
'af_group' => $this->group,
|
|
|
|
],
|
|
|
|
$fname
|
|
|
|
) );
|
|
|
|
},
|
|
|
|
[
|
|
|
|
'checkKeys' => [ $globalRulesKey ],
|
|
|
|
'lockTSE' => 300,
|
|
|
|
'version' => 1
|
|
|
|
]
|
|
|
|
);
|
|
|
|
|
|
|
|
foreach ( $res as $row ) {
|
|
|
|
$matchedFilters[ AbuseFilter::buildGlobalName( $row->af_id ) ] =
|
2019-01-30 19:30:59 +00:00
|
|
|
$this->checkFilter( $row, true );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2015-04-01 04:51:16 +00:00
|
|
|
// Tag the action if the condition limit was hit
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
if ( $this->parser->getCondCount() > $wgAbuseFilterConditionLimit ) {
|
|
|
|
$actionID = $this->getTaggingID();
|
|
|
|
AbuseFilter::bufferTagsToSetByAction( [ $actionID => [ 'abusefilter-condition-limit' ] ] );
|
|
|
|
}
|
|
|
|
|
|
|
|
return $matchedFilters;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Check the conditions of a single filter, and profile it if $this->executeMode is true
|
|
|
|
*
|
|
|
|
* @param stdClass $row
|
2019-01-30 19:30:59 +00:00
|
|
|
* @param bool $global
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
* @return bool
|
|
|
|
*/
|
2019-01-30 19:30:59 +00:00
|
|
|
protected function checkFilter( $row, $global = false ) {
|
|
|
|
$filterName = AbuseFilter::buildGlobalName( $row->af_id, $global );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
$startConds = $this->parser->getCondCount();
|
|
|
|
$startTime = microtime( true );
|
2019-03-25 19:19:17 +00:00
|
|
|
$origExtraTime = AFComputedVariable::$profilingExtraTime;
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
// Store the row somewhere convenient
|
2019-01-30 19:30:59 +00:00
|
|
|
AbuseFilter::cacheFilter( $filterName, $row );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
$pattern = trim( $row->af_pattern );
|
2019-08-10 16:34:42 +00:00
|
|
|
$this->parser->setFilter( $filterName );
|
2019-01-30 19:30:59 +00:00
|
|
|
$result = AbuseFilter::checkConditions( $pattern, $this->parser, true, $filterName );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
2019-03-25 19:19:17 +00:00
|
|
|
$actualExtra = AFComputedVariable::$profilingExtraTime - $origExtraTime;
|
|
|
|
$timeTaken = 1000 * ( microtime( true ) - $startTime - $actualExtra );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$condsUsed = $this->parser->getCondCount() - $startConds;
|
|
|
|
|
2019-01-30 19:30:59 +00:00
|
|
|
$this->profilingData[$filterName] = [
|
|
|
|
'time' => $timeTaken,
|
|
|
|
'conds' => $condsUsed,
|
|
|
|
'result' => $result
|
|
|
|
];
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
2019-01-30 19:30:59 +00:00
|
|
|
return $result;
|
|
|
|
}
|
|
|
|
|
2019-03-24 18:01:35 +00:00
|
|
|
/**
|
|
|
|
* @param array $result Result of the execution, as created in run()
|
|
|
|
* @param string[] $matchedFilters
|
|
|
|
* @param string[] $allFilters
|
|
|
|
*/
|
|
|
|
protected function profileExecution( array $result, array $matchedFilters, array $allFilters ) {
|
|
|
|
$this->checkResetProfiling( $allFilters );
|
|
|
|
$this->recordRuntimeProfilingResult(
|
2019-09-29 09:02:29 +00:00
|
|
|
count( $allFilters ),
|
2019-03-24 18:01:35 +00:00
|
|
|
$result['condCount'],
|
|
|
|
$result['runtime']
|
|
|
|
);
|
|
|
|
$this->recordPerFilterProfiling( $result['profiling'] );
|
|
|
|
$this->recordStats( $result['condCount'], $result['runtime'], (bool)$matchedFilters );
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Check if profiling data for all filters is lesser than the limit. If not, delete it and
|
|
|
|
* also delete per-filter profiling for all filters. Note that we don't need to reset it for
|
|
|
|
* disabled filters too, as their profiling data will be reset upon re-enabling anyway.
|
|
|
|
*
|
|
|
|
* @param array $allFilters
|
|
|
|
*/
|
|
|
|
protected function checkResetProfiling( array $allFilters ) {
|
|
|
|
global $wgAbuseFilterProfileActionsCap;
|
|
|
|
|
|
|
|
$profileKey = AbuseFilter::filterProfileGroupKey( $this->group );
|
|
|
|
$stash = MediaWikiServices::getInstance()->getMainObjectStash();
|
|
|
|
|
|
|
|
$profile = $stash->get( $profileKey );
|
|
|
|
$total = $profile['total'] ?? 0;
|
|
|
|
|
|
|
|
if ( $total > $wgAbuseFilterProfileActionsCap ) {
|
|
|
|
$stash->delete( $profileKey );
|
|
|
|
foreach ( $allFilters as $filter ) {
|
|
|
|
AbuseFilter::resetFilterProfile( $filter );
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-01-30 19:30:59 +00:00
|
|
|
/**
|
|
|
|
* Record per-filter profiling, for all filters
|
|
|
|
*
|
|
|
|
* @param array $data Profiling data, as stored in $this->profilingData
|
2019-10-09 10:37:38 +00:00
|
|
|
* @phan-param array<string,array{time:float,conds:int,result:bool}> $data
|
2019-01-30 19:30:59 +00:00
|
|
|
*/
|
|
|
|
protected function recordPerFilterProfiling( array $data ) {
|
|
|
|
global $wgAbuseFilterSlowFilterRuntimeLimit;
|
|
|
|
|
|
|
|
foreach ( $data as $filterName => $params ) {
|
|
|
|
list( $filterID, $global ) = AbuseFilter::splitGlobalName( $filterName );
|
|
|
|
if ( !$global ) {
|
|
|
|
// @todo Maybe add a parameter to recordProfilingResult to record global filters
|
|
|
|
// data separately (in the foreign wiki)
|
2019-02-10 10:57:37 +00:00
|
|
|
$this->recordProfilingResult( $filterID, $params['time'], $params['conds'], $params['result'] );
|
2019-01-30 19:30:59 +00:00
|
|
|
}
|
|
|
|
|
2019-02-10 12:08:06 +00:00
|
|
|
if ( $params['time'] > $wgAbuseFilterSlowFilterRuntimeLimit ) {
|
|
|
|
$this->recordSlowFilter( $filterName, $params['time'], $params['conds'], $params['result'] );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Record per-filter profiling data
|
|
|
|
*
|
|
|
|
* @param int $filter
|
2019-02-10 12:08:06 +00:00
|
|
|
* @param float $time Time taken, in milliseconds
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
* @param int $conds
|
2019-02-10 10:57:37 +00:00
|
|
|
* @param bool $matched
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
*/
|
2019-02-10 10:57:37 +00:00
|
|
|
protected function recordProfilingResult( $filter, $time, $conds, $matched ) {
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
// Defer updates to avoid massive (~1 second) edit time increases
|
2019-02-10 10:57:37 +00:00
|
|
|
DeferredUpdates::addCallableUpdate( function () use ( $filter, $time, $conds, $matched ) {
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$stash = MediaWikiServices::getInstance()->getMainObjectStash();
|
2019-02-10 10:01:39 +00:00
|
|
|
$profileKey = AbuseFilter::filterProfileKey( $filter );
|
|
|
|
$profile = $stash->get( $profileKey );
|
|
|
|
|
|
|
|
if ( $profile !== false ) {
|
|
|
|
// Number of observed executions of this filter
|
|
|
|
$profile['count']++;
|
2019-02-10 10:57:37 +00:00
|
|
|
if ( $matched ) {
|
|
|
|
// Number of observed matches of this filter
|
|
|
|
$profile['matches']++;
|
|
|
|
}
|
2019-02-10 10:01:39 +00:00
|
|
|
// Total time spent on this filter from all observed executions
|
|
|
|
$profile['total-time'] += $time;
|
|
|
|
// Total number of conditions for this filter from all executions
|
|
|
|
$profile['total-cond'] += $conds;
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
} else {
|
2019-02-10 10:01:39 +00:00
|
|
|
$profile = [
|
|
|
|
'count' => 1,
|
2019-02-10 10:57:37 +00:00
|
|
|
'matches' => (int)$matched,
|
2019-02-10 10:01:39 +00:00
|
|
|
'total-time' => $time,
|
|
|
|
'total-cond' => $conds
|
|
|
|
];
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
}
|
2019-02-10 10:01:39 +00:00
|
|
|
// Note: It is important that all key information be stored together in a single
|
|
|
|
// memcache entry to avoid race conditions where competing Apache instances
|
|
|
|
// partially overwrite the stats.
|
|
|
|
$stash->set( $profileKey, $profile, 3600 );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
} );
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Logs slow filter's runtime data for later analysis
|
|
|
|
*
|
|
|
|
* @param string $filterId
|
|
|
|
* @param float $runtime
|
|
|
|
* @param int $totalConditions
|
|
|
|
* @param bool $matched
|
|
|
|
*/
|
|
|
|
protected function recordSlowFilter( $filterId, $runtime, $totalConditions, $matched ) {
|
|
|
|
$logger = LoggerFactory::getInstance( 'AbuseFilter' );
|
|
|
|
$logger->info(
|
|
|
|
'Edit filter {filter_id} on {wiki} is taking longer than expected',
|
|
|
|
[
|
2020-01-08 13:33:10 +00:00
|
|
|
'wiki' => WikiMap::getCurrentWikiDbDomain()->getId(),
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
'filter_id' => $filterId,
|
|
|
|
'title' => $this->title->getPrefixedText(),
|
|
|
|
'runtime' => $runtime,
|
|
|
|
'matched' => $matched,
|
|
|
|
'total_conditions' => $totalConditions
|
|
|
|
]
|
|
|
|
);
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
2019-01-30 19:30:59 +00:00
|
|
|
* Update global statistics
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
*
|
2019-01-30 19:30:59 +00:00
|
|
|
* @param int $condsUsed The amount of used conditions
|
2015-04-01 04:51:16 +00:00
|
|
|
* @param float $totalTime Time taken, in milliseconds
|
|
|
|
* @param bool $anyMatch Whether at least one filter matched the action
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
*/
|
2015-04-01 04:51:16 +00:00
|
|
|
protected function recordStats( $condsUsed, $totalTime, $anyMatch ) {
|
|
|
|
$profileKey = AbuseFilter::filterProfileGroupKey( $this->group );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$stash = MediaWikiServices::getInstance()->getMainObjectStash();
|
|
|
|
|
2015-04-01 04:51:16 +00:00
|
|
|
// Note: All related data is stored in a single memcache entry and updated via merge()
|
|
|
|
// to avoid race conditions where partial updates on competing instances corrupt the data.
|
|
|
|
$stash->merge(
|
|
|
|
$profileKey,
|
|
|
|
function ( $cache, $key, $profile ) use ( $condsUsed, $totalTime, $anyMatch ) {
|
2019-03-24 18:01:35 +00:00
|
|
|
global $wgAbuseFilterConditionLimit;
|
2015-04-01 04:51:16 +00:00
|
|
|
|
2019-03-24 18:01:35 +00:00
|
|
|
if ( $profile === false ) {
|
2015-04-01 04:51:16 +00:00
|
|
|
$profile = [
|
|
|
|
// Total number of actions observed
|
|
|
|
'total' => 0,
|
|
|
|
// Number of actions ending by exceeding condition limit
|
|
|
|
'overflow' => 0,
|
|
|
|
// Total time of execution of all observed actions
|
|
|
|
'total-time' => 0,
|
|
|
|
// Total number of conditions from all observed actions
|
|
|
|
'total-cond' => 0,
|
|
|
|
// Total number of filters matched
|
|
|
|
'matches' => 0
|
|
|
|
];
|
|
|
|
}
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
2015-04-01 04:51:16 +00:00
|
|
|
$profile['total']++;
|
|
|
|
$profile['total-time'] += $totalTime;
|
|
|
|
$profile['total-cond'] += $condsUsed;
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
2015-04-01 04:51:16 +00:00
|
|
|
// Increment overflow counter, if our condition limit overflowed
|
|
|
|
if ( $condsUsed > $wgAbuseFilterConditionLimit ) {
|
|
|
|
$profile['overflow']++;
|
|
|
|
}
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
2015-04-01 04:51:16 +00:00
|
|
|
// Increment counter by 1 if there was at least one match
|
|
|
|
if ( $anyMatch ) {
|
|
|
|
$profile['matches']++;
|
|
|
|
}
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
2015-04-01 04:51:16 +00:00
|
|
|
return $profile;
|
|
|
|
},
|
|
|
|
AbuseFilter::$statsStoragePeriod
|
|
|
|
);
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
2019-01-30 19:30:59 +00:00
|
|
|
* Record runtime profiling data for all filters together
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
*
|
|
|
|
* @param int $totalFilters
|
|
|
|
* @param int $totalConditions
|
|
|
|
* @param float $runtime
|
|
|
|
*/
|
|
|
|
protected function recordRuntimeProfilingResult( $totalFilters, $totalConditions, $runtime ) {
|
2020-01-08 13:33:10 +00:00
|
|
|
$keyPrefix = 'abusefilter.runtime-profile.' . WikiMap::getCurrentWikiDbDomain()->getId() . '.';
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
$statsd = MediaWikiServices::getInstance()->getStatsdDataFactory();
|
|
|
|
$statsd->timing( $keyPrefix . 'runtime', $runtime );
|
|
|
|
$statsd->timing( $keyPrefix . 'total_filters', $totalFilters );
|
|
|
|
$statsd->timing( $keyPrefix . 'total_conditions', $totalConditions );
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Executes a set of actions.
|
|
|
|
*
|
|
|
|
* @param string[] $filters
|
|
|
|
* @return Status returns the operation's status. $status->isOK() will return true if
|
|
|
|
* there were no actions taken, false otherwise. $status->getValue() will return
|
|
|
|
* an array listing the actions taken. $status->getErrors() etc. will provide
|
|
|
|
* the errors and warnings to be shown to the user to explain the actions.
|
|
|
|
*/
|
|
|
|
protected function executeFilterActions( array $filters ) : Status {
|
|
|
|
global $wgMainCacheType, $wgAbuseFilterDisallowGlobalLocalBlocks, $wgAbuseFilterRestrictions,
|
|
|
|
$wgAbuseFilterBlockDuration, $wgAbuseFilterAnonBlockDuration;
|
|
|
|
|
|
|
|
$actionsByFilter = AbuseFilter::getConsequencesForFilters( $filters );
|
|
|
|
$actionsTaken = array_fill_keys( $filters, [] );
|
|
|
|
|
|
|
|
$messages = [];
|
|
|
|
// Accumulator to track max block to issue
|
|
|
|
$maxExpiry = -1;
|
|
|
|
|
|
|
|
foreach ( $actionsByFilter as $filter => $actions ) {
|
|
|
|
// Special-case handling for warnings.
|
|
|
|
$filterPublicComments = AbuseFilter::getFilter( $filter )->af_public_comments;
|
|
|
|
|
|
|
|
$isGlobalFilter = AbuseFilter::splitGlobalName( $filter )[1];
|
|
|
|
|
|
|
|
// If the filter has "throttle" enabled and throttling is available via object
|
|
|
|
// caching, check to see if the user has hit the throttle.
|
|
|
|
if ( !empty( $actions['throttle'] ) && $wgMainCacheType !== CACHE_NONE ) {
|
|
|
|
$parameters = $actions['throttle']['parameters'];
|
|
|
|
$throttleId = array_shift( $parameters );
|
|
|
|
list( $rateCount, $ratePeriod ) = explode( ',', array_shift( $parameters ) );
|
2019-12-07 17:20:10 +00:00
|
|
|
$rateCount = (int)$rateCount;
|
|
|
|
$ratePeriod = (int)$ratePeriod;
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
$hitThrottle = false;
|
|
|
|
|
|
|
|
// The rest are throttle-types.
|
|
|
|
foreach ( $parameters as $throttleType ) {
|
|
|
|
$hitThrottle = $hitThrottle || $this->isThrottled(
|
|
|
|
$throttleId, $throttleType, $rateCount, $ratePeriod, $isGlobalFilter );
|
|
|
|
}
|
|
|
|
|
|
|
|
unset( $actions['throttle'] );
|
|
|
|
if ( !$hitThrottle ) {
|
|
|
|
$actionsTaken[$filter][] = 'throttle';
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if ( $wgAbuseFilterDisallowGlobalLocalBlocks && $isGlobalFilter ) {
|
|
|
|
$actions = array_diff_key( $actions, array_filter( $wgAbuseFilterRestrictions ) );
|
|
|
|
}
|
|
|
|
|
|
|
|
if ( !empty( $actions['warn'] ) ) {
|
|
|
|
$parameters = $actions['warn']['parameters'];
|
|
|
|
$action = $this->vars->getVar( 'action' )->toString();
|
|
|
|
// Generate a unique key to determine whether the user has already been warned.
|
|
|
|
// We'll warn again if one of these changes: session, page, triggered filter or action
|
|
|
|
$warnKey = 'abusefilter-warned-' . md5( $this->title->getPrefixedText() ) .
|
|
|
|
'-' . $filter . '-' . $action;
|
|
|
|
|
|
|
|
// Make sure the session is started prior to using it
|
|
|
|
$session = SessionManager::getGlobalSession();
|
|
|
|
$session->persist();
|
|
|
|
|
|
|
|
if ( !isset( $session[$warnKey] ) || !$session[$warnKey] ) {
|
|
|
|
$session[$warnKey] = true;
|
|
|
|
|
|
|
|
$msg = $parameters[0] ?? 'abusefilter-warning';
|
|
|
|
$messages[] = [ $msg, $filterPublicComments, $filter ];
|
|
|
|
|
|
|
|
$actionsTaken[$filter][] = 'warn';
|
|
|
|
|
|
|
|
// Don't do anything else.
|
|
|
|
continue;
|
|
|
|
} else {
|
|
|
|
// We already warned them
|
|
|
|
$session[$warnKey] = false;
|
|
|
|
}
|
|
|
|
|
|
|
|
unset( $actions['warn'] );
|
|
|
|
}
|
|
|
|
|
|
|
|
// Prevent double warnings
|
|
|
|
if ( count( array_intersect_key( $actions, array_filter( $wgAbuseFilterRestrictions ) ) ) > 0 &&
|
|
|
|
!empty( $actions['disallow'] )
|
|
|
|
) {
|
|
|
|
unset( $actions['disallow'] );
|
|
|
|
}
|
|
|
|
|
|
|
|
// Find out the max expiry to issue the longest triggered block.
|
|
|
|
// Need to check here since methods like user->getBlock() aren't available
|
|
|
|
if ( !empty( $actions['block'] ) ) {
|
|
|
|
$parameters = $actions['block']['parameters'];
|
|
|
|
|
|
|
|
if ( count( $parameters ) === 3 ) {
|
|
|
|
// New type of filters with custom block
|
|
|
|
if ( $this->user->isAnon() ) {
|
|
|
|
$expiry = $parameters[1];
|
|
|
|
} else {
|
|
|
|
$expiry = $parameters[2];
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
// Old type with fixed expiry
|
|
|
|
if ( $this->user->isAnon() && $wgAbuseFilterAnonBlockDuration !== null ) {
|
|
|
|
// The user isn't logged in and the anon block duration
|
|
|
|
// doesn't default to $wgAbuseFilterBlockDuration.
|
|
|
|
$expiry = $wgAbuseFilterAnonBlockDuration;
|
|
|
|
} else {
|
|
|
|
$expiry = $wgAbuseFilterBlockDuration;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
$currentExpiry = SpecialBlock::parseExpiryInput( $expiry );
|
2019-11-20 17:56:31 +00:00
|
|
|
if ( $maxExpiry === -1 || $currentExpiry > SpecialBlock::parseExpiryInput( $maxExpiry ) ) {
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
// Save the parameters to issue the block with
|
|
|
|
$maxExpiry = $expiry;
|
|
|
|
$blockValues = [
|
|
|
|
AbuseFilter::getFilter( $filter )->af_public_comments,
|
|
|
|
$filter,
|
|
|
|
is_array( $parameters ) && in_array( 'blocktalk', $parameters )
|
|
|
|
];
|
|
|
|
}
|
|
|
|
unset( $actions['block'] );
|
|
|
|
}
|
|
|
|
|
|
|
|
// Do the rest of the actions
|
|
|
|
foreach ( $actions as $action => $info ) {
|
|
|
|
$newMsg = $this->takeConsequenceAction(
|
|
|
|
$action,
|
2019-12-07 17:20:10 +00:00
|
|
|
// @phan-suppress-next-line PhanTypeArraySuspiciousNullable False positive
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$info['parameters'],
|
|
|
|
AbuseFilter::getFilter( $filter )->af_public_comments,
|
|
|
|
$filter
|
|
|
|
);
|
|
|
|
|
|
|
|
if ( $newMsg !== null ) {
|
|
|
|
$messages[] = $newMsg;
|
|
|
|
}
|
|
|
|
$actionsTaken[$filter][] = $action;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// Since every filter has been analysed, we now know what the
|
|
|
|
// longest block duration is, so we can issue the block if
|
|
|
|
// maxExpiry has been changed.
|
|
|
|
if ( $maxExpiry !== -1 ) {
|
|
|
|
$this->doAbuseFilterBlock(
|
|
|
|
[
|
|
|
|
'desc' => $blockValues[0],
|
|
|
|
'number' => $blockValues[1]
|
|
|
|
],
|
|
|
|
$this->user->getName(),
|
|
|
|
$maxExpiry,
|
|
|
|
true,
|
2020-01-28 17:43:26 +00:00
|
|
|
// @phan-suppress-next-line PhanTypeMismatchArgumentNullable
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$blockValues[2]
|
|
|
|
);
|
|
|
|
$message = [
|
|
|
|
'abusefilter-blocked-display',
|
|
|
|
$blockValues[0],
|
|
|
|
$blockValues[1]
|
|
|
|
];
|
|
|
|
// Manually add the message. If we're here, there is one.
|
|
|
|
$messages[] = $message;
|
2020-01-28 17:43:26 +00:00
|
|
|
// @phan-suppress-next-line PhanTypeMismatchDimAssignment
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$actionsTaken[$blockValues[1]][] = 'block';
|
|
|
|
}
|
|
|
|
|
|
|
|
return $this->buildStatus( $actionsTaken, $messages );
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* @param string $throttleId
|
|
|
|
* @param string $types
|
2019-12-07 17:20:10 +00:00
|
|
|
* @param int $rateCount
|
|
|
|
* @param int $ratePeriod
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
* @param bool $global
|
|
|
|
* @return bool
|
|
|
|
*/
|
2019-12-07 17:20:10 +00:00
|
|
|
protected function isThrottled(
|
|
|
|
$throttleId,
|
|
|
|
$types,
|
|
|
|
int $rateCount,
|
|
|
|
int $ratePeriod,
|
|
|
|
$global = false
|
|
|
|
) {
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$stash = MediaWikiServices::getInstance()->getMainObjectStash();
|
|
|
|
$key = $this->throttleKey( $throttleId, $types, $global );
|
2020-01-07 16:52:16 +00:00
|
|
|
$count = (int)$stash->get( $key );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
$logger = LoggerFactory::getInstance( 'AbuseFilter' );
|
|
|
|
$logger->debug( "Got value $count for throttle key $key" );
|
|
|
|
|
2020-01-07 16:52:16 +00:00
|
|
|
$count = $stash->incrWithInit( $key, $ratePeriod );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
if ( $count > $rateCount ) {
|
|
|
|
$logger->debug( "Throttle $key hit value $count -- maximum is $rateCount." );
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
$logger->debug( "Throttle $key not hit!" );
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* @param string $throttleId
|
|
|
|
* @param string $type
|
|
|
|
* @param bool $global
|
|
|
|
* @return string
|
|
|
|
*/
|
|
|
|
protected function throttleKey( $throttleId, $type, $global = false ) {
|
|
|
|
global $wgAbuseFilterIsCentral, $wgAbuseFilterCentralDB;
|
|
|
|
|
|
|
|
$types = explode( ',', $type );
|
|
|
|
|
|
|
|
$identifiers = [];
|
|
|
|
|
|
|
|
foreach ( $types as $subtype ) {
|
|
|
|
$identifiers[] = $this->throttleIdentifier( $subtype );
|
|
|
|
}
|
|
|
|
|
|
|
|
$identifier = sha1( implode( ':', $identifiers ) );
|
|
|
|
|
|
|
|
$cache = MediaWikiServices::getInstance()->getMainWANObjectCache();
|
|
|
|
if ( $global && !$wgAbuseFilterIsCentral ) {
|
|
|
|
return $cache->makeGlobalKey(
|
|
|
|
'abusefilter', 'throttle', $wgAbuseFilterCentralDB, $throttleId, $type, $identifier
|
|
|
|
);
|
|
|
|
}
|
|
|
|
|
|
|
|
return $cache->makeKey( 'abusefilter', 'throttle', $throttleId, $type, $identifier );
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* @param string $type
|
|
|
|
* @return int|string
|
|
|
|
*/
|
|
|
|
protected function throttleIdentifier( $type ) {
|
|
|
|
$request = RequestContext::getMain()->getRequest();
|
|
|
|
|
|
|
|
switch ( $type ) {
|
|
|
|
case 'ip':
|
|
|
|
$identifier = $request->getIP();
|
|
|
|
break;
|
|
|
|
case 'user':
|
|
|
|
$identifier = $this->user->getId();
|
|
|
|
break;
|
|
|
|
case 'range':
|
2020-01-24 17:26:03 +00:00
|
|
|
$identifier = substr( IPUtils::toHex( $request->getIP() ), 0, 4 );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
break;
|
|
|
|
case 'creationdate':
|
2019-12-07 17:20:10 +00:00
|
|
|
$reg = (int)$this->user->getRegistration();
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$identifier = $reg - ( $reg % 86400 );
|
|
|
|
break;
|
|
|
|
case 'editcount':
|
|
|
|
// Hack for detecting different single-purpose accounts.
|
2019-12-07 17:20:10 +00:00
|
|
|
$identifier = (int)$this->user->getEditCount();
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
break;
|
|
|
|
case 'site':
|
|
|
|
$identifier = 1;
|
|
|
|
break;
|
|
|
|
case 'page':
|
|
|
|
$identifier = $this->title->getPrefixedText();
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
// Should never happen
|
|
|
|
// @codeCoverageIgnoreStart
|
|
|
|
$identifier = 0;
|
|
|
|
// @codeCoverageIgnoreEnd
|
|
|
|
}
|
|
|
|
|
|
|
|
return $identifier;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* @param string $action
|
|
|
|
* @param array $parameters
|
|
|
|
* @param string $ruleDescription
|
|
|
|
* @param int|string $ruleNumber
|
|
|
|
*
|
|
|
|
* @return array|null a message describing the action that was taken,
|
|
|
|
* or null if no action was taken. The message is given as an array
|
|
|
|
* containing the message key followed by any message parameters.
|
|
|
|
*/
|
|
|
|
protected function takeConsequenceAction( $action, $parameters, $ruleDescription, $ruleNumber ) {
|
|
|
|
global $wgAbuseFilterCustomActionsHandlers;
|
|
|
|
|
|
|
|
$message = null;
|
|
|
|
|
|
|
|
switch ( $action ) {
|
|
|
|
case 'disallow':
|
|
|
|
$msg = $parameters[0] ?? 'abusefilter-disallowed';
|
|
|
|
$message = [ $msg, $ruleDescription, $ruleNumber ];
|
|
|
|
break;
|
|
|
|
case 'rangeblock':
|
|
|
|
global $wgAbuseFilterRangeBlockSize, $wgBlockCIDRLimit;
|
|
|
|
|
|
|
|
$ip = RequestContext::getMain()->getRequest()->getIP();
|
2020-01-24 17:26:03 +00:00
|
|
|
$type = IPUtils::isIPv6( $ip ) ? 'IPv6' : 'IPv4';
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$CIDRsize = max( $wgAbuseFilterRangeBlockSize[$type], $wgBlockCIDRLimit[$type] );
|
|
|
|
$blockCIDR = $ip . '/' . $CIDRsize;
|
|
|
|
|
|
|
|
$this->doAbuseFilterBlock(
|
|
|
|
[
|
|
|
|
'desc' => $ruleDescription,
|
|
|
|
'number' => $ruleNumber
|
|
|
|
],
|
2020-01-24 17:26:03 +00:00
|
|
|
IPUtils::sanitizeRange( $blockCIDR ),
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
'1 week',
|
|
|
|
false
|
|
|
|
);
|
|
|
|
|
|
|
|
$message = [
|
|
|
|
'abusefilter-blocked-display',
|
|
|
|
$ruleDescription,
|
|
|
|
$ruleNumber
|
|
|
|
];
|
|
|
|
break;
|
|
|
|
case 'degroup':
|
|
|
|
if ( !$this->user->isAnon() ) {
|
|
|
|
// Pull the groups from the VariableHolder, so that they will always be computed.
|
|
|
|
// This allow us to pull the groups from the VariableHolder to undo the degroup
|
|
|
|
// via Special:AbuseFilter/revert.
|
2019-08-11 13:11:20 +00:00
|
|
|
$groups = $this->vars->getVar( 'user_groups', AbuseFilterVariableHolder::GET_LAX );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
if ( $groups->type !== AFPData::DARRAY ) {
|
|
|
|
// Somehow, the variable wasn't set
|
|
|
|
$groups = $this->user->getEffectiveGroups();
|
|
|
|
$this->vars->setVar( 'user_groups', $groups );
|
|
|
|
} else {
|
|
|
|
$groups = $groups->toNative();
|
|
|
|
}
|
|
|
|
$this->vars->setVar( 'user_groups', $groups );
|
|
|
|
|
|
|
|
foreach ( $groups as $group ) {
|
|
|
|
$this->user->removeGroup( $group );
|
|
|
|
}
|
|
|
|
|
|
|
|
$message = [
|
|
|
|
'abusefilter-degrouped',
|
|
|
|
$ruleDescription,
|
|
|
|
$ruleNumber
|
|
|
|
];
|
|
|
|
|
|
|
|
// Don't log it if there aren't any groups being removed!
|
|
|
|
if ( !count( $groups ) ) {
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
|
|
|
$logEntry = new ManualLogEntry( 'rights', 'rights' );
|
|
|
|
$logEntry->setPerformer( AbuseFilter::getFilterUser() );
|
|
|
|
$logEntry->setTarget( $this->user->getUserPage() );
|
|
|
|
$logEntry->setComment(
|
|
|
|
wfMessage(
|
|
|
|
'abusefilter-degroupreason',
|
|
|
|
$ruleDescription,
|
|
|
|
$ruleNumber
|
|
|
|
)->inContentLanguage()->text()
|
|
|
|
);
|
|
|
|
$logEntry->setParameters( [
|
|
|
|
'4::oldgroups' => $groups,
|
|
|
|
'5::newgroups' => []
|
|
|
|
] );
|
|
|
|
$logEntry->publish( $logEntry->insert() );
|
|
|
|
}
|
|
|
|
|
|
|
|
break;
|
|
|
|
case 'blockautopromote':
|
|
|
|
if ( !$this->user->isAnon() ) {
|
2019-07-06 23:35:03 +00:00
|
|
|
// Block for 5 days
|
2019-08-10 13:17:20 +00:00
|
|
|
$blocked = AbuseFilter::blockAutoPromote(
|
2019-07-06 23:35:03 +00:00
|
|
|
$this->user,
|
2018-07-16 12:10:36 +00:00
|
|
|
wfMessage(
|
|
|
|
'abusefilter-blockautopromotereason',
|
|
|
|
$ruleDescription,
|
|
|
|
$ruleNumber
|
|
|
|
)->inContentLanguage()->text()
|
|
|
|
);
|
|
|
|
|
2019-08-10 13:17:20 +00:00
|
|
|
if ( $blocked ) {
|
|
|
|
$message = [
|
|
|
|
'abusefilter-autopromote-blocked',
|
|
|
|
$ruleDescription,
|
|
|
|
$ruleNumber
|
|
|
|
];
|
|
|
|
} else {
|
|
|
|
$logger = LoggerFactory::getInstance( 'AbuseFilter' );
|
|
|
|
$logger->warning(
|
|
|
|
'Cannot block autopromotion to {target}',
|
|
|
|
[ 'target' => $this->user->getName() ]
|
|
|
|
);
|
|
|
|
}
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
}
|
|
|
|
break;
|
|
|
|
|
|
|
|
case 'block':
|
|
|
|
// Do nothing, handled at the end of executeFilterActions. Here for completeness.
|
|
|
|
break;
|
|
|
|
|
|
|
|
case 'tag':
|
|
|
|
// Mark with a tag on recentchanges.
|
|
|
|
$actionID = $this->getTaggingID();
|
|
|
|
AbuseFilter::bufferTagsToSetByAction( [ $actionID => $parameters ] );
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
if ( isset( $wgAbuseFilterCustomActionsHandlers[$action] ) ) {
|
|
|
|
$customFunction = $wgAbuseFilterCustomActionsHandlers[$action];
|
|
|
|
if ( is_callable( $customFunction ) ) {
|
|
|
|
$msg = call_user_func(
|
|
|
|
$customFunction,
|
|
|
|
$action,
|
|
|
|
$parameters,
|
|
|
|
$this->title,
|
|
|
|
$this->vars,
|
|
|
|
$ruleDescription,
|
|
|
|
$ruleNumber
|
|
|
|
);
|
|
|
|
}
|
|
|
|
if ( isset( $msg ) ) {
|
|
|
|
$message = [ $msg ];
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
$logger = LoggerFactory::getInstance( 'AbuseFilter' );
|
|
|
|
$logger->warning( "Unrecognised action $action" );
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return $message;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Perform a block by the AbuseFilter system user
|
|
|
|
* @param array $rule should have 'desc' and 'number'
|
|
|
|
* @param string $target
|
|
|
|
* @param string $expiry
|
|
|
|
* @param bool $isAutoBlock
|
|
|
|
* @param bool $preventEditOwnUserTalk
|
|
|
|
*/
|
|
|
|
private function doAbuseFilterBlock(
|
|
|
|
array $rule,
|
|
|
|
$target,
|
|
|
|
$expiry,
|
|
|
|
$isAutoBlock,
|
|
|
|
$preventEditOwnUserTalk = false
|
|
|
|
) {
|
|
|
|
$filterUser = AbuseFilter::getFilterUser();
|
|
|
|
$reason = wfMessage(
|
|
|
|
'abusefilter-blockreason',
|
|
|
|
$rule['desc'], $rule['number']
|
|
|
|
)->inContentLanguage()->text();
|
|
|
|
|
|
|
|
$block = new DatabaseBlock();
|
|
|
|
$block->setTarget( $target );
|
|
|
|
$block->setBlocker( $filterUser );
|
2019-10-19 23:32:02 +00:00
|
|
|
$block->setReason( $reason );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$block->isHardblock( false );
|
|
|
|
$block->isAutoblocking( $isAutoBlock );
|
|
|
|
$block->isCreateAccountBlocked( true );
|
|
|
|
$block->isUsertalkEditAllowed( !$preventEditOwnUserTalk );
|
2019-10-19 23:32:02 +00:00
|
|
|
$block->setExpiry( SpecialBlock::parseExpiryInput( $expiry ) );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
$success = $block->insert();
|
|
|
|
|
|
|
|
if ( $success ) {
|
|
|
|
// Log it only if the block was successful
|
|
|
|
$logParams = [];
|
2019-10-19 23:32:02 +00:00
|
|
|
$logParams['5::duration'] = ( $block->getExpiry() === 'infinity' )
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
? 'indefinite'
|
|
|
|
: $expiry;
|
|
|
|
$flags = [ 'nocreate' ];
|
2020-01-24 17:26:03 +00:00
|
|
|
if ( !$block->isAutoblocking() && !IPUtils::isIPAddress( $target ) ) {
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
// Conditionally added same as SpecialBlock
|
|
|
|
$flags[] = 'noautoblock';
|
|
|
|
}
|
|
|
|
if ( $preventEditOwnUserTalk === true ) {
|
|
|
|
$flags[] = 'nousertalk';
|
|
|
|
}
|
|
|
|
$logParams['6::flags'] = implode( ',', $flags );
|
|
|
|
|
|
|
|
$logEntry = new ManualLogEntry( 'block', 'block' );
|
|
|
|
$logEntry->setTarget( Title::makeTitle( NS_USER, $target ) );
|
|
|
|
$logEntry->setComment( $reason );
|
|
|
|
$logEntry->setPerformer( $filterUser );
|
|
|
|
$logEntry->setParameters( $logParams );
|
|
|
|
$blockIds = array_merge( [ $success['id'] ], $success['autoIds'] );
|
|
|
|
$logEntry->setRelations( [ 'ipb_id' => $blockIds ] );
|
|
|
|
$logEntry->publish( $logEntry->insert() );
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Constructs a Status object as returned by executeFilterActions() from the list of
|
|
|
|
* actions taken and the corresponding list of messages.
|
|
|
|
*
|
|
|
|
* @param array[] $actionsTaken associative array mapping each filter to the list if
|
|
|
|
* actions taken because of that filter.
|
|
|
|
* @param array[] $messages a list of arrays, where each array contains a message key
|
|
|
|
* followed by any message parameters.
|
|
|
|
*
|
|
|
|
* @return Status
|
|
|
|
*/
|
|
|
|
protected function buildStatus( array $actionsTaken, array $messages ) : Status {
|
|
|
|
$status = Status::newGood( $actionsTaken );
|
|
|
|
|
|
|
|
foreach ( $messages as $msg ) {
|
|
|
|
$status->fatal( ...$msg );
|
|
|
|
}
|
|
|
|
|
|
|
|
return $status;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Creates a template to use for logging taken actions
|
|
|
|
*
|
|
|
|
* @return array
|
|
|
|
*/
|
|
|
|
protected function buildLogTemplate() : array {
|
|
|
|
global $wgAbuseFilterLogIP;
|
|
|
|
|
|
|
|
$request = RequestContext::getMain()->getRequest();
|
|
|
|
$action = $this->vars->getVar( 'action' )->toString();
|
|
|
|
// If $this->user isn't safe to load (e.g. a failure during
|
|
|
|
// AbortAutoAccount), create a dummy anonymous user instead.
|
|
|
|
$user = $this->user->isSafeToLoad() ? $this->user : new User;
|
|
|
|
// Create a template
|
|
|
|
$logTemplate = [
|
|
|
|
'afl_user' => $user->getId(),
|
|
|
|
'afl_user_text' => $user->getName(),
|
|
|
|
'afl_timestamp' => wfGetDB( DB_REPLICA )->timestamp(),
|
|
|
|
'afl_namespace' => $this->title->getNamespace(),
|
|
|
|
'afl_title' => $this->title->getDBkey(),
|
|
|
|
'afl_action' => $action,
|
|
|
|
'afl_ip' => $wgAbuseFilterLogIP ? $request->getIP() : ''
|
|
|
|
];
|
|
|
|
// Hack to avoid revealing IPs of people creating accounts
|
|
|
|
if ( !$user->getId() && ( $action === 'createaccount' || $action === 'autocreateaccount' ) ) {
|
|
|
|
$logTemplate['afl_user_text'] = $this->vars->getVar( 'accountname' )->toString();
|
|
|
|
}
|
|
|
|
return $logTemplate;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Create and publish log entries for taken actions
|
|
|
|
*
|
|
|
|
* @param array[] $actionsTaken
|
|
|
|
* @todo Split this method
|
|
|
|
*/
|
|
|
|
protected function addLogEntries( array $actionsTaken ) {
|
|
|
|
$dbw = wfGetDB( DB_MASTER );
|
|
|
|
$logTemplate = $this->buildLogTemplate();
|
|
|
|
$centralLogTemplate = [
|
2020-01-08 13:33:10 +00:00
|
|
|
'afl_wiki' => WikiMap::getCurrentWikiDbDomain()->getId(),
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
];
|
|
|
|
|
|
|
|
$logRows = [];
|
|
|
|
$centralLogRows = [];
|
|
|
|
$loggedLocalFilters = [];
|
|
|
|
$loggedGlobalFilters = [];
|
|
|
|
|
|
|
|
foreach ( $actionsTaken as $filter => $actions ) {
|
|
|
|
list( $filterID, $global ) = AbuseFilter::splitGlobalName( $filter );
|
|
|
|
$thisLog = $logTemplate;
|
|
|
|
$thisLog['afl_filter'] = $filter;
|
|
|
|
$thisLog['afl_actions'] = implode( ',', $actions );
|
|
|
|
|
|
|
|
// Don't log if we were only throttling.
|
|
|
|
if ( $thisLog['afl_actions'] !== 'throttle' ) {
|
|
|
|
$logRows[] = $thisLog;
|
|
|
|
// Global logging
|
|
|
|
if ( $global ) {
|
|
|
|
$centralLog = $thisLog + $centralLogTemplate;
|
|
|
|
$centralLog['afl_filter'] = $filterID;
|
|
|
|
$centralLog['afl_title'] = $this->title->getPrefixedText();
|
|
|
|
$centralLog['afl_namespace'] = 0;
|
|
|
|
|
|
|
|
$centralLogRows[] = $centralLog;
|
|
|
|
$loggedGlobalFilters[] = $filterID;
|
|
|
|
} else {
|
|
|
|
$loggedLocalFilters[] = $filter;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if ( !count( $logRows ) ) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
// Only store the var dump if we're actually going to add log rows.
|
|
|
|
$varDump = AbuseFilter::storeVarDump( $this->vars );
|
|
|
|
// To distinguish from stuff stored directly
|
|
|
|
$varDump = "stored-text:$varDump";
|
|
|
|
|
|
|
|
$localLogIDs = [];
|
|
|
|
global $wgAbuseFilterNotifications, $wgAbuseFilterNotificationsPrivate;
|
|
|
|
foreach ( $logRows as $data ) {
|
|
|
|
$data['afl_var_dump'] = $varDump;
|
|
|
|
$dbw->insert( 'abuse_filter_log', $data, __METHOD__ );
|
|
|
|
$localLogIDs[] = $data['afl_id'] = $dbw->insertId();
|
|
|
|
// Give grep a chance to find the usages:
|
|
|
|
// logentry-abusefilter-hit
|
|
|
|
$entry = new ManualLogEntry( 'abusefilter', 'hit' );
|
|
|
|
// Construct a user object
|
|
|
|
$user = User::newFromId( $data['afl_user'] );
|
|
|
|
$user->setName( $data['afl_user_text'] );
|
|
|
|
$entry->setPerformer( $user );
|
|
|
|
$entry->setTarget( $this->title );
|
|
|
|
// Additional info
|
|
|
|
$entry->setParameters( [
|
|
|
|
'action' => $data['afl_action'],
|
|
|
|
'filter' => $data['afl_filter'],
|
|
|
|
'actions' => $data['afl_actions'],
|
|
|
|
'log' => $data['afl_id'],
|
|
|
|
] );
|
|
|
|
|
|
|
|
// Send data to CheckUser if installed and we
|
|
|
|
// aren't already sending a notification to recentchanges
|
|
|
|
if ( ExtensionRegistry::getInstance()->isLoaded( 'CheckUser' )
|
|
|
|
&& strpos( $wgAbuseFilterNotifications, 'rc' ) === false
|
|
|
|
) {
|
|
|
|
$rc = $entry->getRecentChange();
|
|
|
|
CheckUserHooks::updateCheckUserData( $rc );
|
|
|
|
}
|
|
|
|
|
|
|
|
if ( $wgAbuseFilterNotifications !== false ) {
|
|
|
|
list( $filterID, $global ) = AbuseFilter::splitGlobalName( $data['afl_filter'] );
|
|
|
|
if ( AbuseFilter::filterHidden( $filterID, $global ) && !$wgAbuseFilterNotificationsPrivate ) {
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
$this->publishEntry( $dbw, $entry, $wgAbuseFilterNotifications );
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
$method = __METHOD__;
|
|
|
|
|
|
|
|
if ( count( $loggedLocalFilters ) ) {
|
|
|
|
// Update hit-counter.
|
|
|
|
$dbw->onTransactionPreCommitOrIdle(
|
|
|
|
function () use ( $dbw, $loggedLocalFilters, $method ) {
|
|
|
|
$dbw->update( 'abuse_filter',
|
|
|
|
[ 'af_hit_count=af_hit_count+1' ],
|
|
|
|
[ 'af_id' => $loggedLocalFilters ],
|
|
|
|
$method
|
|
|
|
);
|
|
|
|
}
|
|
|
|
);
|
|
|
|
}
|
|
|
|
|
|
|
|
$globalLogIDs = [];
|
|
|
|
|
|
|
|
// Global stuff
|
|
|
|
if ( count( $loggedGlobalFilters ) ) {
|
|
|
|
$this->vars->computeDBVars();
|
|
|
|
$globalVarDump = AbuseFilter::storeVarDump( $this->vars, true );
|
|
|
|
$globalVarDump = "stored-text:$globalVarDump";
|
|
|
|
foreach ( $centralLogRows as $index => $data ) {
|
|
|
|
$centralLogRows[$index]['afl_var_dump'] = $globalVarDump;
|
|
|
|
}
|
|
|
|
|
|
|
|
$fdb = AbuseFilter::getCentralDB( DB_MASTER );
|
|
|
|
|
|
|
|
foreach ( $centralLogRows as $row ) {
|
|
|
|
$fdb->insert( 'abuse_filter_log', $row, __METHOD__ );
|
|
|
|
$globalLogIDs[] = $fdb->insertId();
|
|
|
|
}
|
|
|
|
|
|
|
|
$fdb->onTransactionPreCommitOrIdle(
|
|
|
|
function () use ( $fdb, $loggedGlobalFilters, $method ) {
|
|
|
|
$fdb->update( 'abuse_filter',
|
|
|
|
[ 'af_hit_count=af_hit_count+1' ],
|
|
|
|
[ 'af_id' => $loggedGlobalFilters ],
|
|
|
|
$method
|
|
|
|
);
|
|
|
|
}
|
|
|
|
);
|
|
|
|
}
|
|
|
|
|
|
|
|
AbuseFilter::$logIds[ $this->title->getPrefixedText() ] = [
|
|
|
|
'local' => $localLogIDs,
|
|
|
|
'global' => $globalLogIDs
|
|
|
|
];
|
|
|
|
|
|
|
|
$this->checkEmergencyDisable( $loggedLocalFilters );
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Like LogEntry::publish, but doesn't require an ID (which we don't have) and skips the
|
|
|
|
* tagging part
|
|
|
|
*
|
|
|
|
* @param IDatabase $dbw To cancel the callback if the log insertion fails
|
|
|
|
* @param ManualLogEntry $entry
|
|
|
|
* @param string $to One of 'udp', 'rc' and 'rcandudp'
|
|
|
|
*/
|
|
|
|
private function publishEntry( IDatabase $dbw, ManualLogEntry $entry, $to ) {
|
|
|
|
DeferredUpdates::addCallableUpdate(
|
|
|
|
function () use ( $entry, $to ) {
|
|
|
|
$rc = $entry->getRecentChange();
|
|
|
|
|
|
|
|
if ( $to === 'rc' || $to === 'rcandudp' ) {
|
|
|
|
$rc->save( $rc::SEND_NONE );
|
|
|
|
}
|
|
|
|
if ( $to === 'udp' || $to === 'rcandudp' ) {
|
|
|
|
$rc->notifyRCFeeds();
|
|
|
|
}
|
|
|
|
},
|
|
|
|
DeferredUpdates::POSTSEND,
|
|
|
|
$dbw
|
|
|
|
);
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Determine whether a filter must be throttled, i.e. its potentially dangerous
|
|
|
|
* actions must be disabled.
|
|
|
|
*
|
|
|
|
* @param string[] $filters The filters to check
|
|
|
|
*/
|
|
|
|
protected function checkEmergencyDisable( array $filters ) {
|
|
|
|
$stash = MediaWikiServices::getInstance()->getMainObjectStash();
|
|
|
|
// @ToDo this is an amount between 1 and AbuseFilterProfileActionsCap, which means that the
|
|
|
|
// reliability of this number may strongly vary. We should instead use a fixed one.
|
2015-04-01 04:51:16 +00:00
|
|
|
$groupProfile = $stash->get( AbuseFilter::filterProfileGroupKey( $this->group ) );
|
|
|
|
$totalActions = $groupProfile['total'];
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
foreach ( $filters as $filter ) {
|
|
|
|
$threshold = AbuseFilter::getEmergencyValue( 'threshold', $this->group );
|
|
|
|
$hitCountLimit = AbuseFilter::getEmergencyValue( 'count', $this->group );
|
|
|
|
$maxAge = AbuseFilter::getEmergencyValue( 'age', $this->group );
|
|
|
|
|
2019-02-10 10:57:37 +00:00
|
|
|
$filterProfile = $stash->get( AbuseFilter::filterProfileKey( $filter ) );
|
|
|
|
$matchCount = $filterProfile['matches'] ?? 1;
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
// Figure out if the filter is subject to being throttled.
|
2019-12-07 17:20:10 +00:00
|
|
|
$filterAge = (int)wfTimestamp( TS_UNIX, AbuseFilter::getFilter( $filter )->af_timestamp );
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
$exemptTime = $filterAge + $maxAge;
|
|
|
|
|
|
|
|
if ( $totalActions && $exemptTime > time() && $matchCount > $hitCountLimit &&
|
|
|
|
( $matchCount / $totalActions ) > $threshold
|
|
|
|
) {
|
|
|
|
// More than $wgAbuseFilterEmergencyDisableCount matches, constituting more than
|
|
|
|
// $threshold (a fraction) of last few edits. Disable it.
|
|
|
|
DeferredUpdates::addUpdate(
|
|
|
|
new AutoCommitUpdate(
|
|
|
|
wfGetDB( DB_MASTER ),
|
|
|
|
__METHOD__,
|
|
|
|
function ( IDatabase $dbw, $fname ) use ( $filter ) {
|
|
|
|
$dbw->update(
|
|
|
|
'abuse_filter',
|
|
|
|
[ 'af_throttled' => 1 ],
|
|
|
|
[ 'af_id' => $filter ],
|
|
|
|
$fname
|
|
|
|
);
|
|
|
|
}
|
|
|
|
)
|
|
|
|
);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Helper function to get the ID used to identify an action for later tagging it.
|
|
|
|
* @return string
|
|
|
|
*/
|
|
|
|
protected function getTaggingID() {
|
|
|
|
$action = $this->vars->getVar( 'action' )->toString();
|
|
|
|
if ( strpos( $action, 'createaccount' ) === false ) {
|
|
|
|
$username = $this->user->getName();
|
|
|
|
$actionTitle = $this->title;
|
|
|
|
} else {
|
|
|
|
$username = $this->vars->getVar( 'accountname' )->toString();
|
|
|
|
$actionTitle = Title::makeTitleSafe( NS_USER, $username );
|
|
|
|
}
|
2019-12-07 17:20:10 +00:00
|
|
|
'@phan-var Title $actionTitle';
|
Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.
Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2018-12-07 17:46:02 +00:00
|
|
|
|
|
|
|
return AbuseFilter::getTaggingActionId( $action, $actionTitle, $username );
|
|
|
|
}
|
|
|
|
}
|