mediawiki-extensions-AbuseF.../includes/AFComputedVariable.php

431 lines
13 KiB
PHP
Raw Normal View History

2009-02-27 00:03:30 +00:00
<?php
class AFComputedVariable {
public $mMethod, $mParameters;
public static $userCache = array();
public static $articleCache = array();
/**
* @param $method
* @param $parameters
*/
2009-02-27 00:03:30 +00:00
function __construct( $method, $parameters ) {
$this->mMethod = $method;
$this->mParameters = $parameters;
}
/**
* It's like Article::prepareContentForEdit, but not for editing (old wikitext usually)
*
*
* @param string $wikitext
* @param WikiPage $article
*
* @return object
*/
2009-02-27 00:03:30 +00:00
function parseNonEditWikitext( $wikitext, $article ) {
static $cache = array();
$cacheKey = md5( $wikitext ) . ':' . $article->getTitle()->getPrefixedText();
2009-05-22 06:42:10 +00:00
if ( isset( $cache[$cacheKey] ) ) {
2009-02-27 00:03:30 +00:00
return $cache[$cacheKey];
2009-05-22 06:42:10 +00:00
}
2009-02-27 00:03:30 +00:00
global $wgParser;
$edit = (object)array();
$options = new ParserOptions;
$options->setTidy( true );
$edit->output = $wgParser->parse( $wikitext, $article->getTitle(), $options );
2009-02-27 00:03:30 +00:00
$cache[$cacheKey] = $edit;
2009-02-27 00:03:30 +00:00
return $edit;
}
/**
AbuseFilter: Change format of database logging/ performance AF is setting several lazy load variables for the currently editing user. To do this it's passing along the user name extracted from a user object and generating a new user object later from that name which is of course pointless. With this patch I'll pass user objects directly to prevent that. On top of that I've deprecated a method in AFComputedVariable::compute which was redundant as there is a more generic one which can solve that task just fine. Furthermore I've changed the logging behaviour from serializing the whole AbuseFilterVariableHolder object to only store the variables. That has two major advantages: * The amount of data that needs to be saved on a filter hit is reduced to about 1/10 of what the old version needed. * This is much more forward compatible as the old way of saving this relied on the class structure to stay the same while this is a simple array containing the vars. On top of that we now only log variables already set by the time a filter is hit. On top of the obvious performance increasement that makes it easier for the user to spot the relevant data. Another thing this change alters is the way the AbuseFilter internally works with AbuseFilterVariableHolder objects. Right now we use one for testing the filter(s) and later we use another one to compute the same data again in case a filter was hit (for logging)! This is not thoroughly tested yet, but way more sane than what we're currently doing! Change-Id: Ib15e7501bff32a54afe2d103ef5aedb950e58ef6
2013-01-07 00:02:41 +00:00
* For backwards compatibility: Get the user object belonging to a certain name
* in case a user name is given as argument. Nowadays user objects are passed
* directly but many old log entries rely on this.
*
* @param $user string|User
* @return User
*/
AbuseFilter: Change format of database logging/ performance AF is setting several lazy load variables for the currently editing user. To do this it's passing along the user name extracted from a user object and generating a new user object later from that name which is of course pointless. With this patch I'll pass user objects directly to prevent that. On top of that I've deprecated a method in AFComputedVariable::compute which was redundant as there is a more generic one which can solve that task just fine. Furthermore I've changed the logging behaviour from serializing the whole AbuseFilterVariableHolder object to only store the variables. That has two major advantages: * The amount of data that needs to be saved on a filter hit is reduced to about 1/10 of what the old version needed. * This is much more forward compatible as the old way of saving this relied on the class structure to stay the same while this is a simple array containing the vars. On top of that we now only log variables already set by the time a filter is hit. On top of the obvious performance increasement that makes it easier for the user to spot the relevant data. Another thing this change alters is the way the AbuseFilter internally works with AbuseFilterVariableHolder objects. Right now we use one for testing the filter(s) and later we use another one to compute the same data again in case a filter was hit (for logging)! This is not thoroughly tested yet, but way more sane than what we're currently doing! Change-Id: Ib15e7501bff32a54afe2d103ef5aedb950e58ef6
2013-01-07 00:02:41 +00:00
static function getUserObject( $user ) {
if ( $user instanceof User ) {
$username = $user->getName();
} else {
$username = $user;
if ( isset( self::$userCache[$username] ) ) {
return self::$userCache[$username];
}
AbuseFilter: Change format of database logging/ performance AF is setting several lazy load variables for the currently editing user. To do this it's passing along the user name extracted from a user object and generating a new user object later from that name which is of course pointless. With this patch I'll pass user objects directly to prevent that. On top of that I've deprecated a method in AFComputedVariable::compute which was redundant as there is a more generic one which can solve that task just fine. Furthermore I've changed the logging behaviour from serializing the whole AbuseFilterVariableHolder object to only store the variables. That has two major advantages: * The amount of data that needs to be saved on a filter hit is reduced to about 1/10 of what the old version needed. * This is much more forward compatible as the old way of saving this relied on the class structure to stay the same while this is a simple array containing the vars. On top of that we now only log variables already set by the time a filter is hit. On top of the obvious performance increasement that makes it easier for the user to spot the relevant data. Another thing this change alters is the way the AbuseFilter internally works with AbuseFilterVariableHolder objects. Right now we use one for testing the filter(s) and later we use another one to compute the same data again in case a filter was hit (for logging)! This is not thoroughly tested yet, but way more sane than what we're currently doing! Change-Id: Ib15e7501bff32a54afe2d103ef5aedb950e58ef6
2013-01-07 00:02:41 +00:00
wfDebug( "Couldn't find user $username in cache\n" );
}
if ( count( self::$userCache ) > 1000 ) {
self::$userCache = array();
}
AbuseFilter: Change format of database logging/ performance AF is setting several lazy load variables for the currently editing user. To do this it's passing along the user name extracted from a user object and generating a new user object later from that name which is of course pointless. With this patch I'll pass user objects directly to prevent that. On top of that I've deprecated a method in AFComputedVariable::compute which was redundant as there is a more generic one which can solve that task just fine. Furthermore I've changed the logging behaviour from serializing the whole AbuseFilterVariableHolder object to only store the variables. That has two major advantages: * The amount of data that needs to be saved on a filter hit is reduced to about 1/10 of what the old version needed. * This is much more forward compatible as the old way of saving this relied on the class structure to stay the same while this is a simple array containing the vars. On top of that we now only log variables already set by the time a filter is hit. On top of the obvious performance increasement that makes it easier for the user to spot the relevant data. Another thing this change alters is the way the AbuseFilter internally works with AbuseFilterVariableHolder objects. Right now we use one for testing the filter(s) and later we use another one to compute the same data again in case a filter was hit (for logging)! This is not thoroughly tested yet, but way more sane than what we're currently doing! Change-Id: Ib15e7501bff32a54afe2d103ef5aedb950e58ef6
2013-01-07 00:02:41 +00:00
if ( $user instanceof User ) {
$userCache[$username] = $user;
return $user;
}
2009-05-22 06:42:10 +00:00
if ( IP::isIPAddress( $username ) ) {
$u = new User;
$u->setName( $username );
2009-05-22 06:42:10 +00:00
self::$userCache[$username] = $u;
return $u;
}
$user = User::newFromName( $username );
$user->load();
self::$userCache[$username] = $user;
return $user;
2009-02-27 00:03:30 +00:00
}
/**
* @param $namespace
* @param $title Title
* @return Article
*/
2009-02-27 00:03:30 +00:00
static function articleFromTitle( $namespace, $title ) {
2009-05-22 06:42:10 +00:00
if ( isset( self::$articleCache["$namespace:$title"] ) ) {
2009-02-27 00:03:30 +00:00
return self::$articleCache["$namespace:$title"];
2009-05-22 06:42:10 +00:00
}
if ( count( self::$articleCache ) > 1000 ) {
self::$articleCache = array();
}
wfDebug( "Creating article object for $namespace:$title in cache\n" );
// TODO: use WikiPage instead!
2009-02-27 00:03:30 +00:00
$t = Title::makeTitle( $namespace, $title );
self::$articleCache["$namespace:$title"] = new Article( $t );
2009-02-27 00:03:30 +00:00
return self::$articleCache["$namespace:$title"];
}
/**
* @param WikiPage $article
* @return array
*/
static function getLinksFromDB( $article ) {
// Stolen from ConfirmEdit
$id = $article->getId();
2009-05-22 06:42:10 +00:00
if ( !$id ) {
return array();
}
$dbr = wfGetDB( DB_SLAVE );
$res = $dbr->select(
'externallinks',
array( 'el_to' ),
array( 'el_from' => $id ),
__METHOD__
);
$links = array();
foreach ( $res as $row ) {
$links[] = $row->el_to;
}
return $links;
}
/**
* @param $vars AbuseFilterVariableHolder
* @return AFPData|array|int|mixed|null|string
* @throws MWException
* @throws AFPException
*/
2009-02-27 00:03:30 +00:00
function compute( $vars ) {
$parameters = $this->mParameters;
$result = null;
if ( !Hooks::run( 'AbuseFilter-interceptVariable',
array( $this->mMethod, $vars, $parameters, &$result ) ) ) {
return $result instanceof AFPData
? $result : AFPData::newFromPHPVar( $result );
}
switch ( $this->mMethod ) {
2009-02-27 00:03:30 +00:00
case 'diff':
$text1Var = $parameters['oldtext-var'];
$text2Var = $parameters['newtext-var'];
$text1 = $vars->getVar( $text1Var )->toString() . "\n";
$text2 = $vars->getVar( $text2Var )->toString() . "\n";
2009-02-27 00:03:30 +00:00
$result = wfDiff( $text1, $text2 );
break;
case 'diff-split':
$diff = $vars->getVar( $parameters['diff-var'] )->toString();
$line_prefix = $parameters['line-prefix'];
$diff_lines = explode( "\n", $diff );
$interest_lines = array();
foreach ( $diff_lines as $line ) {
if ( substr( $line, 0, 1 ) === $line_prefix ) {
$interest_lines[] = substr( $line, strlen( $line_prefix ) );
2009-02-27 00:03:30 +00:00
}
}
$result = $interest_lines;
2009-02-27 00:03:30 +00:00
break;
case 'links-from-wikitext':
// This should ONLY be used when sharing a parse operation with the edit.
/* @var WikiPage $article */
if ( isset( $parameters['article'] ) ) {
$article = $parameters['article'];
} else {
$article = self::articleFromTitle(
$parameters['namespace'],
$parameters['title']
);
}
if ( $article->getContentModel() === CONTENT_MODEL_WIKITEXT ) {
$textVar = $parameters['text-var'];
// XXX: Use prepareContentForEdit. But we need a Content object for that.
$new_text = $vars->getVar( $textVar )->toString();
$content = ContentHandler::makeContent( $new_text, $article->getTitle() );
$editInfo = $article->prepareContentForEdit( $content );
$links = array_keys( $editInfo->output->getExternalLinks() );
$result = $links;
break;
}
// Otherwise fall back to database
2009-03-19 03:10:18 +00:00
case 'links-from-wikitext-nonedit':
case 'links-from-wikitext-or-database':
// TODO: use Content object instead, if available! In any case, use WikiPage, not Article.
$article = self::articleFromTitle(
$parameters['namespace'],
$parameters['title']
);
if ( $vars->getVar( 'context' )->toString() == 'filter' ) {
$links = $this->getLinksFromDB( $article );
wfDebug( "AbuseFilter: loading old links from DB\n" );
} elseif ( $article->getContentModel() === CONTENT_MODEL_WIKITEXT ) {
wfDebug( "AbuseFilter: loading old links from Parser\n" );
$textVar = $parameters['text-var'];
$wikitext = $vars->getVar( $textVar )->toString();
$editInfo = $this->parseNonEditWikitext( $wikitext, $article );
$links = array_keys( $editInfo->output->getExternalLinks() );
} else {
// TODO: Get links from Content object. But we don't have the content object.
// And for non-text content, $wikitext is usually not going to be a valid
// serialization, but rather some dummy text for filtering.
$links = array();
}
$result = $links;
2009-02-27 00:03:30 +00:00
break;
case 'link-diff-added':
case 'link-diff-removed':
$oldLinkVar = $parameters['oldlink-var'];
$newLinkVar = $parameters['newlink-var'];
2009-02-27 00:03:30 +00:00
$oldLinks = $vars->getVar( $oldLinkVar )->toString();
$newLinks = $vars->getVar( $newLinkVar )->toString();
2009-02-27 00:03:30 +00:00
$oldLinks = explode( "\n", $oldLinks );
$newLinks = explode( "\n", $newLinks );
if ( $this->mMethod == 'link-diff-added' ) {
$result = array_diff( $newLinks, $oldLinks );
}
if ( $this->mMethod == 'link-diff-removed' ) {
$result = array_diff( $oldLinks, $newLinks );
}
2009-02-27 00:03:30 +00:00
break;
case 'parse-wikitext':
// Should ONLY be used when sharing a parse operation with the edit.
if ( isset( $parameters['article'] ) ) {
$article = $parameters['article'];
} else {
$article = self::articleFromTitle(
$parameters['namespace'],
$parameters['title']
);
}
if ( $article->getContentModel() === CONTENT_MODEL_WIKITEXT ) {
$textVar = $parameters['wikitext-var'];
$new_text = $vars->getVar( $textVar )->toString();
$content = ContentHandler::makeContent( $new_text, $article->getTitle() );
$editInfo = $article->prepareContentForEdit( $content );
if ( isset( $parameters['pst'] ) && $parameters['pst'] ) {
$result = $editInfo->pstContent->serialize( $editInfo->format );
} else {
$newHTML = $editInfo->output->getText();
// Kill the PP limit comments. Ideally we'd just remove these by not setting the
// parser option, but then we can't share a parse operation with the edit, which is bad.
$result = preg_replace( '/<!--\s*NewPP limit report[^>]*-->\s*$/si', '', $newHTML );
}
break;
}
// Otherwise fall back to database
2009-02-27 00:03:30 +00:00
case 'parse-wikitext-nonedit':
// TODO: use Content object instead, if available! In any case, use WikiPage, not Article.
2009-02-27 00:03:30 +00:00
$article = self::articleFromTitle( $parameters['namespace'], $parameters['title'] );
$textVar = $parameters['wikitext-var'];
if ( $article->getContentModel() === CONTENT_MODEL_WIKITEXT ) {
if ( isset( $parameters['pst'] ) && $parameters['pst'] ) {
// $textVar is already PSTed when it's not loaded from an ongoing edit.
$result = $vars->getVar( $textVar )->toString();
} else {
$text = $vars->getVar( $textVar )->toString();
$editInfo = $this->parseNonEditWikitext( $text, $article );
$result = $editInfo->output->getText();
}
} else {
// TODO: Parser Output from Content object. But we don't have the content object.
// And for non-text content, $wikitext is usually not going to be a valid
// serialization, but rather some dummy text for filtering.
$result = '';
}
2009-02-27 00:03:30 +00:00
break;
case 'strip-html':
$htmlVar = $parameters['html-var'];
$html = $vars->getVar( $htmlVar )->toString();
$result = StringUtils::delimiterReplace( '<', '>', '', $html );
2009-02-27 00:03:30 +00:00
break;
case 'load-recent-authors':
$title = Title::makeTitle( $parameters['namespace'], $parameters['title'] );
if ( !$title->exists() ) {
$result = '';
break;
}
$result = self::getLastPageAuthors( $title );
break;
case 'load-first-author':
$title = Title::makeTitle( $parameters['namespace'], $parameters['title'] );
$revision = $title->getFirstRevision();
if ( $revision ) {
$result = $revision->getUserText();
} else {
$result = '';
}
2009-02-27 00:03:30 +00:00
break;
case 'get-page-restrictions':
$action = $parameters['action'];
$title = Title::makeTitle( $parameters['namespace'], $parameters['title'] );
2009-02-27 00:03:30 +00:00
$rights = $title->getRestrictions( $action );
$rights = count( $rights ) ? $rights : array();
$result = $rights;
2009-02-27 00:03:30 +00:00
break;
case 'simple-user-accessor':
$user = $parameters['user'];
$method = $parameters['method'];
if ( !$user ) {
throw new MWException( 'No user parameter given.' );
2009-02-27 13:16:34 +00:00
}
AbuseFilter: Change format of database logging/ performance AF is setting several lazy load variables for the currently editing user. To do this it's passing along the user name extracted from a user object and generating a new user object later from that name which is of course pointless. With this patch I'll pass user objects directly to prevent that. On top of that I've deprecated a method in AFComputedVariable::compute which was redundant as there is a more generic one which can solve that task just fine. Furthermore I've changed the logging behaviour from serializing the whole AbuseFilterVariableHolder object to only store the variables. That has two major advantages: * The amount of data that needs to be saved on a filter hit is reduced to about 1/10 of what the old version needed. * This is much more forward compatible as the old way of saving this relied on the class structure to stay the same while this is a simple array containing the vars. On top of that we now only log variables already set by the time a filter is hit. On top of the obvious performance increasement that makes it easier for the user to spot the relevant data. Another thing this change alters is the way the AbuseFilter internally works with AbuseFilterVariableHolder objects. Right now we use one for testing the filter(s) and later we use another one to compute the same data again in case a filter was hit (for logging)! This is not thoroughly tested yet, but way more sane than what we're currently doing! Change-Id: Ib15e7501bff32a54afe2d103ef5aedb950e58ef6
2013-01-07 00:02:41 +00:00
$obj = self::getUserObject( $user );
if ( !$obj ) {
2009-02-27 13:16:34 +00:00
throw new MWException( "Invalid username $user" );
}
$result = call_user_func( array( $obj, $method ) );
2009-02-27 00:03:30 +00:00
break;
case 'user-age':
$user = $parameters['user'];
$asOf = $parameters['asof'];
AbuseFilter: Change format of database logging/ performance AF is setting several lazy load variables for the currently editing user. To do this it's passing along the user name extracted from a user object and generating a new user object later from that name which is of course pointless. With this patch I'll pass user objects directly to prevent that. On top of that I've deprecated a method in AFComputedVariable::compute which was redundant as there is a more generic one which can solve that task just fine. Furthermore I've changed the logging behaviour from serializing the whole AbuseFilterVariableHolder object to only store the variables. That has two major advantages: * The amount of data that needs to be saved on a filter hit is reduced to about 1/10 of what the old version needed. * This is much more forward compatible as the old way of saving this relied on the class structure to stay the same while this is a simple array containing the vars. On top of that we now only log variables already set by the time a filter is hit. On top of the obvious performance increasement that makes it easier for the user to spot the relevant data. Another thing this change alters is the way the AbuseFilter internally works with AbuseFilterVariableHolder objects. Right now we use one for testing the filter(s) and later we use another one to compute the same data again in case a filter was hit (for logging)! This is not thoroughly tested yet, but way more sane than what we're currently doing! Change-Id: Ib15e7501bff32a54afe2d103ef5aedb950e58ef6
2013-01-07 00:02:41 +00:00
$obj = self::getUserObject( $user );
if ( $obj->getId() == 0 ) {
$result = 0;
break;
}
2009-02-27 00:03:30 +00:00
$registration = $obj->getRegistration();
$result = wfTimestamp( TS_UNIX, $asOf ) - wfTimestampOrNull( TS_UNIX, $registration );
2009-02-27 00:03:30 +00:00
break;
case 'user-groups':
AbuseFilter: Change format of database logging/ performance AF is setting several lazy load variables for the currently editing user. To do this it's passing along the user name extracted from a user object and generating a new user object later from that name which is of course pointless. With this patch I'll pass user objects directly to prevent that. On top of that I've deprecated a method in AFComputedVariable::compute which was redundant as there is a more generic one which can solve that task just fine. Furthermore I've changed the logging behaviour from serializing the whole AbuseFilterVariableHolder object to only store the variables. That has two major advantages: * The amount of data that needs to be saved on a filter hit is reduced to about 1/10 of what the old version needed. * This is much more forward compatible as the old way of saving this relied on the class structure to stay the same while this is a simple array containing the vars. On top of that we now only log variables already set by the time a filter is hit. On top of the obvious performance increasement that makes it easier for the user to spot the relevant data. Another thing this change alters is the way the AbuseFilter internally works with AbuseFilterVariableHolder objects. Right now we use one for testing the filter(s) and later we use another one to compute the same data again in case a filter was hit (for logging)! This is not thoroughly tested yet, but way more sane than what we're currently doing! Change-Id: Ib15e7501bff32a54afe2d103ef5aedb950e58ef6
2013-01-07 00:02:41 +00:00
// Deprecated but needed by old log entries
2009-02-27 00:03:30 +00:00
$user = $parameters['user'];
AbuseFilter: Change format of database logging/ performance AF is setting several lazy load variables for the currently editing user. To do this it's passing along the user name extracted from a user object and generating a new user object later from that name which is of course pointless. With this patch I'll pass user objects directly to prevent that. On top of that I've deprecated a method in AFComputedVariable::compute which was redundant as there is a more generic one which can solve that task just fine. Furthermore I've changed the logging behaviour from serializing the whole AbuseFilterVariableHolder object to only store the variables. That has two major advantages: * The amount of data that needs to be saved on a filter hit is reduced to about 1/10 of what the old version needed. * This is much more forward compatible as the old way of saving this relied on the class structure to stay the same while this is a simple array containing the vars. On top of that we now only log variables already set by the time a filter is hit. On top of the obvious performance increasement that makes it easier for the user to spot the relevant data. Another thing this change alters is the way the AbuseFilter internally works with AbuseFilterVariableHolder objects. Right now we use one for testing the filter(s) and later we use another one to compute the same data again in case a filter was hit (for logging)! This is not thoroughly tested yet, but way more sane than what we're currently doing! Change-Id: Ib15e7501bff32a54afe2d103ef5aedb950e58ef6
2013-01-07 00:02:41 +00:00
$obj = self::getUserObject( $user );
$result = $obj->getEffectiveGroups();
2009-02-27 00:03:30 +00:00
break;
case 'length':
$s = $vars->getVar( $parameters['length-var'] )->toString();
$result = strlen( $s );
break;
case 'subtract':
$v1 = $vars->getVar( $parameters['val1-var'] )->toFloat();
$v2 = $vars->getVar( $parameters['val2-var'] )->toFloat();
$result = $v1 - $v2;
break;
case 'revision-text-by-id':
$rev = Revision::newFromId( $parameters['revid'] );
$result = AbuseFilter::revisionToString( $rev );
2009-02-27 00:03:30 +00:00
break;
case 'revision-text-by-timestamp':
$timestamp = $parameters['timestamp'];
$title = Title::makeTitle( $parameters['namespace'], $parameters['title'] );
$dbr = wfGetDB( DB_SLAVE );
$rev = Revision::loadFromTimestamp( $dbr, $title, $timestamp );
$result = AbuseFilter::revisionToString( $rev );
2009-02-27 00:03:30 +00:00
break;
default:
if ( Hooks::run( 'AbuseFilter-computeVariable',
array( $this->mMethod, $vars, $parameters, &$result ) ) ) {
throw new AFPException( 'Unknown variable compute type ' . $this->mMethod );
}
2009-02-27 00:03:30 +00:00
}
return $result instanceof AFPData
? $result : AFPData::newFromPHPVar( $result );
2009-02-27 00:03:30 +00:00
}
/**
* @param Title $title
* @return string[] List of the last 10 (unique) authors from $title
*/
public static function getLastPageAuthors( Title $title ) {
if ( !$title->exists() ) {
return array();
}
$cache = ObjectCache::getMainWANInstance();
return $cache->getWithSetCallback(
$cache->makeKey( 'last-10-authors', 'revision', $title->getLatestRevID() ),
$cache::TTL_MINUTE,
function ( $oldValue, &$ttl, array &$setOpts ) use ( $title ) {
$dbr = wfGetDB( DB_SLAVE );
$setOpts += Database::getCacheSetOptions( $dbr );
// Get the last 100 edit authors with a trivial query (avoid T116557)
$revAuthors = $dbr->selectFieldValues(
'revision',
'rev_user_text',
array( 'rev_page' => $title->getArticleID() ),
__METHOD__,
// Some pages have < 10 authors but many revisions (e.g. bot pages)
array( 'ORDER BY' => 'rev_timestamp DESC', 'LIMIT' => 100 )
);
// Get the last 10 distinct authors within this set of edits
$users = array();
foreach ( $revAuthors as $author ) {
$users[$author] = 1;
if ( count( $users ) >= 10 ) {
break;
}
}
return array_keys( $users );
}
);
}
}