wikimedia/mediawiki-extensions-Linter
1
0
Fork 0
mirror of https://gerrit.wikimedia.org/r/mediawiki/extensions/Linter synced 2024-11-28 01:30:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
mediawiki-extensions-Linter/includes/RecordLintJob.php

69 lines
2 KiB
PHP
Raw Normal View History

Update lint errors via the job queue The job queue will allow us to have better flood control and rate limiting instead of trying to do all the database writes as soon as parsoid contacts MediaWiki. On the downside, this means it may take longer for changes to be reflected in the database and to users, but we already have no promise for that, so it seems okay. Note that if you don't have a job queue runner set up, you'll need to run the runJobs.php script every time to have the jobs execute. Change-Id: I25fd54734aca4dab09711e7f6aee027654931300
2016-11-15 19:31:45 +00:00
<?php
/**
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
* http://www.gnu.org/copyleft/gpl.html
*
* @file
*/
namespace MediaWiki\Linter;
use Job;
use Title;
class RecordLintJob extends Job {
/**
* RecordLintJob constructor.
* @param Title $title
* @param array $params
*/
public function __construct( Title $title, array $params ) {
parent::__construct( 'RecordLintJob', $title, $params );
}
public function run() {
if ( $this->title->getLatestRevID() != $this->params['revision'] ) {
// Outdated now, let a later job handle it
return true;
}
De-duplicate errors and trim excessive errors in the same category It's possible to have duplicate, identical lint errors if the same exact error is repeated in a template transclusion (e.g. {{1x|<b/> <b/>}}) since the position via dsr is the same. In this case, just de-duplicate the errors since we can't differentiate them. At the same time, trim excessive errors on the same page in the same category. It's most likely that if a page has that many of the same errors, the editor or bot will just fix all of them at the same time, so we don't need to include all of them in the database. 20 is kind of a low value, but we can always increase it later on as necessary. Change-Id: I9cded720169870d0eea574e1a930ce4e9b190ac0
2016-11-24 03:47:04 +00:00
// [ 'category' => [ 'id' => LintError ] ]
Update lint errors via the job queue The job queue will allow us to have better flood control and rate limiting instead of trying to do all the database writes as soon as parsoid contacts MediaWiki. On the downside, this means it may take longer for changes to be reflected in the database and to users, but we already have no promise for that, so it seems okay. Note that if you don't have a job queue runner set up, you'll need to run the runJobs.php script every time to have the jobs execute. Change-Id: I25fd54734aca4dab09711e7f6aee027654931300
2016-11-15 19:31:45 +00:00
$errors = [];
De-duplicate errors and trim excessive errors in the same category It's possible to have duplicate, identical lint errors if the same exact error is repeated in a template transclusion (e.g. {{1x|<b/> <b/>}}) since the position via dsr is the same. In this case, just de-duplicate the errors since we can't differentiate them. At the same time, trim excessive errors on the same page in the same category. It's most likely that if a page has that many of the same errors, the editor or bot will just fix all of them at the same time, so we don't need to include all of them in the database. 20 is kind of a low value, but we can always increase it later on as necessary. Change-Id: I9cded720169870d0eea574e1a930ce4e9b190ac0
2016-11-24 03:47:04 +00:00
foreach ( $this->params['errors'] as $errorInfo ) {
$error = new LintError(
$errorInfo['type'],
Enforce category/page/position uniqueness constraint in the database Move location to two separate columns in the database: linter_start and linter_end. This allows us to have the database enforce the uniqueness of those fields, instead of just relying upon the PHP code to do so, which could be bypassed since we have multiple servers and concurrent processes. Change-Id: I3e67ce1b7cb3c93866a388ec3248af4cff2a81e0
2016-12-01 02:47:05 +00:00
$errorInfo['location'],
De-duplicate errors and trim excessive errors in the same category It's possible to have duplicate, identical lint errors if the same exact error is repeated in a template transclusion (e.g. {{1x|<b/> <b/>}}) since the position via dsr is the same. In this case, just de-duplicate the errors since we can't differentiate them. At the same time, trim excessive errors on the same page in the same category. It's most likely that if a page has that many of the same errors, the editor or bot will just fix all of them at the same time, so we don't need to include all of them in the database. 20 is kind of a low value, but we can always increase it later on as necessary. Change-Id: I9cded720169870d0eea574e1a930ce4e9b190ac0
2016-11-24 03:47:04 +00:00
$errorInfo['params']
Update lint errors via the job queue The job queue will allow us to have better flood control and rate limiting instead of trying to do all the database writes as soon as parsoid contacts MediaWiki. On the downside, this means it may take longer for changes to be reflected in the database and to users, but we already have no promise for that, so it seems okay. Note that if you don't have a job queue runner set up, you'll need to run the runJobs.php script every time to have the jobs execute. Change-Id: I25fd54734aca4dab09711e7f6aee027654931300
2016-11-15 19:31:45 +00:00
);
De-duplicate errors and trim excessive errors in the same category It's possible to have duplicate, identical lint errors if the same exact error is repeated in a template transclusion (e.g. {{1x|<b/> <b/>}}) since the position via dsr is the same. In this case, just de-duplicate the errors since we can't differentiate them. At the same time, trim excessive errors on the same page in the same category. It's most likely that if a page has that many of the same errors, the editor or bot will just fix all of them at the same time, so we don't need to include all of them in the database. 20 is kind of a low value, but we can always increase it later on as necessary. Change-Id: I9cded720169870d0eea574e1a930ce4e9b190ac0
2016-11-24 03:47:04 +00:00
// Use unique id as key to get rid of exact dupes
// (e.g. same category of error in same template)
$errors[$error->category][$error->id()] = $error;
Update lint errors via the job queue The job queue will allow us to have better flood control and rate limiting instead of trying to do all the database writes as soon as parsoid contacts MediaWiki. On the downside, this means it may take longer for changes to be reflected in the database and to users, but we already have no promise for that, so it seems okay. Note that if you don't have a job queue runner set up, you'll need to run the runJobs.php script every time to have the jobs execute. Change-Id: I25fd54734aca4dab09711e7f6aee027654931300
2016-11-15 19:31:45 +00:00
}
$lintDb = new Database( $this->title->getArticleID() );
De-duplicate errors and trim excessive errors in the same category It's possible to have duplicate, identical lint errors if the same exact error is repeated in a template transclusion (e.g. {{1x|<b/> <b/>}}) since the position via dsr is the same. In this case, just de-duplicate the errors since we can't differentiate them. At the same time, trim excessive errors on the same page in the same category. It's most likely that if a page has that many of the same errors, the editor or bot will just fix all of them at the same time, so we don't need to include all of them in the database. 20 is kind of a low value, but we can always increase it later on as necessary. Change-Id: I9cded720169870d0eea574e1a930ce4e9b190ac0
2016-11-24 03:47:04 +00:00
$toSet = [];
foreach ( $errors as $category => $catErrors ) {
// If there are too many errors for a category, trim some of them.
if ( count( $catErrors ) > $lintDb::MAX_PER_CAT ) {
$catErrors = array_slice( $catErrors, 0, $lintDb::MAX_PER_CAT );
}
$toSet = array_merge( $toSet, $catErrors );
}
$lintDb->setForPage( $toSet );
Update lint errors via the job queue The job queue will allow us to have better flood control and rate limiting instead of trying to do all the database writes as soon as parsoid contacts MediaWiki. On the downside, this means it may take longer for changes to be reflected in the database and to users, but we already have no promise for that, so it seems okay. Note that if you don't have a job queue runner set up, you'll need to run the runJobs.php script every time to have the jobs execute. Change-Id: I25fd54734aca4dab09711e7f6aee027654931300
2016-11-15 19:31:45 +00:00
return true;
}
}
Reference in a new issue Copy permalink