Commit graph

1063 commits

Author SHA1 Message Date
Kunal Mehta 9ba2839e7d Record totals in categories in statsd
For historical purposes so we can see new errors being added/fixed.

Bug: T152664
Change-Id: Icec62d236351e43ec397ac448805ab16131c943c
2016-12-08 16:52:52 -08:00
Kunal Mehta 782313088d Show error counts on Special:LintErrors
Change-Id: Ib49f2e391ca4b9d1eef5443c011d54a42921ce4e
2016-12-08 16:52:31 -08:00
Translation updater bot ca7a3e85c1 Localisation updates from https://translatewiki.net.
Change-Id: Ib5d48d20a2d5c02df1a42e45d6724a531a1fb450
2016-12-06 22:44:49 +01:00
Kunal Mehta df0f222fa3 Remove redundant index
Because linter_cat is the leftmost part of the linter_cat_page_position
index, we don't need a separate one.

Pointed out by jynus in T148866#2846381.

Change-Id: I3b0e75b02762cc948baf8dbdfee67b611aa3b9c1
2016-12-05 19:40:27 -08:00
Kunal Mehta 08b4c26b7d Add unit tests for LintError
Change-Id: I62ac7c2d652afb7a46e912ad6178ebeb3cff3ff2
2016-12-02 16:27:48 -08:00
Kunal Mehta 81ef6878af Include location in LintError::id()
Since it's no longer in $this->params, we need to explicitly include it.

Change-Id: I45f26db061f02066aa03a501dc022bf33e2c096e
2016-12-02 16:15:17 -08:00
Kunal Mehta 8e2d4e42ee Use INSERT IGNORE when putting new lint errors in the database
The most likely scenario of duplicate key errors is that it's the exact
same lint error and there's just a race condition when calculating which
new errors need to be inserted, so just ignore them.

Follows-up 419610bcdb.

Change-Id: I84749ab221bbd517b474be8875bb6a59e4f3258e
2016-12-02 15:54:02 -08:00
Kunal Mehta fa56bafacf Add basic integration tests for RecordLintJob
Change-Id: I6852d41f481282750b690b2e8122dcb224b4a107
2016-12-02 08:11:16 -08:00
Kunal Mehta a50398d4ce Remove unused LintError::equals()
Change-Id: I0d0d743119fee8a95627c365e1b2f45b8fcf67e7
2016-12-02 07:56:19 -08:00
Kunal Mehta 3a03781a70 Add basic integration tests for Special:LintErrors
Change-Id: I2d527ae6f4f615e73ea5dafa21ba678c1de835f7
2016-12-02 07:55:26 -08:00
Kunal Mehta 9d5759329f Add alias file
So people can localize the special page name

Change-Id: Icd331d02f400dfca465236bb95c1357c58480260
2016-12-02 07:55:13 -08:00
Kunal Mehta 14b53d6281 Add integration tests for Database class
These tests insert variations of fake lint errors into the database, and
then read out of the database to check they round-trip properly.

And while we're at it, improve the setForPage() return value.

These tests can be run with something like:
 php tests/phpunit/phpunit.php extensions/Linter/tests/phpunit/

Change-Id: Ifdba8a8a104d218a822f909bc5d7b3512aca499d
2016-11-30 21:17:51 -08:00
Kunal Mehta 419610bcdb Enforce category/page/position uniqueness constraint in the database
Move location to two separate columns in the database: linter_start and
linter_end. This allows us to have the database enforce the uniqueness
of those fields, instead of just relying upon the PHP code to do so,
which could be bypassed since we have multiple servers and concurrent
processes.

Change-Id: I3e67ce1b7cb3c93866a388ec3248af4cff2a81e0
2016-11-30 18:55:19 -08:00
Kunal Mehta 9550db450d Fix inserting errors if none exist for that page
If no errors existed for the page, inserting new ones would fail with a
database error since $errors was key-ed by unique id, which the database
wrapper interpreted as field names, causing issues. Using array_values()
gets rid of the keys, fixing the issue.

Change-Id: I7645de4e5d3ac4462d7980374c8ef8be6280442b
2016-11-30 18:17:36 -08:00
Translation updater bot 65cdec486c Localisation updates from https://translatewiki.net.
Change-Id: Idc28e0c6c5380c3bd421036d16574c85becd8da8
2016-11-27 22:29:17 +01:00
Translation updater bot ad6efbb7b1 Localisation updates from https://translatewiki.net.
Change-Id: Ie2bcb5ff42bb1947966a30e265d87824831d497d
2016-11-25 22:51:26 +01:00
Translation updater bot 8756b0d573 Localisation updates from https://translatewiki.net.
Change-Id: I3a1690d156832b4b8629b6ac97b0948d0271bb9c
2016-11-24 22:20:23 +01:00
jenkins-bot 603f480f5b Merge "De-duplicate errors and trim excessive errors in the same category" 2016-11-24 14:56:49 +00:00
Kunal Mehta 29379edb0b De-duplicate errors and trim excessive errors in the same category
It's possible to have duplicate, identical lint errors if the same exact
error is repeated in a template transclusion (e.g. {{1x|<b/> <b/>}})
since the position via dsr is the same. In this case, just de-duplicate
the errors since we can't differentiate them.

At the same time, trim excessive errors on the same page in the same
category. It's most likely that if a page has that many of the same
errors, the editor or bot will just fix all of them at the same time, so
we don't need to include all of them in the database. 20 is kind of a
low value, but we can always increase it later on as necessary.

Change-Id: I9cded720169870d0eea574e1a930ce4e9b190ac0
2016-11-23 19:47:04 -08:00
jenkins-bot eb8981bd79 Merge "Delete linter rows when pages are deleted" 2016-11-23 23:58:31 +00:00
Translation updater bot 66c9a0ca2c Localisation updates from https://translatewiki.net.
Change-Id: I3784c6c60a22796ab73ed05b78c58e26f56e7c46
2016-11-23 22:37:53 +01:00
jenkins-bot 5db0873adc Merge "API: Fix lntcategories parameter" 2016-11-23 02:41:52 +00:00
jenkins-bot cd819bdc65 Merge "API: Fix up action=record-lint's getAllowedParams()" 2016-11-23 02:40:22 +00:00
jenkins-bot 78e7e8f8b8 Merge "LintErrorsPager: Set explicit output format on some messages" 2016-11-23 02:39:36 +00:00
jenkins-bot 98ec5c3388 Merge "SpecialLintErrors: Implement getSubpagesForPrefixSearch()" 2016-11-23 02:38:15 +00:00
jenkins-bot d62cd4efc7 Merge "Hardcode category ids" 2016-11-23 02:37:04 +00:00
Kunal Mehta 70cc3ac190 Delete linter rows when pages are deleted
Bug: T151281
Change-Id: I3b113d0729b288683d9bc6c2f552b99ed2890f5c
2016-11-22 18:31:21 -08:00
Kunal Mehta 06671d2034 API: Fix lntcategories parameter
* Treat it as an array in all cases
* Make the default value all categories
* Rename from category to categories

Bug: T151288
Change-Id: I5c0c341112894c5a7ec3aaebb6ac9085353f55bd
2016-11-22 18:31:21 -08:00
Kunal Mehta 2f39ab7fff API: Fix up action=record-lint's getAllowedParams()
Since this module is internal, it doesn't really matter, but it cleans
up the output on api.php?modules=record-lint.

Bug: T151285
Change-Id: I859a2780d6ed1918cc81101e9e2c2cd348a2390a
2016-11-22 18:31:21 -08:00
Kunal Mehta d8437fc9c3 LintErrorsPager: Set explicit output format on some messages
Bug: T151283
Change-Id: Ic52bc11a6aacf7f5bb23d9273fa0bf2a27e0cfc4
2016-11-22 18:31:21 -08:00
Kunal Mehta 96c5e091f1 SpecialLintErrors: Implement getSubpagesForPrefixSearch()
Test plan:
Type "Special:LintErrors/" into the search box, and see that the
autocomplete dropdowns include the subpages for individual error
categories.

Bug: T151289
Change-Id: I919e0e51a3b956f275f9a372b2f2844002972ea7
2016-11-22 18:31:21 -08:00
Kunal Mehta b41e74ce8b Hardcode category ids
Instead of having a complex auto-increment category manager that had
additional caching, just hardcode the (currently 6) ids for each
category, which allows us to simplify a lot of code.

If Parsoid sends a lint error in a category that we don't know about, it
is silently dropped.

Bug: T151287
Change-Id: Ice6edf1b7985390aa0c1c410d357bc565bb69108
2016-11-22 18:31:17 -08:00
Translation updater bot 03be9d61c0 Localisation updates from https://translatewiki.net.
Change-Id: Ife6b67db3375c586d9ad769d76162a4141637d32
2016-11-22 23:25:00 +01:00
Justin Du 9970699910 API: Fix example URL for list=linterrors
The prefix should be lnt, not le.

Bug: T151284
Change-Id: Ib8ffbc9ee181e93da6b990ef8c788da7e49767cd
2016-11-21 20:25:29 -08:00
Translation updater bot 83b1003e6e Localisation updates from https://translatewiki.net.
Change-Id: Id6348fde3f5f397c2cc1c86d1b40f9c0426d629f
2016-11-20 22:19:03 +01:00
Translation updater bot d204de99e7 Localisation updates from https://translatewiki.net.
Change-Id: Ice589f18f39c9e540860e7b9fadda5e4e640fbff
2016-11-19 22:07:09 +01:00
Translation updater bot 18f9409dac Localisation updates from https://translatewiki.net.
Change-Id: I3b494371a68e9188f7850138b95f2ed812e18707
2016-11-18 22:15:18 +01:00
Translation updater bot 95f599ffd4 Localisation updates from https://translatewiki.net.
Change-Id: I7e316101614cf356b2ef2f207a0f7cd59eb92650
2016-11-17 22:53:42 +01:00
Arlo Breault 958623f0fb Display self-closed tags
Change-Id: I6c084a6c72114b57a21cd367d398c673c5c4d991
2016-11-17 12:32:46 -08:00
Translation updater bot 7e4bd54edb Localisation updates from https://translatewiki.net.
Change-Id: I15e06c49f1842cad48113d00e291848a8f51ce5c
2016-11-16 22:48:03 +01:00
jenkins-bot ff8ccb69c6 Merge "Update lint errors via the job queue" 2016-11-16 16:50:39 +00:00
Kunal Mehta 9ecf62ead3 Update lint errors via the job queue
The job queue will allow us to have better flood control and rate
limiting instead of trying to do all the database writes as soon as
parsoid contacts MediaWiki.

On the downside, this means it may take longer for changes to be
reflected in the database and to users, but we already have no promise
for that, so it seems okay.

Note that if you don't have a job queue runner set up, you'll need to
run the runJobs.php script every time to have the jobs execute.

Change-Id: I25fd54734aca4dab09711e7f6aee027654931300
2016-11-15 11:31:45 -08:00
Translation updater bot 86218c9c7f Localisation updates from https://translatewiki.net.
Change-Id: I5ea53b2125a1d09e8a6ad9cc9af21b47a245a9f8
2016-11-13 22:48:33 +01:00
Translation updater bot 881f69a997 Localisation updates from https://translatewiki.net.
Change-Id: I3b98b1b74414453713ec60fd0469feec6263f4c3
2016-11-12 22:54:15 +01:00
Translation updater bot 7b3eba647a Localisation updates from https://translatewiki.net.
Change-Id: I6c1621a7b5a8a392272c73d1bd55ea5c6a7bbb41
2016-11-11 22:57:05 +01:00
Translation updater bot 73a5da149b Localisation updates from https://translatewiki.net.
Change-Id: I44004c2c838b4e7c253bbbe16c37baa270bd6d51
2016-11-10 23:31:39 +01:00
Translation updater bot ad14ea3d98 Localisation updates from https://translatewiki.net.
Change-Id: Ib2cd4e4dedd5f9c3882b53fae1dcb996977d6c6a
2016-11-09 22:43:13 +01:00
Translation updater bot d2b5387c07 Localisation updates from https://translatewiki.net.
Change-Id: I7618c308bbd9dd82ddc864c05acd1f6a26436eb4
2016-11-08 22:46:38 +01:00
Translation updater bot 3a7867e193 Localisation updates from https://translatewiki.net.
Change-Id: I7a2f1891afa1228cd5b3315fbd4818df37f8a1ab
2016-11-04 22:50:40 +01:00
jenkins-bot 56426c2b73 Merge "Add index on linter.linter_cat" 2016-11-03 22:02:54 +00:00