Commit graph

181 commits

Author SHA1 Message Date
Daimona Eaytoy b2af2f0bf5 Fix global caching
Use a more explicit TTL_WEEK, and add the version to avoid breaking the
world if we change the format.

Bug: T227299
Change-Id: I22705496ed8541c3dd9b643d78dff8886f4ff070
2019-07-05 09:17:57 +02:00
Aaron Schulz 2cf7b58434 Convert wfGetDB() calls to using getConnectionRef()
This handles the logic of calling reuseConnection() automatically

Change-Id: I9328e709fe5d81099338a31deef24d34db22d784
2019-07-04 15:09:32 -07:00
Daimona Eaytoy 85b46268f2 Rename the filterAction hook and add a parameter
The 'AbuseFilter-filterAction' hook is deprecated in favour of a new
'AbuseFilterAlterVariables' hook, which provides a User object and has a better name, since it reflects what it should be used for, and doesn't include the name of a function which will be removed. The hook will be hard deprecated in a subsequent patch, to avoid test failures.

Depends-On: Ic0ecc8746e2883c746bef815a0fee4131f1a0646
Change-Id: I212b1e09e9c05d487d96b2f4c28f2a613e6ff3cf
2019-07-04 18:10:47 +00:00
Daimona Eaytoy b7f1a7d459 Enforce saving of full abuse_filter row in cache
This is somehow a follow-up of
Ieb04f019453033c275e211cfc9fd68d5d7c392ef. A new method is introduced to
cache a filter, which checks that all fields are there.

Depends-On: I7c1ea17adf7f42cf9260d416906bfbf3b8a20688
Change-Id: Ic0ecc8746e2883c746bef815a0fee4131f1a0646
2019-07-04 18:10:33 +00:00
Daimona Eaytoy 6ea767f171 Tweak methods related to global filters
To make the switch to afl_filter_id and afl_global easier.

Bug: T227095
Depends-On: Ie550889495232b534c0f9aec31039cf21b2135b1
Change-Id: If557bad8f5c1a6d15e3556e4bfbd0330d7d49c59
2019-07-02 17:02:50 +02:00
Daimona Eaytoy 6100955242 Use more verbose names for filter IDs
Follow-up of Ie550889495232b534c0f9aec31039cf21b2135b1, suggested by
Krinkle.

Change-Id: Ia8f40644c7f4a6ed53186a5edab5df1bd313166a
2019-06-25 18:20:32 +00:00
Daimona Eaytoy 382751a707 Move conditions-related stuff inside AbuseFilterParser
Instead of relying on static methods and members in the AbuseFilter
class, move everything related to conditions inside the Parser, as the
amount of used conditions is something pertaining a single
AbuseFilter(Caching)Parser instance.
This change requires changing some signatures and adding parameters,
but will make introducing the new AbuseFilterRunner class easier (and
that will clean signatures, too).

Depends-On: I5b29ff556eca45fe59d15e2e3df4d06f1f6b3934
Change-Id: I7c1ea17adf7f42cf9260d416906bfbf3b8a20688
2019-06-19 15:14:17 +00:00
Daimona Eaytoy a8e8611509 Remove log_ids meta-variables
This is the second part of removing meta-variables. To achieve this, a
static property is added and another one removed.

Depends-On: I7f60df24dc8e706af289ebbbde7536c0baf8d5c3
Change-Id: I5b29ff556eca45fe59d15e2e3df4d06f1f6b3934
2019-06-19 14:44:56 +00:00
Daimona Eaytoy 246d8e78aa Improve getFirstFilterChange function
Fix the typehint, and use selectField instead of selectRow.
Follow-up of Ie550889495232b534c0f9aec31039cf21b2135b1 suggested by
Krinkle.

Change-Id: I7e74b7b484dfa487db96598ef7aef4895d7bf275
2019-06-17 13:01:56 +02:00
Daimona Eaytoy e7cd4b2a98 Rewrite AbuseFilter::decodeGlobalName
Now it returns an array with a bit more info, and has a different name
to reflect the fact that its input is now split in two parts. Plus, make
it throw whenever it gets an unexpected input, and add a bunch of test
cases for it.

Depends-On: Ib5fdeb75c1324f672b4ded39681f006fde34b4d1
Change-Id: Ie550889495232b534c0f9aec31039cf21b2135b1
2019-06-12 23:56:25 +00:00
Thalia 22ceae7e23 Use MediaWiki\Block\DatabaseBlock instead of Block
This follows the rename of the Block class in I6d96b63ca0.

Change-Id: I44cf9eb68c23a8299316effa4dee7f732486dd84
2019-05-31 16:08:19 +01:00
Daimona Eaytoy 864d2b7e03 Don't run filters with null title
As all title variables would be null, and the result pretty meaningless.

NOTE: Please vote V+2 and submit manually. I359a618ffc4e45ce1fb70f2d
should then be +2ed right after that. This way, there is no need to create
two more patches just for a handful of tests being broken for a minute.

Bug: T144265
Bug: T219030
Depends-On: If6b91711534c0d60e1aa27bd5748c3023e29f376
Change-Id: I5a9db6e7c4356c9662a0b0a51e66252555b3d998
2019-05-25 16:27:08 +00:00
jenkins-bot e5a15ab968 Merge "Add a parameter to generate(User|Title)Vars hooks to specify context" 2019-05-25 11:37:04 +00:00
Daimona Eaytoy 39fc7c12af Restore unit tests for CachingParser and fix it
Added cachingParser back to *all* the parser tests, fixed a couple of
differences with the normal parser, and added a couple of tests so that
any cachingParser-related file has 100% coverage. Also move the remaining
get_matches tests inside parserTests, and specify the parser used in case of failure.
This also adds a new base class for parser-related tests with a couple
of util methods.

Bug: T201193
Change-Id: I980aec3481a52ecc35f1811a366014a5581a7cdb
2019-05-25 10:55:24 +02:00
Daimona Eaytoy a766e39ade Add unit tests for profiling
Yet another important part to have covered. While for normal edits it
already works, for stashed ones it doesn't. That's why we need the patch
for checkAllFilters. Since for stashed edits profiling stats are all
zeros, this may explain T201334.
Changed the timestamp variable to use wfTimestamp instead of time() so
that we can fake it inside unit tests.
In a subsequent patch we should add average runtime conditions to tests
(really tricky).

Bug: T201193
Depends-On: Ib17821240b25c972a187e6b5eae42c5ada6c65e7
Change-Id: I5ee7ba44a6cd82a5ddb24fb4127af04d96e647f4
2019-05-23 08:47:40 +00:00
jenkins-bot a9afdc1f80 Merge "Slightly improve degroup action, remove Phan suppressions" 2019-05-19 17:20:23 +00:00
jenkins-bot c52850aae7 Merge "Add missing limits to explode() calls" 2019-05-15 15:06:18 +00:00
Thiemo Kreuz c6f20a64dd Add missing limits to explode() calls
This is fixing potential bugs where invalid strings with more than one
comma have silently been accepted.

Change-Id: Ib1e7d0c99973f243ef6faad6389bab688187c1cf
2019-05-15 16:14:12 +02:00
Thiemo Kreuz fa3ce90851 Remove comments literally repeating what the code says
I find it obvious that a file called "AbuseFilterTokenizerTest" is a
"test for the AbuseFilterTokenizer class". A comment that is just
repeating this information is typicalls not helpful, but distracting
and a potential source of mistakes, e.g. when stuff is copy-pasted,
but the comment not adjusted.

Change-Id: I1d4cc06e9e5631955ff73bf675090cf9c33c9390
2019-05-15 16:04:32 +02:00
Thalia f23905c402 Remove call to deprecated User::isBlocked
Change-Id: Ibb7412f8aa08a745a211b9b0581ccb6b0ca9eff5
2019-05-14 13:14:57 +01:00
jenkins-bot 06cac3682e Merge "Replace deprecated cache-related functions" 2019-05-01 16:27:56 +00:00
Aaron Schulz bc04dd93fe Avoid sending stashing statsd data for bots in AbuseFilter::filterAction
Change-Id: Ic06f64c22fc94665e58620a98e17264d48c97deb
2019-04-22 17:45:51 -07:00
Daimona Eaytoy 4bcb64b01a Increase code coverage a bit
This patch mostly adds coverageIgnore comments for intendedly
unreachable code etc. Some of them could be made testable by adding a new
filter function (e.g. array cast), but this patch is meant to be
comment-only (aside from the parser test).
Ignoring coverage for these lines makes some methods reach 100%
coverage, which in turn makes it easier to look at the coverage chart
and identify at a glance which parts of the code *really* need to be
covered.

Bug: T201193
Change-Id: Ic30883f7d261d974a2be46308d023e2714119e95
2019-04-13 18:30:14 +02:00
jenkins-bot 903f3db8fe Merge "Beautify old, broken abuse_filter_history rows" 2019-04-10 05:11:38 +00:00
Daimona Eaytoy 25ed009518 Beautify old, broken abuse_filter_history rows
And right when the throttle script seemed complete... Here is another
function! So, this change splits the logic in new functions called
sequentially, and the only actual change is adding the beautifyHistory
function. Its purpose is to search ANY row in abuse_filter_history with
empty/missing parameters and normalize it. More specifically, missing
period and count are inserted as 0, and for missing groups we add
"none", used by a newly introduced message. This way, messages shown on
Special:AbuseFilter/history will be clearer and won't have gaps.

Bug:T209565
Bug:T215787
Change-Id: I38395f4df9d83badfd26cdf584ffba743b6417a9
2019-04-10 04:51:58 +00:00
jenkins-bot efe32b7c93 Merge "Add doc for every class member" 2019-04-06 14:37:19 +00:00
jenkins-bot e03488b66a Merge "Overhaul tag selector" 2019-04-06 12:35:20 +00:00
Daimona Eaytoy a777b681e2 Don't publish LogEntries without ID
Mimic what publish() does, for the part that we need.

Bug: T219951
Bug: T218940
Change-Id: I16dd7071837a6965934d08b770f455f45cd02a6b
2019-04-06 09:46:09 +02:00
Daimona Eaytoy 451666272e Slightly improve degroup action, remove Phan suppressions
Try to get the groups from the var first, and compute them if they don't
exist. Use getEffectiveGroups instead of getGroups as it's done when
setting the lazy var loader. Avoid a pointless array_intersect within an
array_diff. Remove Phan @suppress and add docblock to make it pass.

Change-Id: I49ec6a1264b767cefea55df66ef3b02d4f443b57
2019-03-30 09:53:51 +00:00
Daimona Eaytoy 72c2be7a18 Remove $wgAbuseFilterRuntimeProfiling
The reasoning is similar to the one of the parent patch (Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb). Plus, it records runtime metrics on action different than edits, as there's no reason not to do it.
No performance issues in production.

Bug: T191039
Depends-On: Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb
Change-Id: Ib1112e2fefd0631550d386ba87e5f87db84c3036
2019-03-23 11:31:18 +00:00
Daimona Eaytoy 89520e2353 Remove $wgAbuseFilterProfiling
This variable was introduced to selectively enable profiling because
stats recording was bad for performance. Nowadays, stats are recorded in
a deferredupdate and don't harm performance anymore. Thus, this variable
can be removed and profiling be enabled by default.

Bug: T191039
Depends-On: Ib5fdeb75c1324f672b4ded39681f006fde34b4d1
Change-Id: Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb
2019-03-23 11:31:11 +00:00
Daimona Eaytoy c095069918 Log throttled actions
So that we have some numbers and debugging info.

Change-Id: I490504bfac09d537be5bca36ef52992b6aa4f0cb
2019-03-23 11:52:48 +01:00
jenkins-bot 77a0fd6c0d Merge "Replace double-equals with triple-equals" 2019-03-23 07:56:37 +00:00
Holger Knust 324d0e6aa3 AbuseFilter: Swapped getNative for getText
Use getText method in AbuseFilter.php instead of deprecated getNativeData method.

Bug:  T155582
Change-Id: Ia1af3abea8331e2292d608215946834b282ed753
2019-03-22 17:48:19 -04:00
Daimona Eaytoy f2c1beec44 Replace double-equals with triple-equals
Since double-equals are evil. I left some of them in place where I
wasn't sure, but I may be changed some which were intended to be
doubles. It could be a good idea to delay merging this patch until we'll
have more code coverage.

Change-Id: I1721a3ba532d481e3ecf35f51099c1438b6b73b2
2019-03-22 16:12:13 +01:00
Daimona Eaytoy d1728d62f8 Add a parameter to generate(User|Title)Vars hooks to specify context
These hooks can be called either when filtering the current action, or
to check an action from recentchanges. While AbuseFilter already handles
well these two cases, other extensions don't and need some context.
Depends on the patch below because we're changing generateTitleVars,
which already has a temporary extra parameter that I don't want to mess
up with.
Also follow-up I72933fcc9952fc1aabf6464b2fc0b04ec39c024b for a few
remaining uppercase vars.

Bug: T203166
Depends-On: If5f238cddb41ef92b141e36b4f2f15fd4cc86476
Change-Id: I1983b93bbadabd24d8bf94fa7bb14594d10e731e
2019-03-22 16:06:17 +01:00
jenkins-bot 44d602b9a9 Merge "Use lowercase for built-in variables" 2019-03-22 10:44:24 +00:00
jenkins-bot 85b16f1ecd Merge "Change method used for file_bits_per_channel in /test" 2019-03-22 10:34:11 +00:00
Daimona Eaytoy 45e84bad0e Change method used for file_bits_per_channel in /test
Using MWProps will not raise an 'undefined index' notice and, if the
bitsdepth cannot be established, will return 0. This is way better than
returning the empty string or null, which cannot be treated as integers.

Bug: T218874
Change-Id: I2b585f3a2f257783c15f0fcdd2851f1e1ce4256b
2019-03-21 14:05:54 +01:00
Daimona Eaytoy 4103ff56af Use 'page' prefix for upload vars
Rationale on phabricator.

Bug: T218873
Change-Id: I33128098c186ab716951c72e0d917663721b4740
2019-03-21 13:29:43 +01:00
Daimona Eaytoy d6c649bb0d Overhaul tag selector
If "tag" option is selected and the form is submitted without adding any
tag, just show it blank instead of adding an empty tag to the topbar.
Separately validate the empty tag case (and added a test for it).

Bug: T203353
Depends-On: I3b2e763bd8835207dc5df1db43d3e1881e6961c3
Change-Id: I8884b739fd17fa2eace5aac8775d3524aa606f1f
2019-03-17 14:04:50 +00:00
Daimona Eaytoy bae9c5bb8f Use lowercase for built-in variables
The uppercase is just a leftover from a long time ago. Currently,
variables are case-insensitive, and we already perform a strtolower when
saving them. Since most parts of the code already use lowercase, the
uppercase leftovers only make it harder to grep the code to find
variables. As a bonus, make Ace recognize variables in a
case-insensitive fashion.

Change-Id: I72933fcc9952fc1aabf6464b2fc0b04ec39c024b
2019-03-17 14:23:11 +01:00
jenkins-bot ef8c8e6006 Merge "Make the filter field on ViewHistory of "int" type" 2019-03-17 12:00:47 +00:00
jenkins-bot dc0c5cc5ac Merge "Make uploads testable" 2019-03-17 11:06:43 +00:00
Daimona Eaytoy bedbe36744 Add doc for every class member
Adding PHPdocs to every class members, in every file. This patch only
touches comments, and moved properties on their own lines. Note that
some of these properties would need to be moved, somehow changed, or
just removed (either because they're old, unused leftovers, or just
because we can move them to local scope), but I wanted to keep this
patch doc-only.

Change-Id: I9fe701445bea8f09d82783789ff1ec537ac6704b
2019-03-17 11:40:24 +01:00
jenkins-bot f63b7e8039 Merge "Remove the hacky 'context' variable" 2019-03-17 10:35:31 +00:00
Daimona Eaytoy 3211c71739 Make uploads testable
This patch adds missing methods for testing a filter against uploads in
RC. Please note that (as discussed below) using wfFindFile could be
relatively expensive (as it will be executed for 0-100 RC rows).
If this is true, then we should either use another method (but I
couldn't find a suitable one), or simply reduce the amount of testable
uploads to a reasonable limit.

Bug: T170249
Change-Id: Id406d4e1571873f49bb11e69029311b24ececf49
2019-03-17 11:32:56 +01:00
jenkins-bot de4e971c72 Merge "Remove usage of MakeGlobalVariablesScript hook" 2019-03-17 10:32:30 +00:00
jenkins-bot 4c63c9190c Merge "Remove deprecated DB method" 2019-03-17 10:08:50 +00:00
jenkins-bot e2f1880922 Merge "Don't use wgLang and wgContLang" 2019-03-17 09:53:16 +00:00