Commit graph

296 commits

Author SHA1 Message Date
Daimona Eaytoy 430ba818d0 Add test for multiple conditions inside conditionals
The regression itself was fixed in
I980aec3481a52ecc35f1811a366014a5581a7cdb, so this patch only adds a
test for it.
Also remove a comment about CachingParser failures: we don't want to
encourage people to remove it from tests anymore.

Bug: T152281
Change-Id: I3ad49050ea49bf45d3226878e091da3c8dbefdb1
2019-08-12 18:18:05 +02:00
Daimona Eaytoy 3f171dc0a5 Move keywords handlers to the Parser
Just like we do for functions, it doesn't really make sense to have
keywords separately, in AFPData.

Change-Id: I208a9b1ce2bd12038e9fbcc515c48d604ec80eb8
2019-08-12 14:29:56 +02:00
Daimona Eaytoy 2fdf091eb9 Make several AFPData functions non-static
The keywords-related ones will be handled in a subsequent patch.

Change-Id: Ifcfad438023ef136dc6f2cd5529e867df9b23789
2019-08-12 14:12:16 +02:00
Daimona Eaytoy 69ad23da98 Ban variable variables
As explained on phab, it's not worth the effort of keeping this feature.

Bug: T229947
Change-Id: Ic6067cab8e1ede98545e704888c99e2ed9a004e4
2019-08-11 01:47:35 +00:00
jenkins-bot 8ee442234f Merge "Move "block-autopromote" key from $wgMainStash to 'db-replicated'" 2019-08-07 23:07:45 +00:00
jenkins-bot 1fa5eef94c Merge "Overhaul Blockautopromote action" 2019-08-07 23:03:08 +00:00
Aaron Schulz 9e44f1a9e9 Move "block-autopromote" key from $wgMainStash to 'db-replicated'
Keep the key mutation methods in the AbuseFilter class

Bug: T227376
Change-Id: I03feb05218789a3b73a31c9a94216daafcb7c145
2019-08-07 01:09:13 +00:00
jenkins-bot 5a067f7237 Merge "Add tests for empty operand logging" 2019-08-06 17:38:31 +00:00
Daimona Eaytoy b91db1d7be Add tests for empty operand logging
Follow-up 5f4491f9aa.

Change-Id: I80ca8c3c75f7de23cf9ab16aa66a240e9981c395
2019-08-06 17:17:27 +00:00
Daimona Eaytoy 2ed6272bb2 Partly handle set and set_var in shortcircuit
This is more complicated than the := operator, because the var name
could be a complicated expression, and we have to handle a function
call. This patch only covers the case where the variable name is a
literal, which is enough for WMF production.

Bug: T214674
Change-Id: I6c0f8e95663919a0235b5ccf0c88ad0a539315a7
2019-08-06 16:14:34 +02:00
Daimona Eaytoy 2bdb44d58b Overhaul Blockautopromote action
As for all mostly unused consequences, blockautopromote has a couple of
major problems: first, it blocked the status for a random time between 3
and 7 days, which to me makes no sense at all (is it some sort of
casino?), and this patch fixes it to 5 days. Second, nothing was logged,
not the blocking nor the unblocking. Here I'm adding a LogHandler for
two new sub-actions of 'rights' to keep track of both action.

Bug: T49412
Change-Id: If48a48f5b8baaf9e77c0826466f5d03bb7f691d0
2019-08-05 22:27:49 -04:00
jenkins-bot 19182606c1 Merge "Merge global profiling keys" 2019-08-04 18:40:14 +00:00
rarohde d022377578 Merge global profiling keys
The last step of the profiling overhaul. See T53294 for the original description by Dragons flight.

Note: Here I'm adding a FixMe for a problem which already exists in the code
and the child patch will fix it.

Bug: T53294
Depends-On: I2d8c8f8278073a9420e3eb373fb89a655925618a
Change-Id: Ib12e072a245fcad93c6c6bd452041f3441f68bb7
2019-08-04 17:59:58 +00:00
Daimona Eaytoy 517919fca8 Allow accessing offsets of built-in variables
I5ec4ab44c4e88aaf18c0d7b73355d27050beeda7 almost fixed this bug, but we
also have to make it possible to access builtin variables as arrays.
This will only make sense for a few variables (e.g. added_lines and
removed_lines), but I don't think we should validate it when checking
syntax.

Bug: T198531
Change-Id: I417e1b8d4802bbfccd091ce5c7617659cfd1e4ea
2019-08-04 17:14:44 +00:00
jenkins-bot c0b6267022 Merge "Use milliseconds for time profiling" 2019-08-04 16:12:59 +00:00
jenkins-bot f7fd6a6daf Merge "Move per-filter matches profiling to per-filter data" 2019-08-04 16:07:58 +00:00
Daimona Eaytoy 9049be3609 Specialize empty AFPData types
As described in T156096#5389655.

Change-Id: Ifbf95a6b72a280cd77db6affbd8d642499bbfedc
2019-08-04 15:26:57 +00:00
Daimona Eaytoy c3db63714e Use milliseconds for time profiling
Instead of seconds, and round the average condition at 1dp instead of 0.
Split from child patch by Dragons flight.

Depends-On: I2d8c8f8278073a9420e3eb373fb89a655925618a
Change-Id: I339aed5f8c1d49714e7927ce49286f9ce6c839f5
2019-08-03 23:24:46 +00:00
Daimona Eaytoy 0b7902fe6e Move per-filter matches profiling to per-filter data
They're currently stored separately, so move matches count together with
other per-filter data to keep it consistent. This also removes a
parameter from filterMatchesKey, as it's not needed anymore.
Split from child patch by Dragons flight.

Bug: T53294
Depends-On: I8f47beb73cfc1b63c4b3c809fc6d65a1e66ee334
Change-Id: I2d8c8f8278073a9420e3eb373fb89a655925618a
2019-08-03 23:22:20 +00:00
Daimona Eaytoy a85e1ccc59 Make AbuseFilterParser::$funcCache non-static
Change-Id: I312efe3ce4d1f06e697aa4564aeec1bacbaf97d3
2019-08-03 09:19:49 +00:00
Daimona Eaytoy 09d0254172 Better handling of DNONE
This patch includes:
 * Making it possible to access offsets of a DNONE (returning a DNONE)
 * Initializing user-defined variables as DNONE inside short-circuited branches
 * Make DNONE propagate with other operators
 * Make DNONE count as false for logic operators
 * Remove a now-outaded bit in doLevelAtom. In case of shortcircuit,
   $result is now DNONE instead of DNULL, and thus it's possible to
   access offsets of it. Performance++!
 * Don't allow modifying or adding an element of a DNONE as if it were an
    array (to avoid inconsistencies)

This re-applies Id85c673337fa90a3782fd22eb9690cd996967111 with several fixes.

NOTE: Haven't tested locally, although I'm pretty confident thanks to
the amount of tests added.

Bug: T214674
Bug: T228677
Change-Id: I5ec4ab44c4e88aaf18c0d7b73355d27050beeda7
2019-08-02 21:05:08 +00:00
jenkins-bot e3e157361d Merge "Revert "Initialize user-defined variables during shortcircuit"" 2019-07-29 23:30:50 +00:00
Daimona Eaytoy 13cdb86dd2 Revert "Initialize user-defined variables during shortcircuit"
Reason for revert: T214674#5374806

This reverts commit 56e6117afd.

Bug: T214674
Change-Id: Iccce248d2693cd9877a740b74e72a577e730435e
2019-07-29 23:06:23 +00:00
Daimona Eaytoy 4720c97530 Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.

Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2019-07-23 19:06:27 +00:00
Daimona Eaytoy 56e6117afd Initialize user-defined variables during shortcircuit
Bug: T214674
Depends-On: I5a14d4b2bc3ffd9caaaa095f16f36b9b6009db05
Change-Id: Id85c673337fa90a3782fd22eb9690cd996967111
2019-07-23 12:20:53 +00:00
Daimona Eaytoy 9937f8b050 Remove extra file from parser tests
Added in I5a14d4b2bc3ffd9caaaa095f16f36b9b6009db05, but .r files aren't
used anymore since I6c06e596587750c4ebaabafbd277bc75eeb436a5, and I
forgot to remove the file upon rebasing.

Change-Id: Id688d215b1136bd0a04b8c0d8d8d16de5da1295e
2019-07-15 12:22:09 +02:00
Daimona Eaytoy 18d7d2ed62 Start using AFPData::DNONE
This should allow more flexibility when checking syntax, and a saner
behaviour overall.
Aside from not throwing exception in certain cases, the results should
be almost equal to the ones you would get without this patch. However,
there are still a few things to improve (which for convenience I wrote
inside the parser test) and many to test.

Bug: T204654
Depends-On: I69bfec45c76509fb1112641393f78e8d8834adcd
Change-Id: I5a14d4b2bc3ffd9caaaa095f16f36b9b6009db05
2019-07-14 08:48:47 +00:00
Daimona Eaytoy 7bc566e635 Fix the regex for numbers, start deprecation of non-decimal numbers
Aside from the 14 thingy reported in the task, this syntax is awful! The
fix to the regex should only be intended as a temporary stopgap. A
proper fix would be to introduce a new syntax, like for instance the one
used in PHP.

Bug: T212726
Change-Id: Idc37a17ce539e6c63d67fc07d47d812569debe0e
2019-07-10 13:26:36 +00:00
jenkins-bot 6f0905541a Merge "Make AbuseFilterVariableHolder::mVars private" 2019-07-09 08:42:16 +00:00
jenkins-bot 69bebbb4ff Merge "Simplify action arrays" 2019-07-08 23:07:26 +00:00
Daimona Eaytoy 304b58d46a Make AbuseFilterVariableHolder::mVars private
This property is meant to be private, since it has all kinds of
getters/setters, aside from one which is introduced in this patch.

Change-Id: I217b1e22cabd3c0468c84b1d6a69a6ed3c6fa8e6
2019-07-08 16:25:10 +02:00
Daimona Eaytoy d8d4750e6a Simplify action arrays
The current form is awkward. They're all like
[ actionname => [ 'action' => actionname, 'parameters' => params ] ]
This is greatly confusing since adds a nesting level, and just
duplicates the actionname information (also, we actually never retrieve
it from the internal array). Instead, change all of them to be
[ actionname => params ]
which is a lot shorter and clearer (and easier to handle).
A similar case is handled in I8134ecc41fbecdbed99faf406e9e3ca91b6123b9
(see PS 8..10).

Change-Id: I34c040dbeb3ab01158fb3db22496def6ccaf72d9
2019-07-05 10:00:48 +02:00
Aaron Schulz 2cf7b58434 Convert wfGetDB() calls to using getConnectionRef()
This handles the logic of calling reuseConnection() automatically

Change-Id: I9328e709fe5d81099338a31deef24d34db22d784
2019-07-04 15:09:32 -07:00
Daimona Eaytoy 7398730563 Disallow consecutive comparisons
As explained on phabricator, they don't work with shortcircuit, so they
already fail for all filters using them. Plus IMHO it's an unnecessary
deviation from PHP's behaviour, given that this syntax doesn't do what
users may expect.

Bug: T218906
Change-Id: If9e7545e14044c8dc3b4163bb6fca8ab0683b9fa
2019-07-04 19:15:07 +02:00
Daimona Eaytoy e86d4bc124 Simplify code for stashedEdits tests
Using the new PageEditStash class allows to simplify a bit the
integration tests for edit stashing. As I wrote in a ToDo, it may be
enough to manually run the hook, but that's left to do as a follow-up.

Change-Id: I3389a6961b4f39ecd980be2f429c23f8b7706a15
2019-06-24 11:13:59 +02:00
Daimona Eaytoy 382751a707 Move conditions-related stuff inside AbuseFilterParser
Instead of relying on static methods and members in the AbuseFilter
class, move everything related to conditions inside the Parser, as the
amount of used conditions is something pertaining a single
AbuseFilter(Caching)Parser instance.
This change requires changing some signatures and adding parameters,
but will make introducing the new AbuseFilterRunner class easier (and
that will clean signatures, too).

Depends-On: I5b29ff556eca45fe59d15e2e3df4d06f1f6b3934
Change-Id: I7c1ea17adf7f42cf9260d416906bfbf3b8a20688
2019-06-19 15:14:17 +00:00
petarpetkovic c02590f555 Fix "succesful" typo
Change-Id: Ibd92f6de8b03098e7bdc8c4fc5e3f6cfaba95bdf
2019-06-14 03:08:41 +03:00
Daimona Eaytoy e7cd4b2a98 Rewrite AbuseFilter::decodeGlobalName
Now it returns an array with a bit more info, and has a different name
to reflect the fact that its input is now split in two parts. Plus, make
it throw whenever it gets an unexpected input, and add a bunch of test
cases for it.

Depends-On: Ib5fdeb75c1324f672b4ded39681f006fde34b4d1
Change-Id: Ie550889495232b534c0f9aec31039cf21b2135b1
2019-06-12 23:56:25 +00:00
Thalia 22ceae7e23 Use MediaWiki\Block\DatabaseBlock instead of Block
This follows the rename of the Block class in I6d96b63ca0.

Change-Id: I44cf9eb68c23a8299316effa4dee7f732486dd84
2019-05-31 16:08:19 +01:00
jenkins-bot 369ce36be7 Merge "Tokenizer caching back to APC" 2019-05-28 19:10:22 +00:00
Daimona Eaytoy 53f03e5301 Tokenizer caching back to APC
Partial revert of I4dd81a723e2bdb828b90594ad66a3918d8ec5b6c.
Thinking again of it, I think it's not worth it to have this data over
the network. Plus, given that it's not-that-slow to be computed, I think
there can only be a performance gain in using APC (as opposed to e.g.
memcached/redis) for 99.9% of the filters.

Change-Id: I8c6a4a95ec12c18ede8e6419540f7a2ac943457c
2019-05-28 19:48:26 +02:00
jenkins-bot 112787020d Merge "Support for PermissionManager changes at https://gerrit.wikimedia.org/r/c/mediawiki/core/+/502484" 2019-05-28 11:24:53 +00:00
Vedmaka f293b3b7be Support for PermissionManager changes at
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/502484

Change-Id: I258f02e286b6ba0387e1bff540a744fafb03dc55
2019-05-28 09:33:28 +00:00
Daimona Eaytoy 4b5c7c0198 Add missing covers/group tags
Including CachingParser as follow-up of
I980aec3481a52ecc35f1811a366014a5581a7cdb.

Bug: T201193
Change-Id: I9905efb2b2e61b330c42275c9ccfab2f24750bd4
2019-05-25 14:10:07 +02:00
Daimona Eaytoy 39fc7c12af Restore unit tests for CachingParser and fix it
Added cachingParser back to *all* the parser tests, fixed a couple of
differences with the normal parser, and added a couple of tests so that
any cachingParser-related file has 100% coverage. Also move the remaining
get_matches tests inside parserTests, and specify the parser used in case of failure.
This also adds a new base class for parser-related tests with a couple
of util methods.

Bug: T201193
Change-Id: I980aec3481a52ecc35f1811a366014a5581a7cdb
2019-05-25 10:55:24 +02:00
jenkins-bot 1cb80be0ad Merge "Add tests for various data type casts" 2019-05-24 19:19:20 +00:00
jenkins-bot 058e215882 Merge "Refactor tokenizer caching" 2019-05-24 19:09:03 +00:00
Daimona Eaytoy f56562f583 Add tests for global filters
Another crucial part to have covered. Also clarify that
AbuseFilterCentralDB can be of the form "dbname-prefix".

Remove a filter used for profiling and replace it with a global one:
we're still fine, and the list is kept shorter.

Bug: T201193
Depends-On: I5ee7ba44a6cd82a5ddb24fb4127af04d96e647f4
Change-Id: If6b91711534c0d60e1aa27bd5748c3023e29f376
2019-05-24 16:58:23 +02:00
Daimona Eaytoy b3707106e9 Reset MWTimestamp in tearDown
Follow-up of I5ee7ba44a6cd82a5ddb24fb4127af04d96e647f4.

Change-Id: Icf288d7c4a9d087e7e1cd8a6e8c8cc9dac20e532
2019-05-24 16:54:29 +02:00
Daimona Eaytoy a766e39ade Add unit tests for profiling
Yet another important part to have covered. While for normal edits it
already works, for stashed ones it doesn't. That's why we need the patch
for checkAllFilters. Since for stashed edits profiling stats are all
zeros, this may explain T201334.
Changed the timestamp variable to use wfTimestamp instead of time() so
that we can fake it inside unit tests.
In a subsequent patch we should add average runtime conditions to tests
(really tricky).

Bug: T201193
Depends-On: Ib17821240b25c972a187e6b5eae42c5ada6c65e7
Change-Id: I5ee7ba44a6cd82a5ddb24fb4127af04d96e647f4
2019-05-23 08:47:40 +00:00
Daimona Eaytoy 00b9791349 Add unit tests for stashed edits
This is an important part to cover, and should be further expanded.
Also, fix a couple of minor things around, including making some methods
non-static.

Bug: T201193
Depends-On: I5e35d773904a62105767ce6d7d962ab5525c2d12
Change-Id: Ib17821240b25c972a187e6b5eae42c5ada6c65e7
2019-05-23 08:47:25 +00:00
jenkins-bot c52850aae7 Merge "Add missing limits to explode() calls" 2019-05-15 15:06:18 +00:00
Thiemo Kreuz c6f20a64dd Add missing limits to explode() calls
This is fixing potential bugs where invalid strings with more than one
comma have silently been accepted.

Change-Id: Ib1e7d0c99973f243ef6faad6389bab688187c1cf
2019-05-15 16:14:12 +02:00
Thiemo Kreuz fa3ce90851 Remove comments literally repeating what the code says
I find it obvious that a file called "AbuseFilterTokenizerTest" is a
"test for the AbuseFilterTokenizer class". A comment that is just
repeating this information is typicalls not helpful, but distracting
and a potential source of mistakes, e.g. when stuff is copy-pasted,
but the comment not adjusted.

Change-Id: I1d4cc06e9e5631955ff73bf675090cf9c33c9390
2019-05-15 16:04:32 +02:00
Thalia f23905c402 Remove call to deprecated User::isBlocked
Change-Id: Ibb7412f8aa08a745a211b9b0581ccb6b0ca9eff5
2019-05-14 13:14:57 +01:00
Daimona Eaytoy 2276d8ed2a Refactor tokenizer caching
Split a method, use WAN cache so that we're enabled to use
getWithSetCallback, pass the "version" option there and adapt the test
to it.
Follow-up of I9b3bc36b552901bc6ca7609ee51e80be2979a9c4

Change-Id: I4dd81a723e2bdb828b90594ad66a3918d8ec5b6c
2019-04-23 19:38:10 +02:00
jenkins-bot 968bd9b817 Merge "Add tests for tokenizer caching" 2019-04-17 23:27:19 +00:00
Aryeh Gregor b222330a61 Don't try to move onto an existing page in tests
I didn't fix every case where this happens, just what blocks
I6ddcc9f34a48f997ae39b79cd2df40dd2cc10197 from landing.

Change-Id: I971e619eb76c4474fe037fad258f9c496717bf41
2019-04-17 17:23:23 +03:00
Daimona Eaytoy 4b10a544ab Add tests for tokenizer caching
Caching the result of the tokenization is pretty important
performance-wise, so this test ensures that caching works as expected.
I have also extracted the method used to generate the cache key for
easier testing, and moved the cache instance to a class member because
otherwise that piece of code can't be tested...

Bug: T201193
Change-Id: I9b3bc36b552901bc6ca7609ee51e80be2979a9c4
2019-04-15 16:59:55 +02:00
Daimona Eaytoy ec110c657b Add tests for various data type casts
These are the ones which other tests don't cover, mostly because no
filter syntax can trigger those cases. This patch should bring coverage
for AFPData to 100%.

Bug: T201193
Change-Id: I997576141943959d4602a9f839311108928ec766
2019-04-14 14:08:57 +02:00
Daimona Eaytoy 909eec6716 Tweak coverage part 2
Follow-up of Ic30883f7d261d974a2be46308d023e2714119e95, with two files
that I forgot to git-add and a repositioning of comments to avoid the
last bracket to be reported as uncovered.

Bug: T201193
Change-Id: I6bf7e5892a0f49f6a138792f0aedf230a70c18a8
2019-04-13 19:26:01 +02:00
Daimona Eaytoy 4bcb64b01a Increase code coverage a bit
This patch mostly adds coverageIgnore comments for intendedly
unreachable code etc. Some of them could be made testable by adding a new
filter function (e.g. array cast), but this patch is meant to be
comment-only (aside from the parser test).
Ignoring coverage for these lines makes some methods reach 100%
coverage, which in turn makes it easier to look at the coverage chart
and identify at a glance which parts of the code *really* need to be
covered.

Bug: T201193
Change-Id: Ic30883f7d261d974a2be46308d023e2714119e95
2019-04-13 18:30:14 +02:00
Daimona Eaytoy 8293ec176f Add tests for storing and loading the variables dump
These are specific tests for storeVarDump and loadVarDump, both alone
and in the context of running filters.
Also, include disabled variables in the VariableHolder object if they're
saved in the DB.

Bug: T201193
Depends-On: Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb
Change-Id: I5e35d773904a62105767ce6d7d962ab5525c2d12
2019-04-12 08:03:33 +00:00
jenkins-bot c0da9ff3ac Merge "Clean AbuseFilterParserTests" 2019-04-11 21:46:50 +00:00
Brad Jorsch b59f19d675 AbuseFilterTest: Don't use $wgUser when creating pages
Which means we have to pass a user to WikiPage::doEditContent().

Follows up Ifbcd9adf3.

Change-Id: I1bd0288cc132627d75b4001219522ec5e952eda7
2019-04-09 12:25:34 -04:00
jenkins-bot cc670f0a07 Merge "Clean the AbuseFilterTest class" 2019-04-06 14:47:52 +00:00
jenkins-bot efe32b7c93 Merge "Add doc for every class member" 2019-04-06 14:37:19 +00:00
jenkins-bot d53c84da36 Merge "Restore check for dividebyzero" 2019-04-06 12:35:23 +00:00
jenkins-bot e03488b66a Merge "Overhaul tag selector" 2019-04-06 12:35:20 +00:00
Brad Jorsch 5ace1121b0 Actually create user in AbuseFilterConsequencesTest
If the User passed to $logEntry->setPerformer() represents a creatable
username, then it has to actually exist so the actor row can be created.

Bug: T188327
Change-Id: Iab2fc9593a020ffacd219d644103d685028e3336
2019-04-05 12:35:25 -04:00
Daimona Eaytoy 0ff581e246 Clean AbuseFilterParserTests
Mostly delete result files and assume the result is always true. The few
exceptions were either moved to standalone test, or inverted.

Change-Id: I6c06e596587750c4ebaabafbd277bc75eeb436a5
2019-03-23 12:59:03 +01:00
Daimona Eaytoy 72c2be7a18 Remove $wgAbuseFilterRuntimeProfiling
The reasoning is similar to the one of the parent patch (Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb). Plus, it records runtime metrics on action different than edits, as there's no reason not to do it.
No performance issues in production.

Bug: T191039
Depends-On: Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb
Change-Id: Ib1112e2fefd0631550d386ba87e5f87db84c3036
2019-03-23 11:31:18 +00:00
Daimona Eaytoy 89520e2353 Remove $wgAbuseFilterProfiling
This variable was introduced to selectively enable profiling because
stats recording was bad for performance. Nowadays, stats are recorded in
a deferredupdate and don't harm performance anymore. Thus, this variable
can be removed and profiling be enabled by default.

Bug: T191039
Depends-On: Ib5fdeb75c1324f672b4ded39681f006fde34b4d1
Change-Id: Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb
2019-03-23 11:31:11 +00:00
Daimona Eaytoy 9144f20245 Restore check for dividebyzero
Follow-up of I1721a3ba532d481e3ecf35f51099c1438b6b73b2. This is the only
wrong replacement: strict checking will let 5 / 0.0 pass, with
unexpected results. Adding a regression test for it, too.

Change-Id: I25dbe9fafa92fd9a11bd8bc6ab8e66f305b8d48e
2019-03-23 11:38:39 +01:00
Daimona Eaytoy f2c1beec44 Replace double-equals with triple-equals
Since double-equals are evil. I left some of them in place where I
wasn't sure, but I may be changed some which were intended to be
doubles. It could be a good idea to delay merging this patch until we'll
have more code coverage.

Change-Id: I1721a3ba532d481e3ecf35f51099c1438b6b73b2
2019-03-22 16:12:13 +01:00
Daimona Eaytoy d6c649bb0d Overhaul tag selector
If "tag" option is selected and the form is submitted without adding any
tag, just show it blank instead of adding an empty tag to the topbar.
Separately validate the empty tag case (and added a test for it).

Bug: T203353
Depends-On: I3b2e763bd8835207dc5df1db43d3e1881e6961c3
Change-Id: I8884b739fd17fa2eace5aac8775d3524aa606f1f
2019-03-17 14:04:50 +00:00
Daimona Eaytoy bedbe36744 Add doc for every class member
Adding PHPdocs to every class members, in every file. This patch only
touches comments, and moved properties on their own lines. Note that
some of these properties would need to be moved, somehow changed, or
just removed (either because they're old, unused leftovers, or just
because we can move them to local scope), but I wanted to keep this
patch doc-only.

Change-Id: I9fe701445bea8f09d82783789ff1ec537ac6704b
2019-03-17 11:40:24 +01:00
jenkins-bot 3f3e98fbc5 Merge "Fix shortcircuit for consecutive operations" 2019-03-17 10:04:14 +00:00
Daimona Eaytoy 683e94cdd3 Clean the AbuseFilterTest class
Remove all globals, make methods non-static, improve assertions and
computing some variables, add names to the tests and other minor
improvements.

Change-Id: Ifbcd9adf34d173d0da0aa568fc6f91fdc2d61609
2019-03-17 11:04:10 +01:00
jenkins-bot e2f1880922 Merge "Don't use wgLang and wgContLang" 2019-03-17 09:53:16 +00:00
jenkins-bot 65a4c26804 Merge "Remove exclusions for Generic.Files.LineLength" 2019-03-17 09:49:38 +00:00
Kunal Mehta 577f4dab93 Migrate to new phan
Bug: T216904
Change-Id: I30864bd3d7f9b9ab674bf6589cd9e5e3aed5bb8d
2019-03-16 09:41:23 +00:00
Daimona Eaytoy dd4b579695 Remove exclusions for Generic.Files.LineLength
Keep it only for filters definitions in ConsequencesTests.

Change-Id: I305c7f496a29b20a3ee1d34479d1e4cb9252060a
2019-02-23 10:12:07 +01:00
Thalia 540a557a59 Replace calls to deprecated Block::prevents
Where prevents is used as a setter, use the new setter methods;
where it is used to determine whether a block blocks the target
from editing their talk page, use appliesToUsertalk.

Block::prevents was deprecated and replaced by several other
methods in I0e131696419211.

Bug: T211578
Change-Id: I166cc6f64c0f895ff8c631d2655c1c3208131371
2019-02-22 19:29:02 +00:00
Thiemo Kreuz 3993a7ea15 Replace @expectedException with $this->expectException()
The @expectedException annotation got deprecated in PHPUnit 7.5, and
removed in PHPUnit 8.0. This was done because the annotation does have
two disadvantages:
* The class name is encoded in string, where it is not easy to find for
  all IDEs and tools.
* it did not allow to say exactly *when* the exception is expected.

Change-Id: I85f0b5f44b2f400a121115d402b64827ea534c32
2019-02-19 10:58:16 +01:00
Daimona Eaytoy 6f4bfc9597 Fix shortcircuit for consecutive operations
Using break could halt parsing between operations, instead use continue
to parse all operations.

Bug: T214642
Change-Id: If67ddaffef280c2448c55ae536013758617bba68
2019-02-08 17:55:59 +00:00
Tim Starling c889c2990c In tests that create users, add 'user' to $this->tablesUsed
Change-Id: I7d2c6b304974d487e1b7727f594d0843ff080a7d
2019-02-08 16:40:17 +11:00
Daimona Eaytoy 51120e51c5 Don't use wgLang and wgContLang
For wgLang, there's a Language object available in the proximity, so just pass it.
For wgContLang, use MediaWikiServices.

Change-Id: Ic492007f2d5eeb8048d0919a4b9b7dd98c15c350
2019-02-06 12:00:44 +01:00
jenkins-bot 15a8340ee1 Merge "Reject empty warning and disallow messages when validating a filter" 2019-01-31 21:28:17 +00:00
Daimona Eaytoy 0f041e8282 Split AbuseFilterConsequencesTest tests in several methods
This makes the code easier to maintain and more flexible, plus adds
several tests. Some flaky tests are also improved.

Depends-On: I57ce67c5202c8574fcf1957999a6999fec264cb7
Change-Id: Ibb5322bca93b464e9014b53644c04f2bc1141e72
2019-01-23 21:26:25 +00:00
Daimona Eaytoy 26b783f062 Use data provider's array keys to specify test description
We just passed the description as a parameter, but it's much quicker to
use it as the key in the data provider: PHPUnit will automatically
display it in case of failure, so that we don't have to do that
manually (and still get messages like "failed with data set #7").

Depends-On: I8edcca17ecdcf71397cc9b0d101e8b13ac112047
Change-Id: I57ce67c5202c8574fcf1957999a6999fec264cb7
2019-01-23 21:26:17 +00:00
Daimona Eaytoy 0e6b783ed4 Reject empty warning and disallow messages when validating a filter
Right now, we allow empty messages, and when the "warn" action is
executed we use "abusefilter-warning" if no message is specified.
However, this also produces a PHP notice while editing a filter with
empty message (see Phab). With this patch, empty messages will be
rejected, and a follow-up will be discussed on Phab.

Update: added disallow message as follow-up of
Ic1de03a6944c43a346fa317ee0a217551f0d284a.

Bug: T203353
Depends-On: I8df247f61d9f3769e9580544f324dd174811e939
Change-Id: I71b1f81d10c02de4de141b1ab9b630d05cf4619c
2019-01-21 14:06:54 +01:00
jenkins-bot df2da23d29 Merge "Add unit tests for custom disallow messages" 2019-01-19 12:21:02 +00:00
jenkins-bot b44984c50a Merge "Remove unused stuff" 2019-01-19 12:18:22 +00:00
jenkins-bot 575646393b Merge "Improve code readability" 2019-01-19 12:11:06 +00:00
jenkins-bot a2bee3bcf3 Merge "Simplify parser methods" 2019-01-19 12:11:04 +00:00
jenkins-bot 0d4e982069 Merge "Reduce code duplication" 2019-01-19 12:00:47 +00:00
Daimona Eaytoy 6217ffb928 Remove unused stuff
Variables declared but never used, redundant code, and old leftovers.

Change-Id: Ic51044a45a1b49ad6c7af06c646b11893411a7cd
2019-01-18 17:04:19 +01:00
Daimona Eaytoy 93e8cb5ac5 Tune logging channel
As follow-up of I10b1fd2d9bdfe518089c053d77fef568170ecb65, use
'AbuseFilter' instead of 'AbuseFilterDeprecatedVars' as channel name.
Raise level for null-title filtering. Since with a null title
several things are likely to break, a warning is more appropriate here.
Tweaked the message as well, to include the bug number and to avoid
pointlessly including the title (which is null).
Lower the level for stashedit hit/miss (as it's really spammy and not
that useful right now).
Use 'abusefilter' instead of 'AbuseFilter' for statsd so that everything
has the same prefix.
Also raise the level for parser exceptions and unrecognized
consequences.

Change-Id: I1f9988155e924232b201281795cd322636da8082
2019-01-16 08:56:22 +00:00
Daimona Eaytoy f12fdb4a32 Add unit tests for custom disallow messages
Follow-up of Ic1de03a6944c43a346fa317ee0a217551f0d284a, adding some unit
tests for this newly introduced feature, plus a couple of tweaks for
both tests themselves and i18n.

Change-Id: I8df247f61d9f3769e9580544f324dd174811e939
2019-01-05 10:58:47 +00:00
Thiemo Kreuz 8ccb9839e5 Add test to guarantee tag uniqueness
This is a direct follow up for the bug fixed in Iebbdeac.

Change-Id: I5cc5618aa6161460534804e46a8a3568d1af9af3
2018-12-31 18:26:47 +01:00
daniel 688eccea47 Expose text from all slots to AbuseFilter
This is a first step towards MCR support in AbuseFilter. The textual
representation of all slots is concatenated. Since AbuseFilter uses
getTextForSearchIndex to determine the textual representation of
content, blind concatenation should not break any assumptions
made by AbsueFilter rules: this naive approach is no worse than
AbuseFilters handling of non-textual content in general, and should
work fine for textual content.

Bug: T209291
Change-Id: Ic141085cad2e11bfe106fe83dafcb35ac31206ba
2018-12-05 09:24:08 -08:00
Daimona Eaytoy 206bdc1f6a Use the updated TitleMove hook to filter move actions
For several reasons:
*We're not really checking permissions (and the hook previously used is
meant to be used in such case)
*We'll show a cleaner error message (i.e. without the "You do not have
permission..." part)
*Filtering will happen closer to the actual move

Bug: T208907
Depends-On: I4733724075b7514e9db59e7be772d9409aa9da87
Change-Id: If88f736a446247f8b4b13c055c641d56f544d1ea
2018-12-04 18:58:04 +01:00
Amir Sarabadani fd3e3e78cb Migrate AbuseFilterConsequencesTest from tag_summary to change_tag
Bug: T209525
Change-Id: I6ab0b29800d7654164e8d23fb24b81529b0d2c88
2018-11-28 08:04:51 +01:00
Daimona Eaytoy 7427333ed5 Improve code readability
Simplify some logic constructs, reduce the amount of return statements
inside methods, explicitly declare variables before using them, reduce
code duplication, add names to JS anonymous function to produce clearer
stack traces.

Change-Id: Ife4546a91c30d4c519d09a712ba56a2f33abe579
2018-11-19 16:01:37 +01:00
Daimona Eaytoy e055ecc7c6 Reduce code duplication
Change-Id: I03bd56e4bf455865b27338ac39b3dcef20a88447
2018-11-19 15:50:36 +01:00
Daimona Eaytoy 4480c9493a Remove wgParser and wgRequest
As part of the deprecation process of non-config globals.

Change-Id: Ia84ddc20adbfda72347cf256601050b055b87ecf
2018-11-19 13:40:58 +01:00
jenkins-bot 213c2aa011 Merge "Change throttle selector to restore old functionality, overall improvement" 2018-11-15 00:58:11 +00:00
Daimona Eaytoy d3a8491c3f Change throttle selector to restore old functionality, overall improvement
Long (sigh) explanation in T203587#4569698. Also, simplified the way
TagMultiselect are generated, this one and the one for change tags.
This new selector is back-compat both with the old textarea and the OOUI
checkboxMultiselect; actually, this one is //fully// compatible with the
old textarea.
Add validation for throttle parameters and unit tests for validation
(split from I976c95658cddb2585910b6f8a5f047aadc4e4d47).
Added a trim when retrieving throttle identifier to allow syntax like
'ip, user'.
Improved the message shown on history.
Re-added the maintenance script to clean DB.

As I wrote in the task, a review by two other people would be great, at
least for the maintenance script (it could potentially break the DB).

Bug: T203587
Bug: T203336
Bug: T203584
Bug: T203585
Depends-On: I3b2e763bd8835207dc5df1db43d3e1881e6961c3
Change-Id: I7831dbb0bab55807392ac1f7915d6cb0cb713593
2018-11-14 12:51:36 +01:00
Brad Jorsch f6349e7a32 Update tests that fail with comment/actor migration
* AbuseFilterConsequencesTest is somehow leaving blocks behind. Mark
  ipblocks as being used to avoid that.
* AFComputedVariable::getLastPageAuthors() uses indeterminate order for
  multiple revisions with the same timestamp. Fall back to rev_id
  ordering like MySQL accidentally did before.
* AbuseFilterTest tries to create revisions attributed to users that
  don't exist. Switch to interwiki usernames.

Change-Id: I30f7cdcc3875f3f7af116c1e41e88f62ab9e91d0
2018-11-09 17:03:36 -05:00
jenkins-bot 108ec1117f Merge "Reload the test user instance before checking the edit count" 2018-10-23 18:53:33 +00:00
Aaron Schulz 3191c2adc4 Reload the test user instance before checking the edit count
These are updated in deferred updates and should not rely on the same
User instance being used in those updates. This also avoids convoluted
logic in User to set the new edit count for various cases.

Change-Id: I6d239a5ea286afb10d9e317b2ee1436de60f7e4f
2018-10-23 18:06:12 +00:00
jenkins-bot 7e151f5edc Merge "Unbreak short circuit for arrays" 2018-10-18 04:04:31 +00:00
jenkins-bot c2f5540928 Merge "Use fake timestamps for time-related tests" 2018-10-13 22:19:31 +00:00
Daimona Eaytoy cbd57fe7a1 Simplify test parameters
Instead of having lots of huge arrays, use a fixed one and only
overwrite the needed parameters.

Change-Id: I3b2e763bd8835207dc5df1db43d3e1881e6961c3
2018-10-12 16:40:44 +02:00
Daimona Eaytoy 2ad63c95ef Use fake timestamps for time-related tests
With the hope to finally unbreak such tests, making them much more
stable and clean.

Bug: T206501
Change-Id: I275a088b9b21f47892b4e3c4cd11ef8680a9e6d9
2018-10-10 20:26:52 +02:00
Daimona Eaytoy 70b60e5906 Simplify user_age test
This simplifies the test for user_age, although I'm not totally sure it
will be fixed. AFAICS, there's nothing wrong in there, but we'll see on
future phpunit executions.

Bug: T206501
Change-Id: Iee1a2a65d08c2cffc7a0d655be1eadb018d8bf37
2018-10-09 12:46:47 +02:00
Daimona Eaytoy 6d54b83f2c Simplify parser methods
Use a single function to check parameters amount, avoid duplication
between keywordIn and keywordContains, use if...elseif instead of
if-else when statements have a return inside, simplify some other logic,
add typehinting, and change method visibility according to use of such
methods.

Change-Id: I22225a5cbbb93679a0e78bf6e15866829167fbf4
2018-10-03 17:19:40 +02:00
Daimona Eaytoy e60dacbbea Fix code comments
Fixed some comments adding explanations, fixing syntax, and parameter types
for docblocks. Also fixed some whitespace mess, and added a missing use
statement.

Change-Id: I3547c90bdaa2cab5443e8bf0c63b217fe6ba663f
2018-10-03 16:45:03 +02:00
Daimona Eaytoy d9d5af3890 Unbreak short circuit for arrays
This problem have been making filters potentially fail silently since
2009. Also add tests for arrays to make sure that no problems arise
when short circuit is used.

Bug: T204841
Change-Id: Ie4e2e06498c1202ba73afcc5d164a72427abbca5
2018-10-03 16:44:10 +02:00
jenkins-bot 121df619da Merge "Improve coverage for AbuseFilterTokenizer" 2018-09-09 12:30:49 +00:00
Daimona Eaytoy bffba28713 Add full tests for deprecated variables
This test checks every deprecated variable to be identical to the
newly-named one, and to emit a debug notice. It also changes such debug
to be emitted via logger instead of wfDebug.

Bug: T201193
Bug: T173889
Change-Id: Ie55746bb7731062ae2d46d84857af2a05d78cf4c
2018-08-29 11:00:28 +02:00
Daimona Eaytoy 775c736512 Improve coverage for AbuseFilterTokenizer
This will make tokenizer almost fully covered. The only uncovered parts
are the one with cache and an else condition which I think won't ever be
executed, and thus added a comment for that. Also, remove an obsolete
xxx comment from ComputedVariable (fixed in
I8e420f0259ef6c9e579f7a00beb58f28af9da37d)

Bug: T201193
Change-Id: I6e9a73aa9e437f096f6a1e20d53a7cb50e5ed85d
2018-08-25 10:25:16 +02:00
jenkins-bot 1b5428b9c8 Merge "Improve tests for the AbuseFilter class" 2018-08-23 14:41:00 +00:00
jenkins-bot 826e600731 Merge "Add _age variables to tests" 2018-08-23 14:35:33 +00:00
jenkins-bot 97f98b029d Merge "Improve parser coverage" 2018-08-23 14:33:54 +00:00
Daimona Eaytoy 078e9a3d21 Improve tests for the AbuseFilter class
Add some test cases for conds limit, profiling and other minor things.

Bug: T201193
Change-Id: I9a3035459cafd6537111cf1dea1a2d9a4bd34036
2018-08-23 14:14:57 +00:00
jenkins-bot ad69ea648e Merge "Remove unused function and improve unit test" 2018-08-23 13:46:41 +00:00
Daimona Eaytoy 0e2ae113fb Improve parser coverage
On the way to 100%...

Bug: T201193
Change-Id: I5fd311f861acccb31f346da9acb379b0366488e7
2018-08-23 12:13:47 +02:00
Daimona Eaytoy 03b52c2b37 Remove unused function and improve unit test
AbuseFilterParser::setVars is only used in a parser test. In the past it
was also used in the actual code (see for instance
https://phabricator.wikimedia.org/diffusion/EABF/browse/master/;5cc8dac63ca585c288ca4c8605db810774e39666?grep=setVars), but at the moment it's pretty unuseful.
This patch removes such function and makes the unit test use literals
instead of variables to avoid calling it.

Change-Id: I80cbc4033ff96f2fe8c1da263b1877bfb4c7c0c4
2018-08-23 11:00:16 +02:00
Daimona Eaytoy 90260edad0 Add _age variables to tests
Tests for new variables introduced in
I0993cecc322806382a1b567b60c0a4af69054841.

Change-Id: Iadaa33c20eb26d6e76ac02e3e9c0066b904833bc
2018-08-23 10:50:52 +02:00
Daimona Eaytoy 447d434e2a Improve code coverage
Add some parser tests, improve existing ones, and add missing @covers.

Bug: T201193
Change-Id: I9c0d2d83560baa4a3e1d4465b7919a48c4e26ac1
2018-08-22 19:07:14 +02:00
Daimona Eaytoy d35c42757c Add missing @covers tag
This should help with tracking code coverage and also explains some
coverage discrepancies encountered while writing other tests.

Bug: T201193
Change-Id: I8b20abc46c2d6c6f582953139b9a9f3710b2e4ea
2018-08-22 17:00:38 +02:00
jenkins-bot d94cc34649 Merge "Add deprecated variables to PHPUnit tests" 2018-08-22 12:44:23 +00:00
jenkins-bot a762c82fe7 Merge "Add aliases for "_text" and "article_" variables" 2018-08-22 12:44:20 +00:00
jenkins-bot 777a86314e Merge "Improve code coverage for AbuseFilterParser" 2018-08-22 11:15:00 +00:00
Daimona Eaytoy cd30d5146f Add deprecated variables to PHPUnit tests
Check a bunch of them, they should be computed and be identical to the
ones with new syntax.

Bug: T173889
Depends-On: I5c370b54e6516889624088e27928ad3a1f48a821
Change-Id: I276913a98e06b5f2ff1c5f5f3ba5bcc7b1e8c997
2018-08-22 08:38:31 +00:00
Daimona Eaytoy c962203ad2 Raise tolerance for time-related unit tests to 10 seconds
This helps avoiding failures with tests depending on execution time.

Bug: T202073
Change-Id: I4da859cfb3e49314ca20329e2ad4a3a7c4fae897
2018-08-21 17:18:24 +02:00
Daimona Eaytoy 6bc630cfef Add aliases for "_text" and "article_" variables
Variables regarding title (full list in task description) are quite
deceiving, since they use "text" instead of "title". As proposed in the
task, this is the first patch to add aliases for those variables and
slightly deprecate the old ones. In the future we may be able to replace
every occurrence (either with a search function or directly on the
database), but even a coexistence would be enough to avoid
confusion. A wfDebug log is generated whenever a deprecated variable is
parsed. The "article_" prefix is also changed to "title_", in the same
way as above.
Also, added a hook which other extension may use to specify their
deprecated variables, which will be handled the same as core ones.

Bug: T173889
Change-Id: I5c370b54e6516889624088e27928ad3a1f48a821
2018-08-21 16:59:56 +02:00
Daimona Eaytoy 4f3b020f5d Improve code coverage for AbuseFilterParser
Add some tests and improve others to raise coverage percentage. This
should lead to almost 100% for the AbuseFilterParser class. Aside from
this, a couple of changes:
* Remove an unused function
* Let equals_to_any return a genuine result with empty strings
* Remove an if which will never be true in skipOverBraces, since the
function is called after checking the same conditions.

Bug: T201193
Change-Id: I7020b2ed996236c38c5784d161ad98ec44163406
2018-08-20 14:38:40 +02:00
Umherirrender c954b412c6 Include CheckUser in phan config
Depends-On: I51421184485c3117bbab9ce3dd42f2dbb6c6180c
Change-Id: Ida17580b301ff4a6b0d3d0020c48f65eb1e21026
2018-08-17 17:38:01 +02:00
Daimona Eaytoy bb476e2c45 Fix wrong error message for PHPUnit
We're currently emitting the same error twice, but in one of those cases
it's completely wrong. Damned copy&pasting!

Bug: T202073
Change-Id: I7687826a85f3ef0abaf15d7cd973afc4e55758b2
2018-08-16 17:11:41 +00:00
Daimona Eaytoy 0026a68a8a Add PHPUnit tests for various generic functions
Adding tests for generic functions in AbuseFilter class, ranging from
simple utility function to variable computation.

Bug: T42478
Change-Id: I903fb7ffbc436b27462e3e4611ab65ecb8a543ba
2018-08-09 19:20:46 +00:00
jenkins-bot 4b185b3749 Merge "Add phpunit tests for noparams and notenoughargs exceptions" 2018-08-02 08:41:46 +00:00
jenkins-bot 75729b6195 Merge "Add other phpunit test for AFPUserVisibleException" 2018-08-02 08:07:56 +00:00
Daimona Eaytoy 9440828d13 Add phpunit tests for creating and editing filters
Adding the template for unit tests and some tests. These should cover
all the validation failure cases.

Bug: T42478
Depends-On: Ib7a0335fa7fb3b8a21765438a720205656c1ea09
Change-Id: I3fd0d627295d680ed33b1cbc730435df0446277f
2018-07-18 12:30:55 +02:00
Daimona Eaytoy 5c6007e041 Add tests for filter consequences
The last one of what I think are the must-have tests. This patch
provides the basic tests and the framework, which may be further
expanded later on. Please note that the failures are due to an actual
problem in core, for which there is I7bb0e92b2906a2511fc4290bdc76fc39ec4617fe.

Bug: T42478
Change-Id: I28eb464c63fda7faa3ec7d1f6082f36154d66962
2018-07-15 15:43:18 +00:00
Daimona Eaytoy 4f037c29c2 Add phpunit tests for noparams and notenoughargs exceptions
We're really missing exception tests: in fact, 'noparams' not being
thrown was discovered only a few days ago and worked like that for
years. This patch adds phpunit tests for both noparams and notenoughargs
exception, also checking the returned message.

Depends-On: I484fe2994292970276150d2e417801453339e540
Change-Id: Ia0b9b8fd5c979be06879723b746f9356c628f5cd
2018-07-15 17:35:45 +02:00
Daimona Eaytoy e9921bcda7 Add other phpunit test for AFPUserVisibleException
Follow-up of Iacb8f7a361079e3e117dc6845597c7bd8473e54a for exceptions
thrown outside the parser. With this patch all uses of AFPUserVisibleException
will be covered.

Depends-On: Iacb8f7a361079e3e117dc6845597c7bd8473e54a
Change-Id: Ia7ef6eb832d5725a804a60cb58bc110b06c8abe2
2018-07-01 18:34:01 +02:00
Daimona Eaytoy 7a64280893 Add phpunit tests for all exception thrown in the parser
All uses of "throw" inside AbuseFilterParser are now covered.
Bonus: added a standard suppresswarning when checking regex validity.

Change-Id: Iacb8f7a361079e3e117dc6845597c7bd8473e54a
2018-07-01 18:31:11 +02:00