Commit graph

38 commits

Author SHA1 Message Date
Daimona Eaytoy 69ad23da98 Ban variable variables
As explained on phab, it's not worth the effort of keeping this feature.

Bug: T229947
Change-Id: Ic6067cab8e1ede98545e704888c99e2ed9a004e4
2019-08-11 01:47:35 +00:00
jenkins-bot 5a067f7237 Merge "Add tests for empty operand logging" 2019-08-06 17:38:31 +00:00
Daimona Eaytoy b91db1d7be Add tests for empty operand logging
Follow-up 5f4491f9aa.

Change-Id: I80ca8c3c75f7de23cf9ab16aa66a240e9981c395
2019-08-06 17:17:27 +00:00
Daimona Eaytoy 2ed6272bb2 Partly handle set and set_var in shortcircuit
This is more complicated than the := operator, because the var name
could be a complicated expression, and we have to handle a function
call. This patch only covers the case where the variable name is a
literal, which is enough for WMF production.

Bug: T214674
Change-Id: I6c0f8e95663919a0235b5ccf0c88ad0a539315a7
2019-08-06 16:14:34 +02:00
Daimona Eaytoy 517919fca8 Allow accessing offsets of built-in variables
I5ec4ab44c4e88aaf18c0d7b73355d27050beeda7 almost fixed this bug, but we
also have to make it possible to access builtin variables as arrays.
This will only make sense for a few variables (e.g. added_lines and
removed_lines), but I don't think we should validate it when checking
syntax.

Bug: T198531
Change-Id: I417e1b8d4802bbfccd091ce5c7617659cfd1e4ea
2019-08-04 17:14:44 +00:00
Daimona Eaytoy 9049be3609 Specialize empty AFPData types
As described in T156096#5389655.

Change-Id: Ifbf95a6b72a280cd77db6affbd8d642499bbfedc
2019-08-04 15:26:57 +00:00
Daimona Eaytoy a85e1ccc59 Make AbuseFilterParser::$funcCache non-static
Change-Id: I312efe3ce4d1f06e697aa4564aeec1bacbaf97d3
2019-08-03 09:19:49 +00:00
Daimona Eaytoy 09d0254172 Better handling of DNONE
This patch includes:
 * Making it possible to access offsets of a DNONE (returning a DNONE)
 * Initializing user-defined variables as DNONE inside short-circuited branches
 * Make DNONE propagate with other operators
 * Make DNONE count as false for logic operators
 * Remove a now-outaded bit in doLevelAtom. In case of shortcircuit,
   $result is now DNONE instead of DNULL, and thus it's possible to
   access offsets of it. Performance++!
 * Don't allow modifying or adding an element of a DNONE as if it were an
    array (to avoid inconsistencies)

This re-applies Id85c673337fa90a3782fd22eb9690cd996967111 with several fixes.

NOTE: Haven't tested locally, although I'm pretty confident thanks to
the amount of tests added.

Bug: T214674
Bug: T228677
Change-Id: I5ec4ab44c4e88aaf18c0d7b73355d27050beeda7
2019-08-02 21:05:08 +00:00
Daimona Eaytoy 13cdb86dd2 Revert "Initialize user-defined variables during shortcircuit"
Reason for revert: T214674#5374806

This reverts commit 56e6117afd.

Bug: T214674
Change-Id: Iccce248d2693cd9877a740b74e72a577e730435e
2019-07-29 23:06:23 +00:00
Daimona Eaytoy 56e6117afd Initialize user-defined variables during shortcircuit
Bug: T214674
Depends-On: I5a14d4b2bc3ffd9caaaa095f16f36b9b6009db05
Change-Id: Id85c673337fa90a3782fd22eb9690cd996967111
2019-07-23 12:20:53 +00:00
Daimona Eaytoy 18d7d2ed62 Start using AFPData::DNONE
This should allow more flexibility when checking syntax, and a saner
behaviour overall.
Aside from not throwing exception in certain cases, the results should
be almost equal to the ones you would get without this patch. However,
there are still a few things to improve (which for convenience I wrote
inside the parser test) and many to test.

Bug: T204654
Depends-On: I69bfec45c76509fb1112641393f78e8d8834adcd
Change-Id: I5a14d4b2bc3ffd9caaaa095f16f36b9b6009db05
2019-07-14 08:48:47 +00:00
Daimona Eaytoy 7bc566e635 Fix the regex for numbers, start deprecation of non-decimal numbers
Aside from the 14 thingy reported in the task, this syntax is awful! The
fix to the regex should only be intended as a temporary stopgap. A
proper fix would be to introduce a new syntax, like for instance the one
used in PHP.

Bug: T212726
Change-Id: Idc37a17ce539e6c63d67fc07d47d812569debe0e
2019-07-10 13:26:36 +00:00
Daimona Eaytoy 7398730563 Disallow consecutive comparisons
As explained on phabricator, they don't work with shortcircuit, so they
already fail for all filters using them. Plus IMHO it's an unnecessary
deviation from PHP's behaviour, given that this syntax doesn't do what
users may expect.

Bug: T218906
Change-Id: If9e7545e14044c8dc3b4163bb6fca8ab0683b9fa
2019-07-04 19:15:07 +02:00
Daimona Eaytoy 382751a707 Move conditions-related stuff inside AbuseFilterParser
Instead of relying on static methods and members in the AbuseFilter
class, move everything related to conditions inside the Parser, as the
amount of used conditions is something pertaining a single
AbuseFilter(Caching)Parser instance.
This change requires changing some signatures and adding parameters,
but will make introducing the new AbuseFilterRunner class easier (and
that will clean signatures, too).

Depends-On: I5b29ff556eca45fe59d15e2e3df4d06f1f6b3934
Change-Id: I7c1ea17adf7f42cf9260d416906bfbf3b8a20688
2019-06-19 15:14:17 +00:00
Daimona Eaytoy 39fc7c12af Restore unit tests for CachingParser and fix it
Added cachingParser back to *all* the parser tests, fixed a couple of
differences with the normal parser, and added a couple of tests so that
any cachingParser-related file has 100% coverage. Also move the remaining
get_matches tests inside parserTests, and specify the parser used in case of failure.
This also adds a new base class for parser-related tests with a couple
of util methods.

Bug: T201193
Change-Id: I980aec3481a52ecc35f1811a366014a5581a7cdb
2019-05-25 10:55:24 +02:00
Thiemo Kreuz fa3ce90851 Remove comments literally repeating what the code says
I find it obvious that a file called "AbuseFilterTokenizerTest" is a
"test for the AbuseFilterTokenizer class". A comment that is just
repeating this information is typicalls not helpful, but distracting
and a potential source of mistakes, e.g. when stuff is copy-pasted,
but the comment not adjusted.

Change-Id: I1d4cc06e9e5631955ff73bf675090cf9c33c9390
2019-05-15 16:04:32 +02:00
Daimona Eaytoy 0ff581e246 Clean AbuseFilterParserTests
Mostly delete result files and assume the result is always true. The few
exceptions were either moved to standalone test, or inverted.

Change-Id: I6c06e596587750c4ebaabafbd277bc75eeb436a5
2019-03-23 12:59:03 +01:00
jenkins-bot 3f3e98fbc5 Merge "Fix shortcircuit for consecutive operations" 2019-03-17 10:04:14 +00:00
Thiemo Kreuz 3993a7ea15 Replace @expectedException with $this->expectException()
The @expectedException annotation got deprecated in PHPUnit 7.5, and
removed in PHPUnit 8.0. This was done because the annotation does have
two disadvantages:
* The class name is encoded in string, where it is not easy to find for
  all IDEs and tools.
* it did not allow to say exactly *when* the exception is expected.

Change-Id: I85f0b5f44b2f400a121115d402b64827ea534c32
2019-02-19 10:58:16 +01:00
Daimona Eaytoy 6f4bfc9597 Fix shortcircuit for consecutive operations
Using break could halt parsing between operations, instead use continue
to parse all operations.

Bug: T214642
Change-Id: If67ddaffef280c2448c55ae536013758617bba68
2019-02-08 17:55:59 +00:00
jenkins-bot a2bee3bcf3 Merge "Simplify parser methods" 2019-01-19 12:11:04 +00:00
Daimona Eaytoy 93e8cb5ac5 Tune logging channel
As follow-up of I10b1fd2d9bdfe518089c053d77fef568170ecb65, use
'AbuseFilter' instead of 'AbuseFilterDeprecatedVars' as channel name.
Raise level for null-title filtering. Since with a null title
several things are likely to break, a warning is more appropriate here.
Tweaked the message as well, to include the bug number and to avoid
pointlessly including the title (which is null).
Lower the level for stashedit hit/miss (as it's really spammy and not
that useful right now).
Use 'abusefilter' instead of 'AbuseFilter' for statsd so that everything
has the same prefix.
Also raise the level for parser exceptions and unrecognized
consequences.

Change-Id: I1f9988155e924232b201281795cd322636da8082
2019-01-16 08:56:22 +00:00
Daimona Eaytoy 6d54b83f2c Simplify parser methods
Use a single function to check parameters amount, avoid duplication
between keywordIn and keywordContains, use if...elseif instead of
if-else when statements have a return inside, simplify some other logic,
add typehinting, and change method visibility according to use of such
methods.

Change-Id: I22225a5cbbb93679a0e78bf6e15866829167fbf4
2018-10-03 17:19:40 +02:00
Daimona Eaytoy bffba28713 Add full tests for deprecated variables
This test checks every deprecated variable to be identical to the
newly-named one, and to emit a debug notice. It also changes such debug
to be emitted via logger instead of wfDebug.

Bug: T201193
Bug: T173889
Change-Id: Ie55746bb7731062ae2d46d84857af2a05d78cf4c
2018-08-29 11:00:28 +02:00
jenkins-bot 97f98b029d Merge "Improve parser coverage" 2018-08-23 14:33:54 +00:00
jenkins-bot ad69ea648e Merge "Remove unused function and improve unit test" 2018-08-23 13:46:41 +00:00
Daimona Eaytoy 0e2ae113fb Improve parser coverage
On the way to 100%...

Bug: T201193
Change-Id: I5fd311f861acccb31f346da9acb379b0366488e7
2018-08-23 12:13:47 +02:00
Daimona Eaytoy 03b52c2b37 Remove unused function and improve unit test
AbuseFilterParser::setVars is only used in a parser test. In the past it
was also used in the actual code (see for instance
https://phabricator.wikimedia.org/diffusion/EABF/browse/master/;5cc8dac63ca585c288ca4c8605db810774e39666?grep=setVars), but at the moment it's pretty unuseful.
This patch removes such function and makes the unit test use literals
instead of variables to avoid calling it.

Change-Id: I80cbc4033ff96f2fe8c1da263b1877bfb4c7c0c4
2018-08-23 11:00:16 +02:00
Daimona Eaytoy 447d434e2a Improve code coverage
Add some parser tests, improve existing ones, and add missing @covers.

Bug: T201193
Change-Id: I9c0d2d83560baa4a3e1d4465b7919a48c4e26ac1
2018-08-22 19:07:14 +02:00
Daimona Eaytoy d35c42757c Add missing @covers tag
This should help with tracking code coverage and also explains some
coverage discrepancies encountered while writing other tests.

Bug: T201193
Change-Id: I8b20abc46c2d6c6f582953139b9a9f3710b2e4ea
2018-08-22 17:00:38 +02:00
Daimona Eaytoy 4f3b020f5d Improve code coverage for AbuseFilterParser
Add some tests and improve others to raise coverage percentage. This
should lead to almost 100% for the AbuseFilterParser class. Aside from
this, a couple of changes:
* Remove an unused function
* Let equals_to_any return a genuine result with empty strings
* Remove an if which will never be true in skipOverBraces, since the
function is called after checking the same conditions.

Bug: T201193
Change-Id: I7020b2ed996236c38c5784d161ad98ec44163406
2018-08-20 14:38:40 +02:00
jenkins-bot 4b185b3749 Merge "Add phpunit tests for noparams and notenoughargs exceptions" 2018-08-02 08:41:46 +00:00
Daimona Eaytoy 4f037c29c2 Add phpunit tests for noparams and notenoughargs exceptions
We're really missing exception tests: in fact, 'noparams' not being
thrown was discovered only a few days ago and worked like that for
years. This patch adds phpunit tests for both noparams and notenoughargs
exception, also checking the returned message.

Depends-On: I484fe2994292970276150d2e417801453339e540
Change-Id: Ia0b9b8fd5c979be06879723b746f9356c628f5cd
2018-07-15 17:35:45 +02:00
Daimona Eaytoy e9921bcda7 Add other phpunit test for AFPUserVisibleException
Follow-up of Iacb8f7a361079e3e117dc6845597c7bd8473e54a for exceptions
thrown outside the parser. With this patch all uses of AFPUserVisibleException
will be covered.

Depends-On: Iacb8f7a361079e3e117dc6845597c7bd8473e54a
Change-Id: Ia7ef6eb832d5725a804a60cb58bc110b06c8abe2
2018-07-01 18:34:01 +02:00
Daimona Eaytoy 7a64280893 Add phpunit tests for all exception thrown in the parser
All uses of "throw" inside AbuseFilterParser are now covered.
Bonus: added a standard suppresswarning when checking regex validity.

Change-Id: Iacb8f7a361079e3e117dc6845597c7bd8473e54a
2018-07-01 18:31:11 +02:00
Daimona Eaytoy 3c3a521fec Fix coding conventions exclusion rules
This should fix every error with excluded rules, leaving only the one
for $wgTitle. A double check would be nice in order to avoid regressions
due to stupid mistakes.

Bug: T178007
Change-Id: I22c179f3a01d652640304b59e43fcb5b5a9abac3
2018-04-20 08:40:18 +00:00
Max Semenik a5b92a90c0 Fix license header
Change-Id: Ifb6b2d39fab9375e09c22e87ec818d74bd22fb28
2018-04-03 02:16:33 +00:00
Max Semenik 5c89246fce Rename files to match class name
Change-Id: Ia19bfec6c2289912699b6c90261afda311afb56e
2018-04-02 22:08:13 -04:00
Renamed from tests/phpunit/parserTest.php (Browse further)