Commit graph

789 commits

Author SHA1 Message Date
jenkins-bot 48713c824b Merge "Throw AFPUserVisibleExceptions for empty operands in CachingParser" 2019-09-15 08:36:39 +00:00
Daimona Eaytoy a4e25c1ac9 Throw AFPUserVisibleExceptions for empty operands in CachingParser
Instead of TypeErrors. Basically, only empty parenthesis had to be
fixed.

Bug: T156096
Change-Id: I019615c7bfaa179c2184b5d3ea2c6b5da91366e3
2019-09-14 18:35:40 +00:00
Daimona Eaytoy 5267082c85 Better logging for unset variables
We have many log entries, so we need some more debug data.

Bug: T230256
Change-Id: I0e9638c1ffe537ea6cfd6886ff32ef447fdacc28
2019-09-14 16:49:55 +00:00
Daimona Eaytoy fe395bd96b Use dieBlocked instead of directly using apierror-blocked
This allows us to:
 - Defer handling of the block to the main module
 - Choose the right message depending on the block type
 - Avoid directly using the apierror-blocked message, which could change
 in the future.

Change-Id: If2e32bd2ccf5e314aa51203afd1522b8481377e0
Follows-up: I35f2c6e701a24dccb6e26e3f3c578fd44f68127d
2019-09-14 10:18:01 +02:00
jenkins-bot f8ee9fb959 Merge "Prohibit sitewide blocked users from restoring autopromotion" 2019-09-14 02:59:41 +00:00
jenkins-bot 45d7bd5971 Merge "CachingParser: ensure to catch errors inside short-circuited blocks" 2019-09-14 01:56:35 +00:00
jenkins-bot b8ad85cac7 Merge "Annotate the AST with var names before caching the AST" 2019-09-14 01:03:53 +00:00
Daimona Eaytoy 6e9a9a3bc2 CachingParser: ensure to catch errors inside short-circuited blocks
This is similar to the old parser: when discarding a node, actually
evaluate it if short-circuit is not allowed.
Add a whole lot of tests for all possible exceptions.
Move the logic to extract a message from an AFPUserVisibleException away
from the parser, to keep unit tests working.

Bug: T232498
Change-Id: I31ee4e255c6a87dd693b9bcd582539fdf57acd45
2019-09-13 21:13:15 +00:00
Daimona Eaytoy 004ccfdb5c Annotate the AST with var names before caching the AST
This implements T230982#5475400, and it should speed up the CachingParser by roughly 40%.

Bug: T230982
Change-Id: I803cc58637d50eb90e57decf243f5ca78075d63d
2019-09-13 19:43:50 +00:00
DannyS712 467fba75a0 Prohibit sitewide blocked users from restoring autopromotion
Bug: T232884
Change-Id: I35f2c6e701a24dccb6e26e3f3c578fd44f68127d
2019-09-13 18:32:55 +00:00
jenkins-bot cfad7d6f14 Merge "Actually return errors for action=edit API" 2019-09-10 19:59:03 +00:00
Bartosz Dziewoński 82b6f191d4 Actually return errors for action=edit API
Setting 'apiHookResult' results in a "successful" response; if we want
to report an error, we need to use ApiMessage. We already were doing
this for action=upload. Now our action=edit API responses will be
consistent with MediaWiki and other extensions, and will be able to
take advantage of errorformat=html.

Since this breaks compatibility anyway, also remove some redundant
backwards-compatibility values from the output.

To avoid user interface regressions in VisualEditor, the changes
I3b9c4fef (in VE) and I106dbd3c (in MediaWiki) should be merged first.

Before:
    {
        "edit": {
            "code": "abusefilter-disallowed",
            "message": {
                "key": "abusefilter-disallowed",
                "params": [ ... ]
            },
            "abusefilter": { ... },
            "info": "Hit AbuseFilter: Test filter disallow",
            "warning": "This action has been automatically identified ...",
            "result": "Failure"
        }
    }

After:
    {
        "errors": [
            {
                "code": "abusefilter-disallowed",
                "data": {
                    "abusefilter": { ... },
                },
                "module": "edit",
                "*": "This action has been automatically identified ..."
            }
        ],
        "*": "See http://localhost:3080/w/api.php for API usage. ..."
    }

For comparison, a 'readonly' error:
    {
        "errors": [
            {
                "code": "readonly",
                "data": {
                    "readonlyreason": "foo bar"
                },
                "module": "main",
                "*": "The wiki is currently in read-only mode."
            }
        ],
        "*": "See http://localhost:3080/w/api.php for API usage. ..."
    }

Bug: T229539
Depends-On: I106dbd3cbdbf7082b1d1f1c1106ece6b19c22a86
Depends-On: I3b9c4fefc0869ef7999c21cef754434febd852ec
Change-Id: I5424de387cbbcc9c85026b8cfeaf01635eee34a0
2019-09-09 20:15:19 +02:00
Daimona Eaytoy 173bd089b3 Remove script for blockautopromote entries
It was executed on WMF wikis, and since they were the only affected
wikis we can remove the script.
Also remove a temporary back-compat check in the log formatter.

Bug: T231131
Change-Id: I534acd9c86894eb1bdd96331e9fa85afc7502f88
2019-09-09 13:56:56 +02:00
Daimona Eaytoy 7917354716 Remove redundant logic from special pages
SpecialPage::setHeaders already handles page title, robot policy and
articleRelated. Moreover, avoid having different messages for the H1
title on the special page and the description shown elsewhere, just like
the base SpecialPage class suggests doing. The deleted messages have
been moved to the default message used by SpecialPage::getDescription.

Change-Id: Iab6beaf64b142e30469afd798c569ef40182153e
2019-09-08 12:30:01 +02:00
Daimona Eaytoy 7b06be0204 Allow dangling commas in variargs
This is because there are many filters using this feature. Moreover, it
could make it a little easier to add new arguments, just like dangling
commas in PHP arrays do.
Also re-align the CachingParser code of doLevelFunctions to the one in
the old Parser.

Bug: T153251
Change-Id: Ie4325159f47310788da57415a5e36e62aa4efad0
2019-09-07 11:19:14 +02:00
jenkins-bot 5be19f6f65 Merge "Add a 'strict' option to VariableHolder::getVar" 2019-09-05 19:23:23 +00:00
Daimona Eaytoy 489da0d229 Add a 'strict' option to VariableHolder::getVar
This will help mitigating problems like T230256 by enforcing that the
requested variables must exist. For now, it will only log bad usages,
thus providing a way to identify affected filters and fix them.

Bug: T230256
Change-Id: I7a61916576e444a56f0e07da7b6e5033346226bd
2019-09-04 18:19:23 +00:00
Daimona Eaytoy 13b1e880f2 Hotfix other DUNDEFINED casts to bool
These were spotted on testwiki with wmf.21.

Change-Id: Ic4d67a2b83aedfeb574fa1363a9fc618b2862f95
2019-09-04 18:06:22 +00:00
jenkins-bot f2ae634831 Merge "Fix filter validation in ViewEdit" 2019-09-04 13:34:33 +00:00
Daimona Eaytoy 15e9f34115 Fix filter validation in ViewEdit
Currently it's impossible to create new filters!

Bug: T231985
Change-Id: I92a7739fe9defae6b3d74381792340c7125d9086
2019-09-04 14:04:45 +02:00
Daimona Eaytoy ce8539e2a5 Move parser tests back to /unit
Using `new LanguageEn()` involved a global, so use a MockObject instead.
Also fix LoggerFactory usage in Tokenizer to use DI instead.

Change-Id: I94d03f9459ab6444e239386eb96a0c2434bfe3dc
2019-09-03 13:23:11 +00:00
jenkins-bot 3d319edba9 Merge "Upgrade phan-config to 0.7.1" 2019-09-02 12:46:29 +00:00
jenkins-bot 6441e71ebe Merge "Also catch Error in the hacky workaround for bad rows" 2019-09-02 12:46:27 +00:00
jenkins-bot 2ed5be29e1 Merge "Use permissions accessors" 2019-09-02 10:55:46 +00:00
Daimona Eaytoy 393e47c5a7 Upgrade phan-config to 0.7.1
Change-Id: I859d81eda8601da91602b27a223b6d6d59ecf563
2019-09-01 09:42:26 +00:00
Daimona Eaytoy 2a956bc81a Also catch Error in the hacky workaround for bad rows
PHP7 throws an Error, not a BadMethodCallException. We don't want to
clog the logs with fatals, now that PHP7 is closer.

Bug: T187153
Change-Id: I5a9e581ee0418ae41dd911de02a64d18e4670cd4
2019-08-31 20:42:41 +02:00
jenkins-bot 25d63aa639 Merge "Add new number syntax as experimental" 2019-08-31 17:12:10 +00:00
jenkins-bot 134c75dd5e Merge "Remove AbuseFilter::saveFilter dependency on AbuseFilterViewEdit" 2019-08-31 17:06:00 +00:00
jenkins-bot fc3349df1f Merge "Fix param validation in ViewEdit" 2019-08-30 13:37:21 +00:00
Daimona Eaytoy 933b791ef3 Fix param validation in ViewEdit
We didn't check if the provided ID was valid. While editing an existing
filter (or creating a new one), we check the ID in SpecialAbuseFilter,
so it's guaranteed to get an integer in ViewEdit, and the case of a
non-existing filter is handled later, in buildFilterEditor.
But for links like Special:AbuseFilter/history/foobarbaz/item/1 (where
"foobarbaz" should be the filter ID), no validation was performed. This
caused a useless query to be carried out on the abuse_filter_history table (which would likely return false), then accessing properties of a non-object ('$row->afh_id'), and we ended up showing filter 1. This was spotted because we actually got notices in production.

Bug: T231632
Change-Id: I6436c7d2df8c1f0fc971f4a4079dac9118aa8209
2019-08-30 08:59:49 +00:00
Daimona Eaytoy fdb71b5868 Make AbuseFilterVariableHolder::$mVars public again
+fixme comment per T231542#5451567.

Bug: T231542
Change-Id: If8b7b0568568df93548f12ccdc85fa174ec3558e
2019-08-29 18:51:43 +02:00
Daimona Eaytoy eb91595d8a Use row->afl_action instead of $vars
There's not need to use the variableholder in this hacky (albeit common)
whay, since the row already holds the action.
Note, this doesn't guarantee that the next two lines won't fail - I'd
need to see the actual var dump (T231542#5450720) to determine exactly
why this is failing.

Bug: T231542
Change-Id: I2112b046d00e06b575d15ab3d7da57484fd9cbbd
2019-08-29 13:48:31 +02:00
Daimona Eaytoy d51ca862c6 Move parser tests to /unit
IMHO these can be considered unit tests; they were already fast, but now
they're executed in an instant.
This requires several changes: 1 - delay retrieving messages in
AFPUserVisibleException, to avoid having to deal with i18n whenever we
want to test exceptions; 2 - Use some DI for Parser and Tokenizer.
Equivset-dependend tests are also moved to a new class, thus helping to
fix the AF part of T189560.

Change-Id: If4585bf9bb696857005cf40a0d6985c36ac7e7a8
2019-08-28 16:36:37 +00:00
Daimona Eaytoy 540edc5174 Remove AbuseFilter::saveFilter dependency on AbuseFilterViewEdit
This dependency is wrong, and removing it will also allow creating an
edit API.

Bug: T213037
Depends-On: Id8412e2b8a4e873fd4821ecc1a3c95710be9a870
Change-Id: If8e745a3227cea5093ea3fd8f5b201adedaba3ae
2019-08-27 16:26:18 +00:00
Daimona Eaytoy 8e166f10d6 Refactor and speed up non-parser tests
Some of these are transformed into real unit tests, while the
AbuseFilterSaveTest class is refactored to avoid using the DB and to use
a lot more of mocks and DI.

Depends-On: I22743557e162fd23b3b4e52951a649d8c21109c8
Change-Id: Id8412e2b8a4e873fd4821ecc1a3c95710be9a870
2019-08-27 16:24:27 +00:00
Daimona Eaytoy 87713008d5 Use permissions accessors
There are lots of calls to $user->isAllowed which could be simplified
using available accessors like canEdit(). So simplify those calls and
avoid duplication.

Note that using canEdit also fixes a bug which affected blocked users:
we used to show e.g. the import link, and not to display as disabled
several text fields, while blocked users cannot actually edit filters.

Depends-On: I22743557e162fd23b3b4e52951a649d8c21109c8
Change-Id: I62779e940949ef49018a9c6d901bb6e10aa81da8
2019-08-27 13:21:55 +02:00
Daimona Eaytoy c469fb4b76 Mostly remove $wgUser
There are lots of cases where we can inject a User object without
additional efforts. Now $wgUser is only used inside AFComputedVariable,
which is a little bit harder to handle because some instances of that
class are serialized in the DB, and thus we cannot easily change the
constructor until T213006 is resolved.

This partly copies what Ia474f02dfeee8c7d067ee7e555c08cbfef08f6a6 tried
to do, but adopting a different approach for various can*() methods:
they're now static methods in the AbuseFilter class, so future callers
don't need to instantiate an AbuseFilterView class. This also allows to
re-use those methods in an API module for editing filters (T213037).

Bug: T213037
Bug: T159299
Change-Id: I22743557e162fd23b3b4e52951a649d8c21109c8
2019-08-27 13:20:37 +02:00
Daimona Eaytoy 71730f7d44 Warn if a function has been given too many parameters
While this is not as important as throwing for too few parameters, IMHO
it's still important to fail in this case. Mostly because if a function
receives too many parameters, chances are that who wrote the filter
didn't do that intendedly, and thus there may be a hidden bug.
Bonus: fix a few docblocks.

Bug: T230803
Change-Id: Iac2931f17b50ace8c8f4c2faa44b3f54ca134c54
2019-08-26 20:29:49 +02:00
Daimona Eaytoy 4d86758a49 Add new number syntax as experimental
For now it will only report successful parse. Next step is formally
deprecating the old one (escalated to warning), then removing it in
favour of the new one (in another MW version).

Bug: T212730
Change-Id: I5dd11fd67d8e57d1d0c52ddfa026920ebfc5ee13
2019-08-26 08:15:55 +00:00
jenkins-bot 89524790d5 Merge "Add a hook to determine whether the current action should be filtered" 2019-08-25 18:45:07 +00:00
jenkins-bot ff2f6ee26f Merge "Add a new class for the CachingParser's AST" 2019-08-25 18:00:24 +00:00
Daimona Eaytoy d515af0ae6 Add a new class for the CachingParser's AST
This allows a little bit more of abstraction: we can store other data in the
tree, without having to store it in a specific node (e.g. the variables map,
which is still unused). It also adds a few typehints, and specializes
the return value of eval'ing the AST: previously, it was the one of
evalNode, which wasn't guaranteed to be an AFPData. Now we have this
guarantee. Last but not least, we can now measure runtime metrics for
evalTree, which doesn't recurse.
Bonus: fix a check in the old parser, which used the wrong variable when
reporting outofbounds errors.

Change-Id: Iff806793b1d968e9bb6220f1459f3d0ac587c7da
2019-08-25 17:29:16 +00:00
jenkins-bot 6196801178 Merge "Log more empty operands" 2019-08-24 20:53:01 +00:00
Daimona Eaytoy 2d031d0bee Log more empty operands
And fix a couple of minor bugs.

Bug: T156096
Depends-On: I3b85087677607573f4fa68681735dc35348dcd87
Change-Id: Ia4c713a1d45827f6a8bc5566a8d8835c49f8108a
2019-08-24 19:59:53 +00:00
Daimona Eaytoy 7f554734e6 Don't hardcode blockautopromote duration
As explained on phab, and add a script to fix broken entries.

Bug: T231131
Change-Id: I95d70acb936b5ca987af8f237d236fe47b663919
2019-08-24 11:40:11 +02:00
Huji Lee 1ddb65021b Add links to AbuseFilter logs on Special:Undelete
Depends-On: I671a0479e877e6c37606b688064cb9c893717709
Bug: T231055
Change-Id: Iebf832c513c6a4e954db0ba2633dd8ba6f27b412
2019-08-23 14:56:43 +00:00
Daimona Eaytoy bf61414f88 Don't show empty "Tools:" section in ViewEdit
After having removed the export link in
I72f46247f4323fb5bfe7fa74f332076dbd346187, we don't have any tool to
show for new filters. So avoid outputting an empty section.

Change-Id: Ia07bccdbadb7b874397135bc3f7468d6e0b9eb13
2019-08-21 17:32:43 +02:00
jenkins-bot 47838715fa Merge "Allow if without else" 2019-08-20 20:12:19 +00:00
jenkins-bot 5e605aaa62 Merge "Even better handling of DUNDEFINED" 2019-08-20 20:00:52 +00:00
jenkins-bot bf8ccccade Merge "Fix a bug in the return value of the CachingParser" 2019-08-20 19:58:38 +00:00