Commit graph

128 commits

Author SHA1 Message Date
Daimona Eaytoy c368575af0 Create a base interface for watchers
This will ease adding new watchers, for instance to send Echo
notifications (see T179495 and T100892).

For now, this is just boilerplate, and converting EmergencyWatcher to
the new interface.

Change-Id: I18d62aba53471202b709cdb19033b1729c5c25b4
2020-11-20 23:34:20 +01:00
Daimona Eaytoy 210cf29658 Add an interface for exporting/importing filters
The main benefit of having a dedicated interface is that we can easily
change the output format. So we're now using a custom array without
references to the DB schema, thus making the import/export process
completely independent from the schema.

Change-Id: I4c0de41d914baf1e9a0e588bd31f95b3524a424b
2020-11-18 22:06:09 +00:00
Matěj Suchánek e7813fbafb Introduce EmergencyWatcher service
Change-Id: I45477ca84a99f620d182ef95e5627d421d38f077
2020-11-18 14:20:18 +00:00
Daimona Eaytoy ae29451ab8 Introduce a FilterCompare service
The scope is still quite limited, but as noted in a todo, we might want
to make this completely independent from the database, and add the use
case of ViewDiff.

Change-Id: Ie980fff0983b3e86037265e85da04444c809a6e8
2020-11-18 11:52:44 +00:00
Daimona Eaytoy 1bcfdc3b13 Introduce a FilterValidator
This moves a lot of things away from the AbuseFilter class. There's a
nasty static dependency on ChangeTags, but it's very limited anyway, and
it's going to be fixed once T245964 is resolved.

Change-Id: Ia7df4b4d3289c2722323f59ceecf3fdd38277785
2020-11-18 01:41:31 +00:00
Daimona Eaytoy 725ec052ed Add a FilterLookup service
Some pieces of code were updated to use Filter objects, while other
places are still to be updated. We also need to change the history part
to exclude actions somehow, cleanup the ViewEdit, reduce direct DB
access or anything mentioning DB fields outside of FilterLookup, etc.

Change-Id: I42b7ded685db76eddd45e4b1336f9828cba811ce
2020-11-18 01:17:47 +00:00
Daimona Eaytoy 71a61c2089 Add value objects to represent filters
This is just a start; next step is adding a factory/store method to
get/store these objects. And then use these value objects whenever
applicable.

Note: the actions-related code is still not fully implemented. This is
going to happen as part of the FilterLookup.

Change-Id: I5f33227887c035e301313bbe24d1c1fefb75bc6a
2020-11-04 12:56:14 +01:00
Daimona Eaytoy be75cf1c40 Introduce AbuseFilterParserFactory service
TODO For the future: the final directory for Parser-related classes
should be "Parser", not "ParserNS". However, moving all classes now
would make it harder to rebase changes etc.

Change-Id: Ice335f4723e74f4e5fbe8dcc76ff8ea16310962c
2020-10-31 21:19:00 +01:00
Daimona Eaytoy 1f8df50cb3 Add a service to retrieve the central DB
This is a thin wrapper around LBFactory and the global variable, that
can be injected in classes requiring it (no real class right now, but
that's going to change soon).

Also, remove some DWIM-style returns which made the code harder to
understand.

Change-Id: I1d28ad4a67f914103f3a17cda5f61b28070c7f1c
2020-10-31 12:32:46 +00:00
Daimona Eaytoy 04451d7bde Little cleanup for AbuseFilterRunner
Remove outdated/pointless comments, use already defined variables, etc.

Additionally, make it possible to disable throttling locally.

Change-Id: I98fd5f3eb47b32fc1013360e462a57d932174a95
2020-10-30 21:42:54 +00:00
Daimona Eaytoy cbea88f818 Add a service to retrieve the filter user
Unfortunately, this isn't using DI completely, because of the
User::newSystemUser call. I'm not even sure if we really need to call it
or we can just stick to new UserIdentityValue, but leaving like this for
now.
Also, the types were weakened to UserIdentity, so the transition is
going to be easy anyway.

Change-Id: I08f8fae0fcc622ff0ac3f86771476d06d1c18549
2020-10-26 14:06:53 +01:00
Matěj Suchánek 1445d5962a Introduce BlockAutopromoteStore service
This service is responsible for the blockautopromote feature:
(un)block autopromotion and check status.

The patch mostly moves code from static methods to the new class
and relaxes type hints (e.g. from User to UserIdentity).

Change-Id: I79a72377881cf06717931cd09af12f3b8e5f3e3f
2020-10-24 12:31:44 +00:00
Daimona Eaytoy b309c804fc Add dedicated classes for more hook handlers
The schema changes hook was chosen because the handler is very long. The
test ones were chosen to keep test things away from actual code.

Bug: T261067
Change-Id: Ie06bf62399f6353e3e268cccb3fe4b41bbf951c5
2020-10-22 18:23:09 +02:00
Daimona Eaytoy 4c06dd52c8 Replace $wgAbuseFilterRestrictions with more specific variables
So that sysadmins can further customize the extension. It was also wrong
to use the same variable for many different things.

Note that there's no associated patch in wmf-config because we use the
defaults. However, before merging this patch, please recheck that
AbuseFilterRestrictions and AbuseFilterDisallowGlobalLocalBlocks aren't
used there (https://codesearch.wmflabs.org/operations/?q=AbuseFilterDisallowGlobalLocalBlocks%7CAbuseFilterRestrictions&i=nope&files=&repos=)

Bug: T175221
Change-Id: I7581b3ee6d9d11a6cf1599b8ff874e8c3d54adf4
2020-10-22 13:38:59 +00:00
Matěj Suchánek 2ee3a0d247 Migrate change tags hooks to DI
Bug: T261067
Change-Id: I7b95cd19ab0ae04820e8dcb3481d29a2f9e7a0ca
2020-10-21 16:18:06 +00:00
Matěj Suchánek 85e000c6ed Add ChangeTagsManager service
This service will be resposnsible for loading
and caching change tags used by abuse filters.

Change-Id: I9a710af1dd1ae58c47de1e8509246ed929d0a662
2020-10-21 16:24:32 +02:00
Daimona Eaytoy 9bc885b6b3 Add a ChangeTagger class
The logic about action IDs and the persistent buffer is now encapsulated
inside a single service, which is a step towards getting rid of global
state in the AbuseFilter class, and reducing the responsibilities of the
Runner.

An important change made here is that we now require a LinkTarget rather
than a Title. This removes a dependency on the Title class (a monster
object), makes tests simpler, and denies the need to inject a
TitleFactory. This means living without some bits of context (e.g. we're
no longer using makeTitleSafe to ensure a valid title, and we have to
build a "prefixedtext" manually), but this shouldn't be a problem, given
that the titles are only used to create a cache key: invalid titles are
not a problem, and concatenating namespace + title should always be
sufficient.

Bug: T265370
Change-Id: Iff59cd3d889454a482a89c16691bfefcc5ec0a12
2020-10-21 13:19:30 +02:00
Daimona Eaytoy 1efc324d97 Use new services in AbuseFilterRunner
The first one is UserGroupManager, used for the 'degroup' action. This
is a simple one-line replacement (repeated twice), and the current code
was already using this service under the hood.

The second one is BlockUser, which is not a one-line change (but still
quite simple). In particular, this allows us to avoid duplication with
core logic when constructing the log entry (this is now done by
BlockUser).

Bug: T248743
Change-Id: Ib7c1dc107a169b575f7021e64b6a8fee09529548
2020-10-14 23:08:32 +00:00
Matěj Suchánek 7ef2259228 Migrate a few hook handlers to DI
Bug: T261067
Change-Id: If699917c3d2e9e22525c7d0495554e25f6b45125
2020-10-10 17:23:04 +00:00
Daimona Eaytoy 2026e3ac3a Add an AbuseFilterPermissionManager service
This service should act as a mediator between the AF code and the
permission manager, and it should know what are the permissions required
by each action.

Change-Id: Ieb177d9992147b11fa7b8f05929da6c182cc2286
2020-10-10 14:03:29 +02:00
jenkins-bot c0defc1055 Merge "Add a new FilterProfiler service" 2020-10-10 10:08:58 +00:00
Matěj Suchánek 9e6bc2f4ee Move log formatters to a separate directory and namespace
This will clean up the includes/ directory a little.

Change-Id: I61adacf32257bb2402a272b60b52b69505d981c5
2020-10-07 16:25:38 +02:00
Daimona Eaytoy bc9898f1a1 Add a new FilterProfiler service
Change-Id: Ib66c42ac220731f4e1da9ee6cfb5290759dd6494
2020-10-04 22:00:57 +00:00
Daimona Eaytoy 6c8a29698b Add test traits for uploads and account creation
Ideally, this might live in MediaWikiIntegrationTestCase. For the
createaccount one, AuthManager should also provide a method to log the
creation, because currently we are forced to copypaste that code here.

 - Add the missing tests for 'upload' in RCVariableGenerator, and adjust
the existing ones (delete file afterwards, more tablesUsed, use the
right extension).

 - Exclude from the coverage report a couple of lines which should
theoretically be unreachable. Escalate logging to WARN level, where it's
more likely to be spotted.

 - Remove an unused method (RCVariableGenerator::newFromID). This denies
   the need to maintain and cover it. We also don't want this generator
   to act as a factory.

Overall, this change brings the coverage for RCVariableGenerator to 100%

Bug: T201193
Change-Id: I425c3d9f6800f74eb6e4eda483b90cfb3bbbcb51
2020-10-04 13:16:58 +00:00
Matěj Suchánek 65708afcea Integrate with Renameuser
Register abuse_filter and abuse_filter_history tables.
abuse_filter_log is more difficult (if possible).

Bug: T27377
Bug: T206477
Change-Id: If8289101a08887519d5a90ef84700421b8ed2406
2020-10-01 08:10:22 +00:00
Daimona Eaytoy 55ba083b13 Introduce a KeywordsManager service
This will decouple a bit the huge and chaotic tangle of AF classes. Some
boilerplate code for AbuseFilter services is also added with this patch.

Note that this requires injecting a KeywordsManager in
AbuseFilterVariableHolder, or unit tests would fail. This is still
incomplete, and the Manager is only injected in tests, because
VariableHolder still has to be refactored.

The test for the UpdateVarDumps script had to be updated, because
serializing VHs in there was a bad choice. As pointed out in a comment,
the test is likely going to break again once we remove the BC code, but
I hope that we'll be able to remove the test at that point.

Change-Id: I12a656a310adb8c5f75cab63f6db9e121e109717
2020-09-28 23:03:52 +00:00
Matěj Suchánek f1ecdd4aff Inject PermissionManager to SpecialAbuseLog
Change-Id: I1c80490567ac2d9f716c988ebdad6b59cf28aa06
2020-09-18 23:22:11 +00:00
jenkins-bot 3f8e61b42f Merge "Allow Blockautopromote duration to be configured for wikis." 2020-09-17 17:53:06 +00:00
DannyS712 bf74fd0c23 Allow Blockautopromote duration to be configured for wikis.
Rather than always using 5 days, the length (in days) can be configured by setting
`AbuseFilterBlockAutopromoteDuration` to the desired length.

Bug: T231756
Change-Id: I996e08a9099ab59657fe511ec2934d26edfa5c7b
2020-09-17 17:19:00 +00:00
DannyS712 9c1868d55e Update hook calling to use new HookContainer
Bug: T254306
Change-Id: Ic5c82a367e34135bbc0f00ece5aeef4f2d92881b
2020-09-17 10:05:45 +00:00
Umherirrender f932ba8328 Use LinkBatchFactory in Special:AbuseLog
Change-Id: I2ccf9cd36475a65e61ad0e80ec159f841849089f
2020-09-06 09:31:49 +00:00
Kunal Mehta ad5889ea83 Remove incorrect merge_strategy definitions
Those merge strategies aren't needed because
array_plus is for when the keys of the array are integers, not necessarily
the values.

Change-Id: I164723c4cc6b12cd6e4ce0d49c74d93eda5d871d
2020-07-06 09:00:35 +00:00
DannyS712 4b35336638 Update hooks to use PageSaveComplete
Extension requires MW 1.35+, always available

Bug: T250566
Change-Id: I60cf3cc42db989d8ccb0d06d3cf9eae8a85784ac
2020-06-16 04:18:39 +00:00
Reedy e7a02ddcd0 Replace _merge_strategy with merge_strategy
Change-Id: I313cedec3297725b7711feee20f17621f91561a6
2020-05-12 20:18:16 +01:00
jenkins-bot b118fd50dc Merge "Improve var dumping in /details, /examine and /tools" 2020-04-29 20:00:54 +00:00
DannyS712 6c947609bb Remove unneeded dependency on 'mediawiki.notify'
The single line of code it contained is part of mediawiki.base as of
MW 1.35, and the extension only supports MW 1.35+

Bug: T233676
Change-Id: I4c63fefcd96f7d56be1caf6b37d4ba598e8bdb45
2020-04-09 02:51:57 +00:00
Daimona Eaytoy 4c98aecf4d Improve var dumping in /details, /examine and /tools
Using var_export for better visual effect, especially for arrays.
The result from /tools is much clearer and the 'wrong syntax' message is
a bit more explicative than before.

Bug: T190653
Bug: T239972
Change-Id: I79a17305c7f19f7900f896f895e9365bb5f2fd58
2020-03-28 17:35:43 +01:00
Timo Tijhof c7653905ad Migrate 'user.tokens' dependency to 'user.options'
The 'user.tokens' module is a deprecated alias to 'user.options'
since MW 1.35.

Bug: T235457
Change-Id: I45e8e35edf4e6f901c2045d2592326e531607726
2020-03-27 16:22:57 +00:00
Daimona Eaytoy 2c03c77d9f Add a maintenance script to clean afl_var_dump
This script aims to fix every problem reported in T213006. Subsequent
patches will add new code and drop the back-compat one.

Bug: T213006
Bug: T187153
Bug: T204236
Bug: T187731
Bug: T204235
Bug: T214193
Bug: T214196
Bug: T34478
Depends-On: I5b29ff556eca45fe59d15e2e3df4d06f1f6b3934
Change-Id: I22cf698c5be77506727cbd227c67e037a5d89b5c
2020-02-28 19:41:30 +00:00
Daimona Eaytoy 1686042a91 Move variable generators to new classes
RunVariableGenerator is for generating variables based on the current
action;
RowVariableGenerator is for RC entries;
VariableGenerator is the generic one.

This patch only moves the methods to the new classes, to keep the diff
easier to read, and facilitate conflict resolution. These classes will
then be revamped in I366d598b69ad866496b7cb0059e0835c02e54041.

Note that these classes are now namespaced.

One method, AbuseFilter::getEditVars, was renamed to
AbuseFilterVariableGenerator::generateEditVars, because it would
otherwise conflict with an incompatible method in RunVariableGenerator.

Change-Id: Iff412e5492873d4fae55402939a51609e64d55a8
2020-02-07 19:44:31 +00:00
Daimona Eaytoy f382304aae Add a base class for parser transition
Change-Id: I31282b8632c332b6d46a6bb4a42f57ac0d005b5f
2019-12-15 13:29:56 +00:00
James D. Forrester cff9889549 Move test-only classes to new TestAutoloadClasses declaration
Bug: T196090
Depends-On: I88cf69663362fd599c20649b1df641907a02678d
Change-Id: I5e3ef040628c4ff629a8aabc56a3b8ba259b7e78
2019-12-10 12:18:17 -08:00
DannyS712 a137992900 Restrict viewing Special:Log/AbuseFilter, and remove from recent changes
Only allow users with `abusefilter-view` to see abuse filter log
entries. Note that this is different from the abuse log itself.

Bug: T34959
Change-Id: I587c2eb9bdc2e5ca2652aa338440cdc5b6eb9235
2019-11-10 10:41:17 +00:00
Daimona Eaytoy ce8539e2a5 Move parser tests back to /unit
Using `new LanguageEn()` involved a global, so use a MockObject instead.
Also fix LoggerFactory usage in Tokenizer to use DI instead.

Change-Id: I94d03f9459ab6444e239386eb96a0c2434bfe3dc
2019-09-03 13:23:11 +00:00
Kosta Harlan 984e06639d Move non-unit tests back into default (integration) directory
Follows-up Id8412e2b8a4e873fd4821ecc1a3c95710be9a870.

Change-Id: Ib92cfbb637e0143a5481212f11a6e511929d6801
2019-09-01 19:57:26 +00:00
Daimona Eaytoy d51ca862c6 Move parser tests to /unit
IMHO these can be considered unit tests; they were already fast, but now
they're executed in an instant.
This requires several changes: 1 - delay retrieving messages in
AFPUserVisibleException, to avoid having to deal with i18n whenever we
want to test exceptions; 2 - Use some DI for Parser and Tokenizer.
Equivset-dependend tests are also moved to a new class, thus helping to
fix the AF part of T189560.

Change-Id: If4585bf9bb696857005cf40a0d6985c36ac7e7a8
2019-08-28 16:36:37 +00:00
Daimona Eaytoy d515af0ae6 Add a new class for the CachingParser's AST
This allows a little bit more of abstraction: we can store other data in the
tree, without having to store it in a specific node (e.g. the variables map,
which is still unused). It also adds a few typehints, and specializes
the return value of eval'ing the AST: previously, it was the one of
evalNode, which wasn't guaranteed to be an AFPData. Now we have this
guarantee. Last but not least, we can now measure runtime metrics for
evalTree, which doesn't recurse.
Bonus: fix a check in the old parser, which used the wrong variable when
reporting outofbounds errors.

Change-Id: Iff806793b1d968e9bb6220f1459f3d0ac587c7da
2019-08-25 17:29:16 +00:00
Huji Lee 1ddb65021b Add links to AbuseFilter logs on Special:Undelete
Depends-On: I671a0479e877e6c37606b688064cb9c893717709
Bug: T231055
Change-Id: Iebf832c513c6a4e954db0ba2633dd8ba6f27b412
2019-08-23 14:56:43 +00:00
jenkins-bot 4c4677698b Merge "Rearrange config to provide better experience" 2019-08-15 12:39:59 +00:00
Daimona Eaytoy f71d731b61 Rearrange config to provide better experience
In order to bundle AbuseFilter, we have to provide an optimal
configuration for sysadmins. This patch aims to provide such
configuration, and of course it's open to changes.
Changed values:
*Grant abusefilter-view-private to admins. This is mostly for
redundancy-safety (they already have -modify)
*Grant modify-restricted to admins (see T200032)
*Disable the 'degroup' action. This is really dangerous, not ideal to be
enabled by default.
*Grant the abusefilter-revert right to sysops. This is similar to
assigning the modify-restricted right in its rationale.
*Add abusefilter-log-private to viewrestrictedlogs grant. This right
should go together with abusefilter-view-private. (T226987)
*Remove abusefilter-revert from the rollback grant, as that right is
for reverting AF actions and has nothing to do with rollbacks.

This mostly copies the config of WMF production: degrouping is already
disabled there, and modify-restricted should
be assigned to sysops on WMF wikis as well. The assignment of the
abusefilter-view-private and abusefiliter-revert rights is handled in
I09cb462312c5bbc71ee3a8f0fbd91357908130d4 (parent patch) to keep the
status quo.

NOTE: this patch should be *deployed* together with the config patch to
avoid discontinuities. Thus, please ONLY provide a virtual +2 using a
+1, and I'll coordinate the +2 with a deployer.

Bug: T191740
Bug: T200032
Bug: T226987
Depends-On: I09cb462312c5bbc71ee3a8f0fbd91357908130d4
Change-Id: I58f215b8aa3444602532098a3c9dfd25409f29eb
2019-08-13 09:42:42 +00:00