Commit graph

229 commits

Author SHA1 Message Date
Daimona Eaytoy c368575af0 Create a base interface for watchers
This will ease adding new watchers, for instance to send Echo
notifications (see T179495 and T100892).

For now, this is just boilerplate, and converting EmergencyWatcher to
the new interface.

Change-Id: I18d62aba53471202b709cdb19033b1729c5c25b4
2020-11-20 23:34:20 +01:00
Daimona Eaytoy 9595bd9da5 Introduce a service for saving filters
Change-Id: I6b7d16ad7ea1124989ed67c74413979cfd0275c4
2020-11-20 22:33:21 +01:00
Daimona Eaytoy 210cf29658 Add an interface for exporting/importing filters
The main benefit of having a dedicated interface is that we can easily
change the output format. So we're now using a custom array without
references to the DB schema, thus making the import/export process
completely independent from the schema.

Change-Id: I4c0de41d914baf1e9a0e588bd31f95b3524a424b
2020-11-18 22:06:09 +00:00
Matěj Suchánek e7813fbafb Introduce EmergencyWatcher service
Change-Id: I45477ca84a99f620d182ef95e5627d421d38f077
2020-11-18 14:20:18 +00:00
Daimona Eaytoy ae29451ab8 Introduce a FilterCompare service
The scope is still quite limited, but as noted in a todo, we might want
to make this completely independent from the database, and add the use
case of ViewDiff.

Change-Id: Ie980fff0983b3e86037265e85da04444c809a6e8
2020-11-18 11:52:44 +00:00
Daimona Eaytoy 1bcfdc3b13 Introduce a FilterValidator
This moves a lot of things away from the AbuseFilter class. There's a
nasty static dependency on ChangeTags, but it's very limited anyway, and
it's going to be fixed once T245964 is resolved.

Change-Id: Ia7df4b4d3289c2722323f59ceecf3fdd38277785
2020-11-18 01:41:31 +00:00
Daimona Eaytoy 725ec052ed Add a FilterLookup service
Some pieces of code were updated to use Filter objects, while other
places are still to be updated. We also need to change the history part
to exclude actions somehow, cleanup the ViewEdit, reduce direct DB
access or anything mentioning DB fields outside of FilterLookup, etc.

Change-Id: I42b7ded685db76eddd45e4b1336f9828cba811ce
2020-11-18 01:17:47 +00:00
Daimona Eaytoy 71a61c2089 Add value objects to represent filters
This is just a start; next step is adding a factory/store method to
get/store these objects. And then use these value objects whenever
applicable.

Note: the actions-related code is still not fully implemented. This is
going to happen as part of the FilterLookup.

Change-Id: I5f33227887c035e301313bbe24d1c1fefb75bc6a
2020-11-04 12:56:14 +01:00
Daimona Eaytoy be75cf1c40 Introduce AbuseFilterParserFactory service
TODO For the future: the final directory for Parser-related classes
should be "Parser", not "ParserNS". However, moving all classes now
would make it harder to rebase changes etc.

Change-Id: Ice335f4723e74f4e5fbe8dcc76ff8ea16310962c
2020-10-31 21:19:00 +01:00
Daimona Eaytoy 1f8df50cb3 Add a service to retrieve the central DB
This is a thin wrapper around LBFactory and the global variable, that
can be injected in classes requiring it (no real class right now, but
that's going to change soon).

Also, remove some DWIM-style returns which made the code harder to
understand.

Change-Id: I1d28ad4a67f914103f3a17cda5f61b28070c7f1c
2020-10-31 12:32:46 +00:00
Daimona Eaytoy 04451d7bde Little cleanup for AbuseFilterRunner
Remove outdated/pointless comments, use already defined variables, etc.

Additionally, make it possible to disable throttling locally.

Change-Id: I98fd5f3eb47b32fc1013360e462a57d932174a95
2020-10-30 21:42:54 +00:00
Daimona Eaytoy cbea88f818 Add a service to retrieve the filter user
Unfortunately, this isn't using DI completely, because of the
User::newSystemUser call. I'm not even sure if we really need to call it
or we can just stick to new UserIdentityValue, but leaving like this for
now.
Also, the types were weakened to UserIdentity, so the transition is
going to be easy anyway.

Change-Id: I08f8fae0fcc622ff0ac3f86771476d06d1c18549
2020-10-26 14:06:53 +01:00
Matěj Suchánek 1445d5962a Introduce BlockAutopromoteStore service
This service is responsible for the blockautopromote feature:
(un)block autopromotion and check status.

The patch mostly moves code from static methods to the new class
and relaxes type hints (e.g. from User to UserIdentity).

Change-Id: I79a72377881cf06717931cd09af12f3b8e5f3e3f
2020-10-24 12:31:44 +00:00
Daimona Eaytoy b309c804fc Add dedicated classes for more hook handlers
The schema changes hook was chosen because the handler is very long. The
test ones were chosen to keep test things away from actual code.

Bug: T261067
Change-Id: Ie06bf62399f6353e3e268cccb3fe4b41bbf951c5
2020-10-22 18:23:09 +02:00
Daimona Eaytoy 4c06dd52c8 Replace $wgAbuseFilterRestrictions with more specific variables
So that sysadmins can further customize the extension. It was also wrong
to use the same variable for many different things.

Note that there's no associated patch in wmf-config because we use the
defaults. However, before merging this patch, please recheck that
AbuseFilterRestrictions and AbuseFilterDisallowGlobalLocalBlocks aren't
used there (https://codesearch.wmflabs.org/operations/?q=AbuseFilterDisallowGlobalLocalBlocks%7CAbuseFilterRestrictions&i=nope&files=&repos=)

Bug: T175221
Change-Id: I7581b3ee6d9d11a6cf1599b8ff874e8c3d54adf4
2020-10-22 13:38:59 +00:00
Matěj Suchánek 2ee3a0d247 Migrate change tags hooks to DI
Bug: T261067
Change-Id: I7b95cd19ab0ae04820e8dcb3481d29a2f9e7a0ca
2020-10-21 16:18:06 +00:00
Matěj Suchánek 85e000c6ed Add ChangeTagsManager service
This service will be resposnsible for loading
and caching change tags used by abuse filters.

Change-Id: I9a710af1dd1ae58c47de1e8509246ed929d0a662
2020-10-21 16:24:32 +02:00
Daimona Eaytoy 9bc885b6b3 Add a ChangeTagger class
The logic about action IDs and the persistent buffer is now encapsulated
inside a single service, which is a step towards getting rid of global
state in the AbuseFilter class, and reducing the responsibilities of the
Runner.

An important change made here is that we now require a LinkTarget rather
than a Title. This removes a dependency on the Title class (a monster
object), makes tests simpler, and denies the need to inject a
TitleFactory. This means living without some bits of context (e.g. we're
no longer using makeTitleSafe to ensure a valid title, and we have to
build a "prefixedtext" manually), but this shouldn't be a problem, given
that the titles are only used to create a cache key: invalid titles are
not a problem, and concatenating namespace + title should always be
sufficient.

Bug: T265370
Change-Id: Iff59cd3d889454a482a89c16691bfefcc5ec0a12
2020-10-21 13:19:30 +02:00
Daimona Eaytoy 1efc324d97 Use new services in AbuseFilterRunner
The first one is UserGroupManager, used for the 'degroup' action. This
is a simple one-line replacement (repeated twice), and the current code
was already using this service under the hood.

The second one is BlockUser, which is not a one-line change (but still
quite simple). In particular, this allows us to avoid duplication with
core logic when constructing the log entry (this is now done by
BlockUser).

Bug: T248743
Change-Id: Ib7c1dc107a169b575f7021e64b6a8fee09529548
2020-10-14 23:08:32 +00:00
Matěj Suchánek 7ef2259228 Migrate a few hook handlers to DI
Bug: T261067
Change-Id: If699917c3d2e9e22525c7d0495554e25f6b45125
2020-10-10 17:23:04 +00:00
Daimona Eaytoy 2026e3ac3a Add an AbuseFilterPermissionManager service
This service should act as a mediator between the AF code and the
permission manager, and it should know what are the permissions required
by each action.

Change-Id: Ieb177d9992147b11fa7b8f05929da6c182cc2286
2020-10-10 14:03:29 +02:00
jenkins-bot c0defc1055 Merge "Add a new FilterProfiler service" 2020-10-10 10:08:58 +00:00
Matěj Suchánek 9e6bc2f4ee Move log formatters to a separate directory and namespace
This will clean up the includes/ directory a little.

Change-Id: I61adacf32257bb2402a272b60b52b69505d981c5
2020-10-07 16:25:38 +02:00
Daimona Eaytoy bc9898f1a1 Add a new FilterProfiler service
Change-Id: Ib66c42ac220731f4e1da9ee6cfb5290759dd6494
2020-10-04 22:00:57 +00:00
Daimona Eaytoy 6c8a29698b Add test traits for uploads and account creation
Ideally, this might live in MediaWikiIntegrationTestCase. For the
createaccount one, AuthManager should also provide a method to log the
creation, because currently we are forced to copypaste that code here.

 - Add the missing tests for 'upload' in RCVariableGenerator, and adjust
the existing ones (delete file afterwards, more tablesUsed, use the
right extension).

 - Exclude from the coverage report a couple of lines which should
theoretically be unreachable. Escalate logging to WARN level, where it's
more likely to be spotted.

 - Remove an unused method (RCVariableGenerator::newFromID). This denies
   the need to maintain and cover it. We also don't want this generator
   to act as a factory.

Overall, this change brings the coverage for RCVariableGenerator to 100%

Bug: T201193
Change-Id: I425c3d9f6800f74eb6e4eda483b90cfb3bbbcb51
2020-10-04 13:16:58 +00:00
Matěj Suchánek 65708afcea Integrate with Renameuser
Register abuse_filter and abuse_filter_history tables.
abuse_filter_log is more difficult (if possible).

Bug: T27377
Bug: T206477
Change-Id: If8289101a08887519d5a90ef84700421b8ed2406
2020-10-01 08:10:22 +00:00
Daimona Eaytoy 55ba083b13 Introduce a KeywordsManager service
This will decouple a bit the huge and chaotic tangle of AF classes. Some
boilerplate code for AbuseFilter services is also added with this patch.

Note that this requires injecting a KeywordsManager in
AbuseFilterVariableHolder, or unit tests would fail. This is still
incomplete, and the Manager is only injected in tests, because
VariableHolder still has to be refactored.

The test for the UpdateVarDumps script had to be updated, because
serializing VHs in there was a bad choice. As pointed out in a comment,
the test is likely going to break again once we remove the BC code, but
I hope that we'll be able to remove the test at that point.

Change-Id: I12a656a310adb8c5f75cab63f6db9e121e109717
2020-09-28 23:03:52 +00:00
Matěj Suchánek f1ecdd4aff Inject PermissionManager to SpecialAbuseLog
Change-Id: I1c80490567ac2d9f716c988ebdad6b59cf28aa06
2020-09-18 23:22:11 +00:00
jenkins-bot 3f8e61b42f Merge "Allow Blockautopromote duration to be configured for wikis." 2020-09-17 17:53:06 +00:00
DannyS712 bf74fd0c23 Allow Blockautopromote duration to be configured for wikis.
Rather than always using 5 days, the length (in days) can be configured by setting
`AbuseFilterBlockAutopromoteDuration` to the desired length.

Bug: T231756
Change-Id: I996e08a9099ab59657fe511ec2934d26edfa5c7b
2020-09-17 17:19:00 +00:00
DannyS712 9c1868d55e Update hook calling to use new HookContainer
Bug: T254306
Change-Id: Ic5c82a367e34135bbc0f00ece5aeef4f2d92881b
2020-09-17 10:05:45 +00:00
Umherirrender f932ba8328 Use LinkBatchFactory in Special:AbuseLog
Change-Id: I2ccf9cd36475a65e61ad0e80ec159f841849089f
2020-09-06 09:31:49 +00:00
Kunal Mehta ad5889ea83 Remove incorrect merge_strategy definitions
Those merge strategies aren't needed because
array_plus is for when the keys of the array are integers, not necessarily
the values.

Change-Id: I164723c4cc6b12cd6e4ce0d49c74d93eda5d871d
2020-07-06 09:00:35 +00:00
DannyS712 4b35336638 Update hooks to use PageSaveComplete
Extension requires MW 1.35+, always available

Bug: T250566
Change-Id: I60cf3cc42db989d8ccb0d06d3cf9eae8a85784ac
2020-06-16 04:18:39 +00:00
Reedy e7a02ddcd0 Replace _merge_strategy with merge_strategy
Change-Id: I313cedec3297725b7711feee20f17621f91561a6
2020-05-12 20:18:16 +01:00
jenkins-bot b118fd50dc Merge "Improve var dumping in /details, /examine and /tools" 2020-04-29 20:00:54 +00:00
DannyS712 6c947609bb Remove unneeded dependency on 'mediawiki.notify'
The single line of code it contained is part of mediawiki.base as of
MW 1.35, and the extension only supports MW 1.35+

Bug: T233676
Change-Id: I4c63fefcd96f7d56be1caf6b37d4ba598e8bdb45
2020-04-09 02:51:57 +00:00
Daimona Eaytoy 4c98aecf4d Improve var dumping in /details, /examine and /tools
Using var_export for better visual effect, especially for arrays.
The result from /tools is much clearer and the 'wrong syntax' message is
a bit more explicative than before.

Bug: T190653
Bug: T239972
Change-Id: I79a17305c7f19f7900f896f895e9365bb5f2fd58
2020-03-28 17:35:43 +01:00
Timo Tijhof c7653905ad Migrate 'user.tokens' dependency to 'user.options'
The 'user.tokens' module is a deprecated alias to 'user.options'
since MW 1.35.

Bug: T235457
Change-Id: I45e8e35edf4e6f901c2045d2592326e531607726
2020-03-27 16:22:57 +00:00
Daimona Eaytoy 2c03c77d9f Add a maintenance script to clean afl_var_dump
This script aims to fix every problem reported in T213006. Subsequent
patches will add new code and drop the back-compat one.

Bug: T213006
Bug: T187153
Bug: T204236
Bug: T187731
Bug: T204235
Bug: T214193
Bug: T214196
Bug: T34478
Depends-On: I5b29ff556eca45fe59d15e2e3df4d06f1f6b3934
Change-Id: I22cf698c5be77506727cbd227c67e037a5d89b5c
2020-02-28 19:41:30 +00:00
Daimona Eaytoy 1686042a91 Move variable generators to new classes
RunVariableGenerator is for generating variables based on the current
action;
RowVariableGenerator is for RC entries;
VariableGenerator is the generic one.

This patch only moves the methods to the new classes, to keep the diff
easier to read, and facilitate conflict resolution. These classes will
then be revamped in I366d598b69ad866496b7cb0059e0835c02e54041.

Note that these classes are now namespaced.

One method, AbuseFilter::getEditVars, was renamed to
AbuseFilterVariableGenerator::generateEditVars, because it would
otherwise conflict with an incompatible method in RunVariableGenerator.

Change-Id: Iff412e5492873d4fae55402939a51609e64d55a8
2020-02-07 19:44:31 +00:00
Daimona Eaytoy f382304aae Add a base class for parser transition
Change-Id: I31282b8632c332b6d46a6bb4a42f57ac0d005b5f
2019-12-15 13:29:56 +00:00
James D. Forrester cff9889549 Move test-only classes to new TestAutoloadClasses declaration
Bug: T196090
Depends-On: I88cf69663362fd599c20649b1df641907a02678d
Change-Id: I5e3ef040628c4ff629a8aabc56a3b8ba259b7e78
2019-12-10 12:18:17 -08:00
DannyS712 a137992900 Restrict viewing Special:Log/AbuseFilter, and remove from recent changes
Only allow users with `abusefilter-view` to see abuse filter log
entries. Note that this is different from the abuse log itself.

Bug: T34959
Change-Id: I587c2eb9bdc2e5ca2652aa338440cdc5b6eb9235
2019-11-10 10:41:17 +00:00
Daimona Eaytoy ce8539e2a5 Move parser tests back to /unit
Using `new LanguageEn()` involved a global, so use a MockObject instead.
Also fix LoggerFactory usage in Tokenizer to use DI instead.

Change-Id: I94d03f9459ab6444e239386eb96a0c2434bfe3dc
2019-09-03 13:23:11 +00:00
Kosta Harlan 984e06639d Move non-unit tests back into default (integration) directory
Follows-up Id8412e2b8a4e873fd4821ecc1a3c95710be9a870.

Change-Id: Ib92cfbb637e0143a5481212f11a6e511929d6801
2019-09-01 19:57:26 +00:00
Daimona Eaytoy d51ca862c6 Move parser tests to /unit
IMHO these can be considered unit tests; they were already fast, but now
they're executed in an instant.
This requires several changes: 1 - delay retrieving messages in
AFPUserVisibleException, to avoid having to deal with i18n whenever we
want to test exceptions; 2 - Use some DI for Parser and Tokenizer.
Equivset-dependend tests are also moved to a new class, thus helping to
fix the AF part of T189560.

Change-Id: If4585bf9bb696857005cf40a0d6985c36ac7e7a8
2019-08-28 16:36:37 +00:00
Daimona Eaytoy d515af0ae6 Add a new class for the CachingParser's AST
This allows a little bit more of abstraction: we can store other data in the
tree, without having to store it in a specific node (e.g. the variables map,
which is still unused). It also adds a few typehints, and specializes
the return value of eval'ing the AST: previously, it was the one of
evalNode, which wasn't guaranteed to be an AFPData. Now we have this
guarantee. Last but not least, we can now measure runtime metrics for
evalTree, which doesn't recurse.
Bonus: fix a check in the old parser, which used the wrong variable when
reporting outofbounds errors.

Change-Id: Iff806793b1d968e9bb6220f1459f3d0ac587c7da
2019-08-25 17:29:16 +00:00
Huji Lee 1ddb65021b Add links to AbuseFilter logs on Special:Undelete
Depends-On: I671a0479e877e6c37606b688064cb9c893717709
Bug: T231055
Change-Id: Iebf832c513c6a4e954db0ba2633dd8ba6f27b412
2019-08-23 14:56:43 +00:00
jenkins-bot 4c4677698b Merge "Rearrange config to provide better experience" 2019-08-15 12:39:59 +00:00
Daimona Eaytoy f71d731b61 Rearrange config to provide better experience
In order to bundle AbuseFilter, we have to provide an optimal
configuration for sysadmins. This patch aims to provide such
configuration, and of course it's open to changes.
Changed values:
*Grant abusefilter-view-private to admins. This is mostly for
redundancy-safety (they already have -modify)
*Grant modify-restricted to admins (see T200032)
*Disable the 'degroup' action. This is really dangerous, not ideal to be
enabled by default.
*Grant the abusefilter-revert right to sysops. This is similar to
assigning the modify-restricted right in its rationale.
*Add abusefilter-log-private to viewrestrictedlogs grant. This right
should go together with abusefilter-view-private. (T226987)
*Remove abusefilter-revert from the rollback grant, as that right is
for reverting AF actions and has nothing to do with rollbacks.

This mostly copies the config of WMF production: degrouping is already
disabled there, and modify-restricted should
be assigned to sysops on WMF wikis as well. The assignment of the
abusefilter-view-private and abusefiliter-revert rights is handled in
I09cb462312c5bbc71ee3a8f0fbd91357908130d4 (parent patch) to keep the
status quo.

NOTE: this patch should be *deployed* together with the config patch to
avoid discontinuities. Thus, please ONLY provide a virtual +2 using a
+1, and I'll coordinate the +2 with a deployer.

Bug: T191740
Bug: T200032
Bug: T226987
Depends-On: I09cb462312c5bbc71ee3a8f0fbd91357908130d4
Change-Id: I58f215b8aa3444602532098a3c9dfd25409f29eb
2019-08-13 09:42:42 +00:00
Daimona Eaytoy c34181e548 Add a new API module to retrieve private details from AbuseLog
Follow-up of Iaca492371f48fecf543268c179a651841ed12c3f. This patch adds
the new module, plus some technical changes to private details-related
methods and globals.

Bug: T210329
Depends-On: I613dbadb8f75c8c4116a362607563a436a73d321
Change-Id: I3c45b74c36c191083df184ed57416067a75f6591
2019-08-09 21:10:28 +00:00
Daimona Eaytoy c7ccb68058 Use "privatedetails" instead of "private" where needed
To keep a clear distinction between "private details" (i.e. user's ip)
and "private filters" (i.e. not publicly viewable). This patch renames
rights, i18n keys and methods names.
The patch for renaming globals and rights in WMF config is
I7e6b3d4453403edb6aa602587374b4ff5b6d625f.

Bug: T211004
Change-Id: I613dbadb8f75c8c4116a362607563a436a73d321
2019-08-09 21:10:22 +00:00
jenkins-bot 1fa5eef94c Merge "Overhaul Blockautopromote action" 2019-08-07 23:03:08 +00:00
Daimona Eaytoy 483dab1732 Add a parent class for special pages
This allows us to extract yet another static method from the AbuseFilter
class. This class should be expanded in the future, and an example use
case could be Ia5fd4f0b35fcabf045a7b49fa40fa85b72c92544.

Depends-On: I7c0170167b508132cd16e566c654a6c98dd683e9
Change-Id: I1bb45e47c3b42c01388b99778ce833e4e44419e1
2019-08-06 14:17:38 +00:00
Daimona Eaytoy 2bdb44d58b Overhaul Blockautopromote action
As for all mostly unused consequences, blockautopromote has a couple of
major problems: first, it blocked the status for a random time between 3
and 7 days, which to me makes no sense at all (is it some sort of
casino?), and this patch fixes it to 5 days. Second, nothing was logged,
not the blocking nor the unblocking. Here I'm adding a LogHandler for
two new sub-actions of 'rights' to keep track of both action.

Bug: T49412
Change-Id: If48a48f5b8baaf9e77c0826466f5d03bb7f691d0
2019-08-05 22:27:49 -04:00
Daimona Eaytoy 4720c97530 Add a new class for methods related to running filters
Currently we strongly abuse (pardon the pun) the AbuseFilter class: its
purpose should be to hold static functions intended as generic utility
functions (e.g. to format messages, determine whether a filter is global
etc.), but we actually use it for all methods related to running filters.
This patch creates a new class, AbuseFilterRunner, containing all such
methods, which have been made non-static. This leads to several
improvements (also for related methods and the parser), and opens the
way to further improve the code.
Aside from making the code prettier, less global and easier to test,
this patch could also produce a performance improvement, although I
don't have tools to measure that.
Also note that many public methods have been removed, and almost any of
them has been made protected; a couple of them (the ones used from outside)
are left for back-compat, and will be removed in the future.

Change-Id: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
2019-07-23 19:06:27 +00:00
Daimona Eaytoy 07e5168eb9 Add a maintenance script to fix broken log entries
A very simple fix for some old entries imploded with '\n' instead of
"\n". As for other scripts, this extends LoggedUpdateMaintenance but it
includes a dry run option to be tested, and a subsequent patch (to be
merged after testing) will add it to update.php.

Bug: T208931
Change-Id: I16d629c05c3b9c9055c68dafd261b0a5c0db9c25
2019-07-15 11:59:45 +00:00
MarcoAurelio 253a913746 Add Daimona Eaytoy to authors
Daimona has been an extraordinary maintainer of this tool and has also
implemented new features on Abuse Filter, so I guess it is just and fair
to include his name to this list.

Hyperlink is because `Daimona Eaytoy` is not a real name, according to
his mediawiki.org userpage (can be removed if required; the law accepts
authorship attributed to pseudonyms).

In addition, the list of authors has been sorted alphabetically by name.
If you prefer to order it by family name instead, please let me know.

Change-Id: I6a3b0a757f9fd2af09358298e55a8b0487ce5b32
2019-07-04 10:30:14 +00:00
Thalia 22ceae7e23 Use MediaWiki\Block\DatabaseBlock instead of Block
This follows the rename of the Block class in I6d96b63ca0.

Change-Id: I44cf9eb68c23a8299316effa4dee7f732486dd84
2019-05-31 16:08:19 +01:00
Daimona Eaytoy 39fc7c12af Restore unit tests for CachingParser and fix it
Added cachingParser back to *all* the parser tests, fixed a couple of
differences with the normal parser, and added a couple of tests so that
any cachingParser-related file has 100% coverage. Also move the remaining
get_matches tests inside parserTests, and specify the parser used in case of failure.
This also adds a new base class for parser-related tests with a couple
of util methods.

Bug: T201193
Change-Id: I980aec3481a52ecc35f1811a366014a5581a7cdb
2019-05-25 10:55:24 +02:00
Daimona Eaytoy f56562f583 Add tests for global filters
Another crucial part to have covered. Also clarify that
AbuseFilterCentralDB can be of the form "dbname-prefix".

Remove a filter used for profiling and replace it with a global one:
we're still fine, and the list is kept shorter.

Bug: T201193
Depends-On: I5ee7ba44a6cd82a5ddb24fb4127af04d96e647f4
Change-Id: If6b91711534c0d60e1aa27bd5748c3023e29f376
2019-05-24 16:58:23 +02:00
Daimona Eaytoy 72c2be7a18 Remove $wgAbuseFilterRuntimeProfiling
The reasoning is similar to the one of the parent patch (Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb). Plus, it records runtime metrics on action different than edits, as there's no reason not to do it.
No performance issues in production.

Bug: T191039
Depends-On: Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb
Change-Id: Ib1112e2fefd0631550d386ba87e5f87db84c3036
2019-03-23 11:31:18 +00:00
Daimona Eaytoy 89520e2353 Remove $wgAbuseFilterProfiling
This variable was introduced to selectively enable profiling because
stats recording was bad for performance. Nowadays, stats are recorded in
a deferredupdate and don't harm performance anymore. Thus, this variable
can be removed and profiling be enabled by default.

Bug: T191039
Depends-On: Ib5fdeb75c1324f672b4ded39681f006fde34b4d1
Change-Id: Ia5c477edc8733bb1994cb6d01e1371ed496c8bcb
2019-03-23 11:31:11 +00:00
jenkins-bot de4e971c72 Merge "Remove usage of MakeGlobalVariablesScript hook" 2019-03-17 10:32:30 +00:00
Daimona Eaytoy 0df04cd829 Fix a typo in EmergencyDisable globals description
And specify what we mean with "dangerous actions".

Change-Id: Ida5827978b257cfdf61786c7e19c11a1dc74c664
2019-02-16 14:43:29 +01:00
Daimona Eaytoy fca80fa976 Remove usage of MakeGlobalVariablesScript hook
This is an old leftover, used to add global JS variables in a convoluted
way: using a hook and a total of 3 static properties. We can safely
remove all of this and just call OutputPage::addJsConfigVars, which BTW
is already called where we need it.

Change-Id: Ifad0618fa93b0c7a7e8b23f596234e622aa8846a
2019-01-21 14:27:57 +01:00
jenkins-bot b35ba5af45 Merge "Warn the user if they try to leave the page with unsaved changes" 2019-01-19 12:57:50 +00:00
Daimona Eaytoy 6217ffb928 Remove unused stuff
Variables declared but never used, redundant code, and old leftovers.

Change-Id: Ic51044a45a1b49ad6c7af06c646b11893411a7cd
2019-01-18 17:04:19 +01:00
Bartosz Dziewoński 4610d93c7a Add mobile target to VE plugin modules
Bug: T213774
Change-Id: Idb4dd58b441d24fae7012b79fc049fbd02029c4c
2019-01-15 01:05:20 +01:00
Daimona Eaytoy 206bdc1f6a Use the updated TitleMove hook to filter move actions
For several reasons:
*We're not really checking permissions (and the hook previously used is
meant to be used in such case)
*We'll show a cleaner error message (i.e. without the "You do not have
permission..." part)
*Filtering will happen closer to the actual move

Bug: T208907
Depends-On: I4733724075b7514e9db59e7be772d9409aa9da87
Change-Id: If88f736a446247f8b4b13c055c641d56f544d1ea
2018-12-04 18:58:04 +01:00
Daimona Eaytoy 38749b46bb Warn the user if they try to leave the page with unsaved changes
While editing filters, sometimes it happen that you make some change,
forget about it and then reload/close the page, and no warnings will be
issued. This patch makes use of the core module used for normal page
editing to display a warning if trying to leave a filter editing page
with any unsaved change (both to the filter pattern or other form
elements).

Change-Id: I78d79215565d5c82028b1a2a4276497ccbffdea2
2018-12-04 13:06:46 +01:00
jenkins-bot dff5cd89f9 Merge "Further clarify docs for emergency disable" 2018-12-01 01:32:31 +00:00
Ed Sanders 687106d8af Bring in VE support from VE extension
Change-Id: Ib1354f0404209a15194895026ff9d179d16b1900
2018-11-30 10:59:16 +00:00
Daimona Eaytoy 6aff37fb52 Further clarify docs for emergency disable
This is a follow-up to Ic3bc6e36506973b19a9b1bcecbc1a5080faed2ec. I
believe it's important to specify how many recent actions we're looking
at, and I also think it's not nice to rely on a variable amount of
actions to determine whether a filter should be throttled. Also, require
a $group parameter in filterUsedKey (we always pass one, and there's no
reason not to).

Change-Id: I0384d3f1913ead593f605248950606c81c8f8542
2018-11-28 19:29:15 +01:00
Daimona Eaytoy 1f2b7474ed Clarify code and docs for automatic throttling
For the docs part: make it clear how things work there. For the code
part, these are mostly style changes: shorter variable names, no
unnecessary parameters, make the method private, use clearer variable
names.

Change-Id: Ic3bc6e36506973b19a9b1bcecbc1a5080faed2ec
2018-11-26 16:51:10 +01:00
jenkins-bot 6541d7c5cc Merge "Check that the user block is sitewide when determining permissions" 2018-11-15 17:26:21 +00:00
Daimona Eaytoy 346063eec0 Check that the user block is sitewide when determining permissions
And bump MediaWiki version.

Bug: T208621
Change-Id: Icfcf09c5d7c7498711cb000c3bb16480270efb9c
2018-11-15 17:59:22 +01:00
Daimona Eaytoy d3a8491c3f Change throttle selector to restore old functionality, overall improvement
Long (sigh) explanation in T203587#4569698. Also, simplified the way
TagMultiselect are generated, this one and the one for change tags.
This new selector is back-compat both with the old textarea and the OOUI
checkboxMultiselect; actually, this one is //fully// compatible with the
old textarea.
Add validation for throttle parameters and unit tests for validation
(split from I976c95658cddb2585910b6f8a5f047aadc4e4d47).
Added a trim when retrieving throttle identifier to allow syntax like
'ip, user'.
Improved the message shown on history.
Re-added the maintenance script to clean DB.

As I wrote in the task, a review by two other people would be great, at
least for the maintenance script (it could potentially break the DB).

Bug: T203587
Bug: T203336
Bug: T203584
Bug: T203585
Depends-On: I3b2e763bd8835207dc5df1db43d3e1881e6961c3
Change-Id: I7831dbb0bab55807392ac1f7915d6cb0cb713593
2018-11-14 12:51:36 +01:00
se4598 9d12e1b353 Allow selecting custom disallow message
You can now select a custom message to be displayed for disallowing a edit
the same way as for warn mode. This can be the same or a totally different
message.

This also solves the usecase, when a edit filter is set to warn AND disallow,
to be able to show the user a custom message, but the generic is shown
on the second try (disallow). Now it can be only set to disallow.

Bug: T27086
Change-Id: Ic1de03a6944c43a346fa317ee0a217551f0d284a
2018-10-11 10:35:01 +02:00
Jayprakash12345 8ad2203ce0 Move all apihelp translatable messages to separate files
Should be merged along with
Ib66f4614285411e5fd971dc0cb4291ea8113418c

Bug: T189982
Change-Id: Id9beb08ee34e6861543cf59d80abfac616bf9238
2018-09-21 08:21:38 +00:00
Matěj Suchánek 10ad58a6f3 Migrate AbuseFilter suppress log
Also make entries in Special:Log/suppress filterable.

Change-Id: Ic23e724997e4748c8d0da8138aa73d31b17b7064
2018-08-21 16:05:54 +00:00
tinajohnson.1234 c9003fe1fa Use HistoryPageToolLinks hook to add a log link to history pages
Add an AbuseFilter log link to the subtitle of history pages.

Bug: T28934
Co-authored-by: Matěj Suchánek
Depends-On: I2e0e9e92d3fc303135b0eb9acf06b5fd120178a5
Depends-On: I58a3039b3755648bb0c8aaf87db48ace96ce9344
Change-Id: Ib89c48f2b8f3121ead184844844acee436e2fdd6
2018-07-27 11:25:12 +00:00
Daimona Eaytoy b825e396b5 Switch editing interface to OOUI & improve NoJS usability
This settles almost everything, leaving the tags part ready to be
further improved in the follow-ups.
Also, replaced some fields with totally different ones, improved the
warn preview area and improved a bit nojs experience by hiding unusable
buttons.

Bug: T132284
Bug: T154749
Change-Id: I7a5caa862a32f9792140c6a4d9708a2d20472672
2018-07-17 14:49:50 +02:00
Daimona Eaytoy dd0569c00b Fix extension doc for AbuseFilterPrivateLog
This variable is actually here to determine whether accessing private
info should be logged, not if it's possible to access such info. In
fact, you can still do it if this variable is set to false.

Change-Id: Ieba191bff4ea56eb166bbbd03cb7b1a05ca42c52
2018-07-16 15:32:38 +02:00
jenkins-bot b86208d381 Merge "Add phan seccheck to composer and remove deprecated module" 2018-07-05 17:34:38 +00:00
Daimona Eaytoy 3c1dae9e14 Allow users with abusefilter-view-private to use testing interface
Now the required need will be abusefilter-modify OR
abusefilter-view-private for /tools, /test and /examine.

Bug: T193903
Change-Id: I3f1a91a2cc1df2272e5d4099cefd7c649a0683d5
2018-06-24 14:10:38 +00:00
Daimona Eaytoy 29492513cf Add phan seccheck to composer and remove deprecated module
Seccheck is currently being executed as non-voting, but it always
succeeds because AbuseFilter is not requiring it in composer. Instead,
let's add it: tests will start to fail, but we'll be able to see how
things are going with it. As a bonus, remove the mediawiki.api.parse
module, which is deprecated in 1.32 and replaced by mediawiki.api
(already loaded).

Change-Id: I094a8af4f97c03f8b538ede00420b123de25138a
2018-06-12 15:37:03 +02:00
MarcoAurelio 5724ca14d0 add grant to view private abusefilter log entries
Bug: T191703
Change-Id: I56233786ddaa6c44e6778b56070cf9b13df20dcc
2018-04-11 08:59:36 +00:00
Gergő Tisza ebcc751e7c
Remove abusefilter-revert right from sysops
No point in that right without abusefilter-modify-restricted and
it makes the user interface more confusing.
Undoes part of I4e3125a.

Change-Id: I4afaaa98a5c1b3d0d9518117a28e7e46466f87a1
2018-04-09 19:25:45 +02:00
Gergő Tisza 8f53c11a85
Add default rights config
Add a conservative default configuration so that admins can use
abuse filters without any need for manual setup, and users can
see what's happening. Also expand grants a bit.

Bug: T191740
Change-Id: I4e3125a708277474f416903928397db7f8fb850d
2018-04-09 10:47:35 +02:00
Daimona Eaytoy 55cac6f1b0 Move actions limit to a global variable
This opens the door to further customization and allows every wiki to
set its own value.

Bug: T132925
Change-Id: I63985f2809c3253b07b33caef30fcd8d4c62dfd4
2018-04-05 00:06:40 +00:00
Max Semenik 5c89246fce Rename files to match class name
Change-Id: Ia19bfec6c2289912699b6c90261afda311afb56e
2018-04-02 22:08:13 -04:00
Daimona Eaytoy 2779c9cd3c Move documentation to extension.json
This way we'll be able to go on with
I0e95e9b15e3733d85bdfac2a4d27a4d5ba7db928 and dependencies.

Change-Id: I16aa7c839a71ecae0554ef43fa12423e76118fe0
2018-04-02 17:59:48 +02:00
Daimona Eaytoy 3350183fe3 Revert "Revert "Switch editor to Ace and provide syntax highlight""
Make Ace use a fixed size in em.

This reverts commit 272775ff81.

Change-Id: I9b439b20df91eb367bcef4b6f33ff087aded0b62
2018-03-30 11:10:16 -04:00
Huji 272775ff81 Revert "Switch editor to Ace and provide syntax highlight"
This reverts commit 89e6778793.

Change-Id: I41aee10fdd5633d56692334696fb750f41b15433
2018-03-30 02:07:00 +00:00
Daimona Eaytoy 89e6778793 Switch editor to Ace and provide syntax highlight
Replace the conditions textarea with Ace editor for editing and testing
filter. This uses a soft dependency on CodeEditor; if the latter isn't
installed, the classic textarea is used. The user is still able to
switch between the editors on the go; the new buttons may look a bit
ugly now, but after switching to OOUI they should get much better.
Finally, added a custom syntax highlight for AbuseFilter rules.

Bug: T39192
Change-Id: If3d6a994142e34686bb7fc9f09093f751b599485
2018-03-23 12:39:22 +01:00
Dan Mattern fd3987baee Add option to turn off logging IPs
Adds option to stop logging IPs in afl_ip in the abuse_filter_log table.
Introduces a new global variable: $wgAbuseFilterLogIP

Bug: T187169
Change-Id: I1615ba6949c9f8bcdd6ee6aef580c87a05f43e6a
2018-03-03 23:16:24 +00:00
Huji Lee 146820185c Log accessing private information in abuse filter logs
Bug: T152934
Change-Id: I8049df3b2b9343a6877e9a306d2781d3f27ec657
2018-02-07 18:35:36 +00:00
Jayprakash12345 9ec6c98eb0 Removed deprecated position statements from resource loader module
Bug: T184257
Change-Id: Iadfd2cb039042ca4bcce4926bb4d4508eb9b23ca
2018-02-02 16:33:22 +00:00
MarcoAurelio 59ff4aed5d Actually mark abusefilter creations as such in the AbuseFilter log
Bug: T178283
Co-Authored-By: Matěj Suchánek <matejsuchanek97@gmail.com>
Change-Id: I89776eff2fc420b1fe3e2c3b88671749fadb57b4
2018-01-12 17:03:13 +00:00
Kunal Mehta e5b952449d Use SPDX 3.0 license identifier
SPDX released version 3 of their license list (<https://spdx.org/licenses/>),
which changed the FSF licenses to explicitly end in -only or -or-later
instead of relying on an easy to miss + symbol.

Bug: T183858
Change-Id: Ib7a48d3a432f12ce1c27fe989e85a80335a25801
2018-01-03 19:01:35 -08:00
Umherirrender 80418b1f93 Move classes to own files
Makes MediaWiki.Files.OneClassPerFile.MultipleFound pass

Change-Id: I3b08a69fe7990d6fe5f71cda51d6ac01f11aad2d
2017-12-23 13:43:37 +00:00
Huji Lee 7b7be07957 AbuseFilter block range should not exceed $wgBlockCIDRLimit
This patch introduces a config variable for the range block sizes.
It changes the default IPv6 block size from /16 to /19 using the
same reasoning as  Ia25e156fd8234519c4d74f1d41d93f94a313ce14

Using a config var (as opposed to hardcoded range size) allows
future changes proposed in T179454 to make the range size vary
for different IPs, based on the actual subnet they belong to.

Bug: T179455
Bug: T179456
Change-Id: I8dfa17f553a7af524f0a11c0fd51c48773e27be5
2017-11-02 12:08:49 +00:00
jenkins-bot 689b7abaff Merge "Remove back-compat code paths" 2017-10-31 16:12:40 +00:00
Brad Jorsch 6071e7a43f Remove back-compat code paths
Per T178092, AbuseFilter now maintains compatibility with older versions
of MediaWiki using release branches. Thus, various back-compat code
paths may be removed from the master branch.

Change-Id: Ia1b5eade30d7486e3b1b386b15a7db4e5c8cfead
2017-10-31 09:37:54 -04:00
Dayllan Maza 3e1c5b9099 Add slow filters debug data to the logs.
When $wgAbuseFilterRuntimeProfile is true, all filters taking
longer than $wgAbuseFilterRuntimeLimit will be logged for
later analysis

Bug: T174205
Change-Id: Id81833afa8421476a6cee47eb3393acdb3a38d65
2017-10-12 17:41:49 -04:00
Dayllan Maza c07294cc9c Add runtime metrics to statsd
Metrics per edit:
    - Execution time of all filters
    - Number of filters executed
    - Number of conditions executed

Due to the current structure of abuse filter there was not
a clean way to include filter actions and abuselog creation
as part of the runtime metrics.

Bug: T161059
Change-Id: I6208b620453863133c6623aa419775f63c7d3eb1
2017-08-24 15:58:52 -04:00
matejsuchanek 3b29498f21 Migrate abusefilter/modify log
Bug: T32553
Change-Id: I7e33d1064329124755c77ffe6efbd5d572f43cb9
2017-08-12 10:00:42 +02:00
Matěj Suchánek 55c27a8f6b Require MediaWiki 1.29
After I544cdfa75c7472f2d98b2561bc6f6f9c2d2ad639 (dieWithError
and checkUserRightsAny), this is the oldest MediaWiki version
AbuseFilter can be run on.

AbortMove was removed from MediaWiki in 1.25, UploadVerifyFile
is only relevant for 1.27 and older.

(Replaces I1e962217c3b20d901a5742cf76339a3f488a6e97.)

Change-Id: Iec237b2887f72b115fdcef78d2d7a944ba82c784
2017-08-10 11:01:34 +02:00
Max Semenik 2f250127b4 Normalize file layout
Aka move all code into includes/.

Change-Id: I21f7b80bb6df04abbed6bfccb94f92100dc8f071
2017-08-07 16:11:38 -07:00
Matěj Suchánek eb8541eab5 Use mw.Api.parse() for previewing warnings
And add the filter id as the second parameter to the message.

Bug: T68351
Change-Id: Iee8ac20d3cc9c0f6340ec6f339b0e51d7c88f976
2017-05-31 09:06:31 +02:00
Huji Lee 8f53126853 Remove the "flag the edit in the abuse log" checkbox
Bug: T154091
Change-Id: I40c3176127bb168672b376147bffcdbd2aaff237
2017-02-13 14:27:44 -05:00
Aaron Schulz 2d57141600 Move AbuseFilterVariableHolder up to /includes
This class is not for parsing logic or tree nodes

Change-Id: I07a499cc972c30fc249ec4de3250900a3b703443
2016-12-18 17:13:06 -08:00
Aaron Schulz 9b1021b055 Move various classes to their own files
Change-Id: I5d418b3fa27aa6e04b9a680922e5eab2439ffb20
2016-12-17 11:40:10 -08:00
Paladox 2beb56348c Replace ArticleSaveComplete hook usage
Bug: T147390
Change-Id: Id0fb48d867d7a7542f589deed90bae8195daa556
2016-10-09 17:27:54 +00:00
Victor Vasiliev aa399da279 Implement a tree-caching abuse filter parser
This filter is fully functional.  The old filter is still enabled by
default for a transitional period in case the new one suddenly has
issues.

Change-Id: I4aea5f00c62420108030e60e79d5bf34e913e95d
2016-09-24 02:53:26 +00:00
MusikAnimal d76de81f66 Add basic AbuseFilter reading rights for OAuth
Bug: T126756
Change-Id: Iace64d3b8c2a8ba1eaa5b49efd8362de5d36d3eb
2016-09-02 22:58:17 +00:00
jenkins-bot 6bd39475bb Merge "Only run filters once for direct uploads (without stash)" 2016-08-11 04:44:52 +00:00
Kunal Mehta 722550e431 Remove 'UnitTestsList' hook
Extension unit tests are now autodiscovered.

Bug: T142120
Change-Id: I45dc300b24d27e2a36533600e1232ee56b76b2b8
2016-08-04 13:16:06 -07:00
Bartosz Dziewoński f2e05b105b Only run filters once for direct uploads (without stash)
Uses the new UploadStashFile hook.

Bug: T140522
Depends-On: I2f574b355cd33b2e9fa7ff8e1793503b257cce65
Change-Id: Ic7c2dbc54c6ad300d26172796ee21027a8c372ee
2016-08-03 16:30:18 +02:00
Bartosz Dziewoński 069e0c89a5 Provide page text and edit summary when filtering file uploads
This allows filters using `action='upload'` to use the variables
`summary`, `new_wikitext` and several others that previously were only
provided when editing pages (`action='edit'`).

This is achieved using the new UploadVerifyUpload hook, introduced in
MediaWiki core in change Ie68801b307de8456e1753ba54a29c34c8063bc36.

`action='upload'` is now only used when publishing an upload, and not
for uploads to stash. A new `action='stashupload'` is introduced,
which is used for all uploads, including uploads to stash. This
behaves like `action='upload'` used to, and only provides file
metadata variables.

Filter authors should use `action='stashupload'` when a file can be
checked based only on the file contents, and `action='upload'` only
when the wikitext edit needs to be examined too.

Bug: T87381
Bug: T89252
Bug: T139848
Change-Id: I9654f82ecda82e4917fd0ac6b364b947a1434c73
2016-07-09 13:31:15 +00:00
Bartosz Dziewoński c03ac953ad Remove backwards-compatibility code using APIEditBeforeSave hook
It was only needed for MediaWiki prior to 1.25
(09a5febb7b024c0b6585141bb05cba13a642f3eb).
We no longer support those versions after
d527574d2b.

Bug: T137832
Change-Id: I9d0b7e7713c805ebc7bf59f55456e69c6491e265
2016-06-17 01:45:29 +02:00
Gergő Tisza ed5cc1b5fc Update for AuthManager
Repeats I61e4327ef3c7a31b19feef727de7d683f69e260b (which had to be
reverted due to a problem with an ancestor patch) without any
significant change.

Bug: T110448
Bug: T135360
Change-Id: I1688cf9fbcb04bb56d075c9f0876bd0ffeced4f6
2016-06-06 19:54:26 +00:00
Gergő Tisza 6a2627e944 Change some globals to work better with extension registration
Rename $wgAbuseFilterAvailableActions / $wgAbuseFilterRestrictedActions
to $wgAbuseFilterActions / $wgAbuseFilterRestrictions and make
them an associative array instead of a plain one, as that works more
sanely with extension registration. (The renaming helps to give more
useful errors to sites using the old config.)

Change-Id: I790d39c2849922d7daf7479f298cd90cf30af129
2016-06-06 19:53:53 +00:00
Gergő Tisza d527574d2b Add extension.json, empty php entry point
For ease of review this is the same as
Icff39f3f79f7ffe86214cfa216b53f3d882e1968 and the necessary changes
are done in the child patch. Should not be merged without that.

Bug: T87876
Change-Id: Iefd8d34619bbfb7602c4fea869f3f89fdeb9057b
Co-Authored-By: Paladox <thomasmulhall410@yahoo.com>
2016-06-03 19:16:26 +00:00
Gergő Tisza 8cc1aa78bf Revert extension registration and following changes
Reverts:
* e71808f4c4 ("Add extension.json, empty php entry point")
* 8c937957c0 ("Update for AuthManager")

Bug: T136929
Change-Id: Iefc88cb4e981113119f850ca70b9486675bf8a8d
2016-06-03 14:57:55 +00:00
Gergő Tisza 8c937957c0 Update for AuthManager
Bug: T110448
Bug: T135360
Change-Id: I61e4327ef3c7a31b19feef727de7d683f69e260b
2016-05-28 18:42:28 +00:00
Gergő Tisza e71808f4c4 Add extension.json, empty php entry point
Bug: T87876
Change-Id: Icff39f3f79f7ffe86214cfa216b53f3d882e1968
2016-05-28 18:42:26 +00:00