Commit graph

127 commits

Author SHA1 Message Date
Daimona Eaytoy 45f0a66616 Move remaining classes to own namespace
So everything can be loaded using PSR-4. These classes weren't renamed,
nor the alias for the AbuseFilter class was deprecated, because they
should be refactored first.

Change-Id: Ia328db58eb326968edf5591daac9bacf8c2f75da
2021-01-04 12:11:58 +01:00
Daimona Eaytoy 6081bf90c4 Introduce a VariableGeneratorFactory service
So we can use DI in all generators. Some improvements were deliberately
omitted, e.g. injecting more services and relaxing User/Title to
UserIdentity/LinkTarget, and they'll be included in a subsequent commit.

Depends-On: I1f351071ef2b0b7c80e91407a9c3bb17be293044
Depends-On: Ie71740fac35a86f8fe03023080ae8ca08671243d
Depends-On: I589a0e1c2c5891070ab82cd5adfd9cedec19e67d
Change-Id: I92ef0abd5e45b672e6f297a71b3c2c345d56f136
2021-01-03 14:17:39 +01:00
Daimona Eaytoy 6e27a9ddb3 Cleanup variables-related classes
Change-Id: I20a7fe1a40255043ed0d125dee61ea6052dda69c
2021-01-02 18:19:38 +01:00
Daimona Eaytoy 762d71c51d Create a dedicated namespace for variables-related classes
Some cleanup is left for later to keep the diff easier to read.

Change-Id: Ife445b5e47e707ab77ec867ac3b005866aa74ef2
2021-01-02 18:16:48 +01:00
Daimona Eaytoy fad9a11f7a Add a TextExtractor service
This is an important step towards removing the AbuseFilter class. Note:
proposals for the name of the new service are welcome.

Change-Id: Ib4632173f728b1bdafadef96e01645a833bfceaa
2021-01-01 18:25:32 +01:00
Matěj Suchánek dc7509811a Notify of a throttled filter
Use Echo for delivering the notification to the last
user who edited the filter.

Much boilerplate.

Change-Id: I7a46a03b4f15de20902ec70c62fb4fe750096842
Depends-On: If585b14a6dd6fb8c7d2c3bee1f20d9d08eaac706
2020-12-19 10:31:29 +01:00
jenkins-bot e1a7acf55f Merge "Implement GetAutoPromoteGroupsHook" 2020-12-18 13:41:28 +00:00
Matěj Suchánek 2da4cb4321 Implement GetAutoPromoteGroupsHook
Test coverage included.

Bug: T261067
Change-Id: Iba49f0518d2c1a72405498b2743e5ef300dd6d61
2020-12-15 18:01:04 +01:00
Daimona Eaytoy 3f7dd25fbf Create FilterRunnerFactory
Next step is splitting the Runner into various subclasses.

Change-Id: I766555f31b425cee52fd262c5bfb1c73f3f170d2
2020-12-15 12:47:34 +00:00
Daimona Eaytoy bcbfa66fe8 Write afl_filter_id and afl_global by default
The schema was introduced in 1.34, so there should be no issue in
starting off with writing the new columns.

Bug: T220791
Change-Id: I8f956d4a27692a33368a413fbf4a8eb5da20afe1
2020-12-08 23:06:03 +01:00
Daimona Eaytoy 815ef6051c Split afl_filter in afl_filter_id and afl_global
Add a script to migrate the columns (which can also
be executed in dry run), and a config option with the migration stage
(defaults to SCHEMA_COMPAT_OLD).
Some of the script-related code is stolen from
Ic755526d5f989c4a66b1d37527cda235f61cb437.

Bug: T220791
Change-Id: I7460a2d63f60c2933b36f8383a8abdbba8649e12
2020-12-08 18:31:27 +00:00
Daimona Eaytoy 6305746de3 Use a WikiPage object when filtering edits on a non-existing Title
Remove $title->exists() from the check, so we have the following
changes:
 - The AbuseLog will add a diff link for page creations
 - Searching the AbuseLog for impact:saved will include page creations
 - We don't have to recreate the WikiPage again in RunVariableGenerator

Also remove an old reference to "bug 31656": that comment was added in
rEABFefecf8b2441ae2f31f924ff33103f5affe5d1d62, which changed
Article::getContent() to Article::getRevision()->getRawText(). Nowadays
we don't even use Article anymore, and that conditional isn't even for
retrieving the page content, so the comment is wrong.

Add logging for when the Title object cannot exist, as this should never
happen in the context of the EditFilterMergedContent hook, and always
create a WikiPage. Some signatures were changed to require a WikiPage
object now, and every caller updated to provide it.

Bug: T263104
Bug: T62179
Depends-On: Ic238eaa529ef6bfba06b4dd03924a8e0111d8259
Change-Id: Ibf3bf4f68328ba4a5616ab8f26a8b44d27a25cd7
2020-11-18 20:13:46 +00:00
Daimona Eaytoy 1f8df50cb3 Add a service to retrieve the central DB
This is a thin wrapper around LBFactory and the global variable, that
can be injected in classes requiring it (no real class right now, but
that's going to change soon).

Also, remove some DWIM-style returns which made the code harder to
understand.

Change-Id: I1d28ad4a67f914103f3a17cda5f61b28070c7f1c
2020-10-31 12:32:46 +00:00
Matěj Suchánek 1445d5962a Introduce BlockAutopromoteStore service
This service is responsible for the blockautopromote feature:
(un)block autopromotion and check status.

The patch mostly moves code from static methods to the new class
and relaxes type hints (e.g. from User to UserIdentity).

Change-Id: I79a72377881cf06717931cd09af12f3b8e5f3e3f
2020-10-24 12:31:44 +00:00
Daimona Eaytoy 5890dea4ff Deduplicate cache keys used to check blockautopromote
Previously, AbuseFilterHooks would proxy the data from a slower backend
(db-replicated) to a faster one (hash) reusing the same key. This change
makes it use a dedicated key, so that the "main" key can be kept
internal inside the upcoming BlockAutopromoteStore.

Change-Id: Id46a66991d0e994ee0a83b83b9c95e8951f3041c
2020-10-23 16:43:24 +02:00
Daimona Eaytoy b309c804fc Add dedicated classes for more hook handlers
The schema changes hook was chosen because the handler is very long. The
test ones were chosen to keep test things away from actual code.

Bug: T261067
Change-Id: Ie06bf62399f6353e3e268cccb3fe4b41bbf951c5
2020-10-22 18:23:09 +02:00
Daimona Eaytoy 4c06dd52c8 Replace $wgAbuseFilterRestrictions with more specific variables
So that sysadmins can further customize the extension. It was also wrong
to use the same variable for many different things.

Note that there's no associated patch in wmf-config because we use the
defaults. However, before merging this patch, please recheck that
AbuseFilterRestrictions and AbuseFilterDisallowGlobalLocalBlocks aren't
used there (https://codesearch.wmflabs.org/operations/?q=AbuseFilterDisallowGlobalLocalBlocks%7CAbuseFilterRestrictions&i=nope&files=&repos=)

Bug: T175221
Change-Id: I7581b3ee6d9d11a6cf1599b8ff874e8c3d54adf4
2020-10-22 13:38:59 +00:00
Matěj Suchánek 2ee3a0d247 Migrate change tags hooks to DI
Bug: T261067
Change-Id: I7b95cd19ab0ae04820e8dcb3481d29a2f9e7a0ca
2020-10-21 16:18:06 +00:00
Matěj Suchánek 85e000c6ed Add ChangeTagsManager service
This service will be resposnsible for loading
and caching change tags used by abuse filters.

Change-Id: I9a710af1dd1ae58c47de1e8509246ed929d0a662
2020-10-21 16:24:32 +02:00
jenkins-bot f5950e638f Merge "Performance: don't check autopromotion if blockautopromote is disabled" 2020-10-21 13:20:13 +00:00
Daimona Eaytoy 7e44146781 Performance: don't check autopromotion if blockautopromote is disabled
This hook is called on every request, even for view actions, hence it's
a hot spot and a potential source of performance issues. We can slightly
optimize it by avoiding a cache lookup if blockautopromote is disabled.
Note: this won't really have an impact on WMF wikis since blockautopromote
is enabled almost everywhere.

Bug: T22487
Change-Id: I3743bfea9fe5865a3947cd23a07ae27e2dfa9301
2020-10-21 13:28:41 +02:00
Daimona Eaytoy 9bc885b6b3 Add a ChangeTagger class
The logic about action IDs and the persistent buffer is now encapsulated
inside a single service, which is a step towards getting rid of global
state in the AbuseFilter class, and reducing the responsibilities of the
Runner.

An important change made here is that we now require a LinkTarget rather
than a Title. This removes a dependency on the Title class (a monster
object), makes tests simpler, and denies the need to inject a
TitleFactory. This means living without some bits of context (e.g. we're
no longer using makeTitleSafe to ensure a valid title, and we have to
build a "prefixedtext" manually), but this shouldn't be a problem, given
that the titles are only used to create a cache key: invalid titles are
not a problem, and concatenating namespace + title should always be
sufficient.

Bug: T265370
Change-Id: Iff59cd3d889454a482a89c16691bfefcc5ec0a12
2020-10-21 13:19:30 +02:00
Matěj Suchánek 7ef2259228 Migrate a few hook handlers to DI
Bug: T261067
Change-Id: If699917c3d2e9e22525c7d0495554e25f6b45125
2020-10-10 17:23:04 +00:00
Daimona Eaytoy 2026e3ac3a Add an AbuseFilterPermissionManager service
This service should act as a mediator between the AF code and the
permission manager, and it should know what are the permissions required
by each action.

Change-Id: Ieb177d9992147b11fa7b8f05929da6c182cc2286
2020-10-10 14:03:29 +02:00
Daimona Eaytoy 7a1d6dbdbb Avoid array_filter on explode()
The array_filter is likely meant to empty the array if the empty string
was exploded ( `explode( "\n", '' ) === [ '' ]` ). However, it can also
remove other stuff, e.g. the string '0'. An explicit comparison is
easier to read & interpret, marginally faster, and avoids rare but not
impossible edge cases.

Change-Id: Ie77d65b56319664a2ac370f32341dc72b619a635
2020-10-08 18:56:27 +02:00
Matěj Suchánek b7cda4de4c Prevent cache pollution in fetchAllTags and clean up
Previously, the cached value would depend on the tags
parameter to be updated. The provided value may be
different for each call, so callers may receive
unexpected values.

For example, while core usually calls this with core-defined
hooks, our method AbuseFilter::isAllowedTag calls this
providing an empty array. If core's call happened shortly
after ours and hit cache, its array would be overwritten
with only AbuseFilter's tags, the rest would be lost.

Also do some clean up:

- only call array_filter on explode'd array
- call array_unique on the value, since it's usual that
  multiple filters share the same tag

Noticed when thinking about moving this to a service.

Change-Id: I4f4322e80ec89e48458a3bf46a1146863bec8237
2020-10-07 15:20:41 +02:00
jenkins-bot 2cdec2473e Merge "Actually apply patch-afl_change_deleted_patrolled." 2020-10-05 08:15:48 +00:00
Matěj Suchánek 65708afcea Integrate with Renameuser
Register abuse_filter and abuse_filter_history tables.
abuse_filter_log is more difficult (if possible).

Bug: T27377
Bug: T206477
Change-Id: If8289101a08887519d5a90ef84700421b8ed2406
2020-10-01 08:10:22 +00:00
jenkins-bot a9309c9921 Merge "Revert "Drop duplicate index wiki_timestamp"" 2020-09-23 11:06:12 +00:00
Daimona Eaytoy 01cc79e1d4 Revert "Drop duplicate index wiki_timestamp"
This reverts commit 6a268e7339.

Reason for revert: Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae ended up not renaming the index, so this patch removed an index that was still in use.

Change-Id: Ide4a600a57bcfa4da0c7354b972cc89709ccd660
2020-09-23 10:39:19 +00:00
Alexia E. Smith 422b77ab0e Actually apply patch-afl_change_deleted_patrolled.
This fixes the abuse_filter_log patch-afl_change_deleted_patrolled
not being applied. The patch is provided for (and should work with) all
the supported DBMS.
Additionally, fix the base table files, which would report
afl_patrolled_by as 'NULL', whereas on the WMF cluster it's 'NOT NULL
DEFAULT 0'. The schema patch takes care of converting that column as
well.

Note that this schema change needs not be applied on the WMF cluster, as
that's already up-to-date.

Finally, note that this patch must be backported to 1.33 and 1.34 (and
it might be fairly hard due to the recent schema changes on the
abuse_filter_log table).

Bug: T240895
Change-Id: Ibdbc9b50c25b9e871ebdeae93a54d10877b585f8
2020-09-21 14:52:22 +00:00
jenkins-bot 91f4a1e5a8 Merge "AbuseFilter: Remove duplicate filter log link" 2020-09-15 01:44:59 +00:00
Ammar Abdulhamid 679c777c33 AbuseFilter: Remove duplicate filter log link
For history action, the link would be already added by
HistoryPageToolLinks hook, so it should not be duplicated by this hook.

See images on https://phabricator.wikimedia.org/T261087#6430172

Change-Id: Ia8dd5be49d3ffb48f298ea287e0b2f98c3052015
2020-09-03 17:55:42 +01:00
Daimona Eaytoy f673a04026 Add updateVarDumps to update.php
This shouldn't happen before the script has been tested thoroughly on
WMF wikis with --dry-run.

Bug: T213006
Change-Id: I51425c85bd6932a5c60eb870b02195aae1c24117
2020-08-25 12:49:00 +00:00
Daimona Eaytoy 28ea0e525a Use $user param when filtering edits
This can be different from the User set inside the $context object, as
seen e.g. in Wikibase jobs. Given that the hook provides a $user param,
it makes more sense to use that, rather than extracting it from the
ContextSource kitchen sink.

Bug: T258717
Change-Id: Ib5961068d3df6ae2bfc3f9c6a7b9e555d248b332
2020-08-19 14:24:57 +02:00
Ostrzyciel 99e2766f26 Update PageSaveComplete hook to use EditResult
Bug: T254074
Change-Id: I3922d9a92242c9a6469058a2a2c2a95891e9e429
2020-06-23 14:25:38 +02:00
DannyS712 4b35336638 Update hooks to use PageSaveComplete
Extension requires MW 1.35+, always available

Bug: T250566
Change-Id: I60cf3cc42db989d8ccb0d06d3cf9eae8a85784ac
2020-06-16 04:18:39 +00:00
Daimona Eaytoy 6a268e7339 Drop duplicate index wiki_timestamp
For MySQL it was renamed Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae, but
the old index isn't being deleted, hence creating a duplicate.

Change-Id: I09b9f64759f6a897c393caa77458d63995d5713b
2020-06-12 19:34:41 +02:00
Umherirrender 6e2d7931a5 Pass function name to database functions
Useful for logging

Change-Id: Ib6b3ad814dd24d31aab37be486ff32d3f57c7905
2020-06-07 01:54:41 +02:00
Reedy 7a2373f7d8 Remove some SQLite specific files
Bug: T251613
Change-Id: Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae
2020-05-04 17:50:18 +00:00
Daimona Eaytoy c9a4146f2c Add fixOldLogEntries to update.php
Already done in WMF production, see T228655.

Bug: T208931
Change-Id: Id477387d1607e263a6bf054060dc6dd440e88467
2020-02-27 18:19:50 +00:00
Daimona Eaytoy aa15267c79 Add string typehint to $summary in onParserOutputStashForEdit
Bug: T245928
Depends-On: I8fa287f335e90a59ac18365e7401a5cf703130a3
Change-Id: Ia075e4bdf3a3f011a181c8026ff1cdb8e186f096
2020-02-22 19:27:05 +00:00
Daimona Eaytoy 3f83e57ad7 Factor out variables-related methods
This is another step needed to reduce the size of the gigantic
AbuseFilter and AbuseFilterHooks classes. It also makes many methods
non-static, for more testability.

Note, this layout is still not final. We should somehow merge the
functionality of VariableGenerator and AFComputedVariable, for which
I already have plans.

Change-Id: I366d598b69ad866496b7cb0059e0835c02e54041
2020-02-07 20:27:26 +00:00
Daimona Eaytoy 1686042a91 Move variable generators to new classes
RunVariableGenerator is for generating variables based on the current
action;
RowVariableGenerator is for RC entries;
VariableGenerator is the generic one.

This patch only moves the methods to the new classes, to keep the diff
easier to read, and facilitate conflict resolution. These classes will
then be revamped in I366d598b69ad866496b7cb0059e0835c02e54041.

Note that these classes are now namespaced.

One method, AbuseFilter::getEditVars, was renamed to
AbuseFilterVariableGenerator::generateEditVars, because it would
otherwise conflict with an incompatible method in RunVariableGenerator.

Change-Id: Iff412e5492873d4fae55402939a51609e64d55a8
2020-02-07 19:44:31 +00:00
Daimona Eaytoy 6b7be78534 Use RCDatabaseLogEntry as wrapper in get*VarsFromRCRow
This provides various shortcuts for user, target, comment, etc.,
avoiding direct access to the row, and thus a dependency on the
schema.

Change-Id: I250f94e0ac6cade33441a31ae8a27093a4d937a0
2020-02-07 19:19:10 +00:00
Ammar Abdulhamid 641aeebbcf Replace deprecated IP class with IPUtils
Bug: T242556
Change-Id: If8e9034885726b673d1500fa8b538b5302e66165
2020-01-24 18:27:26 +01:00
jenkins-bot 70d31da673 Merge "Stop using deprecated stuff with easy replacements" 2020-01-22 16:44:57 +00:00
Daimona Eaytoy e9fe252def Fix remaining PHPCS issues
Mainly, add visibility modifiers on constants.

Change-Id: I41e8e2d691b2bad6ea6f244d54517d37d7783181
2020-01-21 12:36:37 +00:00
Daimona Eaytoy 10c2fe7151 Stop using deprecated stuff with easy replacements
This patch is mostly replacing Revision::* constants,
Wikimedia\(restore|suppress)Warnings, and wfWikiId.

Change-Id: I13544cc3e12955a9376ccce3c120e2cee1f2ee2e
2020-01-08 14:59:30 +01:00
Daimona Eaytoy c432f058fd Restore the ability to filter content model changes
Follows-up I3fb7b36ab38ca1544889a4c233b8ffdfc6c80936

Bug: T240485
Change-Id: I2e8337e6f505932a18a5bb5a0d97b9d6bc3f42c8
2019-12-11 19:42:45 +01:00