Commit graph

113 commits

Author SHA1 Message Date
Daimona Eaytoy 5890dea4ff Deduplicate cache keys used to check blockautopromote
Previously, AbuseFilterHooks would proxy the data from a slower backend
(db-replicated) to a faster one (hash) reusing the same key. This change
makes it use a dedicated key, so that the "main" key can be kept
internal inside the upcoming BlockAutopromoteStore.

Change-Id: Id46a66991d0e994ee0a83b83b9c95e8951f3041c
2020-10-23 16:43:24 +02:00
Daimona Eaytoy b309c804fc Add dedicated classes for more hook handlers
The schema changes hook was chosen because the handler is very long. The
test ones were chosen to keep test things away from actual code.

Bug: T261067
Change-Id: Ie06bf62399f6353e3e268cccb3fe4b41bbf951c5
2020-10-22 18:23:09 +02:00
Daimona Eaytoy 4c06dd52c8 Replace $wgAbuseFilterRestrictions with more specific variables
So that sysadmins can further customize the extension. It was also wrong
to use the same variable for many different things.

Note that there's no associated patch in wmf-config because we use the
defaults. However, before merging this patch, please recheck that
AbuseFilterRestrictions and AbuseFilterDisallowGlobalLocalBlocks aren't
used there (https://codesearch.wmflabs.org/operations/?q=AbuseFilterDisallowGlobalLocalBlocks%7CAbuseFilterRestrictions&i=nope&files=&repos=)

Bug: T175221
Change-Id: I7581b3ee6d9d11a6cf1599b8ff874e8c3d54adf4
2020-10-22 13:38:59 +00:00
Matěj Suchánek 2ee3a0d247 Migrate change tags hooks to DI
Bug: T261067
Change-Id: I7b95cd19ab0ae04820e8dcb3481d29a2f9e7a0ca
2020-10-21 16:18:06 +00:00
Matěj Suchánek 85e000c6ed Add ChangeTagsManager service
This service will be resposnsible for loading
and caching change tags used by abuse filters.

Change-Id: I9a710af1dd1ae58c47de1e8509246ed929d0a662
2020-10-21 16:24:32 +02:00
jenkins-bot f5950e638f Merge "Performance: don't check autopromotion if blockautopromote is disabled" 2020-10-21 13:20:13 +00:00
Daimona Eaytoy 7e44146781 Performance: don't check autopromotion if blockautopromote is disabled
This hook is called on every request, even for view actions, hence it's
a hot spot and a potential source of performance issues. We can slightly
optimize it by avoiding a cache lookup if blockautopromote is disabled.
Note: this won't really have an impact on WMF wikis since blockautopromote
is enabled almost everywhere.

Bug: T22487
Change-Id: I3743bfea9fe5865a3947cd23a07ae27e2dfa9301
2020-10-21 13:28:41 +02:00
Daimona Eaytoy 9bc885b6b3 Add a ChangeTagger class
The logic about action IDs and the persistent buffer is now encapsulated
inside a single service, which is a step towards getting rid of global
state in the AbuseFilter class, and reducing the responsibilities of the
Runner.

An important change made here is that we now require a LinkTarget rather
than a Title. This removes a dependency on the Title class (a monster
object), makes tests simpler, and denies the need to inject a
TitleFactory. This means living without some bits of context (e.g. we're
no longer using makeTitleSafe to ensure a valid title, and we have to
build a "prefixedtext" manually), but this shouldn't be a problem, given
that the titles are only used to create a cache key: invalid titles are
not a problem, and concatenating namespace + title should always be
sufficient.

Bug: T265370
Change-Id: Iff59cd3d889454a482a89c16691bfefcc5ec0a12
2020-10-21 13:19:30 +02:00
Matěj Suchánek 7ef2259228 Migrate a few hook handlers to DI
Bug: T261067
Change-Id: If699917c3d2e9e22525c7d0495554e25f6b45125
2020-10-10 17:23:04 +00:00
Daimona Eaytoy 2026e3ac3a Add an AbuseFilterPermissionManager service
This service should act as a mediator between the AF code and the
permission manager, and it should know what are the permissions required
by each action.

Change-Id: Ieb177d9992147b11fa7b8f05929da6c182cc2286
2020-10-10 14:03:29 +02:00
Daimona Eaytoy 7a1d6dbdbb Avoid array_filter on explode()
The array_filter is likely meant to empty the array if the empty string
was exploded ( `explode( "\n", '' ) === [ '' ]` ). However, it can also
remove other stuff, e.g. the string '0'. An explicit comparison is
easier to read & interpret, marginally faster, and avoids rare but not
impossible edge cases.

Change-Id: Ie77d65b56319664a2ac370f32341dc72b619a635
2020-10-08 18:56:27 +02:00
Matěj Suchánek b7cda4de4c Prevent cache pollution in fetchAllTags and clean up
Previously, the cached value would depend on the tags
parameter to be updated. The provided value may be
different for each call, so callers may receive
unexpected values.

For example, while core usually calls this with core-defined
hooks, our method AbuseFilter::isAllowedTag calls this
providing an empty array. If core's call happened shortly
after ours and hit cache, its array would be overwritten
with only AbuseFilter's tags, the rest would be lost.

Also do some clean up:

- only call array_filter on explode'd array
- call array_unique on the value, since it's usual that
  multiple filters share the same tag

Noticed when thinking about moving this to a service.

Change-Id: I4f4322e80ec89e48458a3bf46a1146863bec8237
2020-10-07 15:20:41 +02:00
jenkins-bot 2cdec2473e Merge "Actually apply patch-afl_change_deleted_patrolled." 2020-10-05 08:15:48 +00:00
Matěj Suchánek 65708afcea Integrate with Renameuser
Register abuse_filter and abuse_filter_history tables.
abuse_filter_log is more difficult (if possible).

Bug: T27377
Bug: T206477
Change-Id: If8289101a08887519d5a90ef84700421b8ed2406
2020-10-01 08:10:22 +00:00
jenkins-bot a9309c9921 Merge "Revert "Drop duplicate index wiki_timestamp"" 2020-09-23 11:06:12 +00:00
Daimona Eaytoy 01cc79e1d4 Revert "Drop duplicate index wiki_timestamp"
This reverts commit 6a268e7339.

Reason for revert: Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae ended up not renaming the index, so this patch removed an index that was still in use.

Change-Id: Ide4a600a57bcfa4da0c7354b972cc89709ccd660
2020-09-23 10:39:19 +00:00
Alexia E. Smith 422b77ab0e Actually apply patch-afl_change_deleted_patrolled.
This fixes the abuse_filter_log patch-afl_change_deleted_patrolled
not being applied. The patch is provided for (and should work with) all
the supported DBMS.
Additionally, fix the base table files, which would report
afl_patrolled_by as 'NULL', whereas on the WMF cluster it's 'NOT NULL
DEFAULT 0'. The schema patch takes care of converting that column as
well.

Note that this schema change needs not be applied on the WMF cluster, as
that's already up-to-date.

Finally, note that this patch must be backported to 1.33 and 1.34 (and
it might be fairly hard due to the recent schema changes on the
abuse_filter_log table).

Bug: T240895
Change-Id: Ibdbc9b50c25b9e871ebdeae93a54d10877b585f8
2020-09-21 14:52:22 +00:00
jenkins-bot 91f4a1e5a8 Merge "AbuseFilter: Remove duplicate filter log link" 2020-09-15 01:44:59 +00:00
Ammar Abdulhamid 679c777c33 AbuseFilter: Remove duplicate filter log link
For history action, the link would be already added by
HistoryPageToolLinks hook, so it should not be duplicated by this hook.

See images on https://phabricator.wikimedia.org/T261087#6430172

Change-Id: Ia8dd5be49d3ffb48f298ea287e0b2f98c3052015
2020-09-03 17:55:42 +01:00
Daimona Eaytoy f673a04026 Add updateVarDumps to update.php
This shouldn't happen before the script has been tested thoroughly on
WMF wikis with --dry-run.

Bug: T213006
Change-Id: I51425c85bd6932a5c60eb870b02195aae1c24117
2020-08-25 12:49:00 +00:00
Daimona Eaytoy 28ea0e525a Use $user param when filtering edits
This can be different from the User set inside the $context object, as
seen e.g. in Wikibase jobs. Given that the hook provides a $user param,
it makes more sense to use that, rather than extracting it from the
ContextSource kitchen sink.

Bug: T258717
Change-Id: Ib5961068d3df6ae2bfc3f9c6a7b9e555d248b332
2020-08-19 14:24:57 +02:00
Ostrzyciel 99e2766f26 Update PageSaveComplete hook to use EditResult
Bug: T254074
Change-Id: I3922d9a92242c9a6469058a2a2c2a95891e9e429
2020-06-23 14:25:38 +02:00
DannyS712 4b35336638 Update hooks to use PageSaveComplete
Extension requires MW 1.35+, always available

Bug: T250566
Change-Id: I60cf3cc42db989d8ccb0d06d3cf9eae8a85784ac
2020-06-16 04:18:39 +00:00
Daimona Eaytoy 6a268e7339 Drop duplicate index wiki_timestamp
For MySQL it was renamed Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae, but
the old index isn't being deleted, hence creating a duplicate.

Change-Id: I09b9f64759f6a897c393caa77458d63995d5713b
2020-06-12 19:34:41 +02:00
Umherirrender 6e2d7931a5 Pass function name to database functions
Useful for logging

Change-Id: Ib6b3ad814dd24d31aab37be486ff32d3f57c7905
2020-06-07 01:54:41 +02:00
Reedy 7a2373f7d8 Remove some SQLite specific files
Bug: T251613
Change-Id: Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae
2020-05-04 17:50:18 +00:00
Daimona Eaytoy c9a4146f2c Add fixOldLogEntries to update.php
Already done in WMF production, see T228655.

Bug: T208931
Change-Id: Id477387d1607e263a6bf054060dc6dd440e88467
2020-02-27 18:19:50 +00:00
Daimona Eaytoy aa15267c79 Add string typehint to $summary in onParserOutputStashForEdit
Bug: T245928
Depends-On: I8fa287f335e90a59ac18365e7401a5cf703130a3
Change-Id: Ia075e4bdf3a3f011a181c8026ff1cdb8e186f096
2020-02-22 19:27:05 +00:00
Daimona Eaytoy 3f83e57ad7 Factor out variables-related methods
This is another step needed to reduce the size of the gigantic
AbuseFilter and AbuseFilterHooks classes. It also makes many methods
non-static, for more testability.

Note, this layout is still not final. We should somehow merge the
functionality of VariableGenerator and AFComputedVariable, for which
I already have plans.

Change-Id: I366d598b69ad866496b7cb0059e0835c02e54041
2020-02-07 20:27:26 +00:00
Daimona Eaytoy 1686042a91 Move variable generators to new classes
RunVariableGenerator is for generating variables based on the current
action;
RowVariableGenerator is for RC entries;
VariableGenerator is the generic one.

This patch only moves the methods to the new classes, to keep the diff
easier to read, and facilitate conflict resolution. These classes will
then be revamped in I366d598b69ad866496b7cb0059e0835c02e54041.

Note that these classes are now namespaced.

One method, AbuseFilter::getEditVars, was renamed to
AbuseFilterVariableGenerator::generateEditVars, because it would
otherwise conflict with an incompatible method in RunVariableGenerator.

Change-Id: Iff412e5492873d4fae55402939a51609e64d55a8
2020-02-07 19:44:31 +00:00
Daimona Eaytoy 6b7be78534 Use RCDatabaseLogEntry as wrapper in get*VarsFromRCRow
This provides various shortcuts for user, target, comment, etc.,
avoiding direct access to the row, and thus a dependency on the
schema.

Change-Id: I250f94e0ac6cade33441a31ae8a27093a4d937a0
2020-02-07 19:19:10 +00:00
Ammar Abdulhamid 641aeebbcf Replace deprecated IP class with IPUtils
Bug: T242556
Change-Id: If8e9034885726b673d1500fa8b538b5302e66165
2020-01-24 18:27:26 +01:00
jenkins-bot 70d31da673 Merge "Stop using deprecated stuff with easy replacements" 2020-01-22 16:44:57 +00:00
Daimona Eaytoy e9fe252def Fix remaining PHPCS issues
Mainly, add visibility modifiers on constants.

Change-Id: I41e8e2d691b2bad6ea6f244d54517d37d7783181
2020-01-21 12:36:37 +00:00
Daimona Eaytoy 10c2fe7151 Stop using deprecated stuff with easy replacements
This patch is mostly replacing Revision::* constants,
Wikimedia\(restore|suppress)Warnings, and wfWikiId.

Change-Id: I13544cc3e12955a9376ccce3c120e2cee1f2ee2e
2020-01-08 14:59:30 +01:00
Daimona Eaytoy c432f058fd Restore the ability to filter content model changes
Follows-up I3fb7b36ab38ca1544889a4c233b8ffdfc6c80936

Bug: T240485
Change-Id: I2e8337e6f505932a18a5bb5a0d97b9d6bc3f42c8
2019-12-11 19:42:45 +01:00
Daimona Eaytoy 6be070e5a2 Strengthen the check for null edits
Even if the Content objects are different, the normalized text contents
may be identical.
Also, stop misattributing null edits by adding the last revision of the
page as afl_rev_id.

Bug: T240115
Change-Id: I3fb7b36ab38ca1544889a4c233b8ffdfc6c80936
2019-12-10 17:03:49 +01:00
Petr Pchelko 915b9a1538 Remove usages of deprecated User methods
Bug: T220191
Change-Id: I54e20870a32ff98b41a98495694ff563c4c4c5ca
2019-10-30 12:51:01 +00:00
jenkins-bot 0e30c1c34e Merge "Add new schemas for splitting afl_filter" 2019-09-27 15:41:06 +00:00
Daimona Eaytoy 0119108ee7 Fix params to ParserOutputStashForEdit
$summary and $user are always guaranteed to be passed, and $user is
guaranteed to be a User object. Hence, update the hook handler to
reflect that.

Change-Id: I3a7fcb074b460b77210de5a6bad43f500aff3249
2019-09-23 23:33:51 +02:00
Daimona Eaytoy 9a6dd1307c Add new schemas for splitting afl_filter
It'd be great if we could get this included in 1.34.

Bug: T220791
Change-Id: I62d429d0eb6a7adc51cc37fe18f878077f85a006
2019-09-22 16:04:45 +00:00
Daimona Eaytoy 4c8be4d374 Add profiling points throughout the code for the CachingParser switch
Bug: T156095
Change-Id: Ib934be34a953166fe1b94cfe8ed216afe3b906ca
2019-09-18 10:02:55 +00:00
jenkins-bot cfad7d6f14 Merge "Actually return errors for action=edit API" 2019-09-10 19:59:03 +00:00
Bartosz Dziewoński 82b6f191d4 Actually return errors for action=edit API
Setting 'apiHookResult' results in a "successful" response; if we want
to report an error, we need to use ApiMessage. We already were doing
this for action=upload. Now our action=edit API responses will be
consistent with MediaWiki and other extensions, and will be able to
take advantage of errorformat=html.

Since this breaks compatibility anyway, also remove some redundant
backwards-compatibility values from the output.

To avoid user interface regressions in VisualEditor, the changes
I3b9c4fef (in VE) and I106dbd3c (in MediaWiki) should be merged first.

Before:
    {
        "edit": {
            "code": "abusefilter-disallowed",
            "message": {
                "key": "abusefilter-disallowed",
                "params": [ ... ]
            },
            "abusefilter": { ... },
            "info": "Hit AbuseFilter: Test filter disallow",
            "warning": "This action has been automatically identified ...",
            "result": "Failure"
        }
    }

After:
    {
        "errors": [
            {
                "code": "abusefilter-disallowed",
                "data": {
                    "abusefilter": { ... },
                },
                "module": "edit",
                "*": "This action has been automatically identified ..."
            }
        ],
        "*": "See http://localhost:3080/w/api.php for API usage. ..."
    }

For comparison, a 'readonly' error:
    {
        "errors": [
            {
                "code": "readonly",
                "data": {
                    "readonlyreason": "foo bar"
                },
                "module": "main",
                "*": "The wiki is currently in read-only mode."
            }
        ],
        "*": "See http://localhost:3080/w/api.php for API usage. ..."
    }

Bug: T229539
Depends-On: I106dbd3cbdbf7082b1d1f1c1106ece6b19c22a86
Depends-On: I3b9c4fefc0869ef7999c21cef754434febd852ec
Change-Id: I5424de387cbbcc9c85026b8cfeaf01635eee34a0
2019-09-09 20:15:19 +02:00
Daimona Eaytoy 393e47c5a7 Upgrade phan-config to 0.7.1
Change-Id: I859d81eda8601da91602b27a223b6d6d59ecf563
2019-09-01 09:42:26 +00:00
Huji Lee 1ddb65021b Add links to AbuseFilter logs on Special:Undelete
Depends-On: I671a0479e877e6c37606b688064cb9c893717709
Bug: T231055
Change-Id: Iebf832c513c6a4e954db0ba2633dd8ba6f27b412
2019-08-23 14:56:43 +00:00
Santhosh Thottingal 1176e3a465
Fix the warning about permission name changes
Change-Id: I16463550328eb19d33270d8677404e012e5c80df
2019-08-13 14:40:17 +05:30
jenkins-bot 3748c41e79 Merge "Remove outdated comment, add a new one" 2019-08-12 08:26:30 +00:00
Daimona Eaytoy 200905f1cf Remove outdated comment, add a new one
As explained in T230093#5408119.

Bug: T230295
Change-Id: Ic0beaf9d126d14fbb7efbd2d0ec55e10c0fbcc99
2019-08-11 14:35:38 +00:00
Daimona Eaytoy ff40204ef1 Gracefully handle blockautopromote failures
Instead of returning a successful message, return null and log a
warning. Also, make autopromoteBlockKey public + internal and use it
from Hooks instead of duplicating the logic.
Follow-up: I03feb05218789a3b73a31c9a94216daafcb7c145

Change-Id: I8ce96d1bd0239003f8ee6a45f412b9502d542a18
2019-08-10 15:18:30 +02:00