Commit graph

5752 commits

Author SHA1 Message Date
Matěj Suchánek 77f6ecce13 Cleanup FilterProfiler API
Make FilterProfiler::getFilterProfile return stats unchanged,
in a structured way. Move computations to AbuseFilterViewEdit,
as they are only useful there. Don't return false on cache
misses, return arrays with zero values instead.

Bug: T266531
Change-Id: I8718cc31a5004340bf742315c7075e10a61fcbfd
2020-10-28 12:48:30 +00:00
jenkins-bot 5f38ddd5cf Merge "Add typehints to hook handlers" 2020-10-28 12:43:10 +00:00
Daimona Eaytoy be4ef544c4 Merge "Simplify ViewEdit, last round" 2020-10-28 10:38:15 +00:00
Daimona Eaytoy ccf8afe75b Add typehints to hook handlers
Needed after core change I95bb47104ad3dc0a69c812c627ffa631c5dc6ace to
make phan pass on master.

Change-Id: I6202212493340064945a559799e248130f418d6e
2020-10-28 11:37:07 +01:00
Translation updater bot 59962e8835 Localisation updates from https://translatewiki.net.
Change-Id: I1b52349c89fa213233f34d0a1fcf28cf13a97130
2020-10-28 08:35:12 +01:00
Matěj Suchánek be0268f200 Unbreak EmergencyDisable
FilterProfiler::getFilterProfile returns data in a different
format than the data is really stored.

Bug: T266531
Change-Id: I0d961a1ae67769da61f841df2462d47f81849972
2020-10-27 10:07:15 +01:00
Translation updater bot 2f5566d624 Localisation updates from https://translatewiki.net.
Change-Id: I737e3101420d1daf951debbb86ac2f5569bad854
2020-10-27 08:22:59 +01:00
Daimona Eaytoy 916234598d Simplify ViewEdit, last round
This deals with data inconsistencies in buildFilterEditor. Every
property of $row was tested in all 5 scenarios (also using Selenium) to
check when it's set. The result is in the normalizeRow method, which
aims to remove any inconsistencies, so that buildFilterEditor always
receives a "complete" row with all defaults set.

The code in buildFilterEditor is now cleaner (because there are no
isset() checks), and it gives us a unique place where we can set
defaults (rather than partly doing that in
loadRequest/loadFilterData/loadImport, and partly relying on isset).

This will be especially useful when introducing value objects to
represent filters, because now you just have to look at normalizeRow()
to tell which properties are allowed to be missing, and thus what "kind"
of filter object you need (see
I5f33227887c035e301313bbe24d1c1fefb75bc6a).

Additionally, reduce the properties that get passed around during
export/import, and make the selenium test try a roundtrip, rather than
relying on hardcoded data that may get outdated. A future patch will
refactor the import/export code.

Change-Id: Id52c466baaf6da18e2981f27a81ffdad3a509e78
2020-10-26 13:07:29 +00:00
jenkins-bot 711f949b95 Merge "Cleanup for AbuseFilter class" 2020-10-26 11:25:01 +00:00
Translation updater bot b553f9b203 Localisation updates from https://translatewiki.net.
Change-Id: I46bd3a8127fe39a264018a9fcca9e5b978131533
2020-10-26 08:29:24 +01:00
Daimona Eaytoy 0d751dde04 Cleanup for AbuseFilter class
Remove unused property, move to AbuseFilterView a method that's only
used there.

Change-Id: I16658521e32eeaafc1d601528d52bef17e1bf3b5
2020-10-25 15:55:21 +01:00
Daimona Eaytoy 6c9fc516aa ViewRevert: avoid needless query
The previous code would call getUserGroups again once creating the log
entry, but this was slightly flawed: we're updating groups on master,
but the read happens on a replica that might be outdated, hence
resulting in broken logging. Instead of reading from master, we can just
keep a list of the groups that were actually added, and use that
afterwards.

Change-Id: I7cc282e15561de3a3d3e183808a65991aa27d2bb
2020-10-25 10:29:59 +01:00
jenkins-bot 8fe9902af3 Merge "Use UserGroupManager when reverting degroup action" 2020-10-25 09:24:15 +00:00
jenkins-bot 50ae561641 Merge "Simplify ViewEdit, round 2" 2020-10-25 09:10:11 +00:00
Daimona Eaytoy 18ade98339 tests: Move any profiling-related test to FilterProfilerTest
This commit removes several tests from AbuseFilterConsequences, thus
speeding it up a lot (especially because these tests were very slow,
with each test *case* taking up to 30s in the coverage job).

Everything is now covered by the new AbuseFilterFilterProfilerTest
which, although not being a pure unit test, is much much faster than
*Consequences.

Change-Id: Ic6b16d23ec99abee287f36093b8573505f9c613a
2020-10-24 18:09:26 +02:00
Matěj Suchánek 6d81fca76b Improve FilterProfiler coverage
Also improve documentation of some FilterProfiler methods.

Change-Id: I08198c643a7d2dac10e928914e8a5c7413f2543d
2020-10-24 16:23:47 +02:00
jenkins-bot d7770ad520 Merge "Introduce BlockAutopromoteStore service" 2020-10-24 13:16:57 +00:00
jenkins-bot ba9e461ed0 Merge "Deduplicate cache keys used to check blockautopromote" 2020-10-24 12:57:11 +00:00
Matěj Suchánek 1445d5962a Introduce BlockAutopromoteStore service
This service is responsible for the blockautopromote feature:
(un)block autopromotion and check status.

The patch mostly moves code from static methods to the new class
and relaxes type hints (e.g. from User to UserIdentity).

Change-Id: I79a72377881cf06717931cd09af12f3b8e5f3e3f
2020-10-24 12:31:44 +00:00
jenkins-bot dfc9cc2a19 Merge "Code cleanup for FilterProfiler" 2020-10-23 14:43:26 +00:00
Daimona Eaytoy 5890dea4ff Deduplicate cache keys used to check blockautopromote
Previously, AbuseFilterHooks would proxy the data from a slower backend
(db-replicated) to a faster one (hash) reusing the same key. This change
makes it use a dedicated key, so that the "main" key can be kept
internal inside the upcoming BlockAutopromoteStore.

Change-Id: Id46a66991d0e994ee0a83b83b9c95e8951f3041c
2020-10-23 16:43:24 +02:00
Daimona Eaytoy 416dcd9ba3 Simplify ViewEdit, round 2
- Add a helper method to output an unrecoverable error, comprising a
   button to go back to the filters list;
- Move the token check to attemptSave, so to make the conditionals
  easier to read, and group errors together
- Make buildFilterEditor take an HTML parameter for the error, so the
  caller can specify whether it's error or warning
- Move the check for non-existing filters out of buildFilterEditor
- Add a bunch of typehints
- Don't set af_throttled and af_hit_count in the empty row template, but
  set af_deleted (these are only used in buildFilterEditor)
- Make AbuseFilter::translateFromHistory consistently include the af_global
  property (previously it would only be set for global filters; this error
  was introduced when first implementing global filters)
- The only user-facing change is that, when trying to use a custom
  warning/disallow message on a global filter, this is now considered a
  non-fatal error, so we now show the editing interface (and not just an
  unrecoverable error).

The next step is resolving the @todo in buildFilterEditor about null
checks.

Change-Id: I9d217dcac3f4cc0b26e53eca735cc327d5efc76d
2020-10-23 13:00:43 +00:00
Daimona Eaytoy 4de4ef358b Use UserGroupManager when reverting degroup action
This commit avoids direct queries on the DB, which is already an
improvement. It also adds some TODO comments for future improvements,
mostly things that depend on core changes.

Bug: T265224
Change-Id: I8eb76a0c463751976c2c5deedb3570305f1ab4f0
2020-10-23 12:07:45 +00:00
jenkins-bot cc7763f760 Merge "Add dedicated classes for more hook handlers" 2020-10-23 11:38:20 +00:00
Daimona Eaytoy 6724227182 Flatten the array returned by getConsequencesForFilters
There's no point in repeating the action name, because it's already used
as key. We can then flatten the array and just keep the parameters in
the third nesting level.

Change-Id: I54abcc49322f432cedd361abeedb72e067d3de41
2020-10-22 16:36:11 +00:00
Daimona Eaytoy b309c804fc Add dedicated classes for more hook handlers
The schema changes hook was chosen because the handler is very long. The
test ones were chosen to keep test things away from actual code.

Bug: T261067
Change-Id: Ie06bf62399f6353e3e268cccb3fe4b41bbf951c5
2020-10-22 18:23:09 +02:00
jenkins-bot b4b83c9ddc Merge "Replace $wgAbuseFilterRestrictions with more specific variables" 2020-10-22 16:19:49 +00:00
Matěj Suchánek 6b1b879da8 Code cleanup for FilterProfiler
Follows up Ib66c42ac220731f4e1da9ee6cfb5290759dd6494.

Apply DannyS712's suggestions from that patch.

Change-Id: Ib9f19969a888bd29f9f46e90fb52b49ce883c667
2020-10-22 15:39:00 +02:00
Daimona Eaytoy 4c06dd52c8 Replace $wgAbuseFilterRestrictions with more specific variables
So that sysadmins can further customize the extension. It was also wrong
to use the same variable for many different things.

Note that there's no associated patch in wmf-config because we use the
defaults. However, before merging this patch, please recheck that
AbuseFilterRestrictions and AbuseFilterDisallowGlobalLocalBlocks aren't
used there (https://codesearch.wmflabs.org/operations/?q=AbuseFilterDisallowGlobalLocalBlocks%7CAbuseFilterRestrictions&i=nope&files=&repos=)

Bug: T175221
Change-Id: I7581b3ee6d9d11a6cf1599b8ff874e8c3d54adf4
2020-10-22 13:38:59 +00:00
Translation updater bot 673fe625c5 Localisation updates from https://translatewiki.net.
Change-Id: Idf8571b10ee47187c153b19598b7fd9c6197bd40
2020-10-22 08:29:45 +02:00
jenkins-bot 1c10edb80f Merge "Migrate change tags hooks to DI" 2020-10-21 18:04:20 +00:00
jenkins-bot 1c1b40f322 Merge "Inject ChangeTagsManager to ChangeTagger" 2020-10-21 17:21:23 +00:00
jenkins-bot c7e1d11c74 Merge "Add ChangeTagsManager service" 2020-10-21 17:15:40 +00:00
jenkins-bot c865e210de Merge "Simplify ViewEdit::loadRequest" 2020-10-21 16:39:18 +00:00
Matěj Suchánek 2ee3a0d247 Migrate change tags hooks to DI
Bug: T261067
Change-Id: I7b95cd19ab0ae04820e8dcb3481d29a2f9e7a0ca
2020-10-21 16:18:06 +00:00
Matěj Suchánek 93556284a0 Inject ChangeTagsManager to ChangeTagger
We decided to have the tag name provided by ChangeTagsManager,
so make ChangeTagger depend on it.

Change-Id: If3cbfd992f45651f47477031befffc0fd30f4a28
2020-10-21 16:30:43 +02:00
Matěj Suchánek 85e000c6ed Add ChangeTagsManager service
This service will be resposnsible for loading
and caching change tags used by abuse filters.

Change-Id: I9a710af1dd1ae58c47de1e8509246ed929d0a662
2020-10-21 16:24:32 +02:00
jenkins-bot f5950e638f Merge "Performance: don't check autopromotion if blockautopromote is disabled" 2020-10-21 13:20:13 +00:00
Daimona Eaytoy 7e44146781 Performance: don't check autopromotion if blockautopromote is disabled
This hook is called on every request, even for view actions, hence it's
a hot spot and a potential source of performance issues. We can slightly
optimize it by avoiding a cache lookup if blockautopromote is disabled.
Note: this won't really have an impact on WMF wikis since blockautopromote
is enabled almost everywhere.

Bug: T22487
Change-Id: I3743bfea9fe5865a3947cd23a07ae27e2dfa9301
2020-10-21 13:28:41 +02:00
Daimona Eaytoy 9bc885b6b3 Add a ChangeTagger class
The logic about action IDs and the persistent buffer is now encapsulated
inside a single service, which is a step towards getting rid of global
state in the AbuseFilter class, and reducing the responsibilities of the
Runner.

An important change made here is that we now require a LinkTarget rather
than a Title. This removes a dependency on the Title class (a monster
object), makes tests simpler, and denies the need to inject a
TitleFactory. This means living without some bits of context (e.g. we're
no longer using makeTitleSafe to ensure a valid title, and we have to
build a "prefixedtext" manually), but this shouldn't be a problem, given
that the titles are only used to create a cache key: invalid titles are
not a problem, and concatenating namespace + title should always be
sufficient.

Bug: T265370
Change-Id: Iff59cd3d889454a482a89c16691bfefcc5ec0a12
2020-10-21 13:19:30 +02:00
Translation updater bot f53a7d2275 Localisation updates from https://translatewiki.net.
Change-Id: I5edcd144146b0ab26014062888f20a9040c94061
2020-10-21 08:25:57 +02:00
Daimona Eaytoy 215f16a177 Prevent uncaught warnings/exception on Special:AbuseFilter
This patch addresses two issues observed in WMF production:
 - Specifying a search mode without a search pattern would result in a
   call to mb_stripos (in AbuseFilterPager) with an empty delimiter,
   which triggers a PHP warning. Avoid this by checking that the search
   pattern is not the empty string, and unset the search mode if that's
   the case.
 - Trying to use an invalid search mode would result in an unhandled
   LogicException. We have some code in place to check the validity of
   the URL parameter, but the relevant code didn't reset the search mode
   to null, hence AbuseFilterPager would throw before we can show a
   pretty error to the user.

Bug: T265994
Change-Id: Ib19d36d6265981097bbb551783fdac8bdaa98854
2020-10-20 13:59:45 +02:00
Translation updater bot 5f0345477e Localisation updates from https://translatewiki.net.
Change-Id: I6d43e208612685bcc11ea0c79660a54225ec29b0
2020-10-20 08:28:55 +02:00
jenkins-bot 3b59156b4c Merge "Minor updates related to var dumps" 2020-10-19 08:27:05 +00:00
jenkins-bot e002cbb4fa Merge "Exclude implicit groups when degrouping the user" 2020-10-19 07:56:41 +00:00
jenkins-bot 21ccbae8d4 Merge "Avoid direct coupling between SpecialAbuseFilter and AbuseFilterView" 2020-10-19 07:55:57 +00:00
Translation updater bot 62be32fe07 Localisation updates from https://translatewiki.net.
Change-Id: Ib3c077654f1f34477ecda240926eea25f42eba83
2020-10-19 08:33:57 +02:00
Daimona Eaytoy a330d0c454 Exclude implicit groups when degrouping the user
It doesn't make much sense to try to remove implicit groups like 'user'
and '*'. As a matter of fact, these groups are also excluded in
AbuseFilterViewRevert when undoing degroups.

Change-Id: I292499611ccfbd12df28b713d4244530db15c26d
2020-10-18 15:34:04 +02:00
Daimona Eaytoy 3a85e03c72 Simplify ViewEdit::loadRequest
This method was divided into multiple, shorter methods. We now have a
dedicated method for imports, and one for everything else, plus a method
for loading actions. Merged a conditional for when the token didn't
match. Avoid returning Status objects with data inside as it's too
difficult to properly infer types for those.

This is still not perfect, and another round of simplification might be
necessary before this class can be updated to use the upcoming Filter
value objects.

Change-Id: I2de1de1982105e5b9b817a893c357615ffb7db86
2020-10-18 11:06:30 +00:00
Daimona Eaytoy f589629b12 Avoid direct coupling between SpecialAbuseFilter and AbuseFilterView
While this might seem a small change, it removes the last remaining
coupling between SpecialAbuseFilter and the *View classes, that were
forming a huge tangle.

Change-Id: I5a9d6516e3fa2d3efc4bb2e19b05379dc33cd84d
2020-10-17 00:37:11 +02:00