Commit graph

324 commits

Author SHA1 Message Date
Matěj Suchánek 0eff5a3fa0 Separate loose and backwards compatible code
Small refactoring. Create checkAllFiltersInternal and process
its return value in checkAllFilters to ensure compatibility.

Also fix some annotations.

Change-Id: If9d296de48f08d719f1700f88870002b814c5b31
2021-02-08 11:18:53 +01:00
Daimona Eaytoy a04a601240 Introduce an EditRevUpdater service
This service allows linking the EditFilterMergedContent and
PageSaveComplete hooks for the same edit, so we can update rev IDs in
the abuse_filter_log table. Having such a services also avoids two hacky
static props, and should allow separating the hook handlers easily.

Change-Id: I622d15225ee3af202cb5730a7112652aef8ca71a
2021-01-27 00:24:39 +01:00
Daimona Eaytoy 005cc83642 Increase coverage for more classes
Change-Id: Iae6a24291f821fda77a45d8c1584de010af6a834
2021-01-17 17:38:58 +00:00
Daimona Eaytoy 8639e0c368 Introduce subclasses of Filter with specific use cases
In particular, this brings stronger typing for getID(), and we can get
rid of many phan suppressions.

Change-Id: Icbf3a6f7db8105082646ec227f62c09449fb165d
2021-01-17 00:47:29 +00:00
Daimona Eaytoy 4c0690b4b1 Move getFirstFilterChange to FilterLookup
Additionally:
- Add typehints for stronger typing, and use strict comparison in the
  callers
- Use MIN instead of sorting, as the former is optimized by the DBMS;
  sorting was also happening on the wrong key, i.e. afh_timestamp, as
  opposed to afh_id

Change-Id: I631772fdfeb510b0bc8b582b84bcf2533d7bc097
2021-01-04 14:52:42 -08:00
Daimona Eaytoy 45f0a66616 Move remaining classes to own namespace
So everything can be loaded using PSR-4. These classes weren't renamed,
nor the alias for the AbuseFilter class was deprecated, because they
should be refactored first.

Change-Id: Ia328db58eb326968edf5591daac9bacf8c2f75da
2021-01-04 12:11:58 +01:00
Daimona Eaytoy 6e27a9ddb3 Cleanup variables-related classes
Change-Id: I20a7fe1a40255043ed0d125dee61ea6052dda69c
2021-01-02 18:19:38 +01:00
Daimona Eaytoy 762d71c51d Create a dedicated namespace for variables-related classes
Some cleanup is left for later to keep the diff easier to read.

Change-Id: Ife445b5e47e707ab77ec867ac3b005866aa74ef2
2021-01-02 18:16:48 +01:00
Daimona Eaytoy fad9a11f7a Add a TextExtractor service
This is an important step towards removing the AbuseFilter class. Note:
proposals for the name of the new service are welcome.

Change-Id: Ib4632173f728b1bdafadef96e01645a833bfceaa
2021-01-01 18:25:32 +01:00
Daimona Eaytoy b85f464530 Introduce a VariablesFormatter service
Moves more methods away from the AbuseFilter class. Testing
buildVarDumpTable is not easy because we'd have to parse the generated HTML.

Change-Id: I073a537201de150ba9dd7bf15a99f3a009dc6ba1
2021-01-01 15:45:52 +01:00
Daimona Eaytoy c7f06750d6 Add a LazyVariableComputer service
See task for a description of the plan. Also note that
AFComputedVariable should be renamed and its properties made private.

This commit includes some adjustments for taint-check in
AbuseFilter::buildVarDumpTable and ::revisionToString.

There's some space for improvement in the new LazyVariableComputer, but
that's left for another commit.

Bug: T261069
Change-Id: Ia44f6e079d39f44cf0122dec5ddb5513ab54f0c6
2020-12-31 14:05:52 +01:00
jenkins-bot 534ad9cfe6 Merge "Add a service to format filter specs" 2020-12-20 10:21:39 +00:00
Daimona Eaytoy 7bcb5ec2d5 Add a service to format filter specs
This requires a MessageLocalizer, which currently means providing the
main RequestContext. This is the only alternative right now, until core
provides a proper MessageLocalizer service (see T247127).

Change-Id: I8c93e2ae7e7bd4fc561c5e8490ed2feb1ef0edc2
2020-12-19 14:22:08 +01:00
libraryupgrader 682b6136e7 build: Updating mediawiki/mediawiki-phan-config to 0.10.6
Change-Id: I910c56e32829cea54a16105253004901e00a1885
2020-12-19 13:09:09 +00:00
Daimona Eaytoy 3f7dd25fbf Create FilterRunnerFactory
Next step is splitting the Runner into various subclasses.

Change-Id: I766555f31b425cee52fd262c5bfb1c73f3f170d2
2020-12-15 12:47:34 +00:00
Daimona Eaytoy c52ef337d7 Add a VariablesBlobStore service
Change-Id: If0c1eab2391819f8b4c801d12275d9ec14490f7a
2020-12-15 02:35:15 +00:00
daniel dfeff89317 Use a BlobStore for storing var dumps
AbuseFilter emulates the storage mechanism also used for page content.
Instead of duplicating the relevant code, AbuseFilter should use the
same BlobStore service also used by RevisionStore.

Note that this change is not strictly needed to resolve T198341, but is
needed to unblock T183490

Bug: T261889
Bug: T198341
Bug: T183490
Change-Id: I3fc8475dd8d50d73d705b706ff597a130267e990
2020-12-15 02:35:05 +00:00
Daimona Eaytoy 9d288478fe Remove a bunch of deprecated methods
These are all unused.

Depends-On: Id99da02a98bb392cafed370768edcc8ac3d712ab
Change-Id: I9ff17714f94e1fbbd52da32ebf4f054a551edc1f
2020-12-13 18:31:27 +00:00
Daimona Eaytoy 5e609eb537 Add GlobalNameUtils class
This is just a temporary location for these two methods. Since they're
used a lot, having them in the AbuseFilter class means that the
dependency graph is unnecessarily complicated. Thus, since these methods
aren't doing much, they were moved to a dedicated class. Future todo is
finding an appropriate location, that might be either as part of another
service, or keep them in a Utilities class, perhaps a single class with
all util methods, rather than a specific class.

Change-Id: I52cc47a6b9a387cd1e68c5127f6598a4c43ca428
2020-12-12 17:49:48 +00:00
Daimona Eaytoy 1c625eeae4 Drop back-compat code
This should be merged once T246539 is done.

Bug: T213006
Change-Id: I4444cada720ab62d187f2dd0c4760697e465f2ff
2020-12-08 17:15:47 +00:00
Daimona Eaytoy c786c4adc6 Add ConsequencesRegistry
Change-Id: I91f4f28e09fa46b9ab2457b2a241b6f105320bdd
2020-12-03 22:39:25 +01:00
Matěj Suchánek de997fe98e Improve type safety of filter ids
Also fix a bug in FilterProfiler. It would attempt to reset
stats for global filters but we do not record them (yet?).

Change-Id: I0228d8c85dab146deb877dfce506f1e8e7711a9f
2020-12-03 14:58:51 +01:00
Daimona Eaytoy c957188866 Add ConsequencesLookup
The class is used to retrieve consequences from the Database.

Change-Id: I46b3925aac47554723649c076eff64707a2ea2e6
2020-11-27 16:43:44 +01:00
Daimona Eaytoy 9595bd9da5 Introduce a service for saving filters
Change-Id: I6b7d16ad7ea1124989ed67c74413979cfd0275c4
2020-11-20 22:33:21 +01:00
Daimona Eaytoy a71ea3aa38 Remove AbuseFilter::getFilter
Needs the patch in ContentTranslation first.

Depends-On: I0b74db70ad4e9768e4dcb84b9decb9c737e942e5
Change-Id: Id186ea99fcf69aa4348e404677ce5da998d83170
2020-11-19 15:11:32 +00:00
Daimona Eaytoy 210cf29658 Add an interface for exporting/importing filters
The main benefit of having a dedicated interface is that we can easily
change the output format. So we're now using a custom array without
references to the DB schema, thus making the import/export process
completely independent from the schema.

Change-Id: I4c0de41d914baf1e9a0e588bd31f95b3524a424b
2020-11-18 22:06:09 +00:00
Matěj Suchánek e7813fbafb Introduce EmergencyWatcher service
Change-Id: I45477ca84a99f620d182ef95e5627d421d38f077
2020-11-18 14:20:18 +00:00
Daimona Eaytoy ae29451ab8 Introduce a FilterCompare service
The scope is still quite limited, but as noted in a todo, we might want
to make this completely independent from the database, and add the use
case of ViewDiff.

Change-Id: Ie980fff0983b3e86037265e85da04444c809a6e8
2020-11-18 11:52:44 +00:00
jenkins-bot 914f0f4a13 Merge "Remove AbuseFilter::filterHidden and ::getGlobalFilterDescription" 2020-11-18 09:36:03 +00:00
jenkins-bot 3158a7ebc7 Merge "Remove temporary parameter" 2020-11-18 09:09:02 +00:00
Daimona Eaytoy 6376394713 Remove AbuseFilter::filterHidden and ::getGlobalFilterDescription
They've been replaced by getters in the Filter class.

Note, the Lookup is not injected in this patch because some places would
need careful thought, so it's left to do later.

Change-Id: I40b8c8452d9df741217d7fa090a5e746a2f46994
2020-11-18 08:43:22 +00:00
Daimona Eaytoy 1bcfdc3b13 Introduce a FilterValidator
This moves a lot of things away from the AbuseFilter class. There's a
nasty static dependency on ChangeTags, but it's very limited anyway, and
it's going to be fixed once T245964 is resolved.

Change-Id: Ia7df4b4d3289c2722323f59ceecf3fdd38277785
2020-11-18 01:41:31 +00:00
Daimona Eaytoy 725ec052ed Add a FilterLookup service
Some pieces of code were updated to use Filter objects, while other
places are still to be updated. We also need to change the history part
to exclude actions somehow, cleanup the ViewEdit, reduce direct DB
access or anything mentioning DB fields outside of FilterLookup, etc.

Change-Id: I42b7ded685db76eddd45e4b1336f9828cba811ce
2020-11-18 01:17:47 +00:00
Daimona Eaytoy bad5a9a29c Make AbuseFilterViewEdit work with Filter objects
This requires adjusting some methods to work with Filter objects. Some
methods and tests are left in an inconsistent/suboptimal state, plus some todos
were added, but all of this is going to be remediated in another commit.

Change-Id: Id063ee73d97c7aef56323e1457d99704f77ab943
2020-11-18 00:52:37 +00:00
Daimona Eaytoy e8947970ce Remove temporary parameter
The only usage outside of AbuseFilter (in ContentTranslation) was fixed with
Ifc9ede277791398290786cdb6743137004b5c713.

Change-Id: I22cf9c76ef3b007502045a02c82255ba6c9fd0f2
2020-11-04 15:06:32 +00:00
Daimona Eaytoy 71a61c2089 Add value objects to represent filters
This is just a start; next step is adding a factory/store method to
get/store these objects. And then use these value objects whenever
applicable.

Note: the actions-related code is still not fully implemented. This is
going to happen as part of the FilterLookup.

Change-Id: I5f33227887c035e301313bbe24d1c1fefb75bc6a
2020-11-04 12:56:14 +01:00
Daimona Eaytoy be75cf1c40 Introduce AbuseFilterParserFactory service
TODO For the future: the final directory for Parser-related classes
should be "Parser", not "ParserNS". However, moving all classes now
would make it harder to rebase changes etc.

Change-Id: Ice335f4723e74f4e5fbe8dcc76ff8ea16310962c
2020-10-31 21:19:00 +01:00
Daimona Eaytoy 1f8df50cb3 Add a service to retrieve the central DB
This is a thin wrapper around LBFactory and the global variable, that
can be injected in classes requiring it (no real class right now, but
that's going to change soon).

Also, remove some DWIM-style returns which made the code harder to
understand.

Change-Id: I1d28ad4a67f914103f3a17cda5f61b28070c7f1c
2020-10-31 12:32:46 +00:00
jenkins-bot ec5b9bef44 Merge "Add a service to retrieve the filter user" 2020-10-29 09:52:56 +00:00
Daimona Eaytoy 916234598d Simplify ViewEdit, last round
This deals with data inconsistencies in buildFilterEditor. Every
property of $row was tested in all 5 scenarios (also using Selenium) to
check when it's set. The result is in the normalizeRow method, which
aims to remove any inconsistencies, so that buildFilterEditor always
receives a "complete" row with all defaults set.

The code in buildFilterEditor is now cleaner (because there are no
isset() checks), and it gives us a unique place where we can set
defaults (rather than partly doing that in
loadRequest/loadFilterData/loadImport, and partly relying on isset).

This will be especially useful when introducing value objects to
represent filters, because now you just have to look at normalizeRow()
to tell which properties are allowed to be missing, and thus what "kind"
of filter object you need (see
I5f33227887c035e301313bbe24d1c1fefb75bc6a).

Additionally, reduce the properties that get passed around during
export/import, and make the selenium test try a roundtrip, rather than
relying on hardcoded data that may get outdated. A future patch will
refactor the import/export code.

Change-Id: Id52c466baaf6da18e2981f27a81ffdad3a509e78
2020-10-26 13:07:29 +00:00
Daimona Eaytoy cbea88f818 Add a service to retrieve the filter user
Unfortunately, this isn't using DI completely, because of the
User::newSystemUser call. I'm not even sure if we really need to call it
or we can just stick to new UserIdentityValue, but leaving like this for
now.
Also, the types were weakened to UserIdentity, so the transition is
going to be easy anyway.

Change-Id: I08f8fae0fcc622ff0ac3f86771476d06d1c18549
2020-10-26 14:06:53 +01:00
Daimona Eaytoy 0d751dde04 Cleanup for AbuseFilter class
Remove unused property, move to AbuseFilterView a method that's only
used there.

Change-Id: I16658521e32eeaafc1d601528d52bef17e1bf3b5
2020-10-25 15:55:21 +01:00
jenkins-bot 50ae561641 Merge "Simplify ViewEdit, round 2" 2020-10-25 09:10:11 +00:00
Matěj Suchánek 1445d5962a Introduce BlockAutopromoteStore service
This service is responsible for the blockautopromote feature:
(un)block autopromotion and check status.

The patch mostly moves code from static methods to the new class
and relaxes type hints (e.g. from User to UserIdentity).

Change-Id: I79a72377881cf06717931cd09af12f3b8e5f3e3f
2020-10-24 12:31:44 +00:00
Daimona Eaytoy 416dcd9ba3 Simplify ViewEdit, round 2
- Add a helper method to output an unrecoverable error, comprising a
   button to go back to the filters list;
- Move the token check to attemptSave, so to make the conditionals
  easier to read, and group errors together
- Make buildFilterEditor take an HTML parameter for the error, so the
  caller can specify whether it's error or warning
- Move the check for non-existing filters out of buildFilterEditor
- Add a bunch of typehints
- Don't set af_throttled and af_hit_count in the empty row template, but
  set af_deleted (these are only used in buildFilterEditor)
- Make AbuseFilter::translateFromHistory consistently include the af_global
  property (previously it would only be set for global filters; this error
  was introduced when first implementing global filters)
- The only user-facing change is that, when trying to use a custom
  warning/disallow message on a global filter, this is now considered a
  non-fatal error, so we now show the editing interface (and not just an
  unrecoverable error).

The next step is resolving the @todo in buildFilterEditor about null
checks.

Change-Id: I9d217dcac3f4cc0b26e53eca735cc327d5efc76d
2020-10-23 13:00:43 +00:00
Daimona Eaytoy 6724227182 Flatten the array returned by getConsequencesForFilters
There's no point in repeating the action name, because it's already used
as key. We can then flatten the array and just keep the parameters in
the third nesting level.

Change-Id: I54abcc49322f432cedd361abeedb72e067d3de41
2020-10-22 16:36:11 +00:00
Daimona Eaytoy 4c06dd52c8 Replace $wgAbuseFilterRestrictions with more specific variables
So that sysadmins can further customize the extension. It was also wrong
to use the same variable for many different things.

Note that there's no associated patch in wmf-config because we use the
defaults. However, before merging this patch, please recheck that
AbuseFilterRestrictions and AbuseFilterDisallowGlobalLocalBlocks aren't
used there (https://codesearch.wmflabs.org/operations/?q=AbuseFilterDisallowGlobalLocalBlocks%7CAbuseFilterRestrictions&i=nope&files=&repos=)

Bug: T175221
Change-Id: I7581b3ee6d9d11a6cf1599b8ff874e8c3d54adf4
2020-10-22 13:38:59 +00:00
Matěj Suchánek 85e000c6ed Add ChangeTagsManager service
This service will be resposnsible for loading
and caching change tags used by abuse filters.

Change-Id: I9a710af1dd1ae58c47de1e8509246ed929d0a662
2020-10-21 16:24:32 +02:00
Daimona Eaytoy 9bc885b6b3 Add a ChangeTagger class
The logic about action IDs and the persistent buffer is now encapsulated
inside a single service, which is a step towards getting rid of global
state in the AbuseFilter class, and reducing the responsibilities of the
Runner.

An important change made here is that we now require a LinkTarget rather
than a Title. This removes a dependency on the Title class (a monster
object), makes tests simpler, and denies the need to inject a
TitleFactory. This means living without some bits of context (e.g. we're
no longer using makeTitleSafe to ensure a valid title, and we have to
build a "prefixedtext" manually), but this shouldn't be a problem, given
that the titles are only used to create a cache key: invalid titles are
not a problem, and concatenating namespace + title should always be
sufficient.

Bug: T265370
Change-Id: Iff59cd3d889454a482a89c16691bfefcc5ec0a12
2020-10-21 13:19:30 +02:00
jenkins-bot 3b59156b4c Merge "Minor updates related to var dumps" 2020-10-19 08:27:05 +00:00