Commit graph

357 commits

Author SHA1 Message Date
Matěj Suchánek d76affb1db Move ChangeTags stuff to separate namespace
Change-Id: I6d7bed0e62f001f82c00a3528cc0018388c9c70e
2020-11-27 15:13:34 +00:00
jenkins-bot a6e96ed915 Merge "Introduce ChangeTagValidator service" 2020-11-27 15:13:06 +00:00
Matěj Suchánek 872b6118f4 Introduce ChangeTagValidator service
Just moving code around. Without a unit test because DI
coverage of change tags in core isn't available yet.

Change-Id: Iac861e1e24dae13581b8d9173357a1d6c94be88a
2020-11-27 15:11:48 +01:00
Matěj Suchánek 1ad77dc9fb Introduce EditBoxBuilderFactory service and EditBoxBuilder
It makes sense to look at this and Iedd7a5dca24 together,
as this patch itself doesn't really fix anything.

Change-Id: Ifef5266b1803d1a96489789b08d9beed044d908f
2020-11-26 14:49:04 +01:00
Daimona Eaytoy 904d9cddbb Represent Consequences with command objects
The consequence-taking logic is moved away from AbuseFilterRunner, to
dedicated classes. There's now one class per consequence, encapsulating
everything it needs to take the consequence.

Several interfaces allow customizing different types of consequences.
Every "special check" in AbuseFilter was generalized to use these
interfaces, rather than knowing how to handle each consequence.

Adding more consequences from other extensions will also be easier, and
it should happen via a hook (not a global), returning a class that
implements Consequence. The BCConsequence class was temporarily added
for legacy custom consequences.

A ConsequenceFactory class is added to instantiate consequences; this
would possibly benefit from using ObjectFactory, but it doesn't because
it would also reduce readability (although we might do that in the
future).

These classes are still not covered by unit tests, and this is left to
do for later. The new unit tests should mostly replace
AbuseFilterConsequencesTest. @covers tag were added to keep the status
quo (i.e. code that was considered covered while in AbuseFilterRunner
will still be considered covered), although we'll have to adjust them.

Change-Id: Ia1a9a8bbf55ddd875dfd5bbc55fcd612cff568ef
2020-11-25 17:35:36 +00:00
jenkins-bot 26d3abfab4 Merge "Introduce a service for saving filters" 2020-11-25 14:47:21 +00:00
Daimona Eaytoy c368575af0 Create a base interface for watchers
This will ease adding new watchers, for instance to send Echo
notifications (see T179495 and T100892).

For now, this is just boilerplate, and converting EmergencyWatcher to
the new interface.

Change-Id: I18d62aba53471202b709cdb19033b1729c5c25b4
2020-11-20 23:34:20 +01:00
Daimona Eaytoy 9595bd9da5 Introduce a service for saving filters
Change-Id: I6b7d16ad7ea1124989ed67c74413979cfd0275c4
2020-11-20 22:33:21 +01:00
Daimona Eaytoy 3f7fff56e8 Adjust code coverage
-Exclude methods and classes that cannot be meaningfully covered
-Add a simple test for AbuseFilterServices
-Exclude ServiceWiring because there's no way to tell PHPUnit it's
covered

Change-Id: I4c67b0d3fea68c7a3b3cbe01b5608f87e1b492db
2020-11-19 22:40:26 +00:00
Daimona Eaytoy eab1f13696 Make VariableGeneratorTest an integration test
It's actually using MediaWikiServices.

Change-Id: I6ec1b4723ff3f187eccf44a8b4ac286572fdfbbe
2020-11-19 13:55:16 +01:00
jenkins-bot 31f4607790 Merge "Handle DUNDEFINED in array offsets" 2020-11-18 23:30:58 +00:00
jenkins-bot 8f47259285 Merge "Add an interface for exporting/importing filters" 2020-11-18 23:13:53 +00:00
Daimona Eaytoy 3fc30021d2 Handle DUNDEFINED in array offsets
The behaviour is:
- When assigning to an undefined offset, delete the whole array and turn
it into another DUNDEFINED
- When retrieving from an undefined offset, just return DUNDEFINED.

Bug: T237214
Change-Id: I621ee7a16c90bb86a57be04e7ce0a748ecdbfcc7
2020-11-18 14:20:49 -08:00
Daimona Eaytoy 210cf29658 Add an interface for exporting/importing filters
The main benefit of having a dedicated interface is that we can easily
change the output format. So we're now using a custom array without
references to the DB schema, thus making the import/export process
completely independent from the schema.

Change-Id: I4c0de41d914baf1e9a0e588bd31f95b3524a424b
2020-11-18 22:06:09 +00:00
Daimona Eaytoy 7a24c94d6e Evaluate left-to-right when adding elements to array
Bug: T237090
Change-Id: I5fb72dec0ea12240b6563e66b69e399edc4c72d6
2020-11-18 21:25:45 +00:00
Daimona Eaytoy df017d478c Factor out another method from AbuseFilterRunner::getFilteredConsequences
This is a no-op, moving code around, introducing another distinction re
"filtering actions", which now happens in 2 steps:

 - The first step only uses "generic" information available by looking
   at enabled actions as a "group". This includes keeping only the
   longest block, and removing 'disallow' if other blocking actions are
   enabled.
 - The second step uses information that is only available after having
   "partly executed" (named "pre-checked") a consequence. For instance,
   we need to pre-check 'throttle' to see if the throttle was hit, and
   remove any other actions if not.

Change-Id: I7be5cfaa61e942a06f97ed52f50e9c8c70a120e8
2020-11-18 16:49:26 +00:00
Daimona Eaytoy ef9e828fbe Filter out actions to execute before actually executing them
This way we don't have special cases in executeFilterActions, and instead, we execute
all actions in the same place. In turn, this is going to ease the
transition to a new consequences system: next step is refactoring this
code into a service with proper DI etc.

Bug: T204447
Change-Id: I8134ecc41fbecdbed99faf406e9e3ca91b6123b9
2020-11-18 16:49:01 +00:00
Matěj Suchánek e7813fbafb Introduce EmergencyWatcher service
Change-Id: I45477ca84a99f620d182ef95e5627d421d38f077
2020-11-18 14:20:18 +00:00
Daimona Eaytoy ae29451ab8 Introduce a FilterCompare service
The scope is still quite limited, but as noted in a todo, we might want
to make this completely independent from the database, and add the use
case of ViewDiff.

Change-Id: Ie980fff0983b3e86037265e85da04444c809a6e8
2020-11-18 11:52:44 +00:00
Daimona Eaytoy 1bcfdc3b13 Introduce a FilterValidator
This moves a lot of things away from the AbuseFilter class. There's a
nasty static dependency on ChangeTags, but it's very limited anyway, and
it's going to be fixed once T245964 is resolved.

Change-Id: Ia7df4b4d3289c2722323f59ceecf3fdd38277785
2020-11-18 01:41:31 +00:00
Daimona Eaytoy 725ec052ed Add a FilterLookup service
Some pieces of code were updated to use Filter objects, while other
places are still to be updated. We also need to change the history part
to exclude actions somehow, cleanup the ViewEdit, reduce direct DB
access or anything mentioning DB fields outside of FilterLookup, etc.

Change-Id: I42b7ded685db76eddd45e4b1336f9828cba811ce
2020-11-18 01:17:47 +00:00
Daimona Eaytoy bad5a9a29c Make AbuseFilterViewEdit work with Filter objects
This requires adjusting some methods to work with Filter objects. Some
methods and tests are left in an inconsistent/suboptimal state, plus some todos
were added, but all of this is going to be remediated in another commit.

Change-Id: Id063ee73d97c7aef56323e1457d99704f77ab943
2020-11-18 00:52:37 +00:00
Daimona Eaytoy 71a61c2089 Add value objects to represent filters
This is just a start; next step is adding a factory/store method to
get/store these objects. And then use these value objects whenever
applicable.

Note: the actions-related code is still not fully implemented. This is
going to happen as part of the FilterLookup.

Change-Id: I5f33227887c035e301313bbe24d1c1fefb75bc6a
2020-11-04 12:56:14 +01:00
Daimona Eaytoy 1f8df50cb3 Add a service to retrieve the central DB
This is a thin wrapper around LBFactory and the global variable, that
can be injected in classes requiring it (no real class right now, but
that's going to change soon).

Also, remove some DWIM-style returns which made the code harder to
understand.

Change-Id: I1d28ad4a67f914103f3a17cda5f61b28070c7f1c
2020-10-31 12:32:46 +00:00
jenkins-bot ec5b9bef44 Merge "Add a service to retrieve the filter user" 2020-10-29 09:52:56 +00:00
Matěj Suchánek 77f6ecce13 Cleanup FilterProfiler API
Make FilterProfiler::getFilterProfile return stats unchanged,
in a structured way. Move computations to AbuseFilterViewEdit,
as they are only useful there. Don't return false on cache
misses, return arrays with zero values instead.

Bug: T266531
Change-Id: I8718cc31a5004340bf742315c7075e10a61fcbfd
2020-10-28 12:48:30 +00:00
Daimona Eaytoy 916234598d Simplify ViewEdit, last round
This deals with data inconsistencies in buildFilterEditor. Every
property of $row was tested in all 5 scenarios (also using Selenium) to
check when it's set. The result is in the normalizeRow method, which
aims to remove any inconsistencies, so that buildFilterEditor always
receives a "complete" row with all defaults set.

The code in buildFilterEditor is now cleaner (because there are no
isset() checks), and it gives us a unique place where we can set
defaults (rather than partly doing that in
loadRequest/loadFilterData/loadImport, and partly relying on isset).

This will be especially useful when introducing value objects to
represent filters, because now you just have to look at normalizeRow()
to tell which properties are allowed to be missing, and thus what "kind"
of filter object you need (see
I5f33227887c035e301313bbe24d1c1fefb75bc6a).

Additionally, reduce the properties that get passed around during
export/import, and make the selenium test try a roundtrip, rather than
relying on hardcoded data that may get outdated. A future patch will
refactor the import/export code.

Change-Id: Id52c466baaf6da18e2981f27a81ffdad3a509e78
2020-10-26 13:07:29 +00:00
Daimona Eaytoy cbea88f818 Add a service to retrieve the filter user
Unfortunately, this isn't using DI completely, because of the
User::newSystemUser call. I'm not even sure if we really need to call it
or we can just stick to new UserIdentityValue, but leaving like this for
now.
Also, the types were weakened to UserIdentity, so the transition is
going to be easy anyway.

Change-Id: I08f8fae0fcc622ff0ac3f86771476d06d1c18549
2020-10-26 14:06:53 +01:00
jenkins-bot 50ae561641 Merge "Simplify ViewEdit, round 2" 2020-10-25 09:10:11 +00:00
Daimona Eaytoy 18ade98339 tests: Move any profiling-related test to FilterProfilerTest
This commit removes several tests from AbuseFilterConsequences, thus
speeding it up a lot (especially because these tests were very slow,
with each test *case* taking up to 30s in the coverage job).

Everything is now covered by the new AbuseFilterFilterProfilerTest
which, although not being a pure unit test, is much much faster than
*Consequences.

Change-Id: Ic6b16d23ec99abee287f36093b8573505f9c613a
2020-10-24 18:09:26 +02:00
Matěj Suchánek 6d81fca76b Improve FilterProfiler coverage
Also improve documentation of some FilterProfiler methods.

Change-Id: I08198c643a7d2dac10e928914e8a5c7413f2543d
2020-10-24 16:23:47 +02:00
Matěj Suchánek 1445d5962a Introduce BlockAutopromoteStore service
This service is responsible for the blockautopromote feature:
(un)block autopromotion and check status.

The patch mostly moves code from static methods to the new class
and relaxes type hints (e.g. from User to UserIdentity).

Change-Id: I79a72377881cf06717931cd09af12f3b8e5f3e3f
2020-10-24 12:31:44 +00:00
Daimona Eaytoy 416dcd9ba3 Simplify ViewEdit, round 2
- Add a helper method to output an unrecoverable error, comprising a
   button to go back to the filters list;
- Move the token check to attemptSave, so to make the conditionals
  easier to read, and group errors together
- Make buildFilterEditor take an HTML parameter for the error, so the
  caller can specify whether it's error or warning
- Move the check for non-existing filters out of buildFilterEditor
- Add a bunch of typehints
- Don't set af_throttled and af_hit_count in the empty row template, but
  set af_deleted (these are only used in buildFilterEditor)
- Make AbuseFilter::translateFromHistory consistently include the af_global
  property (previously it would only be set for global filters; this error
  was introduced when first implementing global filters)
- The only user-facing change is that, when trying to use a custom
  warning/disallow message on a global filter, this is now considered a
  non-fatal error, so we now show the editing interface (and not just an
  unrecoverable error).

The next step is resolving the @todo in buildFilterEditor about null
checks.

Change-Id: I9d217dcac3f4cc0b26e53eca735cc327d5efc76d
2020-10-23 13:00:43 +00:00
Daimona Eaytoy 4c06dd52c8 Replace $wgAbuseFilterRestrictions with more specific variables
So that sysadmins can further customize the extension. It was also wrong
to use the same variable for many different things.

Note that there's no associated patch in wmf-config because we use the
defaults. However, before merging this patch, please recheck that
AbuseFilterRestrictions and AbuseFilterDisallowGlobalLocalBlocks aren't
used there (https://codesearch.wmflabs.org/operations/?q=AbuseFilterDisallowGlobalLocalBlocks%7CAbuseFilterRestrictions&i=nope&files=&repos=)

Bug: T175221
Change-Id: I7581b3ee6d9d11a6cf1599b8ff874e8c3d54adf4
2020-10-22 13:38:59 +00:00
Matěj Suchánek 93556284a0 Inject ChangeTagsManager to ChangeTagger
We decided to have the tag name provided by ChangeTagsManager,
so make ChangeTagger depend on it.

Change-Id: If3cbfd992f45651f47477031befffc0fd30f4a28
2020-10-21 16:30:43 +02:00
Daimona Eaytoy 9bc885b6b3 Add a ChangeTagger class
The logic about action IDs and the persistent buffer is now encapsulated
inside a single service, which is a step towards getting rid of global
state in the AbuseFilter class, and reducing the responsibilities of the
Runner.

An important change made here is that we now require a LinkTarget rather
than a Title. This removes a dependency on the Title class (a monster
object), makes tests simpler, and denies the need to inject a
TitleFactory. This means living without some bits of context (e.g. we're
no longer using makeTitleSafe to ensure a valid title, and we have to
build a "prefixedtext" manually), but this shouldn't be a problem, given
that the titles are only used to create a cache key: invalid titles are
not a problem, and concatenating namespace + title should always be
sufficient.

Bug: T265370
Change-Id: Iff59cd3d889454a482a89c16691bfefcc5ec0a12
2020-10-21 13:19:30 +02:00
Matěj Suchánek d91ddd2169 Cleanup filter id handling on Special:AbuseFilter/history
In particular, the interface shouldn't generate links to
"Special:AbuseFilter/history/0" (AbuseFilterHistoryPager::getTitle,
can be seen when visiting "Special:AbuseFilter/history").

Change-Id: Id3dc1bb4fc3c5e853603bf0ec04a6b1751f7d862
2020-10-10 11:40:46 +00:00
Daimona Eaytoy f0539e0c1e Represent new filters with null instead of 'new'
PHP is not strongly typed, so it's not a good idea to use scalars of
different types (here it's an integer vs the string 'new') to represent
different possibilities. This can have bad effects when type juggling
occurs, and it's also harder to figure out what the type of the
parameter can be (because a numeric ID might have been passed as a
string). Using integer vs null avoids all of this, and also allows us to
use nullable typehints.

These changes were partly copied from
If981cb35bf19a8469aa6c43c907e107cf8c65bc2 and should help with the
migration to the Filter value objects.

Change-Id: I8837d46c3c33761fea53f67b530b721dc7bd49b0
2020-10-10 12:23:50 +02:00
jenkins-bot c0defc1055 Merge "Add a new FilterProfiler service" 2020-10-10 10:08:58 +00:00
Daimona Eaytoy 9f2906e34b Reduce dependencies of AbuseFilter::saveFilter
This patch removes the dependency of saveFilter on the ContextSource
kitchen sink. It also removes some unneded dependency, and adds
$originalRow/$originalActions as parameter, rather than hacky properties
in $newRow that are easy to forget. The related test can also be greatly
simplified.

This also introduces a behaviour change: checking $newRow instead of the Request allows us
to account for values normalization done in
AbuseFilterViewEdit::loadRequest, and to also work correctly for imports
(and generally speaking, it makes the method suitable for an
AbuseFilterEdit API module, too).

Next step is moving this method to a service. Some signatures,
indenting, name choices etc. are subpar, but this is just because these
methods are temporary anyway.

Bug: T213037
Change-Id: I235b928d7b9c2ef1c46ea0bf3e3ed212500b4161
2020-10-09 11:52:02 +00:00
jenkins-bot c6ab7b8e1f Merge "Use triple equals in abuse filter parser tests" 2020-10-06 17:42:29 +00:00
Huji Lee d07717dd0d Use triple equals in abuse filter parser tests
The only exception is mwexamples-comparisons.t which intentionally
includes examples with = and == to test the "weak" version of the
comparison operator.

Bug: T262063
Change-Id: I6f92aadc69489da481a606bfda89617b8efbb261
2020-10-06 12:29:47 -04:00
Daimona Eaytoy bc9898f1a1 Add a new FilterProfiler service
Change-Id: Ib66c42ac220731f4e1da9ee6cfb5290759dd6494
2020-10-04 22:00:57 +00:00
Daimona Eaytoy 6c8a29698b Add test traits for uploads and account creation
Ideally, this might live in MediaWikiIntegrationTestCase. For the
createaccount one, AuthManager should also provide a method to log the
creation, because currently we are forced to copypaste that code here.

 - Add the missing tests for 'upload' in RCVariableGenerator, and adjust
the existing ones (delete file afterwards, more tablesUsed, use the
right extension).

 - Exclude from the coverage report a couple of lines which should
theoretically be unreachable. Escalate logging to WARN level, where it's
more likely to be spotted.

 - Remove an unused method (RCVariableGenerator::newFromID). This denies
   the need to maintain and cover it. We also don't want this generator
   to act as a factory.

Overall, this change brings the coverage for RCVariableGenerator to 100%

Bug: T201193
Change-Id: I425c3d9f6800f74eb6e4eda483b90cfb3bbbcb51
2020-10-04 13:16:58 +00:00
Daimona Eaytoy 2e13d58c74 Add tests for retrieving RC variables
This was also long overdue. Also fix a bug that caused page creations to
not be shown when examining past edits (using rc_last_oldid doesn't work
for page creations).

Bug: T201193
Bug: T262903
Change-Id: I5f7a994add12332c950904146248c5de7c2beee5
2020-10-04 12:43:04 +00:00
jenkins-bot a9654c3ab3 Merge "Refactor AbuseFilterView instantiation" 2020-10-04 12:28:24 +00:00
Matěj Suchánek eb81b92c06 Refactor AbuseFilterView instantiation
- Make a separate method which determines the view
  to be shown from subpage syntax and test it.
- Reduce circular dependency between SpecialAbuseFilter
  and AbuseFilterView. Use params to transfer information
  to views.

Change-Id: Ib9442ea5f9990a5c48f9b9e04055aa22bf7e456e
2020-10-04 13:15:04 +02:00
Daimona Eaytoy 60519e1886 updateVarDumps: avoid using services in the constructor
This is I4df27f3d02432c201c04d9fa118f0129b0a79778 striking again. Fool
me once, shame on thee, fool me twice...

Change-Id: Icea025a2c81e3b413b7bd9ece52866aeaf42937d
2020-10-03 23:23:37 +00:00
Daimona Eaytoy 7bc5248ed7 Selenium tests: wait for clickable button
It's possible that we try clicking this button before it's ready. This
theory is hard to verify because I get no problem locally, but this
shouldn't hurt.

Also specialize input selectors for a couple elements that might not be
univocally determined within the page.

Change-Id: Ida65c3c5fd4d8b3b35ecbee7e99977c71c7c4b96
2020-10-01 00:51:35 +02:00
jenkins-bot fa412f4e7e Merge "Rewrite the VariableHolder code to translate deprecated variables" 2020-09-30 09:10:37 +00:00