Commit graph

790 commits

Author SHA1 Message Date
Umherirrender da7683bcbc tests: Improve tests for postgres
Change-Id: I9720b6c7d096ae8415c00eb0ac1ddc461ea0a8dc
2022-07-09 21:40:27 +00:00
jenkins-bot c3c70f7fa0 Merge "FilterProfiler: use WRStats" 2022-07-06 00:05:15 +00:00
Tim Starling cdf2f474e8 FilterProfiler: use WRStats
A new core facility written for this use case.

Bug: T310662
Depends-On: I26b1cdba0a06ad16ad8bb71b455e1b6180924d17
Change-Id: I2b902d034a8c3308c0ba9878b69e873ca8fbda52
2022-07-06 09:35:08 +10:00
Matěj Suchánek e7492a230f Replace unnecessary use of User
In action=abusefilterunblockautopromote, leave UserIdentity
instantiation to the parent. Note that this changes the "code"
in the response from "baduser_user" to "baduser".

Change-Id: I97d2bf3fa3c5486e461823f840cad2763e1bcfea
2022-07-02 23:58:08 +00:00
Matěj Suchánek 799e1db093 Convert remaining permissions checks to use Authority
Change-Id: I5e996cac37bc806db6c3d7ad5c666a606cd79236
2022-07-02 14:49:47 +02:00
DannyS712 139ca18efe Migrate AbuseFilterPermissionManager to authority
Almost all callers already provide an Authority in the form
of a User object, so mostly just need to change the typehints

Depends-On: I58661943c7e1acb6ff09798ee1a30be0fde3f459
Change-Id: I2ad86859c8194c14d7331f58db62b7cff4698085
2022-07-01 06:58:17 +00:00
jenkins-bot af93d83b51 Merge "tests: Improve RCVariableGeneratorTest" 2022-06-30 07:29:48 +00:00
Umherirrender b833d740fd tests: Improve RCVariableGeneratorTest
- use unique ids to find rc entry, to support parallel unit tests and
rdbms where the auto increment value must not increase in time
- Change from Title::newFromText to Title::makeTitle to avoid parsing
the title
- Pass the title to editPage() to avoid reparse of the title
- Use assertSame to compare values

Change-Id: I455b4412a6669475463dee7dea0969ae1cbd8ebb
2022-06-29 22:34:44 +02:00
jenkins-bot 8d4c5d4d33 Merge "Use LinkTarget in ConsequencesExecutor" 2022-06-29 08:52:37 +00:00
jenkins-bot 2314785568 Merge "tests: Avoid Title::newFromText/title parsing" 2022-06-28 21:43:11 +00:00
jenkins-bot 00944567c6 Merge "tests: MWTimestamp::setFakeTime is reset by core" 2022-06-28 21:43:09 +00:00
Matěj Suchánek 4beca85154 Compute user and page age relative to recent change timestamp
These are apparently the only two variables for which we can
quickly determine their value in such simple way.

Later, we can also try it for recent contributions.

Bug: T102944
Change-Id: Iecfa9e5c5ba8c078691334b676cc6f289790cb74
2022-06-28 20:53:33 +00:00
Umherirrender 32a97e8d15 tests: MWTimestamp::setFakeTime is reset by core
It is in MediaWikiTestCaseTrait since 438b392

Change-Id: Ib89406fdbad0c9fecada50c8f1ee45e27d17c522
2022-06-28 20:48:31 +00:00
Umherirrender 637a88316b tests: Avoid Title::newFromText/title parsing
Using Title::newFromText is parsing the string, which is expensive.
Just use Title::makeTitle when the result is known.
editPage() can take a Title or WikiPage instead of a string, avoid
creation of Title there.
The default ns on editPage() is only needed when giving a string

Change-Id: Ie303b9e6d6b8d6ac80286059f8e86bfc76b779af
2022-06-28 22:46:45 +02:00
Matěj Suchánek b381636974 Extend RCVariableGeneratorTest
Make an edit, retrieve the recent change and test computed variables.

Change-Id: I04beed0b1f7c5adb47e71fd9b03102cb23838e16
2022-06-28 19:50:11 +00:00
Matěj Suchánek 7ae2060b27 Avoid array to object cast in filterToDatabaseRow
Both callers immediately call get_object_vars
to cast it back to array. Avoid this roundtrip.

Change-Id: I6525d76f8a03a4d28c2b50b580c539affe98064f
2022-06-28 18:46:28 +00:00
Umherirrender 20fd8f7b07 Use LinkTarget in ConsequencesExecutor
The Parameters class already only needs a LinkTarget

Change-Id: I4e8e1d7c92f41502a084be3359b97e0d434f08c0
2022-06-28 19:46:50 +02:00
Umherirrender 30fefb75bf Use UserIdentity in ConsequencesExecutor
Change-Id: I281a30610595ed3e984f43aa747eff37abe72939
2022-06-27 22:05:18 +02:00
Daimona Eaytoy f33bc5868c Set the 'timestamp' var in addGenericVars
This was most definitely my intention when I introduced the concept of
"generic vars", so it's a bit surprising to discover, 3.5 years later,
that the timestamp isn't computed there.

Also make the timestamp always be a string for consistency, since that's
the type documented on mw.org. I've manually checked all filters on
Wikimedia wikis using the timestamp variable, and added explicit int
casts where needed (although I think they'd still work due to implicit
casts).

Change-Id: Ib6e15225dd95c2eead7e48c200d203d6918e0c18
2022-06-26 14:49:40 +02:00
Umherirrender 3d3c45f348 tests: Mock WikiPage in unit test
Bug: T297688
Change-Id: Ic1655141564f02530b1ae6b625a1d3e261a00304
2022-06-24 22:22:24 +02:00
Matěj Suchánek 40564ca635 Remove $info argument from ReversibleConsequence::revert
It was a temporary catch-all variable, but we can replace it
(and probably won't need it).

Change-Id: Ie1a64455c47445050bd83c853b3cafd283d5d020
2022-06-08 11:59:18 +02:00
jenkins-bot 1a6985469b Merge "Inline/simplify smaller pieces of duplicate/complex PHP code" 2022-06-03 20:38:22 +00:00
Thiemo Kreuz bbded6231c Inline/simplify smaller pieces of duplicate/complex PHP code
Change-Id: I59d0f17b77c8c3d47bc532bdefd9d8c0883f180b
2022-06-03 21:04:38 +02:00
jenkins-bot bb94c0914c Merge "Add support for regex string replacements." 2022-05-31 14:54:33 +00:00
Daimona Eaytoy a46db47bd5 Fix validation for ip_in_ranges
We want to make sure that all parameters are valid regardless of whether
there's a match.

Also make the minimum number of parameters = 2, so it's easier to switch
between this function and ip_in_range.

Change-Id: I141558a7ef4533485e315b3d93ea9b64f0959db7
2022-05-21 15:39:21 +02:00
fossifer b1739a588f Add ip_in_ranges function
Added support for ip_in_ranges which allow multiple ranges to be
checked at the same time. If the IP is in any of the ranges, the
function returns true.

Bug: T305017
Change-Id: Ic75c87ecd4cacf47ce2ff1b04173405230ff81d0
2022-05-11 12:27:16 +08:00
proc 1d1215bafb
Add support for regex string replacements.
Bug: T285468
Change-Id: I25f8ad1b58cc10f4c6f6ef5ebab99fe58ec71b1e
2022-04-20 18:38:24 +01:00
Daimona Eaytoy 59eb3b70fb Inject dependencies into the authentication provider
- Define it with the extension.json key, instead of using the
  registration callback
- Inject the services it needs
- Replace direct User instantiation with UserFactory
- Move log subtypes to extension.json as well

Change-Id: I86a761c7fa844b1f417b974798373622a15f6411
2022-04-09 18:44:25 +02:00
Matěj Suchánek 686d7ea88c Use RestrictionStore instead of deprecated method
Also restructure the unit test a bit.

Change-Id: If5ce26f1bc4efdb29653aed3fc47335dddc1e44c
2022-03-29 16:11:55 +02:00
jenkins-bot bd309bb220 Merge "Clean up test files" 2022-03-25 21:28:10 +00:00
jenkins-bot def507f6d3 Merge "Refactor ConsequencesExecutor to process consequences in more steps" 2022-03-23 09:06:55 +00:00
stang f20699935a Replace (error|warning|success)box in test cases of AbuseFilter
Bug: T304243
Change-Id: Iae2b968fc4c84bf360489ec8ff3491afd476c898
2022-03-20 20:16:31 +00:00
Daimona Eaytoy 8ee9a21750 Clean up test files
Convert a few integration tests to unit tests now that it's possible,
split the AbuseFilterSaveTest file into three different classes.

Change-Id: Ia2c0d7ab878b20a89324336a532abdc44f1e6b74
2022-03-20 17:40:49 +00:00
Daimona Eaytoy 2de5fce177 Refactor ConsequencesExecutor to process consequences in more steps
Introduce shorter methods, one for each steps, so that it's easier to
understand what the code is doing and figure out if the order makes
sense. The ConsequencesExecutor test is now a proper unit test. Also
simplify AbuseFilterConsequencesTest, removing old/wrong logic and
fixing two expected values that were actually wrong (but worked because
of the aforementioned wrong logic).

The only functional changes should be:
 - We pick the longest block *after* checking the ConsequenceDisabler
   consequences, so e.g. if a filter has a long block + warn and another
   filter has a shorter block, we still keep the second one if warn will
   disable the block.
 - Remove disallow in presence of dangerous actions after checking
   ConsequenceDisabler's and deduplicating blocks. Otherwise we may
   remove disallow for filters where block (etc.) doesn't end up being
   disabled. We may also want to consider not removing disallow at all,
   now that messages are customizable.

Bug: T303059
Change-Id: If00adbf2056758222eaaea70b16d3b4f89502c20
2022-03-19 15:49:36 +00:00
Alexander Vorwerk 4aedfe8d91 Use updated ObjectFactory namespace
Change-Id: I99c5e5664d2401c36a9890f148eba7c25e6e8324
2022-03-09 22:17:07 +00:00
jenkins-bot 894b94bf7d Merge "Add logging when the 'block' action fails" 2022-03-07 09:26:42 +00:00
jenkins-bot dad1fff238 Merge "Overhaul throttle identifiers" 2022-03-06 13:50:43 +00:00
Daimona Eaytoy a0fd0bae01 Overhaul throttle identifiers
- Use a /64 range for IPv6 instead of /16.
- Fix a curious and serious bug for IPv6, where grouping by range
  would only use the first (!) number of the IP address, due to the
  'v6-' prefix returned by IP::toHex.
- Fail hard if the identifier is unknown -- it's not something that's
  supposed to happen.
- Include the type name in each identifier, instead of prefixing all
  type names to all identifiers. This makes it easier to understand the
  parts of the key.
- Test the whole lot.

Bug: T211101
Change-Id: I54c4209f2f0d5a4c5e7b81bed240ca3e28a2ded7
2022-03-06 13:31:06 +00:00
daniel a512ed31a7 Rename private assertion method
assertStatusMessage is being added to MediaWikiTestCaseTrait, rename
a method of the same name in FilterValidatorTest to avoid conflicts.

Change-Id: I642a3b620ab4d8ad620f7a1253fed98d6796883d
NeededBy: Ic01715b9a55444d3df6b5d4097e78cb8ac082b3e
2022-03-05 21:48:18 +00:00
Daimona Eaytoy 496c2ee370 Add logging when the 'block' action fails
Also avoid using User, use Authority instead.

Bug: T303059
Change-Id: I419ab3726d95ef600e2aa14dca5fa14066d245e3
2022-03-05 19:12:53 +00:00
Daimona Eaytoy b5c22f2b77 Improve wording for throttled filter warnings
List which actions were disabled, or explicitly say that no actions were
disabled if that's the case. Also avoid the word "throttle" in messages
as it may be hard to translate. Also don't suggest optimizations to the
filter conditions -- unoptimized rules have nothing to do with a filter
being throttled.

Bug: T200036
Change-Id: Id989fb185453d068b7685241ee49189a2df67b5f
2022-02-22 11:10:19 +00:00
Daimona Eaytoy 167f6cb642 Introduce ActionSpecifier
This is a plain value object that represents the action being filtered,
replacing associative arrays that were being used up to this point.

We should now check whether it's possible to make it not require an
accountname (which complicates things), and then use it in related
classes as well, e.g. Parameters.

Change-Id: I9550c14819b600c97c46b632cc1c2d447972d69c
2022-02-18 11:30:56 +00:00
Huji 52827acbab Make rmspecials preserve whitespace
The existing filters on WMF wikis has been changes such that calls
to rmspecials() are now rmspecials(rmwhitespace()) to ensure no change
is made in behaviour. Filter admins can change this back if filter is
not meant to trigger when part of the input is contains spaces.

Bug: T263024
Change-Id: Idde09b50fb8eda357afbedc1199a5483fa8217c1
2022-02-06 06:07:46 +00:00
Kosta Harlan bc19e738f6 selenium: Run test suites concurrently
Bug: T226869
Change-Id: I0d7435bd6ee9b0893ea387722eaa18a6e120c67a
2022-01-06 15:09:09 +00:00
Alexander Vorwerk ccb85c9a55 Avoid using WikiPage::factory()
WikiPage::factory() is deprecated since 1.36 and should be replaced
with WikiPageFactory::newFromTitle().

Bug: T297688
Change-Id: I85d3566519ab977aad8c517cc48fc8c271e5589a
2021-12-17 09:22:26 +00:00
jenkins-bot 13db4c34e5 Merge "MediaWikiTestCase -> MediaWikiIntegrationTestCase" 2021-10-12 02:16:38 +00:00
Alexander Vorwerk 7cc7cfa806 MediaWikiTestCase -> MediaWikiIntegrationTestCase
MediaWikiTestCase has been renamed to MediaWikiIntegrationTestCase in 1.34.

Bug: T293043
Change-Id: I6e7c5a34ae49d56a8e7b5ac7d06fa9c0283bed5e
2021-10-11 23:32:14 +02:00
jenkins-bot a332b3ff0f Merge "Remove afl_filter entirely" 2021-09-25 01:39:08 +00:00
Daimona Eaytoy e8471a717c Add method to properly check visibility of AbuseLog entries
This replaces the previous pattern of callers having to use
RevisionLookup if the result was 'implicit'. Also, in some cases where
we were just hiding things if the visibility was !== true, properly
handle the implicit case by using the new method. Make the new method
return string constants rather than bool|string.

The new method also fixes some potential info leaks which happened when
the row was hidden, the user could view suppressed AbuseLog entries, but
the associated revision was also deleted and the user couldn't see it
(this shouldn't be relevant for WMF wikis since AF deletion is
oversight-level).

Also add a bunch of tests for the various cases to ensure we don't
regress again.

Bug: T261532
Change-Id: I929f865acf5d207b739cb3af043f70cb59243ee0
2021-09-25 00:08:33 +00:00
Daimona Eaytoy dae374aec2 Remove afl_filter entirely
As per T220791, the old schema and the flag can be removed in 1.38.

Bug: T220791
Change-Id: Ic6b1c8a22d17a301faf32d2e23778d90c41c39de
2021-09-18 11:06:10 +00:00
Daimona Eaytoy b2dc2c4dd8 Refactor ParserStatus
ParserStatus is now more lightweight, and doesn't know about "result"
and "from cache". Instead, it has an isValid() method which is merely a
shorthand for checking whether getException() is null.

Introduce a child class, RuleCheckerStatus, which knows about result and
cache and can be (un)serialized.

This removes the ambiguity of the $result field, and helps the
transition to a new RuleChecker class.

Change-Id: I0dac7ab4febbfdabe72596631db630411d967ab5
2021-09-17 11:25:54 +00:00
jenkins-bot 5475cae543 Merge "Rename AbuseFilterVariableGeneratorTest" 2021-09-15 17:10:27 +00:00
Matěj Suchánek 3ffbfb63f2 Rename AbuseFilterVariableGeneratorTest
We don't need the AbuseFilter prefix anymore.

Change-Id: Ia54016000895fd22dec5f397ab2d42d20bfd1816
2021-09-15 18:17:36 +02:00
Daimona Eaytoy 7c26c4b8d5 More cleanup for parser-related classes
Change-Id: I6a2bbf519e1d5c6fe2778f69624bd80b9ea1ef86
2021-09-10 12:50:20 +00:00
Daimona Eaytoy a722dfe1a4 Rename ParserFactory -> RuleCheckerFactory
The old parser now has the correct name "Evaluator", so the
ParserFactory name was outdated. Additionally, the plan is to create a
new RuleChecker class, acting as a facade for the different
parsing-related stages (lexer, parser, evaluator, etc.), which is what
most if not all callers should use. The RuleCheckerFactory still returns
a FilterEvaluator for now.
Also, "Parser" is a specific term defining *how* things happen
internally, whereas "RuleChecker" describes *what* callers should expect
from the new class.

Change-Id: I25b47a162d933c1e385175aae715ca38872b1442
2021-09-08 21:59:34 +02:00
Daimona Eaytoy 357ddd498c Clean up / simplify parser-related classes
Remove unnecessary setters, injecting everything in the constructor.
These were leftovers from before the introduction of ParserFactory.
Remove public access to the conds used, include the information inside
the returned ParserStatus instead, and consequently simplify callers.

Change-Id: I0a30e044877c6c858af3ff73f819d5ec7c4cc769
2021-09-08 13:41:52 +02:00
Daimona Eaytoy f8e9ac7e2a Rename AbuseFilterCachingParser -> FilterEvaluator
It's an evaluator, not a parser.

Change-Id: Ib6d33e8423ea72709cf5a33f4397ba33e352ea80
2021-09-08 13:40:47 +02:00
Daimona Eaytoy 6684ea6450 Remove AFPTransitionBase
Also cleanup the mPos hack in the CachingParser.

Change-Id: Ib5693802a3ceb80cb736880ed65e27340abef689
2021-09-06 19:33:48 +00:00
jenkins-bot 199cf1edf8 Merge "Add a static analyzer for the filter language" 2021-09-03 19:51:58 +00:00
Matěj Suchánek 0af21948fc Replace WikiPage::factory in non-test code
Change-Id: I1442ca6603ce5151b98fc88cd84c25af0f34e4f6
2021-09-01 04:55:25 +00:00
Daimona Eaytoy 86257d825c tests: Use DBConnRef, not IDatabase, as retval of getConnectionRef
So that the method can be typehinted in core.

Also add phan-var to fix broken master build due to typehint additions
in core.

Change-Id: I4a072e00ffeeb437753fc3d3c1f15de9929df510
2021-08-31 21:45:10 +02:00
Sorawee Porncharoenwase 320e3d696f Add a static analyzer for the filter language
This commit adds a class AFPSyntaxChecker which can statically analyze
a filter code to detect the following errors:

- unbound variables (which comes in two modes: conservative and liberal,
  default to conservative)
- unused variables (disabled by default for compatibilty)
- assignment on built-in identifiers
- function application's arity mismatch
- function application's invalid function name
- non-string literal in the first argument of set / set_var

The existing parser and evaluator are modified as follows:

- The new (caching) evaluator no longer needs to perform variable
  hoisting at runtime.
  - Note that for array assignment, this changes the semantics.
- The new parser is more lenient, reducing parsing errors.
  The static analyzer will catch these errors instead, allowing us
  to give a much better error message and reduces the complexity of
  the parser.
  * The parser now allows function name to be any identifier.
  * The parser now allows arity mismatch to occur.
  * The parser now allows the first argument of set to be any expression.

Concretely, obvious changes that users will see are:

1. a := [1]; false & (a[] := 2); a[0] === 1

   would evaluate to true, while it used to evaluate to the undefined value
   due to hoisting

2. f(1)

   will now error with 'f is not a valid function' as opposed to
   'Unexpected "T_BRACE"'

3. length

   will now error with 'Illegal use of built-in identifier "length"'
   as opposed to 'Expected a ('

Appendix: conservative and liberal mode

The conservative mode is completely compatible with the current evaluator.
That is,

false & (a := 1); a

will not deem `a` as unbound, though this is actually undesirable because
`a` would then be bound to the troublesome undefined value.

The liberal mode rejects the above pattern by deeming `a` as unbound.
However, it also rejects

true & (a := 1); a

even though (a := 1) is always executed. Since there are several filters
in Wikimedia projects that rely on this behavior, we default the mode
to conservative for now.

Note that even the liberal mode doesn't really respect lexical scope
appeared in some other programming languages (see also T234690).
For instance:

(if true then (a := 1) else (a := 2) end); a

would be accepted by the liberal checker, even though under lexical scope,
`a` would be unbound. However, it is unlikely that lexical scope
will be suitable for the filter language, as most filters in
Wikimedia projects that have user-defined variable do violate lexical scope.

Bug: T260903
Bug: T238709
Bug: T237610
Bug: T234690
Bug: T231536
Change-Id: Ic6d030503e554933f8d220c6f87b680505918ae2
2021-08-31 03:28:24 +02:00
Daimona Eaytoy 704364a5e7 Move parser exceptions to specific namespace and rename them
Create a dedicated "Exception" sub-namespace and remove the "AFP"
prefix, a leftover from the pre-namespace era.

Change-Id: I7e5fded9316d8b7d1628bc1a6ba8b1879ac901e1
2021-08-29 23:38:31 +00:00
jenkins-bot 9b93b0256a Merge "Avoid passing invalid offset to mb_strpos" 2021-08-18 18:45:12 +00:00
Daimona Eaytoy e9795468c4 Switch filterable actions hooks to the new system
Bug: T261067
Bug: T211680
Change-Id: I0e7e4a48b56c3e5fde56f50693fd0cdc19c30dd0
2021-08-16 14:18:56 +00:00
TChin bfa72b9caf Use MovePageFactory
Bug: T252934
Change-Id: I39440ef05d9318f9ab4abd34990887971197a045
2021-08-10 16:31:05 -04:00
Matěj Suchánek ace6f652af AbuseFilterConsequencesTest: Don't call non-static method statically
Change-Id: I0b4ed2f456bf4a52756eb0b98a29994a4a53812c
2021-07-30 01:24:15 +00:00
libraryupgrader 5377ebe819 build: Updating dependencies
composer:
* mediawiki/mediawiki-codesniffer: 36.0.0 → 37.0.0

npm:
* postcss: 7.0.35 → 7.0.36
  * https://npmjs.com/advisories/1693 (CVE-2021-23368)

Change-Id: I2b382f3bb236fb44eb24c6a257b13b8fd886541c
2021-07-21 18:51:18 +00:00
DannyS712 745d911d68 Add tests for afl_rev_id being set
Regression tests to make sure T286140 does not
happen again.

In the process, discovered what caused that bug
with afl_rev_id not being set: EditRevUpdater::updateRev()
compares the WikiPage given in the PageSaveComplete hook
to the one given to it by AbuseFilterHooks from
onEditFilterMergedContent, and compares the two using
`===`, meaning that they must refer to the same underlying
object. That bug was caused because AbuseFilterHooks
changed to providing a different object, despite still
referring to the same underlying page.

We should probably change that behavior in EditRevUpdater,
but for now updated AbuseFilterConsequencesTest to pass
the same object around by using RequestContext::setWikiPage()
and providing the WikiPage object to
MediaWikiIntegrationTestCase::editPage().

Bug: T286140
Change-Id: I6562f513c463538af6b59b12a64564b254024613
2021-07-04 08:04:06 +00:00
Daimona Eaytoy 069fa064f5 Avoid passing invalid offset to mb_strpos
Bug: T285978
Change-Id: I3d100fd05f34fe3b01ecbbce5361badc613f9406
2021-07-02 14:07:46 +00:00
DannyS712 47f861b6f6 Pass a user to WikiPage::prepareContentForEdit()
Bug: T285447
Change-Id: I4d277419106c3af5222377a863c80dd866ba188b
2021-06-24 04:01:33 +00:00
jenkins-bot 805ea5b4ff Merge "User mock must return Block instance from getBlock." 2021-06-08 19:14:52 +00:00
jenkins-bot 405e6a7771 Merge "selenium: Update wdio-mediawiki" 2021-06-08 17:35:08 +00:00
daniel 54285fe984 User mock must return Block instance from getBlock.
Change-Id: I569e91dd07b8f89af42344b6d6df87560dcb6bbe
Needed-By: T271494
2021-06-08 17:12:48 +02:00
jenkins-bot 997e665530 Merge "Don't use p class="success" for success messages" 2021-06-04 08:59:58 +00:00
Roman Stolar c347a0d33e Update DatabaseBlock construct option 'by' to use User Identity only
Bug: T283641
Change-Id: I16b07e18441143a5d5d470eef4c28037a150b605
2021-05-31 17:36:55 +03:00
sahil 5fca7c2f51 selenium: Update wdio-mediawiki
wdio-mediawiki v1.1.1:
- Includes wdio-defaults.conf.js file that vastly simplifies wdio.conf.js.
- Replaces @wdio/spec-reporter with @wdio/dot-reporter.
- Introduces video recording.

Bug: T283597
Change-Id: I603626d20eb060ba6d90b1a91d49afe13e29d945
2021-05-28 01:04:55 +05:30
jenkins-bot 7e8cf0d101 Merge "Selenium: update README.md file" 2021-05-18 15:45:15 +00:00
jenkins-bot 7d0e50a2cd Merge "Use FauxRequest::setUpload in AbuseFilterUploadTestTrait::doUpload" 2021-05-15 12:22:10 +00:00
sahil 96706a53fd Selenium: update README.md file
Bug: T282237
Change-Id: I93d7538bf191e42460e5d54532f6775a05dac661
2021-05-07 18:45:07 +05:30
Daimona Eaytoy 58ad3d1542 Replace deprecated User::getEffectiveGroups
Bug: T281824
Change-Id: I5487d143277a44742048668c920bbad57ebe6af1
2021-05-06 15:35:35 +02:00
libraryupgrader 06cdddc9d0 build: Updating composer dependencies
* mediawiki/mediawiki-codesniffer: 35.0.0 → 36.0.0
* php-parallel-lint/php-parallel-lint: 1.2.0 → 1.3.0

Change-Id: I92d6f6d6f817765df24f845103a489624f4290f2
2021-05-02 06:41:54 +00:00
Umherirrender 1fa7a83f60 Use static closures where safe to use
Created by I25a17fb22b6b669e817317a0f45051ae9c608208

Change-Id: I533690311ca559685de8a4bf123348c9bcfa5931
2021-04-30 20:55:35 +02:00
mainframe98 a32d483ef4 Don't use p class="success" for success messages
These are part of legacy styles and aren't provided by all skins.
Using Html::successbox abstracts the classes away.
Internally that uses div class="successbox" instead.

Bug: T280766
Change-Id: I0cca59e2f391510095c2c6fb187ace5e91fdde8b
2021-04-30 18:19:31 +00:00
jenkins-bot 0dc93136d6 Merge "Improve test coverage of API modules" 2021-04-18 16:03:25 +00:00
Matěj Suchánek a2ee8c41e2 Improve test coverage of API modules
Also solve one a TODO.

Change-Id: I61a38f3c741274f00ad0ad4789106a943daef222
2021-04-18 10:37:38 +02:00
jenkins-bot 5cd39a51fa Merge "Remove the old parser" 2021-04-17 15:21:54 +00:00
Daimona Eaytoy f67c2d5434 Remove deprecated $wgAbuseFilterCustomActionsHandlers
Extensions should now specify custom actions using the
AbuseFilterCustomActions hook.

Change-Id: Id21640d406b18c627eedff39d3f246cf21e042b3
2021-04-11 14:49:50 +00:00
Daimona Eaytoy f8438a4647 Remove the old parser
All methods were moved to the new parser. Tests and other pieces were
adjusted to expect just a single parser. There are still some TODOs
(remove AFPTransitionBase, remove $this->mCur), but these are left for
another commit.

Note that the new parser was not renamed: this is because the names are
wrong anyway (CachingParser is more of an Evaluator than a Parser, and
AFPTreeParser is the real parser, and should be renamed as well).

NOTE to reviewers: this patch looks quite big, but if you diff the old
parser with the new version of the CachingParser, you'll notice that the
diff is actually small, since everything was basically copied verbatim.

Bug: T239990
Change-Id: Ie914ef64c70503a201b4d2dec698ca2fa8e69b10
2021-04-09 13:23:07 +00:00
Umherirrender 32f7ae140e Use ::class for class name
This works also for non-existing classes,
because it is resolved on compile time

Change-Id: Ia3a1484c9c4f46a128c367ddd057c41dd560111d
2021-04-08 20:54:48 +02:00
Daimona Eaytoy 3e2153b86b Update userCanViewRev to use Authority
Change-Id: Ia10acf499ce33af03eeea45e34779a00e6628fe1
2021-04-07 13:55:10 +02:00
DannyS712 5d8ac68310 Convert AbuseFilterDBTest to pure unit tests
No integration needed, use a mock user.

Change-Id: I206d019aec626e6e4c16de10ecf30a29d5ab12e5
2021-04-06 16:28:35 +00:00
jenkins-bot e616e5b71c Merge "selenium: Remove ChromeDriver dependency" 2021-03-26 19:02:24 +00:00
daniel 65c5fd6b51 Construct UserIdentityValue without actorId
The actorId parameter to the UserIdentityValue constructor has been
deprecated.

Change-Id: I4a22e761276a9fefa15c7b1554a0d03980d0c663
Needed-By: I9925906d11e47efaec3c1f48d5cb3f9896a982c1
2021-03-26 11:00:56 +01:00
daniel e954ecfe19 Fix UserIdentityValue constructor call in test
Change-Id: If905e442444d319114e86ce78007bdc6dbc9c9f2
Needed-By: I9925906d11e47efaec3c1f48d5cb3f9896a982c1
2021-03-25 15:13:15 +01:00
jenkins-bot 92ffc1a7ed Merge "Don't directly instantiate PageEditStash" 2021-03-25 10:51:09 +00:00
Petr Pchelko e92d74ad51 Don't directly instantiate PageEditStash
Needed-By: I7164b914299441bd0f82e764252c8b5d30b45fbe
Change-Id: Iab96a35be8f50fdbc66194bd8956d98b5b6b0032
2021-03-24 21:03:46 -06:00
Željko Filipin 682b76662e selenium: Remove ChromeDriver dependency
Since WebdriverIO v5, Puppeteer is available as a Chromedriver
alternative.

Puppeteer is bundled with WebdriverIO. Chromedriver needs to be installed
(and started/stopped) separately. Getting rid of Chromedriver simplifies
our documentation, among other things.

The commit updates tests/selenium/wdio.conf.js to use Puppeteer.

Bug: T269566
Change-Id: Ib2a547792a34e6d40137432f7800b5f71c254c36
2021-03-23 18:03:29 +00:00
jenkins-bot fa8358ce0c Merge "Replace RecentChange::getPerformer with RecentChange::getPerformerIdentity" 2021-03-18 14:15:11 +00:00
Vadim Kovalenko 85be3c57bc Replace RecentChange::getPerformer with RecentChange::getPerformerIdentity
Bug: T276412
Change-Id: I8a55bd5cb17bbc259ec36c40261058e0b46ee4a6
2021-03-15 16:57:40 +02:00
jenkins-bot c0cb48abaf Merge "UserIdentityValue: do not provide actor ID" 2021-03-13 21:30:27 +00:00
jenkins-bot 1c5e5eb1e2 Merge "Create distinct builders for plain and ace editor" 2021-03-13 13:36:53 +00:00
daniel 05e416ef11 UserIdentityValue: do not provide actor ID
Depends-On: Ic2ca6d489db821fc2334e53bf2496c7b0d3ea5b1
Change-Id: I76e0a5b0dd68b763bdd0c5e4fe5d09f98e350ce1
Needed-By: I9925906d11e47efaec3c1f48d5cb3f9896a982c1
Bug: T274179
2021-03-12 16:38:51 +01:00
Daimona Eaytoy c5d19577a4 Fix method names of hook interfaces
The hook names contain a dash, which is mapped to an underscore by the
hook runner (see Ie8c8fb603b33ff95c8f8d52f392227f147c528d8), and the
previous method names weren't matching this.

Follow-up: Ic5c82a367e34135bbc0f00ece5aeef4f2d92881b

Change-Id: Ie80b62c49b2f4aaea49d5a1883f513348689d16a
2021-03-09 17:03:14 +00:00
Vadim Kovalenko 62376f437e Replace UserIdentity::getUserId with ::getId
Bug: T275482
Change-Id: Ie9fd52005ca1eb264dc791a2f87d7308a9e8810e
2021-03-08 09:18:49 -05:00
Daimona Eaytoy 6ba8e93537 Create distinct builders for plain and ace editor
Change-Id: I9d2b7572fed6e0b3660d3b0d5dad324d6b75fde9
2021-03-08 09:44:58 +00:00
jenkins-bot 12f4e81964 Merge "Simplify AbuseFilterBlockTest" 2021-03-06 09:23:26 +00:00
Daimona Eaytoy 92ecccbdc7 Simplify AbuseFilterBlockTest
Requires injecting a temporary block factory, and excluding
ManualLogEntry::insert from the test, but it's now much cleaner and
quicker.

It still cannot be a unit test due to the usage of User.

Change-Id: Iba9732d6d79733b31b45eb4d0187b1c8a82499dc
2021-03-05 14:18:01 +00:00
Daimona Eaytoy 124031fe09 Stop using deprecated User::addGroup in tests
Bug: T276094
Change-Id: I299f89fbb4a4f6ba36ef4b77987bda9f2159d311
2021-03-01 15:28:08 +01:00
jenkins-bot ef4a5c4115 Merge "Make FilterProfiler independent of DeferredUpdate" 2021-02-28 13:18:51 +00:00
Matěj Suchánek 709803eb46 Make FilterProfiler independent of DeferredUpdate
Schedule the deferred update from FilterRunner, just like
we do with EmergencyCache.

Change-Id: I121211bb02a77c191001d11d4af3796e8572967e
2021-02-28 12:03:05 +01:00
Daimona Eaytoy 3365a648f2 Reject filters with invalid groups
It is currently possible to save a filter with an invalid group, if you
manually change the form data. So prevent this by validating the group
before saving.

Change-Id: I03f80b8c6ab583a357273f7b2679a424ac784db7
2021-02-27 16:01:09 +00:00
Matěj Suchánek b8ac52c51c Use independent stats for emergency disable
Bug: T264629
Change-Id: I64b611243b6a4c136b82b09f2ccf588d1c3e3426
2021-02-26 18:10:49 +01:00
jenkins-bot 1f3597f925 Merge "Update hit counts in a DeferredUpdate" 2021-02-23 06:34:17 +00:00
jenkins-bot 1b6e209ce6 Merge "Create a new method for authorizing access to test tools" 2021-02-22 18:00:06 +00:00
jenkins-bot 63a9c86607 Merge "Improve test coverage metrics" 2021-02-22 17:00:23 +00:00
daniel 63a497fb56 Don't set actor on UserIdentityValue in tests.
The actor ID is being removed from UserIdentityValue. Non-zero values
are triggering a deprecation warning now.

Needed-By: I9925906d11e47efaec3c1f48d5cb3f9896a982c1
Change-Id: Id60e56e70f6e4b44f49887d9e5ae5a23b1fd19a2
2021-02-22 11:30:54 +00:00
Matěj Suchánek c2376efddc Improve test coverage metrics
Change-Id: I1618883e3ade7dde538242fb51a36c22999df76d
2021-02-21 09:59:52 +01:00
jenkins-bot b050e36843 Merge "Align arg counting between the parsers" 2021-02-21 03:37:52 +00:00
Daimona Eaytoy 2470bea0d9 Update hit counts in a DeferredUpdate
Bug: T274455
Change-Id: Iadd383f150c5f9b736e37bfd8bdad56298f3d8d5
2021-02-21 03:15:16 +00:00
Matěj Suchánek 5d4025d8c9 Create a new method for authorizing access to test tools
This commit doesn't change any permissions for anybody.
It's the first step to achieve what the task asks for.

Bug: T242821
Change-Id: I8060ca926e6769b11d470fe4037854cda496000d
2021-02-20 17:54:35 +01:00
jenkins-bot 8575201ab2 Merge "Do not serialize RunnerData to array in FilterRunner" 2021-02-20 14:47:15 +00:00
Daimona Eaytoy 2bb5c3c7b5 Align arg counting between the parsers
1 - Change the structure of if/elseif for readability
2 - In the old parser, if there's an empty argument, never add it (the
new parser was already doing that).

Bug: T156095
Bug: T156096
Change-Id: I4237b1a0ba01e7ce04dcc945f7daf34612fcf07d
2021-02-20 14:33:56 +00:00
Daimona Eaytoy e64049c30b Create dedicated types of parser exceptions
Introduce a clear distinction between internal exceptions and
user-visible exceptions, leaving AFPException as base abstract class.

Later, it should be possible to narrow some types around, e.g. in
ParserStatus (that might work with user-visible exceptions only).

Also a future TODO is putting all the exceptions in their own namespace
(probably ...\Parser\Exception).

Change-Id: I4e33a45117f0a3e73af03cc1e3f2734beaf2b5e1
2021-02-12 13:56:02 +00:00
Matěj Suchánek c18e4a4a5f Do not serialize RunnerData to array in FilterRunner
Change-Id: Ia803042224959e516bc14bdc034421b8e80390a8
2021-02-12 14:03:50 +01:00
Matěj Suchánek a51b9bf903 Serialize all data for edit stash
Thanks to this, we will be able to provide more information
to consequences and watchers, which will open door for new
features and possibly cleaner code.

Change-Id: I7135509823ea84b2a2923d2c1831ce293b98a9f9
2021-02-11 15:09:50 +01:00
Matěj Suchánek 2b3af752ef Get rid of hitCondLimit parameter and field
As the todo says, move the check to the callers.

Change-Id: I5c5fbf772ca57758c901a1ae8068a0f119b4f26f
2021-02-08 15:48:59 +00:00
jenkins-bot 57671efea3 Merge "Create RunnerData class to store data about filter runtime and results" 2021-02-08 15:35:10 +00:00
Matěj Suchánek 1a0a702641 Create RunnerData class to store data about filter runtime and results
Get rid of the $profilingData property.

Change-Id: I608e7b9bcf9e91d9afaadfb8cd191e60d47f9db7
2021-02-08 16:06:25 +01:00
Umherirrender 4714a4ed80 Rework time handling in AbuseFilterConsequencesTest::testStashedEdit
Avoid strtotime and compare TS_MW timestamps
Set a fake time to get the same block expiry for relative times

Bug: T272236
Depends-On: I1357d3a78538b8bcb2a3507d86f35371e3f26d47
Change-Id: I5447953c5a0d7ecf4534f4ac4bc2260fa9f42117
2021-02-07 14:27:17 +01:00
jenkins-bot a7b24b1dee Merge "Clean up EditStashCache and test" 2021-02-07 01:32:26 +00:00
Daimona Eaytoy a5d79f426c Clean up EditStashCache and test
Change-Id: I952b7bb32d8697c89988f4e0eda8d3177cb30972
2021-02-06 23:16:32 +00:00
jenkins-bot d7204eaf73 Merge "Use a different message prefix for parser warnings" 2021-02-06 19:19:19 +00:00
Daimona Eaytoy 4dbde4dcf0 Use a different message prefix for parser warnings
The abusefilter-warning prefix is reserved for filter warnings. Pointed
out by Matěj.

Change-Id: I169e4c3d29b08c7f5af2136a683fc4427f8e93f5
2021-02-06 15:42:33 +00:00
Matěj Suchánek 6bb44fd088 Introduce EditStashCache
This class is responsible for interaction with edit stash.

Bug: T271520
Change-Id: I7cc32de0494e76cd9ba12220235c1cdb6b1d5ee1
2021-02-06 12:43:34 +01:00
jenkins-bot a94b2247f6 Merge "Cover some API modules by tests" 2021-02-04 23:43:02 +00:00
jenkins-bot 07612675f4 Merge "Use Authority in TextExtractor" 2021-02-04 17:53:52 +00:00
Matěj Suchánek a0fcfbcc32 Cover some API modules by tests
Change-Id: Icc57e260b3b06a58fc05f304d6e63dc40f970fe9
2021-02-04 15:17:00 +01:00
Daimona Eaytoy b0058c0f1b Use Authority in TextExtractor
And make its test a pure unit test, as per TODO comment.

Change-Id: Ia3ca38702ea61c5e551a581248d2b9471ef881fb
2021-02-02 00:43:01 +00:00
Daimona Eaytoy bf9142a644 Partial integration of EditBoxBuilder with HTMLForm
This patch adds a transparent HTMLForm field that can be used to insert
the edit box inside an HTMLForm, and updates /test and /tools to use
that. The field class, together with the other editbox-related classes,
is now in a dedicated namespace. A future TODO is making it a real
HTMLForm field.

Also improve a bit the form in /test: add section labels and
avoid reusing the same label message used on Special:AbuseFilter.

Bug: T261584
Change-Id: Ib74bb5fdba4f8476169b754030fce6d4f72ce65a
2021-02-01 16:23:42 +00:00
Umherirrender bcf6abb052 Use FauxRequest::setUpload in AbuseFilterUploadTestTrait::doUpload
Avoid direct use of super global $_FILES

Depends-On: I7392acc9bb682ec6b7025dbed0734c142f45c91a
Change-Id: I7d067da8572a0f412612fee12aaa2f0e128ea41f
2021-01-30 00:16:14 +01:00
Reedy 4f9982fe4e Add test case for space around - in IP range
Change-Id: Ie4563a0b150a95a7c460f4f46066fd79a3b7f2fd
2021-01-27 00:43:53 +00:00
jenkins-bot d96f0ea3f2 Merge "Introduce an EditRevUpdater service" 2021-01-27 00:33:29 +00:00
Daimona Eaytoy a04a601240 Introduce an EditRevUpdater service
This service allows linking the EditFilterMergedContent and
PageSaveComplete hooks for the same edit, so we can update rev IDs in
the abuse_filter_log table. Having such a services also avoids two hacky
static props, and should allow separating the hook handlers easily.

Change-Id: I622d15225ee3af202cb5730a7112652aef8ca71a
2021-01-27 00:24:39 +01:00
Daimona Eaytoy 5c43c0ab35 Allow single IPs in ip_in_range
Also add a bunch of tests for this function.

REMINDER: Change the docs on mw.org when this will be merged.

Bug: T218074
Depends-On: I155024341e8e6b13240e37b30c31b95dc83a47e0
Change-Id: I979e45110bc0e76b499679184993085062ffcac5
2021-01-26 04:37:51 +00:00
Petr Pchelko 6aa8f6f67b Do not mock User in TextExtractorTest.
In I63d9807264d7e2295afef51fc9d982447f92fcbd we are
changing how the permission checks are applied for revision,
so it uses passed User instance as Authority. However, when
user is mocked, the tests are breaking since the new user methods
are not mocked. Pass a real user for now to fix the test. Once
Authority reaches maturity and is ok to use in extensions, the
test should be rewritten to use authority directly.

Bug: T271458
Change-Id: Iacab813b253cc6e1439007e573e8ace06645860f
2021-01-20 09:32:18 -06:00
Daimona Eaytoy 005cc83642 Increase coverage for more classes
Change-Id: Iae6a24291f821fda77a45d8c1584de010af6a834
2021-01-17 17:38:58 +00:00
jenkins-bot 3306c341fc Merge "Make User::get* calls explicit in LazyVariableComputer" 2021-01-17 02:49:20 +00:00
Daimona Eaytoy 22b408d9e6 Use Filter objects in ViewDiff
And cleanup weird spacing, conditionals, etc.

Change-Id: I747216df65c2f34f7167612e90506890bc61880a
2021-01-17 00:47:33 +00:00
Daimona Eaytoy 8639e0c368 Introduce subclasses of Filter with specific use cases
In particular, this brings stronger typing for getID(), and we can get
rid of many phan suppressions.

Change-Id: Icbf3a6f7db8105082646ec227f62c09449fb165d
2021-01-17 00:47:29 +00:00
Daimona Eaytoy ed49f86b74 Make User::get* calls explicit in LazyVariableComputer
With explicit calls it's easier to see what method is being used,
whether it's deprecated, etc. Some methods here are in fact deprecated
or already have a proper replacement, but this is left for a follow-up.

Change-Id: Iee3154855f86c76aab98e7c14250c14e8b9ee939
2021-01-17 00:35:40 +00:00
Daimona Eaytoy 5fcc5ef357 Improve coverage for several classes
Change-Id: I257524dd22a5617ac47a3565a5b8fe4855aa67c7
2021-01-16 15:01:40 +00:00
Umherirrender 717abf4300 Improve function and property documentation
Removed some wrong @param from test function,
the @dataProvider should be a enough here

Change-Id: I82d81d066d916d3326169315aa099548f26d115a
2021-01-16 00:31:09 +01:00
jenkins-bot 76ae26d51e Merge "Add unit tests for load-first-author method" 2021-01-15 18:02:15 +00:00
jenkins-bot e35ab7c3e7 Merge "Improve coverage of parser-related classes" 2021-01-15 16:45:10 +00:00
Daimona Eaytoy 01b61b5e6a Re-enable AbuseFilterBlockTest::testRevert
Bug: T272124
Change-Id: Ic8ee8d2ce2b04e5cd0345efdb1e8f9653ef83bf3
2021-01-15 13:01:19 +01:00
Kosta Harlan 77656948ba tests: Mark testRevert as skipped due to unrelated failure
Bug: T272124
Change-Id: I99985cbf0a37f0d6c7f30dffa6d34408c0ec72ed
2021-01-15 09:53:26 +01:00
jenkins-bot b12778316a Merge "Improve coverage for hooks" 2021-01-15 03:43:52 +00:00
Daimona Eaytoy a9722868ab Improve coverage of parser-related classes
Change-Id: I229c528505f0208b34f37d8c969450731e5a08a3
2021-01-15 03:16:48 +00:00
Daimona Eaytoy 159046fc5a Misc coverage improvements
Change-Id: I656d9c9eedf4e8b8dfb7a13513d699e5ced22423
2021-01-15 00:51:16 +00:00
Daimona Eaytoy ab2ad164ff Improve coverage around consequences
Add a lot more unit tests, improve code testability, remove duplicated
integration tests.

Change-Id: Id8c9266ae107217047f267296070f26f575889d1
2021-01-15 00:51:04 +00:00
Daimona Eaytoy 10c7a11077 Improve coverage for hooks
- Exclude a couple of classes from coverage reports
- Add tests for all handlers
- Add tests for the runner, copied from core
- Make AbuseFilterRunner a real service

Change-Id: I7a0fe3cd8300faef5ef72d7f986b1734c324d8d1
2021-01-14 22:49:24 +01:00
jenkins-bot 192fb15e4d Merge "Refactor VariableGenerator and LazyVariableComputer tests" 2021-01-14 21:47:13 +00:00
Daimona Eaytoy 8646db7573 Add unit tests for load-first-author method
This is using core methods, so it can be unit tested. The same isn't
true for load-recent-authors, which performs a custom DB query and whose
test is probably the slowest AbuseFilter test. Simplify it for now,
until the method is moved to MW core.

Change-Id: Ifbdae1a06aabca996eeac151a6d029fd991ad64d
2021-01-14 17:23:54 +01:00
jenkins-bot bbe2eb6145 Merge "Reorder and cleanup some test files" 2021-01-10 16:13:05 +00:00
jenkins-bot 57fbeddbd0 Merge "Add pure unit tests for FilterRunner" 2021-01-09 13:56:24 +00:00
Daimona Eaytoy 9afc968523 Refactor VariableGenerator and LazyVariableComputer tests
Additionally, avoid building Title objects in LazyVariableComputer, it
just adds a dependency on TitleFactory and creating mocks is more
complicated, but it's pointless because the caller already has a Title
object.

And also stop using Title::getEarliestRevTime(), since the replacement
is easy (we already have a RevisionLookup).

Note for reviewers about renames:
- Code VariableGeneratorDBTest was moved to LazyVariableComputerDBTest,
  RCVariableGeneratorTest, and AbuseFilterVariableGeneratorTest
- AbuseFilterVariableGenerator test was moved into a dedicated
  directory, methods were changed not to test the var values

Change-Id: I3dff8739a9b79f33321d836449b082c3ce63f277
2021-01-09 11:26:24 +00:00
Daimona Eaytoy 72fcf1605a Reorder and cleanup some test files
Change-Id: Ib18b161dc938d6861904bf136570de1a2bd8dd46
2021-01-08 16:14:54 +01:00
jenkins-bot ac1e84df9b Merge "Use overrideUserPermissions in TextExtractorTest" 2021-01-08 09:44:40 +00:00
Daimona Eaytoy 72a23b4e5c Add pure unit tests for FilterRunner
Mainly constructor and conditions limit, which can be removed from
ConsequencesTest (where it was very slow).

Additionally, inject globals into FilterRunner.

Change-Id: I56ca67de6878dbc2185038faae3eb2b04fb56be9
2021-01-07 12:15:11 +00:00
Daimona Eaytoy 49dbb21bc2 Improve coverage for Consequences methods
In particular getMessage

Change-Id: I1c0e533bccde458e64008a4e02556273ce11d5e7
2021-01-06 14:07:02 +01:00
Daimona Eaytoy 8368b5d9b7 Use overrideUserPermissions in TextExtractorTest
This allows merging I1acd55c07d07b4a0d43fd838e11374b6d9be98d9.

Change-Id: I99ab3a69c41b3ec6721f9504ad6c77d3122df591
2021-01-06 12:46:11 +01:00
jenkins-bot 93ec5951e2 Merge "Move remaining classes to own namespace" 2021-01-04 21:08:01 +00:00
Daimona Eaytoy a5eab82204 Add a bunch of tests
Code change: in buildVarDumpTable remove special-cased null value. This
was used to avoid passing null to Html::element, but is no longer
necessary, since we now pretty-print the value.

Change-Id: I6180f6c53448d2a8c8c6066f222e9fd9df577554
2021-01-04 15:54:54 +01:00
Daimona Eaytoy 45f0a66616 Move remaining classes to own namespace
So everything can be loaded using PSR-4. These classes weren't renamed,
nor the alias for the AbuseFilter class was deprecated, because they
should be refactored first.

Change-Id: Ia328db58eb326968edf5591daac9bacf8c2f75da
2021-01-04 12:11:58 +01:00
jenkins-bot 21890a7bea Merge "Introduce a VariableGeneratorFactory service" 2021-01-03 22:59:47 +00:00
Daimona Eaytoy 6081bf90c4 Introduce a VariableGeneratorFactory service
So we can use DI in all generators. Some improvements were deliberately
omitted, e.g. injecting more services and relaxing User/Title to
UserIdentity/LinkTarget, and they'll be included in a subsequent commit.

Depends-On: I1f351071ef2b0b7c80e91407a9c3bb17be293044
Depends-On: Ie71740fac35a86f8fe03023080ae8ca08671243d
Depends-On: I589a0e1c2c5891070ab82cd5adfd9cedec19e67d
Change-Id: I92ef0abd5e45b672e6f297a71b3c2c345d56f136
2021-01-03 14:17:39 +01:00
jenkins-bot b0e8a76b2e Merge "DI for AbuseFilterSpecialPage" 2021-01-03 12:40:04 +00:00
Daimona Eaytoy 6e27a9ddb3 Cleanup variables-related classes
Change-Id: I20a7fe1a40255043ed0d125dee61ea6052dda69c
2021-01-02 18:19:38 +01:00
Daimona Eaytoy 762d71c51d Create a dedicated namespace for variables-related classes
Some cleanup is left for later to keep the diff easier to read.

Change-Id: Ife445b5e47e707ab77ec867ac3b005866aa74ef2
2021-01-02 18:16:48 +01:00
Daimona Eaytoy d3b330b6d4 Create a VariablesManager service
This makes VariableHolder a true value object, and introduces a
stateless service, VariableManager, to operate on it.

Note, in theory, this new service is still cyclically coupled with
LazyVariableComputed. However, it's now two stateless service being
coupled, not two smart/god value objects, so we've still earned
something. For now, the dependency is hidden by using a callback. Some
alternatives for that are mentioned in a code comment.

Bug: T261069
Change-Id: I2f2c84c8e91472ba36084a8bbb4a923f6e04354b
2021-01-02 17:15:31 +00:00
Matěj Suchánek de5b7ee8ea DI for AbuseFilterSpecialPage
Change-Id: I5c702990398e0adb5fa73be54638cb8b6b268beb
2021-01-02 11:13:41 +01:00
Matěj Suchánek f5b18a36bf Move special page classes to own namespace
Change-Id: Ic2d13518924e77b1be96d1a7489abcd07e6d1dab
2021-01-02 10:54:13 +01:00
jenkins-bot d2884049be Merge "Add a TextExtractor service" 2021-01-01 19:36:42 +00:00
jenkins-bot b43fc2387c Merge "Inject the condition limit into AbuseFilterParser" 2021-01-01 19:20:57 +00:00
jenkins-bot 0760c730db Merge "Rename AbuseFilterTest to GlobalNameUtilsTest" 2021-01-01 18:46:47 +00:00
jenkins-bot 80e6e582e4 Merge "Make tests pass on SQLite" 2021-01-01 18:08:58 +00:00
Daimona Eaytoy aafd3bcfcd Inject the condition limit into AbuseFilterParser
Change-Id: I487ba25ca3f3ac4b84c3afaf88b35678944cdb4d
2021-01-01 18:27:06 +01:00
Daimona Eaytoy fad9a11f7a Add a TextExtractor service
This is an important step towards removing the AbuseFilter class. Note:
proposals for the name of the new service are welcome.

Change-Id: Ib4632173f728b1bdafadef96e01645a833bfceaa
2021-01-01 18:25:32 +01:00
Matěj Suchánek ec2beb97b7 Rename AbuseFilterTest to GlobalNameUtilsTest
The suite doesn't anything from AbuseFilter class anymore.

Change-Id: If54de5ad2f4c68803768848e08154f8a3199a63a
2021-01-01 18:11:39 +01:00
jenkins-bot bac48babd8 Merge "Partly reorg integration tests" 2021-01-01 17:07:18 +00:00
jenkins-bot 6fcd3e147d Merge "Introduce a VariablesFormatter service" 2021-01-01 17:03:12 +00:00
Daimona Eaytoy e381d1995b Partly reorg integration tests
Move to 'integration' all tests that are meant to stay there. Move
SaveTest outside because, while we might want to finalize it as an
integration test, some parts can still be moved to a unit test.

Change-Id: Id4b6deaac6875fdd85eebbebf0c5fb952d1fbb06
2021-01-01 15:54:52 +00:00
Daimona Eaytoy b85f464530 Introduce a VariablesFormatter service
Moves more methods away from the AbuseFilter class. Testing
buildVarDumpTable is not easy because we'd have to parse the generated HTML.

Change-Id: I073a537201de150ba9dd7bf15a99f3a009dc6ba1
2021-01-01 15:45:52 +01:00
Daimona Eaytoy f7d74a0745 Fix path of parser tests
Follow-up: Ie46ef18d6ba1017e25c76b1762f678e5452264d9

Tests were silently ignored.

Change-Id: Ice0b7c78759add387afd50e5f7724d454a2b0c71
2021-01-01 15:42:09 +01:00
Daimona Eaytoy 0e69c18545 Organize unit tests
Add namespaces, shorten class names.

Non-unit tests and AbuseFilterTest are untouched because those should be
refactored first.

Change-Id: Ie46ef18d6ba1017e25c76b1762f678e5452264d9
2021-01-01 13:26:37 +00:00
jenkins-bot 544911a78d Merge "Add a LazyVariableComputer service" 2021-01-01 00:18:32 +00:00
Daimona Eaytoy a6176399b1 Make tests pass on SQLite
Skip a test that fails with
  Wikimedia\Rdbms\DBQueryError: Error 5: database is locked
  Function: Wikimedia\Rdbms\Database::beginIfImplied (MediaWiki\Extension\AbuseFilter\FilterLookup::getAllActiveFiltersInGroupFromDB)
  
Probably due to some concurrency issue caused by the duplicate connection, and also with

  Wikimedia\Rdbms\DBQueryError: Error 1: no such table: unittest_external_abuse_filter
  Function: MediaWiki\Extension\AbuseFilter\FilterLookup::getAllActiveFiltersInGroupFromDB
  
for unknown reasons.

Move the mwGlobals override inside the test to avoid the same "database is locked" error
on every other test in that class.

Bug: T251967
Change-Id: I552a8d1fa532941f630fd734e590993e7462aeb0
2020-12-31 20:11:10 +00:00
Matěj Suchánek 2793e7f1cf Reversible consequences
Introduce ReversibleConsequence interface for Consequence classes
whose potentially destructive actions can be reverted using
Special:AbuseFilter/revert. This allows moving reverting logic from
AbuseFilterViewRevert to individual Consequence classes and testing.

Unfortunately, the code is definitely not very clean now.

Change-Id: I558da711f1645ccf64792c6102cf743827171320
2020-12-31 14:43:32 +01:00
Daimona Eaytoy c7f06750d6 Add a LazyVariableComputer service
See task for a description of the plan. Also note that
AFComputedVariable should be renamed and its properties made private.

This commit includes some adjustments for taint-check in
AbuseFilter::buildVarDumpTable and ::revisionToString.

There's some space for improvement in the new LazyVariableComputer, but
that's left for another commit.

Bug: T261069
Change-Id: Ia44f6e079d39f44cf0122dec5ddb5513ab54f0c6
2020-12-31 14:05:52 +01:00
jenkins-bot 534ad9cfe6 Merge "Add a service to format filter specs" 2020-12-20 10:21:39 +00:00
jenkins-bot 16ef710ee3 Merge "Test some Consequence classes and clean up" 2020-12-20 09:25:39 +00:00
Matěj Suchánek 63b950e5b6 Test some Consequence classes and clean up
Sadly, these are not unit tests.

Bug: T201193
Change-Id: I4c977ab14b273b02803a63f0a7b152a581a838b2
2020-12-19 16:31:22 +01:00
Matěj Suchánek 352a207c70 Improve code coverage
Bug: T201193
Change-Id: Ie086fd525bec19c63c13f8710a27897229cc33c8
2020-12-19 16:28:34 +01:00
Daimona Eaytoy 7bcb5ec2d5 Add a service to format filter specs
This requires a MessageLocalizer, which currently means providing the
main RequestContext. This is the only alternative right now, until core
provides a proper MessageLocalizer service (see T247127).

Change-Id: I8c93e2ae7e7bd4fc561c5e8490ed2feb1ef0edc2
2020-12-19 14:22:08 +01:00
Matěj Suchánek dc7509811a Notify of a throttled filter
Use Echo for delivering the notification to the last
user who edited the filter.

Much boilerplate.

Change-Id: I7a46a03b4f15de20902ec70c62fb4fe750096842
Depends-On: If585b14a6dd6fb8c7d2c3bee1f20d9d08eaac706
2020-12-19 10:31:29 +01:00
jenkins-bot d386633103 Merge "Create a dedicated namespace for all consequences-related classes" 2020-12-18 22:57:00 +00:00
Daimona Eaytoy b394956c22 Create a dedicated namespace for all consequences-related classes
Change-Id: Ibc39593e34da36e57b640af0b5bbf2145f725e92
2020-12-18 19:27:33 +00:00
Daimona Eaytoy 7c1d1c6d7d Return warnings from the parser, add warning for catch-all regexps
This commit introduces some boilerplate for emitting warnings from the
AbuseFilter parser, and also code for showing these warnings in the ace
editor. Adding new warnings should be as simple as appending to
AbuseFilterParser::warnings (and adding the relevant i18n).

Bug: T264768
Bug: T269770
Change-Id: Ic11021b379f997a89f59c8c0572338d957e089a6
2020-12-18 18:22:41 +01:00
jenkins-bot e1a7acf55f Merge "Implement GetAutoPromoteGroupsHook" 2020-12-18 13:41:28 +00:00
Matěj Suchánek 2da4cb4321 Implement GetAutoPromoteGroupsHook
Test coverage included.

Bug: T261067
Change-Id: Iba49f0518d2c1a72405498b2743e5ef300dd6d61
2020-12-15 18:01:04 +01:00
Daimona Eaytoy 3f7dd25fbf Create FilterRunnerFactory
Next step is splitting the Runner into various subclasses.

Change-Id: I766555f31b425cee52fd262c5bfb1c73f3f170d2
2020-12-15 12:47:34 +00:00
Daimona Eaytoy 68adaa5cb1 Introduce ConsequencesExecutor
This is the last big step towards moving Consequences-related things away from
AbuseFilterRunner. There's still some cleanup to do (+ write proper tests), but
this should really be the last important code change.

Change-Id: I347795fe93ba496c43b1d5cfc9ba6e1326842c06
2020-12-15 13:47:21 +01:00
Daimona Eaytoy c52ef337d7 Add a VariablesBlobStore service
Change-Id: If0c1eab2391819f8b4c801d12275d9ec14490f7a
2020-12-15 02:35:15 +00:00
daniel dfeff89317 Use a BlobStore for storing var dumps
AbuseFilter emulates the storage mechanism also used for page content.
Instead of duplicating the relevant code, AbuseFilter should use the
same BlobStore service also used by RevisionStore.

Note that this change is not strictly needed to resolve T198341, but is
needed to unblock T183490

Bug: T261889
Bug: T198341
Bug: T183490
Change-Id: I3fc8475dd8d50d73d705b706ff597a130267e990
2020-12-15 02:35:05 +00:00
DannyS712 7ccf758c4b AbuseFilterConsequencesTest: stop setting $wgUser
Shouldn't be needed anymore, not read by extension

Bug: T246733
Change-Id: I10dc21ad34402d83d57f23bc754a437b8a015af7
2020-12-14 20:53:00 +00:00
Daimona Eaytoy 9d288478fe Remove a bunch of deprecated methods
These are all unused.

Depends-On: Id99da02a98bb392cafed370768edcc8ac3d712ab
Change-Id: I9ff17714f94e1fbbd52da32ebf4f054a551edc1f
2020-12-13 18:31:27 +00:00
Daimona Eaytoy 5e609eb537 Add GlobalNameUtils class
This is just a temporary location for these two methods. Since they're
used a lot, having them in the AbuseFilter class means that the
dependency graph is unnecessarily complicated. Thus, since these methods
aren't doing much, they were moved to a dedicated class. Future todo is
finding an appropriate location, that might be either as part of another
service, or keep them in a Utilities class, perhaps a single class with
all util methods, rather than a specific class.

Change-Id: I52cc47a6b9a387cd1e68c5127f6598a4c43ca428
2020-12-12 17:49:48 +00:00
jenkins-bot 93c477d4b8 Merge "Move parser classes to a dedicated namespace" 2020-12-09 10:48:36 +00:00
Daimona Eaytoy da1c71ec4c Move parser classes to a dedicated namespace
Names were kept for now.

Change-Id: Ib2eb5d7b523a64f2a0f72fdcdde2043a76cc9a37
2020-12-09 01:30:20 +00:00
Daimona Eaytoy ca3f652cd7 Almost kill the last use of wgUser
This is the last use, and it was a bit harder to remove because it was
buried inside AFComputedVariable. Starting with
I4444cada720ab62d187f2dd0c4760697e465f2ff, we can freely change the
parameters to AFComputedVariable without breaking old log entries.

Note, we still need a fallback for other extensions calling this
method...

Bug: T246733
Depends-On: I4444cada720ab62d187f2dd0c4760697e465f2ff
Change-Id: I5d786a518ef88fad9c8d9c25ef4553a0bf30b2b2
2020-12-08 23:28:24 +01:00
Daimona Eaytoy 815ef6051c Split afl_filter in afl_filter_id and afl_global
Add a script to migrate the columns (which can also
be executed in dry run), and a config option with the migration stage
(defaults to SCHEMA_COMPAT_OLD).
Some of the script-related code is stolen from
Ic755526d5f989c4a66b1d37527cda235f61cb437.

Bug: T220791
Change-Id: I7460a2d63f60c2933b36f8383a8abdbba8649e12
2020-12-08 18:31:27 +00:00
Daimona Eaytoy 1c625eeae4 Drop back-compat code
This should be merged once T246539 is done.

Bug: T213006
Change-Id: I4444cada720ab62d187f2dd0c4760697e465f2ff
2020-12-08 17:15:47 +00:00
Daimona Eaytoy 600f1735f2 Add a hook for extensions to add custom actions
The global is now deprecated, and it will be removed soon.

Bug: T265794
Change-Id: I4e6c9b143744cb72c441017921bac9cd1960609c
2020-12-04 16:10:44 +00:00
Daimona Eaytoy d351d7150b Always take into account custom actions
$wgAbuseFilterActions shouldn't be used normally, as it excludes actions
registered by other extensions.

Note: mw:Extension:AbuseFilter#Integration_with_other_extensions should
be updated after merging.

Bug: T239348
Change-Id: I89b3f0228eacdf145e8f2dd2a5602d0c7ce75a86
2020-12-03 21:39:35 +00:00
Daimona Eaytoy c786c4adc6 Add ConsequencesRegistry
Change-Id: I91f4f28e09fa46b9ab2457b2a241b6f105320bdd
2020-12-03 22:39:25 +01:00
jenkins-bot d02b10db97 Merge "Improve type safety of filter ids" 2020-12-03 17:50:31 +00:00
Thiemo Kreuz 34968d783e Simplify a few pieces of code
… mostly by inlining pieces, instead of assigning them to
a variable first.

Change-Id: Ibc432ed05f7b853a44fc1a301ef820984facb067
2020-12-03 16:50:55 +01:00
Matěj Suchánek de997fe98e Improve type safety of filter ids
Also fix a bug in FilterProfiler. It would attempt to reset
stats for global filters but we do not record them (yet?).

Change-Id: I0228d8c85dab146deb877dfce506f1e8e7711a9f
2020-12-03 14:58:51 +01:00
Matěj Suchánek 0f062fca06 Move AbuseFilterView classes to separate namespace
Change-Id: I569281b13ec81d9f35038c7ef17a2d98f16f9b5c
2020-12-03 13:06:53 +01:00
Matěj Suchánek a1ba43f586 DI for AbuseFilterView and subclasses
Also add a test to ensure all views can be instantiated without problem.

Change-Id: Iedd7a5dca240efab1077fa51a3522c983b0ba4fa
2020-12-03 13:04:35 +01:00
Thiemo Kreuz e17e1b7e01 Remove comments that literally repeat what the code says
Such comments don't add anything.

Change-Id: I7530d6693293fbdd06ca3ee077c6e783fd9a4ac1
2020-12-03 09:50:56 +01:00
Daimona Eaytoy c957188866 Add ConsequencesLookup
The class is used to retrieve consequences from the Database.

Change-Id: I46b3925aac47554723649c076eff64707a2ea2e6
2020-11-27 16:43:44 +01:00
Matěj Suchánek d76affb1db Move ChangeTags stuff to separate namespace
Change-Id: I6d7bed0e62f001f82c00a3528cc0018388c9c70e
2020-11-27 15:13:34 +00:00
jenkins-bot a6e96ed915 Merge "Introduce ChangeTagValidator service" 2020-11-27 15:13:06 +00:00
Matěj Suchánek 872b6118f4 Introduce ChangeTagValidator service
Just moving code around. Without a unit test because DI
coverage of change tags in core isn't available yet.

Change-Id: Iac861e1e24dae13581b8d9173357a1d6c94be88a
2020-11-27 15:11:48 +01:00
Matěj Suchánek 1ad77dc9fb Introduce EditBoxBuilderFactory service and EditBoxBuilder
It makes sense to look at this and Iedd7a5dca24 together,
as this patch itself doesn't really fix anything.

Change-Id: Ifef5266b1803d1a96489789b08d9beed044d908f
2020-11-26 14:49:04 +01:00
Daimona Eaytoy 904d9cddbb Represent Consequences with command objects
The consequence-taking logic is moved away from AbuseFilterRunner, to
dedicated classes. There's now one class per consequence, encapsulating
everything it needs to take the consequence.

Several interfaces allow customizing different types of consequences.
Every "special check" in AbuseFilter was generalized to use these
interfaces, rather than knowing how to handle each consequence.

Adding more consequences from other extensions will also be easier, and
it should happen via a hook (not a global), returning a class that
implements Consequence. The BCConsequence class was temporarily added
for legacy custom consequences.

A ConsequenceFactory class is added to instantiate consequences; this
would possibly benefit from using ObjectFactory, but it doesn't because
it would also reduce readability (although we might do that in the
future).

These classes are still not covered by unit tests, and this is left to
do for later. The new unit tests should mostly replace
AbuseFilterConsequencesTest. @covers tag were added to keep the status
quo (i.e. code that was considered covered while in AbuseFilterRunner
will still be considered covered), although we'll have to adjust them.

Change-Id: Ia1a9a8bbf55ddd875dfd5bbc55fcd612cff568ef
2020-11-25 17:35:36 +00:00
jenkins-bot 26d3abfab4 Merge "Introduce a service for saving filters" 2020-11-25 14:47:21 +00:00
Daimona Eaytoy c368575af0 Create a base interface for watchers
This will ease adding new watchers, for instance to send Echo
notifications (see T179495 and T100892).

For now, this is just boilerplate, and converting EmergencyWatcher to
the new interface.

Change-Id: I18d62aba53471202b709cdb19033b1729c5c25b4
2020-11-20 23:34:20 +01:00
Daimona Eaytoy 9595bd9da5 Introduce a service for saving filters
Change-Id: I6b7d16ad7ea1124989ed67c74413979cfd0275c4
2020-11-20 22:33:21 +01:00
Daimona Eaytoy 3f7fff56e8 Adjust code coverage
-Exclude methods and classes that cannot be meaningfully covered
-Add a simple test for AbuseFilterServices
-Exclude ServiceWiring because there's no way to tell PHPUnit it's
covered

Change-Id: I4c67b0d3fea68c7a3b3cbe01b5608f87e1b492db
2020-11-19 22:40:26 +00:00
Daimona Eaytoy eab1f13696 Make VariableGeneratorTest an integration test
It's actually using MediaWikiServices.

Change-Id: I6ec1b4723ff3f187eccf44a8b4ac286572fdfbbe
2020-11-19 13:55:16 +01:00
jenkins-bot 31f4607790 Merge "Handle DUNDEFINED in array offsets" 2020-11-18 23:30:58 +00:00
jenkins-bot 8f47259285 Merge "Add an interface for exporting/importing filters" 2020-11-18 23:13:53 +00:00
Daimona Eaytoy 3fc30021d2 Handle DUNDEFINED in array offsets
The behaviour is:
- When assigning to an undefined offset, delete the whole array and turn
it into another DUNDEFINED
- When retrieving from an undefined offset, just return DUNDEFINED.

Bug: T237214
Change-Id: I621ee7a16c90bb86a57be04e7ce0a748ecdbfcc7
2020-11-18 14:20:49 -08:00
Daimona Eaytoy 210cf29658 Add an interface for exporting/importing filters
The main benefit of having a dedicated interface is that we can easily
change the output format. So we're now using a custom array without
references to the DB schema, thus making the import/export process
completely independent from the schema.

Change-Id: I4c0de41d914baf1e9a0e588bd31f95b3524a424b
2020-11-18 22:06:09 +00:00
Daimona Eaytoy 7a24c94d6e Evaluate left-to-right when adding elements to array
Bug: T237090
Change-Id: I5fb72dec0ea12240b6563e66b69e399edc4c72d6
2020-11-18 21:25:45 +00:00
Daimona Eaytoy df017d478c Factor out another method from AbuseFilterRunner::getFilteredConsequences
This is a no-op, moving code around, introducing another distinction re
"filtering actions", which now happens in 2 steps:

 - The first step only uses "generic" information available by looking
   at enabled actions as a "group". This includes keeping only the
   longest block, and removing 'disallow' if other blocking actions are
   enabled.
 - The second step uses information that is only available after having
   "partly executed" (named "pre-checked") a consequence. For instance,
   we need to pre-check 'throttle' to see if the throttle was hit, and
   remove any other actions if not.

Change-Id: I7be5cfaa61e942a06f97ed52f50e9c8c70a120e8
2020-11-18 16:49:26 +00:00
Daimona Eaytoy ef9e828fbe Filter out actions to execute before actually executing them
This way we don't have special cases in executeFilterActions, and instead, we execute
all actions in the same place. In turn, this is going to ease the
transition to a new consequences system: next step is refactoring this
code into a service with proper DI etc.

Bug: T204447
Change-Id: I8134ecc41fbecdbed99faf406e9e3ca91b6123b9
2020-11-18 16:49:01 +00:00