Commit graph

1706 commits

Author SHA1 Message Date
Daimona Eaytoy c52ef337d7 Add a VariablesBlobStore service
Change-Id: If0c1eab2391819f8b4c801d12275d9ec14490f7a
2020-12-15 02:35:15 +00:00
daniel dfeff89317 Use a BlobStore for storing var dumps
AbuseFilter emulates the storage mechanism also used for page content.
Instead of duplicating the relevant code, AbuseFilter should use the
same BlobStore service also used by RevisionStore.

Note that this change is not strictly needed to resolve T198341, but is
needed to unblock T183490

Bug: T261889
Bug: T198341
Bug: T183490
Change-Id: I3fc8475dd8d50d73d705b706ff597a130267e990
2020-12-15 02:35:05 +00:00
jenkins-bot a7e965bbad Merge "Overhaul the interface for hiding AbuseLog entries" 2020-12-15 02:33:33 +00:00
jenkins-bot 3668ac1bd9 Merge "Simplify page handling in AFComputedVariable" 2020-12-13 19:31:18 +00:00
Daimona Eaytoy 9d288478fe Remove a bunch of deprecated methods
These are all unused.

Depends-On: Id99da02a98bb392cafed370768edcc8ac3d712ab
Change-Id: I9ff17714f94e1fbbd52da32ebf4f054a551edc1f
2020-12-13 18:31:27 +00:00
Umherirrender 90254368fc build: Remove unneeded phan suppression
Depends-On: Iab770702fd3a1d20f25f54057770dc03e28f1b0e
Change-Id: I3ce1816abb0f64843e9c8e3f866a6ef84c66285f
2020-12-13 00:37:20 +01:00
Daimona Eaytoy 5e609eb537 Add GlobalNameUtils class
This is just a temporary location for these two methods. Since they're
used a lot, having them in the AbuseFilter class means that the
dependency graph is unnecessarily complicated. Thus, since these methods
aren't doing much, they were moved to a dedicated class. Future todo is
finding an appropriate location, that might be either as part of another
service, or keep them in a Utilities class, perhaps a single class with
all util methods, rather than a specific class.

Change-Id: I52cc47a6b9a387cd1e68c5127f6598a4c43ca428
2020-12-12 17:49:48 +00:00
Daimona Eaytoy 2e703bdbb7 Simplify page handling in AFComputedVariable
Change-Id: I8a992a67222928e3a412470d1cb31653da4d172a
2020-12-12 17:17:10 +00:00
jenkins-bot f7f34924d1 Merge "Partly decouple SpecialAbuseLog and AbuseLogPager" 2020-12-11 21:17:57 +00:00
jenkins-bot c992529f3d Merge "Create a separate view for hiding AbuseLog entries" 2020-12-11 21:15:47 +00:00
Daimona Eaytoy 21d7c08aa7 Overhaul the interface for hiding AbuseLog entries
The main change is the addition of checkboxes to hide/show multiple
entries at the same time. Also, tweaked some i18n and made the process
return more useful success/error messages.

This patch introduces some technical debt, caused by SpecialAbuseLog and
AbuseLogPager being tightly coupled (which is a pre-existing problem,
but it got worse here).

Bug: T260904
Bug: T144096
Bug: T206945
Bug: T206938
Change-Id: I13f476d8126f81b0417e7509784c83d4f21cf348
2020-12-11 20:55:08 +00:00
Daimona Eaytoy 86f308c6f0 Partly decouple SpecialAbuseLog and AbuseLogPager
Move to the latter some methods that make more sense in there. Inject
some more services, don't require a SpecialAbuseLog to be passed in the
constructor.

There are still a couple of static calls, but fixing those would require
factoring more classes out of SpecialAbuseLog (e.g. a service to
determine visibility of AbuseLog entries).

Change-Id: I1b3012ca85bf049a07e0433fc0b357f502c355ad
2020-12-11 20:53:53 +00:00
Daimona Eaytoy 7243dd6cf9 Create a separate view for hiding AbuseLog entries
This is moving code away from SpecialAbuseLog, which is already too big
and has too many purposes. As such, the behaviour is not changed,
including for now bugs that were already present in the old version.

Change-Id: Idc13f7f746ada2e425662c6948c32aa744edac61
2020-12-11 20:34:52 +00:00
jenkins-bot 6f848578ea Merge "Allow the parsers to return extra info" 2020-12-11 16:35:25 +00:00
Daimona Eaytoy 3e0c30ff92 Allow the parsers to return extra info
This is achieved by creating a new ParserStatus class. Aside from the
result of parse(), it contains whether the cache was warm. This can be
used to differentiate profiling data as part of T231112.

Another use case is returning non-fatal warnings (T269770).

Change-Id: Ifcbda861ce1a44bbe9bffba5b83cd9ef338a8dba
2020-12-11 15:03:23 +00:00
jenkins-bot 4e5b97b6d2 Merge "build: Updating mediawiki/mediawiki-phan-config to 0.10.5" 2020-12-11 00:29:04 +00:00
libraryupgrader 281eec8e4d build: Updating mediawiki/mediawiki-phan-config to 0.10.5
Change-Id: Ie3fcfdf733885aac2ef0ee07cc1a8d4f3fedb7d7
2020-12-10 18:28:54 +00:00
jenkins-bot 8c4265b4a0 Merge "Write afl_filter_id and afl_global by default" 2020-12-10 17:36:33 +00:00
jenkins-bot 93c477d4b8 Merge "Move parser classes to a dedicated namespace" 2020-12-09 10:48:36 +00:00
Daimona Eaytoy 8b21c994fd Remove more BC code
Bug: T187153
Change-Id: I0c74b855893c7da8ab531e67a0aac298aed32c2c
2020-12-09 02:40:55 +01:00
Daimona Eaytoy da1c71ec4c Move parser classes to a dedicated namespace
Names were kept for now.

Change-Id: Ib2eb5d7b523a64f2a0f72fdcdde2043a76cc9a37
2020-12-09 01:30:20 +00:00
Daimona Eaytoy f41fe76df7 Kill $wgUser
Depends-On: Iadbce7501e42971901f6d9efcb2810ae42be51d8
Depends-On: I624610cb2372db200995c8d01d62b1d74efca19e
Change-Id: I51d99c30fbc0e87c038013bf5b8c27b1c735e977
2020-12-08 23:23:13 +00:00
Daimona Eaytoy ca3f652cd7 Almost kill the last use of wgUser
This is the last use, and it was a bit harder to remove because it was
buried inside AFComputedVariable. Starting with
I4444cada720ab62d187f2dd0c4760697e465f2ff, we can freely change the
parameters to AFComputedVariable without breaking old log entries.

Note, we still need a fallback for other extensions calling this
method...

Bug: T246733
Depends-On: I4444cada720ab62d187f2dd0c4760697e465f2ff
Change-Id: I5d786a518ef88fad9c8d9c25ef4553a0bf30b2b2
2020-12-08 23:28:24 +01:00
Daimona Eaytoy bcbfa66fe8 Write afl_filter_id and afl_global by default
The schema was introduced in 1.34, so there should be no issue in
starting off with writing the new columns.

Bug: T220791
Change-Id: I8f956d4a27692a33368a413fbf4a8eb5da20afe1
2020-12-08 23:06:03 +01:00
jenkins-bot 5de037dea6 Merge "Simplify User handling in AFComputedVariable" 2020-12-08 20:58:30 +00:00
Daimona Eaytoy 815ef6051c Split afl_filter in afl_filter_id and afl_global
Add a script to migrate the columns (which can also
be executed in dry run), and a config option with the migration stage
(defaults to SCHEMA_COMPAT_OLD).
Some of the script-related code is stolen from
Ic755526d5f989c4a66b1d37527cda235f61cb437.

Bug: T220791
Change-Id: I7460a2d63f60c2933b36f8383a8abdbba8649e12
2020-12-08 18:31:27 +00:00
Daimona Eaytoy e91ab70175 Simplify User handling in AFComputedVariable
This is some more back-compat code that we can get rid of after cleaning
old entries.

Change-Id: I374fc4496c27cf50e6960a9ab07b9c80431b4a3a
2020-12-08 17:51:08 +00:00
Daimona Eaytoy 1c625eeae4 Drop back-compat code
This should be merged once T246539 is done.

Bug: T213006
Change-Id: I4444cada720ab62d187f2dd0c4760697e465f2ff
2020-12-08 17:15:47 +00:00
Matěj Suchánek 08db320afe Fix Special:AbuseLog not to throw when global filters are not available anymore
There is a try-catch block but the same call was also done
unconditionally after it, making it throw when global filters
are disabled.

Change-Id: Ic8032592799756521a59ee23c0e76cb03a510b94
2020-12-07 20:12:06 +01:00
Daimona Eaytoy 600f1735f2 Add a hook for extensions to add custom actions
The global is now deprecated, and it will be removed soon.

Bug: T265794
Change-Id: I4e6c9b143744cb72c441017921bac9cd1960609c
2020-12-04 16:10:44 +00:00
jenkins-bot 1c8eb3963e Merge "Always take into account custom actions" 2020-12-04 14:07:09 +00:00
jenkins-bot edd998f532 Merge "Move API modules to separate namespace and rename them" 2020-12-04 12:33:05 +00:00
Daimona Eaytoy af8c237559 Move API modules to separate namespace and rename them
Change-Id: I436e8fed4a1cbe4f1b36a16b213ede7976e871ba
2020-12-03 22:11:09 +00:00
Daimona Eaytoy d351d7150b Always take into account custom actions
$wgAbuseFilterActions shouldn't be used normally, as it excludes actions
registered by other extensions.

Note: mw:Extension:AbuseFilter#Integration_with_other_extensions should
be updated after merging.

Bug: T239348
Change-Id: I89b3f0228eacdf145e8f2dd2a5602d0c7ce75a86
2020-12-03 21:39:35 +00:00
Daimona Eaytoy c786c4adc6 Add ConsequencesRegistry
Change-Id: I91f4f28e09fa46b9ab2457b2a241b6f105320bdd
2020-12-03 22:39:25 +01:00
jenkins-bot a7670f2bb5 Merge "Move pagers to their own namespace" 2020-12-03 20:26:59 +00:00
jenkins-bot c560217865 Merge "Use 'default' as default group when reading filters from history" 2020-12-03 18:04:34 +00:00
jenkins-bot d02b10db97 Merge "Improve type safety of filter ids" 2020-12-03 17:50:31 +00:00
jenkins-bot 6f7a9965a0 Merge "Add UpdateHitCountWatcher" 2020-12-03 17:46:37 +00:00
Thiemo Kreuz e45ce1f5bd Mark two private methods as such
Not used anywhere else:
https://codesearch.wmcloud.org/search/?q=stringifyActions
https://codesearch.wmcloud.org/search/?q=normalizeBlocks

This patch also moves that [ '' ] fallback out of a method
where it was misplaced. That fallback is very specific for
the diff algorithm, but not something one would expect from
a method called "stringifyActions".

Change-Id: I458eef61c6b6741bbd433ea26a012aaeb01cea3f
2020-12-03 16:52:17 +01:00
Thiemo Kreuz 34968d783e Simplify a few pieces of code
… mostly by inlining pieces, instead of assigning them to
a variable first.

Change-Id: Ibc432ed05f7b853a44fc1a301ef820984facb067
2020-12-03 16:50:55 +01:00
DannyS712 0fb033775f Use 'default' as default group when reading filters from history
This was NULL for old entries, because no default was added
in I758795f01eaf3ff56c5720d660cd989ef95764a7 (see T263324)

Bug: T269314
Change-Id: I5af8b0d3a9d7b6d2570cf79bbbe8b5b170ba1230
2020-12-03 15:16:17 +00:00
Daimona Eaytoy 1a3bd4b2b3 Move pagers to their own namespace
Change-Id: I5d3da5e51bbc54179c0618f1877f2eabb8302542
2020-12-03 14:17:09 +00:00
Matěj Suchánek de997fe98e Improve type safety of filter ids
Also fix a bug in FilterProfiler. It would attempt to reset
stats for global filters but we do not record them (yet?).

Change-Id: I0228d8c85dab146deb877dfce506f1e8e7711a9f
2020-12-03 14:58:51 +01:00
Daimona Eaytoy ed1195ea23 Add UpdateHitCountWatcher
Change-Id: I61c40312022c1037abb03819d06e5e220dd07e15
2020-12-03 13:47:10 +00:00
Daimona Eaytoy 22dc4af459 Create an AbuseLogger service
For now this is just moving code around.

Change-Id: Ie61a1c122b4e93a74b465eb781f9cbf49f0b32e6
2020-12-03 14:46:57 +01:00
Matěj Suchánek 0f062fca06 Move AbuseFilterView classes to separate namespace
Change-Id: I569281b13ec81d9f35038c7ef17a2d98f16f9b5c
2020-12-03 13:06:53 +01:00
Matěj Suchánek a1ba43f586 DI for AbuseFilterView and subclasses
Also add a test to ensure all views can be instantiated without problem.

Change-Id: Iedd7a5dca240efab1077fa51a3522c983b0ba4fa
2020-12-03 13:04:35 +01:00
Thiemo Kreuz 7047dba1f1 Update a few unspecific PHPDoc comments
Change-Id: I363d7f854c550654c5d4345f381e3731de6f2d3c
2020-12-03 09:50:09 +01:00
jenkins-bot 753a0dc482 Merge "Add ConsequencesLookup" 2020-12-02 15:49:33 +00:00
DannyS712 ace8a613a4 Consequences cleanup, should be a no-op
Reduce use of User objects in favor of UserIdentity
Use `use` statements

Change-Id: Icdd1b8be2d1345a8dd48b8d5d3af8781c6658c7c
2020-11-30 01:12:25 +00:00
Reedy 95d8278b58 Clean up SchemaChangeHandler
* Move all SQL files into db_patches (or below)
** Remove db type from filename
* Remove a lot of duplicated code and simplify

Change-Id: If22f2a2c46a59ac24c89ce612c74d169f053ab26
2020-11-29 19:14:24 +00:00
Matěj Suchánek db87a68260 Remove unused variable and comment
Change-Id: I76f1d08aadda89dc9ba22eb35469c33cf9c86a33
2020-11-29 11:11:39 +01:00
Reedy f11bbe1c9b Use ::class rather than stringified class name
Change-Id: I5695f40627122b76d792a3a05c97c42d921a5536
2020-11-28 11:19:51 +00:00
DannyS712 66a43948f3 EditBoxBuilder: Minor cleanup for readability
Add use statements for OOUI classes,
clean up line breaks
Should be a no-op

Change-Id: I7e83a41a138557f99fadcef38e992bfc1bff7f7f
2020-11-27 23:33:11 +00:00
Daimona Eaytoy c957188866 Add ConsequencesLookup
The class is used to retrieve consequences from the Database.

Change-Id: I46b3925aac47554723649c076eff64707a2ea2e6
2020-11-27 16:43:44 +01:00
Matěj Suchánek d76affb1db Move ChangeTags stuff to separate namespace
Change-Id: I6d7bed0e62f001f82c00a3528cc0018388c9c70e
2020-11-27 15:13:34 +00:00
jenkins-bot a6e96ed915 Merge "Introduce ChangeTagValidator service" 2020-11-27 15:13:06 +00:00
Matěj Suchánek 872b6118f4 Introduce ChangeTagValidator service
Just moving code around. Without a unit test because DI
coverage of change tags in core isn't available yet.

Change-Id: Iac861e1e24dae13581b8d9173357a1d6c94be88a
2020-11-27 15:11:48 +01:00
Matěj Suchánek 1ad77dc9fb Introduce EditBoxBuilderFactory service and EditBoxBuilder
It makes sense to look at this and Iedd7a5dca24 together,
as this patch itself doesn't really fix anything.

Change-Id: Ifef5266b1803d1a96489789b08d9beed044d908f
2020-11-26 14:49:04 +01:00
Daimona Eaytoy 904d9cddbb Represent Consequences with command objects
The consequence-taking logic is moved away from AbuseFilterRunner, to
dedicated classes. There's now one class per consequence, encapsulating
everything it needs to take the consequence.

Several interfaces allow customizing different types of consequences.
Every "special check" in AbuseFilter was generalized to use these
interfaces, rather than knowing how to handle each consequence.

Adding more consequences from other extensions will also be easier, and
it should happen via a hook (not a global), returning a class that
implements Consequence. The BCConsequence class was temporarily added
for legacy custom consequences.

A ConsequenceFactory class is added to instantiate consequences; this
would possibly benefit from using ObjectFactory, but it doesn't because
it would also reduce readability (although we might do that in the
future).

These classes are still not covered by unit tests, and this is left to
do for later. The new unit tests should mostly replace
AbuseFilterConsequencesTest. @covers tag were added to keep the status
quo (i.e. code that was considered covered while in AbuseFilterRunner
will still be considered covered), although we'll have to adjust them.

Change-Id: Ia1a9a8bbf55ddd875dfd5bbc55fcd612cff568ef
2020-11-25 17:35:36 +00:00
jenkins-bot 26d3abfab4 Merge "Introduce a service for saving filters" 2020-11-25 14:47:21 +00:00
Daimona Eaytoy c368575af0 Create a base interface for watchers
This will ease adding new watchers, for instance to send Echo
notifications (see T179495 and T100892).

For now, this is just boilerplate, and converting EmergencyWatcher to
the new interface.

Change-Id: I18d62aba53471202b709cdb19033b1729c5c25b4
2020-11-20 23:34:20 +01:00
Daimona Eaytoy 9595bd9da5 Introduce a service for saving filters
Change-Id: I6b7d16ad7ea1124989ed67c74413979cfd0275c4
2020-11-20 22:33:21 +01:00
jenkins-bot ca54f0b2e3 Merge "EmergencyWatcher: update data for all filters at the same time" 2020-11-20 07:04:58 +00:00
jenkins-bot a16cca0ecb Merge "Adjust code coverage" 2020-11-20 06:48:12 +00:00
jenkins-bot b6b90c07cb Merge "Remove AbuseFilter::getFilter" 2020-11-20 06:46:01 +00:00
Daimona Eaytoy cdbe9260c7 EmergencyWatcher: update data for all filters at the same time
This will avoid unneeded queries, in theory. In practice, it will
almost never happen to have more than one filter to throttle.

Change-Id: I5b8df51215463ce4464f6a2d0390f58612a5a213
2020-11-20 06:41:56 +00:00
Daimona Eaytoy 3f7fff56e8 Adjust code coverage
-Exclude methods and classes that cannot be meaningfully covered
-Add a simple test for AbuseFilterServices
-Exclude ServiceWiring because there's no way to tell PHPUnit it's
covered

Change-Id: I4c67b0d3fea68c7a3b3cbe01b5608f87e1b492db
2020-11-19 22:40:26 +00:00
libraryupgrader e5c9bf119d build: Updating mediawiki/mediawiki-phan-config to 0.10.4
Change-Id: I8309c5ed36536f5304e1429c4c24553b456ddc8e
2020-11-19 20:33:25 +00:00
Matěj Suchánek 83c2ccb1b3 Optimize EmergencyWatcher
Avoid queries for profiler data when the filter hasn't
been changed recently.

Change-Id: I691d3922436e80264403f9c5b8b822be729e1d94
2020-11-19 18:20:16 +01:00
Daimona Eaytoy a71ea3aa38 Remove AbuseFilter::getFilter
Needs the patch in ContentTranslation first.

Depends-On: I0b74db70ad4e9768e4dcb84b9decb9c737e942e5
Change-Id: Id186ea99fcf69aa4348e404677ce5da998d83170
2020-11-19 15:11:32 +00:00
jenkins-bot 97019239bc Merge "FilterProfiler: allow searching for slow global filters" 2020-11-18 23:43:20 +00:00
jenkins-bot 31f4607790 Merge "Handle DUNDEFINED in array offsets" 2020-11-18 23:30:58 +00:00
DannyS712 09c3a9df05 FilterProfiler: allow searching for slow global filters
The slow filter hits are logged for the target wiki, but
the fix would be on meta, so make it possible to filter
for those

Change-Id: I6e02866479e77d707f4fa951ec909c325b944158
2020-11-18 23:20:30 +00:00
jenkins-bot 8f47259285 Merge "Add an interface for exporting/importing filters" 2020-11-18 23:13:53 +00:00
Daimona Eaytoy 3fc30021d2 Handle DUNDEFINED in array offsets
The behaviour is:
- When assigning to an undefined offset, delete the whole array and turn
it into another DUNDEFINED
- When retrieving from an undefined offset, just return DUNDEFINED.

Bug: T237214
Change-Id: I621ee7a16c90bb86a57be04e7ce0a748ecdbfcc7
2020-11-18 14:20:49 -08:00
Daimona Eaytoy 210cf29658 Add an interface for exporting/importing filters
The main benefit of having a dedicated interface is that we can easily
change the output format. So we're now using a custom array without
references to the DB schema, thus making the import/export process
completely independent from the schema.

Change-Id: I4c0de41d914baf1e9a0e588bd31f95b3524a424b
2020-11-18 22:06:09 +00:00
Daimona Eaytoy 7a24c94d6e Evaluate left-to-right when adding elements to array
Bug: T237090
Change-Id: I5fb72dec0ea12240b6563e66b69e399edc4c72d6
2020-11-18 21:25:45 +00:00
Daimona Eaytoy b5ae7360bc AbuseLog: Use a radio button not checkbox for suppressing entries
Add a radio to select between "hide" and "show" instead of a single,
cryptic checkbox which doesn't really explain what it does.
Also wrap the list in a form which will later be used to mass-delete
entries.

Depends-On: I1bb45e47c3b42c01388b99778ce833e4e44419e1
Change-Id: Ie2d019fad5af7c626d722dc348f40eb0db21e527
2020-11-18 20:57:39 +00:00
jenkins-bot cc10f76bfa Merge "Use a WikiPage object when filtering edits on a non-existing Title" 2020-11-18 20:52:53 +00:00
Daimona Eaytoy 6305746de3 Use a WikiPage object when filtering edits on a non-existing Title
Remove $title->exists() from the check, so we have the following
changes:
 - The AbuseLog will add a diff link for page creations
 - Searching the AbuseLog for impact:saved will include page creations
 - We don't have to recreate the WikiPage again in RunVariableGenerator

Also remove an old reference to "bug 31656": that comment was added in
rEABFefecf8b2441ae2f31f924ff33103f5affe5d1d62, which changed
Article::getContent() to Article::getRevision()->getRawText(). Nowadays
we don't even use Article anymore, and that conditional isn't even for
retrieving the page content, so the comment is wrong.

Add logging for when the Title object cannot exist, as this should never
happen in the context of the EditFilterMergedContent hook, and always
create a WikiPage. Some signatures were changed to require a WikiPage
object now, and every caller updated to provide it.

Bug: T263104
Bug: T62179
Depends-On: Ic238eaa529ef6bfba06b4dd03924a8e0111d8259
Change-Id: Ibf3bf4f68328ba4a5616ab8f26a8b44d27a25cd7
2020-11-18 20:13:46 +00:00
Daimona Eaytoy df017d478c Factor out another method from AbuseFilterRunner::getFilteredConsequences
This is a no-op, moving code around, introducing another distinction re
"filtering actions", which now happens in 2 steps:

 - The first step only uses "generic" information available by looking
   at enabled actions as a "group". This includes keeping only the
   longest block, and removing 'disallow' if other blocking actions are
   enabled.
 - The second step uses information that is only available after having
   "partly executed" (named "pre-checked") a consequence. For instance,
   we need to pre-check 'throttle' to see if the throttle was hit, and
   remove any other actions if not.

Change-Id: I7be5cfaa61e942a06f97ed52f50e9c8c70a120e8
2020-11-18 16:49:26 +00:00
Daimona Eaytoy ef9e828fbe Filter out actions to execute before actually executing them
This way we don't have special cases in executeFilterActions, and instead, we execute
all actions in the same place. In turn, this is going to ease the
transition to a new consequences system: next step is refactoring this
code into a service with proper DI etc.

Bug: T204447
Change-Id: I8134ecc41fbecdbed99faf406e9e3ca91b6123b9
2020-11-18 16:49:01 +00:00
Matěj Suchánek e7813fbafb Introduce EmergencyWatcher service
Change-Id: I45477ca84a99f620d182ef95e5627d421d38f077
2020-11-18 14:20:18 +00:00
Daimona Eaytoy ae29451ab8 Introduce a FilterCompare service
The scope is still quite limited, but as noted in a todo, we might want
to make this completely independent from the database, and add the use
case of ViewDiff.

Change-Id: Ie980fff0983b3e86037265e85da04444c809a6e8
2020-11-18 11:52:44 +00:00
jenkins-bot 914f0f4a13 Merge "Remove AbuseFilter::filterHidden and ::getGlobalFilterDescription" 2020-11-18 09:36:03 +00:00
jenkins-bot 3158a7ebc7 Merge "Remove temporary parameter" 2020-11-18 09:09:02 +00:00
Daimona Eaytoy 6376394713 Remove AbuseFilter::filterHidden and ::getGlobalFilterDescription
They've been replaced by getters in the Filter class.

Note, the Lookup is not injected in this patch because some places would
need careful thought, so it's left to do later.

Change-Id: I40b8c8452d9df741217d7fa090a5e746a2f46994
2020-11-18 08:43:22 +00:00
jenkins-bot de67c30d96 Merge "Don't show form for reverting filter actions when no actions were found" 2020-11-18 02:06:34 +00:00
Matěj Suchánek 8955c55dc7 Don't show form for reverting filter actions when no actions were found
Change-Id: I779a318a9daaf6d3a17335914a7fd85877765625
2020-11-18 01:42:38 +00:00
Daimona Eaytoy 1bcfdc3b13 Introduce a FilterValidator
This moves a lot of things away from the AbuseFilter class. There's a
nasty static dependency on ChangeTags, but it's very limited anyway, and
it's going to be fixed once T245964 is resolved.

Change-Id: Ia7df4b4d3289c2722323f59ceecf3fdd38277785
2020-11-18 01:41:31 +00:00
Daimona Eaytoy 725ec052ed Add a FilterLookup service
Some pieces of code were updated to use Filter objects, while other
places are still to be updated. We also need to change the history part
to exclude actions somehow, cleanup the ViewEdit, reduce direct DB
access or anything mentioning DB fields outside of FilterLookup, etc.

Change-Id: I42b7ded685db76eddd45e4b1336f9828cba811ce
2020-11-18 01:17:47 +00:00
Daimona Eaytoy bad5a9a29c Make AbuseFilterViewEdit work with Filter objects
This requires adjusting some methods to work with Filter objects. Some
methods and tests are left in an inconsistent/suboptimal state, plus some todos
were added, but all of this is going to be remediated in another commit.

Change-Id: Id063ee73d97c7aef56323e1457d99704f77ab943
2020-11-18 00:52:37 +00:00
Daimona Eaytoy 555383a5c6 Unbreak master build
Phan is failing on master with

  includes/Views/AbuseFilterViewEdit.php:506 PhanTypeMismatchArgument Argument 1 ($salt) is ['abusefilter',$filter] of type array{0:'abusefilter',1:?int} but \User::getEditToken() takes string|string[] defined at ../../includes/user/User.php:3735

due to a documentation change in core.

Change-Id: Ibc01332c67224e3efc7922d1be882615c2de5d9a
2020-11-18 00:15:54 +00:00
jenkins-bot 524555c400 Merge "Add value objects to represent filters" 2020-11-05 15:08:53 +00:00
Daimona Eaytoy e8947970ce Remove temporary parameter
The only usage outside of AbuseFilter (in ContentTranslation) was fixed with
Ifc9ede277791398290786cdb6743137004b5c713.

Change-Id: I22cf9c76ef3b007502045a02c82255ba6c9fd0f2
2020-11-04 15:06:32 +00:00
jenkins-bot 1d06f5fc4c Merge "Use HTMLForm features instead of mSubmit" 2020-11-04 13:07:47 +00:00
Daimona Eaytoy 71a61c2089 Add value objects to represent filters
This is just a start; next step is adding a factory/store method to
get/store these objects. And then use these value objects whenever
applicable.

Note: the actions-related code is still not fully implemented. This is
going to happen as part of the FilterLookup.

Change-Id: I5f33227887c035e301313bbe24d1c1fefb75bc6a
2020-11-04 12:56:14 +01:00
Daimona Eaytoy 0f17e47d88 Use HTMLForm features instead of mSubmit
Rely on modern HTMLForm features instead of using a dedicated class
property. The form identifiers are necessary, because these forms are
GET forms, and HTMLForm will always think that the form was submitted,
if it doesn't have an identifier (see T238467 and related
tasks/patches).

Additionally, make the first form on ViewRevert a GET form, like on
Special:AbuseLog.

Bug: T263627
Change-Id: Ia6ca45896732742ef73e401b09663728b9e7dda2
2020-11-04 11:51:27 +00:00
jenkins-bot 3b9a79cabc Merge "Introduce AbuseFilterParserFactory service" 2020-11-04 10:34:43 +00:00
jenkins-bot 648c2f8001 Merge "Divide AbuseFilterPermissionManager::canSeeLogDetails" 2020-11-04 10:32:46 +00:00
jenkins-bot 94ef2b3ad4 Merge "Resolve/remove a few todos in AbuseFilterRunner" 2020-11-04 10:00:33 +00:00
jenkins-bot 16c6cba1f5 Merge "Remove exclusions for new PHPCS rules, bump PHPCS to 33.0.0" 2020-11-03 19:53:16 +00:00
Daimona Eaytoy 4cc3934a73 Remove exclusions for new PHPCS rules, bump PHPCS to 33.0.0
Change-Id: I346c5e41b76322c4bcbc6b2402f1316e73c45681
2020-11-03 19:26:11 +00:00
Huji 9c9d2885a4 Correct the documentation of publishEntry() method
The publish() method that it resembles is not a method defined in
the LogEntry class, and not even in LogEntryBase class. It is
defined in the ManualLogEntry class. Let's reference it correctly.

Change-Id: I60cfceac7c19047e299cf9f704dda8d8ef2f2ba6
2020-11-02 15:53:55 -05:00
jenkins-bot 8946ba54a2 Merge "Remove dead line of code from AbuseFilterExaminePager" 2020-11-01 14:44:12 +00:00
Daimona Eaytoy c1c3daa031 Resolve/remove a few todos in AbuseFilterRunner
Some were outdated/not doable, others were resolved.

Change-Id: Ice524a4d31f8d90ab507801562787b946c59d651
2020-11-01 14:08:25 +01:00
Daimona Eaytoy be75cf1c40 Introduce AbuseFilterParserFactory service
TODO For the future: the final directory for Parser-related classes
should be "Parser", not "ParserNS". However, moving all classes now
would make it harder to rebase changes etc.

Change-Id: Ice335f4723e74f4e5fbe8dcc76ff8ea16310962c
2020-10-31 21:19:00 +01:00
Matěj Suchánek 5efbf80034 Remove dead line of code from AbuseFilterExaminePager
Ordering is done by in IndexPager::buildQueryInfo. In fact,
this key is unconditionally overridden there and the query
is sorted by rc_id (specified in ::getIndexField). It would
probably deserve some performance analysis because
the ordering and filtering don't seem to use matching indices.

Change-Id: I9e73d44d868ddf5beba6dc6e4550e851a6df5119
2020-10-31 18:00:03 +01:00
jenkins-bot bec7c44d12 Merge "Move mCounter property from AbuseFilterViewExamine to AbuseFilterExaminePager" 2020-10-31 16:27:12 +00:00
Matěj Suchánek 3e8a4b63ab Move mCounter property from AbuseFilterViewExamine to AbuseFilterExaminePager
It is only used there. Reduce coupling.

Change-Id: I1fad101c4cd971914a031b08f10114cd7278cc66
2020-10-31 15:31:23 +00:00
Daimona Eaytoy 1f8df50cb3 Add a service to retrieve the central DB
This is a thin wrapper around LBFactory and the global variable, that
can be injected in classes requiring it (no real class right now, but
that's going to change soon).

Also, remove some DWIM-style returns which made the code harder to
understand.

Change-Id: I1d28ad4a67f914103f3a17cda5f61b28070c7f1c
2020-10-31 12:32:46 +00:00
jenkins-bot 6a081ade68 Merge "Little cleanup for AbuseFilterRunner" 2020-10-31 11:42:31 +00:00
jenkins-bot 4f30f4e188 Merge "Process 'throttle' action if object caching is disabled" 2020-10-30 22:24:02 +00:00
Daimona Eaytoy 04451d7bde Little cleanup for AbuseFilterRunner
Remove outdated/pointless comments, use already defined variables, etc.

Additionally, make it possible to disable throttling locally.

Change-Id: I98fd5f3eb47b32fc1013360e462a57d932174a95
2020-10-30 21:42:54 +00:00
jenkins-bot f0962ccd51 Merge "Use MainObjectStash for generating throttle keys" 2020-10-30 20:06:37 +00:00
Daimona Eaytoy 91f2cf9439 Process 'throttle' action if object caching is disabled
See a longer explanation on phabricator.

Bug: T265216
Change-Id: I8e0054ba523f993aeb48a7e1533bbb913b46c435
2020-10-30 20:20:58 +01:00
Matěj Suchánek 59f507b16c Use MainObjectStash for generating throttle keys
Keys should be generated for a cache that will use
them.

Change-Id: Ic634410e2521b02c1b50c798a7f2d5b96705af8c
2020-10-30 18:41:45 +01:00
Daimona Eaytoy d73a94ad30 Create helper methods for the 'warn' action
Change-Id: I62e752e0dbed4f723cc6f600085a1689f3962bd3
2020-10-29 11:10:47 +00:00
Daimona Eaytoy 7dd10ff348 Split checking vs setting throttle
This is still not very useful, but it's going to come up handy when
we'll be refactoring this code.

Additionally, fix a shortcircuit issue which caused additional throttle
types to not be processed if a type was already triggered.

Change-Id: Ied44d9300b3fa2ad00fe95c9c3da3c3f8faa650b
2020-10-29 10:17:43 +00:00
jenkins-bot ec5b9bef44 Merge "Add a service to retrieve the filter user" 2020-10-29 09:52:56 +00:00
Matěj Suchánek 77f6ecce13 Cleanup FilterProfiler API
Make FilterProfiler::getFilterProfile return stats unchanged,
in a structured way. Move computations to AbuseFilterViewEdit,
as they are only useful there. Don't return false on cache
misses, return arrays with zero values instead.

Bug: T266531
Change-Id: I8718cc31a5004340bf742315c7075e10a61fcbfd
2020-10-28 12:48:30 +00:00
jenkins-bot 5f38ddd5cf Merge "Add typehints to hook handlers" 2020-10-28 12:43:10 +00:00
Daimona Eaytoy be4ef544c4 Merge "Simplify ViewEdit, last round" 2020-10-28 10:38:15 +00:00
Daimona Eaytoy ccf8afe75b Add typehints to hook handlers
Needed after core change I95bb47104ad3dc0a69c812c627ffa631c5dc6ace to
make phan pass on master.

Change-Id: I6202212493340064945a559799e248130f418d6e
2020-10-28 11:37:07 +01:00
Daimona Eaytoy e0b187a546 Divide AbuseFilterPermissionManager::canSeeLogDetails
This commit splits this method into a version that doesn't need a
filter, and another version which requires one. This latter version has
a single mandatory parameter, $filterHidden, and it's up to the callers
to retrieve the value to pass in.

As mentioned in a TODO, this should eventually be changed to take a
Filter object (still under review as
I5f33227887c035e301313bbe24d1c1fefb75bc6a), which is also why
AbuseFilter::filterHidden is not being used here.

Change-Id: Id47a80131e12a5f7e1e93676299641dbf1e2b0ad
2020-10-27 19:51:01 +00:00
Matěj Suchánek be0268f200 Unbreak EmergencyDisable
FilterProfiler::getFilterProfile returns data in a different
format than the data is really stored.

Bug: T266531
Change-Id: I0d961a1ae67769da61f841df2462d47f81849972
2020-10-27 10:07:15 +01:00
Daimona Eaytoy 916234598d Simplify ViewEdit, last round
This deals with data inconsistencies in buildFilterEditor. Every
property of $row was tested in all 5 scenarios (also using Selenium) to
check when it's set. The result is in the normalizeRow method, which
aims to remove any inconsistencies, so that buildFilterEditor always
receives a "complete" row with all defaults set.

The code in buildFilterEditor is now cleaner (because there are no
isset() checks), and it gives us a unique place where we can set
defaults (rather than partly doing that in
loadRequest/loadFilterData/loadImport, and partly relying on isset).

This will be especially useful when introducing value objects to
represent filters, because now you just have to look at normalizeRow()
to tell which properties are allowed to be missing, and thus what "kind"
of filter object you need (see
I5f33227887c035e301313bbe24d1c1fefb75bc6a).

Additionally, reduce the properties that get passed around during
export/import, and make the selenium test try a roundtrip, rather than
relying on hardcoded data that may get outdated. A future patch will
refactor the import/export code.

Change-Id: Id52c466baaf6da18e2981f27a81ffdad3a509e78
2020-10-26 13:07:29 +00:00
Daimona Eaytoy cbea88f818 Add a service to retrieve the filter user
Unfortunately, this isn't using DI completely, because of the
User::newSystemUser call. I'm not even sure if we really need to call it
or we can just stick to new UserIdentityValue, but leaving like this for
now.
Also, the types were weakened to UserIdentity, so the transition is
going to be easy anyway.

Change-Id: I08f8fae0fcc622ff0ac3f86771476d06d1c18549
2020-10-26 14:06:53 +01:00
jenkins-bot 711f949b95 Merge "Cleanup for AbuseFilter class" 2020-10-26 11:25:01 +00:00
Daimona Eaytoy 0d751dde04 Cleanup for AbuseFilter class
Remove unused property, move to AbuseFilterView a method that's only
used there.

Change-Id: I16658521e32eeaafc1d601528d52bef17e1bf3b5
2020-10-25 15:55:21 +01:00
Daimona Eaytoy 6c9fc516aa ViewRevert: avoid needless query
The previous code would call getUserGroups again once creating the log
entry, but this was slightly flawed: we're updating groups on master,
but the read happens on a replica that might be outdated, hence
resulting in broken logging. Instead of reading from master, we can just
keep a list of the groups that were actually added, and use that
afterwards.

Change-Id: I7cc282e15561de3a3d3e183808a65991aa27d2bb
2020-10-25 10:29:59 +01:00
jenkins-bot 8fe9902af3 Merge "Use UserGroupManager when reverting degroup action" 2020-10-25 09:24:15 +00:00
jenkins-bot 50ae561641 Merge "Simplify ViewEdit, round 2" 2020-10-25 09:10:11 +00:00
Matěj Suchánek 6d81fca76b Improve FilterProfiler coverage
Also improve documentation of some FilterProfiler methods.

Change-Id: I08198c643a7d2dac10e928914e8a5c7413f2543d
2020-10-24 16:23:47 +02:00
jenkins-bot d7770ad520 Merge "Introduce BlockAutopromoteStore service" 2020-10-24 13:16:57 +00:00
jenkins-bot ba9e461ed0 Merge "Deduplicate cache keys used to check blockautopromote" 2020-10-24 12:57:11 +00:00
Matěj Suchánek 1445d5962a Introduce BlockAutopromoteStore service
This service is responsible for the blockautopromote feature:
(un)block autopromotion and check status.

The patch mostly moves code from static methods to the new class
and relaxes type hints (e.g. from User to UserIdentity).

Change-Id: I79a72377881cf06717931cd09af12f3b8e5f3e3f
2020-10-24 12:31:44 +00:00
jenkins-bot dfc9cc2a19 Merge "Code cleanup for FilterProfiler" 2020-10-23 14:43:26 +00:00
Daimona Eaytoy 5890dea4ff Deduplicate cache keys used to check blockautopromote
Previously, AbuseFilterHooks would proxy the data from a slower backend
(db-replicated) to a faster one (hash) reusing the same key. This change
makes it use a dedicated key, so that the "main" key can be kept
internal inside the upcoming BlockAutopromoteStore.

Change-Id: Id46a66991d0e994ee0a83b83b9c95e8951f3041c
2020-10-23 16:43:24 +02:00
Daimona Eaytoy 416dcd9ba3 Simplify ViewEdit, round 2
- Add a helper method to output an unrecoverable error, comprising a
   button to go back to the filters list;
- Move the token check to attemptSave, so to make the conditionals
  easier to read, and group errors together
- Make buildFilterEditor take an HTML parameter for the error, so the
  caller can specify whether it's error or warning
- Move the check for non-existing filters out of buildFilterEditor
- Add a bunch of typehints
- Don't set af_throttled and af_hit_count in the empty row template, but
  set af_deleted (these are only used in buildFilterEditor)
- Make AbuseFilter::translateFromHistory consistently include the af_global
  property (previously it would only be set for global filters; this error
  was introduced when first implementing global filters)
- The only user-facing change is that, when trying to use a custom
  warning/disallow message on a global filter, this is now considered a
  non-fatal error, so we now show the editing interface (and not just an
  unrecoverable error).

The next step is resolving the @todo in buildFilterEditor about null
checks.

Change-Id: I9d217dcac3f4cc0b26e53eca735cc327d5efc76d
2020-10-23 13:00:43 +00:00
Daimona Eaytoy 4de4ef358b Use UserGroupManager when reverting degroup action
This commit avoids direct queries on the DB, which is already an
improvement. It also adds some TODO comments for future improvements,
mostly things that depend on core changes.

Bug: T265224
Change-Id: I8eb76a0c463751976c2c5deedb3570305f1ab4f0
2020-10-23 12:07:45 +00:00
jenkins-bot cc7763f760 Merge "Add dedicated classes for more hook handlers" 2020-10-23 11:38:20 +00:00
Daimona Eaytoy 6724227182 Flatten the array returned by getConsequencesForFilters
There's no point in repeating the action name, because it's already used
as key. We can then flatten the array and just keep the parameters in
the third nesting level.

Change-Id: I54abcc49322f432cedd361abeedb72e067d3de41
2020-10-22 16:36:11 +00:00
Daimona Eaytoy b309c804fc Add dedicated classes for more hook handlers
The schema changes hook was chosen because the handler is very long. The
test ones were chosen to keep test things away from actual code.

Bug: T261067
Change-Id: Ie06bf62399f6353e3e268cccb3fe4b41bbf951c5
2020-10-22 18:23:09 +02:00
Matěj Suchánek 6b1b879da8 Code cleanup for FilterProfiler
Follows up Ib66c42ac220731f4e1da9ee6cfb5290759dd6494.

Apply DannyS712's suggestions from that patch.

Change-Id: Ib9f19969a888bd29f9f46e90fb52b49ce883c667
2020-10-22 15:39:00 +02:00
Daimona Eaytoy 4c06dd52c8 Replace $wgAbuseFilterRestrictions with more specific variables
So that sysadmins can further customize the extension. It was also wrong
to use the same variable for many different things.

Note that there's no associated patch in wmf-config because we use the
defaults. However, before merging this patch, please recheck that
AbuseFilterRestrictions and AbuseFilterDisallowGlobalLocalBlocks aren't
used there (https://codesearch.wmflabs.org/operations/?q=AbuseFilterDisallowGlobalLocalBlocks%7CAbuseFilterRestrictions&i=nope&files=&repos=)

Bug: T175221
Change-Id: I7581b3ee6d9d11a6cf1599b8ff874e8c3d54adf4
2020-10-22 13:38:59 +00:00
jenkins-bot 1c10edb80f Merge "Migrate change tags hooks to DI" 2020-10-21 18:04:20 +00:00
jenkins-bot 1c1b40f322 Merge "Inject ChangeTagsManager to ChangeTagger" 2020-10-21 17:21:23 +00:00
jenkins-bot c7e1d11c74 Merge "Add ChangeTagsManager service" 2020-10-21 17:15:40 +00:00
jenkins-bot c865e210de Merge "Simplify ViewEdit::loadRequest" 2020-10-21 16:39:18 +00:00
Matěj Suchánek 2ee3a0d247 Migrate change tags hooks to DI
Bug: T261067
Change-Id: I7b95cd19ab0ae04820e8dcb3481d29a2f9e7a0ca
2020-10-21 16:18:06 +00:00
Matěj Suchánek 93556284a0 Inject ChangeTagsManager to ChangeTagger
We decided to have the tag name provided by ChangeTagsManager,
so make ChangeTagger depend on it.

Change-Id: If3cbfd992f45651f47477031befffc0fd30f4a28
2020-10-21 16:30:43 +02:00
Matěj Suchánek 85e000c6ed Add ChangeTagsManager service
This service will be resposnsible for loading
and caching change tags used by abuse filters.

Change-Id: I9a710af1dd1ae58c47de1e8509246ed929d0a662
2020-10-21 16:24:32 +02:00
jenkins-bot f5950e638f Merge "Performance: don't check autopromotion if blockautopromote is disabled" 2020-10-21 13:20:13 +00:00
Daimona Eaytoy 7e44146781 Performance: don't check autopromotion if blockautopromote is disabled
This hook is called on every request, even for view actions, hence it's
a hot spot and a potential source of performance issues. We can slightly
optimize it by avoiding a cache lookup if blockautopromote is disabled.
Note: this won't really have an impact on WMF wikis since blockautopromote
is enabled almost everywhere.

Bug: T22487
Change-Id: I3743bfea9fe5865a3947cd23a07ae27e2dfa9301
2020-10-21 13:28:41 +02:00
Daimona Eaytoy 9bc885b6b3 Add a ChangeTagger class
The logic about action IDs and the persistent buffer is now encapsulated
inside a single service, which is a step towards getting rid of global
state in the AbuseFilter class, and reducing the responsibilities of the
Runner.

An important change made here is that we now require a LinkTarget rather
than a Title. This removes a dependency on the Title class (a monster
object), makes tests simpler, and denies the need to inject a
TitleFactory. This means living without some bits of context (e.g. we're
no longer using makeTitleSafe to ensure a valid title, and we have to
build a "prefixedtext" manually), but this shouldn't be a problem, given
that the titles are only used to create a cache key: invalid titles are
not a problem, and concatenating namespace + title should always be
sufficient.

Bug: T265370
Change-Id: Iff59cd3d889454a482a89c16691bfefcc5ec0a12
2020-10-21 13:19:30 +02:00
Daimona Eaytoy 215f16a177 Prevent uncaught warnings/exception on Special:AbuseFilter
This patch addresses two issues observed in WMF production:
 - Specifying a search mode without a search pattern would result in a
   call to mb_stripos (in AbuseFilterPager) with an empty delimiter,
   which triggers a PHP warning. Avoid this by checking that the search
   pattern is not the empty string, and unset the search mode if that's
   the case.
 - Trying to use an invalid search mode would result in an unhandled
   LogicException. We have some code in place to check the validity of
   the URL parameter, but the relevant code didn't reset the search mode
   to null, hence AbuseFilterPager would throw before we can show a
   pretty error to the user.

Bug: T265994
Change-Id: Ib19d36d6265981097bbb551783fdac8bdaa98854
2020-10-20 13:59:45 +02:00
jenkins-bot 3b59156b4c Merge "Minor updates related to var dumps" 2020-10-19 08:27:05 +00:00
jenkins-bot e002cbb4fa Merge "Exclude implicit groups when degrouping the user" 2020-10-19 07:56:41 +00:00
Daimona Eaytoy a330d0c454 Exclude implicit groups when degrouping the user
It doesn't make much sense to try to remove implicit groups like 'user'
and '*'. As a matter of fact, these groups are also excluded in
AbuseFilterViewRevert when undoing degroups.

Change-Id: I292499611ccfbd12df28b713d4244530db15c26d
2020-10-18 15:34:04 +02:00
Daimona Eaytoy 3a85e03c72 Simplify ViewEdit::loadRequest
This method was divided into multiple, shorter methods. We now have a
dedicated method for imports, and one for everything else, plus a method
for loading actions. Merged a conditional for when the token didn't
match. Avoid returning Status objects with data inside as it's too
difficult to properly infer types for those.

This is still not perfect, and another round of simplification might be
necessary before this class can be updated to use the upcoming Filter
value objects.

Change-Id: I2de1de1982105e5b9b817a893c357615ffb7db86
2020-10-18 11:06:30 +00:00
Daimona Eaytoy f589629b12 Avoid direct coupling between SpecialAbuseFilter and AbuseFilterView
While this might seem a small change, it removes the last remaining
coupling between SpecialAbuseFilter and the *View classes, that were
forming a huge tangle.

Change-Id: I5a9d6516e3fa2d3efc4bb2e19b05379dc33cd84d
2020-10-17 00:37:11 +02:00
jenkins-bot 94af753348 Merge "Use new services in AbuseFilterRunner" 2020-10-16 13:20:08 +00:00
jenkins-bot c094da9cec Merge "Simplify code for tagging the action on cache hit if the cond limit was hit" 2020-10-16 11:49:20 +00:00
Matěj Suchánek adbe9bcbce Improve display of log entries when global filters are not enabled
Don't create <a> tags without a href. Show a placeholder
message instead of nothing (alternatively, we could create
a new message for each existing one).

Bug: T174000
Change-Id: Id55b90881aacc620ff3c519ad6eedf212f36c4ed
2020-10-15 15:05:16 +02:00
Daimona Eaytoy 1efc324d97 Use new services in AbuseFilterRunner
The first one is UserGroupManager, used for the 'degroup' action. This
is a simple one-line replacement (repeated twice), and the current code
was already using this service under the hood.

The second one is BlockUser, which is not a one-line change (but still
quite simple). In particular, this allows us to avoid duplication with
core logic when constructing the log entry (this is now done by
BlockUser).

Bug: T248743
Change-Id: Ib7c1dc107a169b575f7021e64b6a8fee09529548
2020-10-14 23:08:32 +00:00
Daimona Eaytoy a7182acafd Simplify code for tagging the action on cache hit if the cond limit was hit
This code was simply caching the AbuseFilter::$tagsToSet property, but
this is not necessary. The only tag that can be buffered during edit
stashing is the conds limit tag. So we just save whether the conds limit
was hit, and apply the tag from a single point afterwards.

Also avoid checking whether 'tag' is enabled as an action, since this tag
should always be added when applicable.

Next step is creating some sort of Watcher service that will do
everything on its own: check whether the limit was hit, save this
information, and tag the action later.

Bug: T265370
Change-Id: I90319a658736fad7d564cb51152061709c230411
2020-10-13 16:05:18 +00:00
Daimona Eaytoy 45d80bc7e5 Clean up view classes
- Depend on a generic IContextSource rather than SpecialAbuseFilter
  (lower coupling);
- Inject a LinkRenderer (IContextSource doesn't have a ::getLinkRenderer
  method)
- Add a helper method in SpecialAbuseFilter to get the page title, that
  can also be used elsewhere (and the name constant can be made private
  now)
- Pull down the mFilter property (and rename it to just 'filter') to
  classes that actually need it. Some classes didn't need this at all
  and the types were different among subclasses

Now the only cause of coupling between the View classes and
SpecialAbuseFilter is the static call in getTitle.

Change-Id: I3df0c3a7621f0cc9a64a16b0a402a15aae2d5d73
2020-10-13 10:38:43 +02:00
jenkins-bot 95766762c4 Merge "Migrate a few hook handlers to DI" 2020-10-13 08:36:27 +00:00
jenkins-bot 3e61e886ba Merge "Add an AbuseFilterPermissionManager service" 2020-10-13 08:36:25 +00:00
jenkins-bot 51ce0bacf6 Merge "Delegate some switch cases to the parent in GlobalAFPager" 2020-10-12 10:13:25 +00:00
Matěj Suchánek 7ef2259228 Migrate a few hook handlers to DI
Bug: T261067
Change-Id: If699917c3d2e9e22525c7d0495554e25f6b45125
2020-10-10 17:23:04 +00:00
jenkins-bot 42525e4d5a Merge "Cleanup filter id handling on Special:AbuseFilter/history" 2020-10-10 12:28:05 +00:00
Daimona Eaytoy 2026e3ac3a Add an AbuseFilterPermissionManager service
This service should act as a mediator between the AF code and the
permission manager, and it should know what are the permissions required
by each action.

Change-Id: Ieb177d9992147b11fa7b8f05929da6c182cc2286
2020-10-10 14:03:29 +02:00
jenkins-bot f1de9145f5 Merge "Remove sorting by user from Special:AbuseFilter/history" 2020-10-10 11:58:54 +00:00
Matěj Suchánek d91ddd2169 Cleanup filter id handling on Special:AbuseFilter/history
In particular, the interface shouldn't generate links to
"Special:AbuseFilter/history/0" (AbuseFilterHistoryPager::getTitle,
can be seen when visiting "Special:AbuseFilter/history").

Change-Id: Id3dc1bb4fc3c5e853603bf0ec04a6b1751f7d862
2020-10-10 11:40:46 +00:00
Daimona Eaytoy f0539e0c1e Represent new filters with null instead of 'new'
PHP is not strongly typed, so it's not a good idea to use scalars of
different types (here it's an integer vs the string 'new') to represent
different possibilities. This can have bad effects when type juggling
occurs, and it's also harder to figure out what the type of the
parameter can be (because a numeric ID might have been passed as a
string). Using integer vs null avoids all of this, and also allows us to
use nullable typehints.

These changes were partly copied from
If981cb35bf19a8469aa6c43c907e107cf8c65bc2 and should help with the
migration to the Filter value objects.

Change-Id: I8837d46c3c33761fea53f67b530b721dc7bd49b0
2020-10-10 12:23:50 +02:00
jenkins-bot c0defc1055 Merge "Add a new FilterProfiler service" 2020-10-10 10:08:58 +00:00
jenkins-bot f4570e232d Merge "Hide filter group selector when filter selector is hidden" 2020-10-09 23:02:04 +00:00
jenkins-bot da3bfa3314 Merge "Reduce dependencies of AbuseFilter::saveFilter" 2020-10-09 14:58:30 +00:00
Matěj Suchánek 826f03d928 Remove sorting by user from Special:AbuseFilter/history
This feature didn't work and even if we fixed it as suggested
in the task, it would still be bogus. For deterministic paging,
the afh_user_text field should be in an index together with
another field(s). But currently it's indexed alone.

By the way, the indexes on abuse_filter_history should be fixed
anyway. Special:AbuseFilter/history also allows filtering by
filter/user which require index on the fields. They are present
but are not composite, so either the sorting is done
inefficiently without an index or there is a fullscan.

Also remove the getIndexField override. TablePager knows best
what value can be used there, we don't really have to override
it.

Bug: T204210
Change-Id: I7335f82c917a1d219fd7f0999da5b62433f14bd8
2020-10-09 15:41:19 +02:00
Matěj Suchánek 2c650f7710 Hide filter group selector when filter selector is hidden
This was a means to bypass the limitation to filter by
triggered filter (for example, when a group contains
a single filter).

Change-Id: Icd7b0b64ff16b4ce26f4d52ad9d9abce62972e60
2020-10-09 13:33:13 +00:00
Daimona Eaytoy 9f2906e34b Reduce dependencies of AbuseFilter::saveFilter
This patch removes the dependency of saveFilter on the ContextSource
kitchen sink. It also removes some unneded dependency, and adds
$originalRow/$originalActions as parameter, rather than hacky properties
in $newRow that are easy to forget. The related test can also be greatly
simplified.

This also introduces a behaviour change: checking $newRow instead of the Request allows us
to account for values normalization done in
AbuseFilterViewEdit::loadRequest, and to also work correctly for imports
(and generally speaking, it makes the method suitable for an
AbuseFilterEdit API module, too).

Next step is moving this method to a service. Some signatures,
indenting, name choices etc. are subpar, but this is just because these
methods are temporary anyway.

Bug: T213037
Change-Id: I235b928d7b9c2ef1c46ea0bf3e3ed212500b4161
2020-10-09 11:52:02 +00:00
jenkins-bot 73d6544bc6 Merge "Avoid array_filter on explode()" 2020-10-09 07:49:24 +00:00
jenkins-bot f2648afb15 Merge "Prevent cache pollution in fetchAllTags and clean up" 2020-10-08 18:02:21 +00:00
Daimona Eaytoy 7a1d6dbdbb Avoid array_filter on explode()
The array_filter is likely meant to empty the array if the empty string
was exploded ( `explode( "\n", '' ) === [ '' ]` ). However, it can also
remove other stuff, e.g. the string '0'. An explicit comparison is
easier to read & interpret, marginally faster, and avoids rare but not
impossible edge cases.

Change-Id: Ie77d65b56319664a2ac370f32341dc72b619a635
2020-10-08 18:56:27 +02:00
Matěj Suchánek 9e6bc2f4ee Move log formatters to a separate directory and namespace
This will clean up the includes/ directory a little.

Change-Id: I61adacf32257bb2402a272b60b52b69505d981c5
2020-10-07 16:25:38 +02:00
Matěj Suchánek b7cda4de4c Prevent cache pollution in fetchAllTags and clean up
Previously, the cached value would depend on the tags
parameter to be updated. The provided value may be
different for each call, so callers may receive
unexpected values.

For example, while core usually calls this with core-defined
hooks, our method AbuseFilter::isAllowedTag calls this
providing an empty array. If core's call happened shortly
after ours and hit cache, its array would be overwritten
with only AbuseFilter's tags, the rest would be lost.

Also do some clean up:

- only call array_filter on explode'd array
- call array_unique on the value, since it's usual that
  multiple filters share the same tag

Noticed when thinking about moving this to a service.

Change-Id: I4f4322e80ec89e48458a3bf46a1146863bec8237
2020-10-07 15:20:41 +02:00
Daimona Eaytoy 58538103c9 Delegate some switch cases to the parent in GlobalAFPager
af_actions and af_hidden are treated in the same way, so avoid
duplicating that code. Some of the remaining cases are also quite
similar (although not identical), so we might want to merge them in the
future.

Change-Id: I1b48502e077e58eb9ff459326bba18bb1d127242
2020-10-07 12:46:52 +02:00
jenkins-bot 1b0fa6bc6e Merge "Remove useless param to wfMessage" 2020-10-06 11:21:14 +00:00
jenkins-bot b0e68af5f5 Merge "ViewEdit: account for empty actions in imported data" 2020-10-06 09:34:30 +00:00
Daimona Eaytoy 11d922d1bd Remove useless param to wfMessage
Copying my investigation from I8c93e2ae7e7bd4fc561c5e8490ed2feb1ef0edc2:

This code was introduced in 2009, see rEABF0f1eb8db78bfa83ddb93427f39aad619523d8f25:

  $display = wfMsg( "abusefilter-action-$action" );
  $display = wfEmptyMsg( "abusefilter-action-$action", $display ) ? $action : $display;

And wfEmptyMsg looked like this:

  function wfEmptyMsg( $msg, $wfMsgOut ) {
    return $wfMsgOut === htmlspecialchars( "<$msg>" );
  }

so this made sense. But then, in 2010 (rMWae3ced88e535c7fd046f0ad6f0710cc87f0004ea) the function was changed:

  function wfEmptyMsg( $key ) {
    global $wgMessageCache;
    return $wgMessageCache->get( $key ) === false;
  }

without anyone removing the parameter from AbuseFilter.

Finally, in 2012 (rEABF176227e721c9475de2c2163d3b6e20ca4769c406) the usage of wfEmptyMsg was removed, and $display became a parameter to wfMessage().

Long story short, no need to pass that parameter.

Change-Id: Iad875f0c0ab5aaa06c795232638f52e9ca62786e
2020-10-05 23:39:23 +02:00
jenkins-bot 2cdec2473e Merge "Actually apply patch-afl_change_deleted_patrolled." 2020-10-05 08:15:48 +00:00
jenkins-bot 2e1afd5a51 Merge "Add test traits for uploads and account creation" 2020-10-05 07:49:38 +00:00
Daimona Eaytoy bc9898f1a1 Add a new FilterProfiler service
Change-Id: Ib66c42ac220731f4e1da9ee6cfb5290759dd6494
2020-10-04 22:00:57 +00:00
jenkins-bot b45c9205e9 Merge "Add tests for retrieving RC variables" 2020-10-04 13:38:32 +00:00
Daimona Eaytoy 6c8a29698b Add test traits for uploads and account creation
Ideally, this might live in MediaWikiIntegrationTestCase. For the
createaccount one, AuthManager should also provide a method to log the
creation, because currently we are forced to copypaste that code here.

 - Add the missing tests for 'upload' in RCVariableGenerator, and adjust
the existing ones (delete file afterwards, more tablesUsed, use the
right extension).

 - Exclude from the coverage report a couple of lines which should
theoretically be unreachable. Escalate logging to WARN level, where it's
more likely to be spotted.

 - Remove an unused method (RCVariableGenerator::newFromID). This denies
   the need to maintain and cover it. We also don't want this generator
   to act as a factory.

Overall, this change brings the coverage for RCVariableGenerator to 100%

Bug: T201193
Change-Id: I425c3d9f6800f74eb6e4eda483b90cfb3bbbcb51
2020-10-04 13:16:58 +00:00
jenkins-bot 9530684878 Merge "Use null defaults for search options on Special:AbuseFilter" 2020-10-04 12:56:14 +00:00
Daimona Eaytoy 2e13d58c74 Add tests for retrieving RC variables
This was also long overdue. Also fix a bug that caused page creations to
not be shown when examining past edits (using rc_last_oldid doesn't work
for page creations).

Bug: T201193
Bug: T262903
Change-Id: I5f7a994add12332c950904146248c5de7c2beee5
2020-10-04 12:43:04 +00:00
Matěj Suchánek eb81b92c06 Refactor AbuseFilterView instantiation
- Make a separate method which determines the view
  to be shown from subpage syntax and test it.
- Reduce circular dependency between SpecialAbuseFilter
  and AbuseFilterView. Use params to transfer information
  to views.

Change-Id: Ib9442ea5f9990a5c48f9b9e04055aa22bf7e456e
2020-10-04 13:15:04 +02:00
Daimona Eaytoy f8c525fc52 Minor updates related to var dumps
- Include an attempt to restore the dump in case the text table
   contains a truncated dump (not 100% sure that this can really
   happen, nor do I know the cause, but it shouldn't hurt)
 - Remove a check for 'action'. The variable might be missing in case of
   a corrupted dump. Having an array at that point can only mean "new
   format".
 - Don't assume that old_wikitext and new_wikitext are set when showing
   past filter hits (again, might be unset due to data corruption).

Bug: T264513
Change-Id: I7510d28fc3f43f985a1283e23b413f07adfe7921
2020-10-04 01:18:14 +02:00
Daimona Eaytoy 04d735117f Use ::class in SpecialAbuseFilter
This is a simple change but with tons of benefits:
 - Easier to track usages for IDEs
 - Easier to understand in static analysis (phan)
 - Can be analyzed by phpda
 - Ensures no typos
 - These classes can be namespaced without affecting readability here

Change-Id: Ic04d19dfbe9184baf2ef4bac53011521e2e44953
2020-10-02 12:55:19 +02:00
Matěj Suchánek ce41ddb85b Remove void unset statement
The key isn't set in the above declaration.

Change-Id: If256acc85913fca10062d00e46092e298b6553f7
2020-10-01 15:01:06 +02:00
Daimona Eaytoy 752492c2ba Use null defaults for search options on Special:AbuseFilter
- Use null instead of empty strings
- Check the mode, and not the pattern, to decide whether the user
  searched for something
- The call to parent::__construct can now be moved up
- Note in a comment how this code is problematic due to "smart"
  constructors
- Avoid caching the headers, as that's not going to work anymore.

Change-Id: I420ab0215d53354a67d9d130ebd8d85dfbd2778b
2020-10-01 13:35:36 +02:00
jenkins-bot edb1c5289a Merge "Integrate with Renameuser" 2020-10-01 11:33:08 +00:00
Matěj Suchánek 65708afcea Integrate with Renameuser
Register abuse_filter and abuse_filter_history tables.
abuse_filter_log is more difficult (if possible).

Bug: T27377
Bug: T206477
Change-Id: If8289101a08887519d5a90ef84700421b8ed2406
2020-10-01 08:10:22 +00:00
Matěj Suchánek f851b529b3 Deduplicate instance variables in Pagers
These have been saved in the parent class for quite some time.
Refactor accessors in method overrides.

Change-Id: I9819caa5ab87ac3a8e47efb32b00d89c3e2a61af
2020-10-01 08:05:49 +00:00
jenkins-bot fa412f4e7e Merge "Rewrite the VariableHolder code to translate deprecated variables" 2020-09-30 09:10:37 +00:00
Daimona Eaytoy 1bdf4e5351 Rewrite the VariableHolder code to translate deprecated variables
The current code was more of a subpar, temporary solution. However, we
need a stable solution in case more variables will be deprecated in the
future (T213006 fixes the problem for the past deprecation round). So,
instead of setting a hacky property, directly translate all variables
when loading the var dump. This is not only stable, but has a couple
micro-performance advantages:
 - Calling getDeprecatedVariables happens only once when loading the
   dump, and not every time a variable is accessed
 - No checks are needed when retrieving a variable,
   because names can always assumed to be new

Some simple benchmarks reveals a runtime reduction of 8-15% compared to
the old code (8% when it had varsVersion = 2, 15% for varsVersion = 1),
which comes at no cost together with increased readability and
stability. It ain't much, but it's honest work.

Change-Id: Ib32a92c4ad939790633aa63eb3ef8d4629488bea
2020-09-29 15:06:14 +00:00
jenkins-bot 32baa3a166 Merge "Hide rule search if global filters are to be shown" 2020-09-29 14:18:17 +00:00
jenkins-bot 7a684c487c Merge "Move some misplaced AbuseFilterParser entry points" 2020-09-29 13:51:17 +00:00
jenkins-bot 261e551856 Merge "Fix check for central wiki" 2020-09-29 13:25:03 +00:00
Matěj Suchánek aa7f381bca Fix check for central wiki
It should be the other way around.

Change-Id: Ia55c70a9f5ac17d791352899ee0d38c4969ea6dd
2020-09-29 12:59:34 +00:00
Matěj Suchánek 86ad51a57a Hide rule search if global filters are to be shown
When the user selects to see global rules and it's a remote wiki, hide the rule search field. (Note that the list of search modes needs to listen to this setting as well.)

This was discussed during reviewing I0771fa048.

Also move local/global filters setting to the top as it's more important than that for disabled and deleted filters (which will both stay together).

Change-Id: I0912aa1f5d7a5d75e6ae5a2a3362b8d38260c611
2020-09-29 13:49:11 +02:00
Daimona Eaytoy 55ba083b13 Introduce a KeywordsManager service
This will decouple a bit the huge and chaotic tangle of AF classes. Some
boilerplate code for AbuseFilter services is also added with this patch.

Note that this requires injecting a KeywordsManager in
AbuseFilterVariableHolder, or unit tests would fail. This is still
incomplete, and the Manager is only injected in tests, because
VariableHolder still has to be refactored.

The test for the UpdateVarDumps script had to be updated, because
serializing VHs in there was a bad choice. As pointed out in a comment,
the test is likely going to break again once we remove the BC code, but
I hope that we'll be able to remove the test at that point.

Change-Id: I12a656a310adb8c5f75cab63f6db9e121e109717
2020-09-28 23:03:52 +00:00
Daimona Eaytoy a1626a0d7f Move some misplaced AbuseFilterParser entry points
These methods had no reals reason to be static and belong to the
AbuseFilter class. Most of them were moved to Parser class as common
variations of the existing entry points. One was specific to the
EvalExpression API module and was moved there.

This change comes at no cost, and will make it possible to inject a
parser where needed.

Change-Id: Ifd169cfc99df8a5eb4ca94ac330f301ca28a2442
2020-09-29 00:36:08 +02:00
Daimona Eaytoy 8fa9e6625a Add tests for 'upload' action
This adds some coverage for the *VariableGenerator classes. It's still
not perfect, but something to start with in sight of future
refactorings.

Bug: T201193
Change-Id: Iafa85fb8623ea278ce6e42118df72751806382c2
2020-09-28 11:53:53 +00:00
jenkins-bot f07f7348ee Merge "Move link to /import in a button on ViewList" 2020-09-27 08:50:58 +00:00
Matěj Suchánek 5fcb826519 Hide rules search options if the pattern is empty
This reduces the consumed space if the user is not going to search through filter rules.

Change-Id: Ic53edeab75f8110871bdf69afc1184ea7d72cee9
2020-09-25 17:47:34 +02:00
Martin Urbanec eeb7ee8cef HookRunner: onAbuseFilterGenerateUserVars should run generateUserVars
Bug: T263750
Change-Id: I23751b78f363f35ca47f9af5c0c70129c838f4c6
2020-09-24 16:40:03 +02:00
jenkins-bot a9309c9921 Merge "Revert "Drop duplicate index wiki_timestamp"" 2020-09-23 11:06:12 +00:00
Daimona Eaytoy 01cc79e1d4 Revert "Drop duplicate index wiki_timestamp"
This reverts commit 6a268e7339.

Reason for revert: Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae ended up not renaming the index, so this patch removed an index that was still in use.

Change-Id: Ide4a600a57bcfa4da0c7354b972cc89709ccd660
2020-09-23 10:39:19 +00:00
jenkins-bot b6c21df589 Merge "Inject a user into RCVariableGenerator" 2020-09-22 12:10:29 +00:00
jenkins-bot 8e75557ac8 Merge "ViewTools: hide the result box when empty" 2020-09-22 11:59:21 +00:00
DannyS712 801e1f57e5 Inject a user into RCVariableGenerator
Needed in ::addUploadVars

Bug: T263033
Change-Id: Iedde4a39dcc3192616e36a45690a0619efeb7309
2020-09-21 16:15:21 +00:00
Alexia E. Smith 422b77ab0e Actually apply patch-afl_change_deleted_patrolled.
This fixes the abuse_filter_log patch-afl_change_deleted_patrolled
not being applied. The patch is provided for (and should work with) all
the supported DBMS.
Additionally, fix the base table files, which would report
afl_patrolled_by as 'NULL', whereas on the WMF cluster it's 'NOT NULL
DEFAULT 0'. The schema patch takes care of converting that column as
well.

Note that this schema change needs not be applied on the WMF cluster, as
that's already up-to-date.

Finally, note that this patch must be backported to 1.33 and 1.34 (and
it might be fairly hard due to the recent schema changes on the
abuse_filter_log table).

Bug: T240895
Change-Id: Ibdbc9b50c25b9e871ebdeae93a54d10877b585f8
2020-09-21 14:52:22 +00:00
jenkins-bot ed160a69a7 Merge "Let sysops see difflinks to deleted revisions on Special:AbuseLog" 2020-09-20 14:44:16 +00:00
jenkins-bot 6140d688f7 Merge "parser: Add a BC option to get DNULL for unset variables" 2020-09-20 13:58:27 +00:00
Matěj Suchánek 4605baa289 Let sysops see difflinks to deleted revisions on Special:AbuseLog
Bug: T261630
Change-Id: I01eeecea28cbd3520702155860b340ea673bab0d
2020-09-20 15:41:12 +02:00
Daimona Eaytoy c1b4f1084c ViewTools: hide the result box when empty
The <pre> element is now hidden with CSS, and is only shown after the
user clicks the "Eval" button.
Moreover, make the button primary and progressive, as to indicate that
it activates the primary function of that page.

Bug: T253492
Change-Id: I300ce6ec0a84ea73025a5af9173024df7c291e03
2020-09-19 12:37:06 +00:00
Matěj Suchánek f1ecdd4aff Inject PermissionManager to SpecialAbuseLog
Change-Id: I1c80490567ac2d9f716c988ebdad6b59cf28aa06
2020-09-18 23:22:11 +00:00
jenkins-bot f1ab4a1777 Merge "Cleanup abuse log code and join it with revision" 2020-09-18 23:05:37 +00:00
Daimona Eaytoy f8c9b8fa36 Move link to /import in a button on ViewList
We have many topnav links, and future patches may add others (e.g.
Ia5fd4f0b35fcabf045a7b49fa40fa85b72c92544). The "import" feature is
probably the less used, and is also pretty similar to creating a new
filter.
Thus, remove its link from the topbar and move it to a button next to
the "Create a new filter" button.
Note that the old message is reusable, and thus it should be moved on
translatewiki after merge.

Change-Id: I52042d62b2bab7e4a1e9bbc027e7de5addec8157
2020-09-18 14:59:00 +00:00
Daimona Eaytoy 8a7a576cb0 ViewEdit: account for empty actions in imported data
Empty actions are JSONified as [], not {}, hence they're not objects.

Bug: T252181
Change-Id: Ieb5e315ad87bd3a489ade26f5f0dd202810ae896
2020-09-18 14:52:43 +00:00
Daimona Eaytoy e5746bbb0e parser: Add a BC option to get DNULL for unset variables
While checking a filter, if a variable is not set (e.g. added_lines for
an account creation), the VariableHolder will return a DNULL, rather
than a DUNDEFINED. This means that some filters will resume working, and
the WMF servers will stop getting AF warnings at a rate of 4 millions per
day. This also requires adjusting some tests to reflect the new
behaviour (which is actually the OLD behaviour, that filters had until
last year when we introduced the DUNDEFINED data type). It also requires
adjusting a check in the old parser, but that's not really relevant
because the plan is to remove the old parser before 1.36 is released
(see I0e75f334c7e0dfc1239f2e5f5f7d7452b0bbf29e).

Bug: T230256
Change-Id: I4d06303047397674c1edbfc32628f1bc83ac3340
2020-09-18 15:05:58 +02:00
jenkins-bot 36a0f41873 Merge "Add separate abusefilter-log-search-filter-help-central message" 2020-09-18 07:55:02 +00:00
DannyS712 a75e01dcb6 Add separate abusefilter-log-search-filter-help-central message
On the central abuse filter wiki, show a different help message

Bug: T238510
Change-Id: I7f60e279f0301b1636e19a31535cb3bac87c241a
2020-09-17 23:50:35 +00:00
Umherirrender bd45434102 Add MessageLocalizer to AbuseFilter::getActionDisplay
Avoid global state when parsing messages

Change-Id: Ib65182f6d41430fb87337082a16b8006a73fe95d
2020-09-17 22:45:52 +02:00
jenkins-bot 3f8e61b42f Merge "Allow Blockautopromote duration to be configured for wikis." 2020-09-17 17:53:06 +00:00
DannyS712 bf74fd0c23 Allow Blockautopromote duration to be configured for wikis.
Rather than always using 5 days, the length (in days) can be configured by setting
`AbuseFilterBlockAutopromoteDuration` to the desired length.

Bug: T231756
Change-Id: I996e08a9099ab59657fe511ec2934d26edfa5c7b
2020-09-17 17:19:00 +00:00
Matěj Suchánek 02962b9665 Cleanup abuse log code and join it with revision
This is an intermediate step for better "diff" links
on abuse log. With this first change, only links
to existing revisions are shown.

Change-Id: Ib420d46fd34dc38d8c7fd3d511a905738e49db0b
2020-09-17 16:36:31 +02:00
jenkins-bot 12586812c2 Merge "Hide "User:" prefix from autopromote log entries" 2020-09-17 11:26:57 +00:00
jenkins-bot 6bf5e2ce6f Merge "Standardize the order of options in dropdown filters for search" 2020-09-17 11:26:55 +00:00
Matěj Suchánek cba29fe85b Hide "User:" prefix from autopromote log entries
Bug: T247173
Change-Id: I40aa888bc45d8274d03eb7ead7bedaf1d087fb1c
2020-09-17 12:12:00 +02:00
DannyS712 9c1868d55e Update hook calling to use new HookContainer
Bug: T254306
Change-Id: Ic5c82a367e34135bbc0f00ece5aeef4f2d92881b
2020-09-17 10:05:45 +00:00
jenkins-bot 001d8b92fb Merge "Hardcode 'abusefilter-view' right when adding CU log entry" 2020-09-15 11:16:33 +00:00
jenkins-bot 91f4a1e5a8 Merge "AbuseFilter: Remove duplicate filter log link" 2020-09-15 01:44:59 +00:00
proc 3e5b9f18fd Hardcode 'abusefilter-view' right when adding CU log entry
Bug: T255506
Change-Id: I397e1160d0fee28873ff73404fefa8edd08652ac
2020-09-15 01:20:12 +00:00
DannyS712 1601bbf0f6 Reduce direct references to $wgUser
Bug: T246733
Change-Id: I2c919fcb01476e8299e15046789023b42cccc6ee
2020-09-13 22:49:46 +00:00
C. Scott Ananian 1bd6f2aa94 AbuseFilterViewEdit: only invoke Language::filterNum on a numeric string
Bug: T237467
Change-Id: I9dcbe91fa926dba1cfd24d9bf075ee1ebef36b9e
2020-09-09 01:38:20 -04:00
Umherirrender f932ba8328 Use LinkBatchFactory in Special:AbuseLog
Change-Id: I2ccf9cd36475a65e61ad0e80ec159f841849089f
2020-09-06 09:31:49 +00:00
Ammar Abdulhamid 679c777c33 AbuseFilter: Remove duplicate filter log link
For history action, the link would be already added by
HistoryPageToolLinks hook, so it should not be duplicated by this hook.

See images on https://phabricator.wikimedia.org/T261087#6430172

Change-Id: Ia8dd5be49d3ffb48f298ea287e0b2f98c3052015
2020-09-03 17:55:42 +01:00
jenkins-bot 03f1190bf9 Merge "SpecialAbuseLog: Add redirect=no to page link" 2020-08-31 06:34:01 +00:00
jenkins-bot 08f68c77c2 Merge "Use LogFormatter::getLinkRenderer in AbuseLogHitFormatter" 2020-08-29 00:23:40 +00:00
Umherirrender 64ef0fe00c Use LogFormatter::getLinkRenderer in AbuseLogHitFormatter
Available since 1.30

Change-Id: Ia5e0d5561692f78ac91feca5dddcb67d2809a9ba
2020-08-29 00:03:26 +00:00
DannyS712 f06a632c3d SpecialAbuseLog: Add redirect=no to page link
Bug: T247615
Change-Id: Ifa8f7b949336ae735fd0067dbc2dec15748be7cf
2020-08-28 23:19:54 +00:00
DannyS712 bf0ed6d631 Fix abusefilter-log-cannot-see-private-details
Should be privatedetails, not private-details

Change-Id: I58d8f0ce760e92739876cc783b8dd4258965cd1e
2020-08-28 21:29:37 +00:00
jenkins-bot e5cefc7d18 Merge "Hard-deprecate a few methods in the AbuseFilter class" 2020-08-25 23:48:33 +00:00
Daimona Eaytoy f673a04026 Add updateVarDumps to update.php
This shouldn't happen before the script has been tested thoroughly on
WMF wikis with --dry-run.

Bug: T213006
Change-Id: I51425c85bd6932a5c60eb870b02195aae1c24117
2020-08-25 12:49:00 +00:00
jenkins-bot 14be09701b Merge "Use $user param when filtering edits" 2020-08-20 07:10:52 +00:00
Daimona Eaytoy 28ea0e525a Use $user param when filtering edits
This can be different from the User set inside the $context object, as
seen e.g. in Wikibase jobs. Given that the hook provides a $user param,
it makes more sense to use that, rather than extracting it from the
ContextSource kitchen sink.

Bug: T258717
Change-Id: Ib5961068d3df6ae2bfc3f9c6a7b9e555d248b332
2020-08-19 14:24:57 +02:00
Daimona Eaytoy 5faea5ee58 Add BC hack for some 2009 AbuseLog entries causing a fatal error
Some AbuseLog entries from 2009 are missing the 'timestamp' parameter
used to compute the old wikitext of the page. This was only used for a
short amount of time before
https://phabricator.wikimedia.org/rEABFd1d27eede6536067c5180b2515ea937d71525d4d.

Nowadays, it's causing a fatal error when we try to migrate the affected
entries, see T246539#6388362.

Since we only have a Title available, we cannot rebuild what the old
wikitext would look like, so a placeholder text is used (this should
hopefully be clearer than showing an empty string).

Bug: T246539
Change-Id: I5230f2fdc84da121728a5a75da458f1a4ef1ecd3
2020-08-19 12:37:40 +02:00
C. Scott Ananian a135c2f4da Remove calls to ParserOptions::setTidy()
ParserOptions::setTidy() was already a no-op in MW 1.35, and
AbuseFilter already requires MW >= 1.35 in extension.json.
ParserOptions::setTidy() was deprecated in MW 1.35 and will be removed
in a future release.

Bug: T198214
Change-Id: I269e829cf1f33e233bfcf7f95388e041180c2556
2020-08-12 23:39:36 -04:00
Umherirrender e10b7e4208 Fix broken PHPDoc comments not starting with /**
Change-Id: Id9579ad6ca4dc75921e8fd4aaaccdffd530c2e35
2020-08-09 01:04:29 +02:00
Timo Tijhof 314a1f419f Avoid use of unfiltered 'getTraceAsString' in debug logs
These render string arguments with potentially sensitive information
that we don't want to store in debug logs. Use the standard
'exception' field instead per T233342, letting the central
logic be in charge of creating 'exception.trace' with the normalised
trace rendering and filtering logic we have there.

Bug: T233342
Change-Id: I4620f36229fd5076b4370d20149c890030bf4c64
2020-07-23 22:41:27 +01:00
proc a31f4e46af
Strict type comparison
Bug: T248806
Change-Id: I039ab7f103bb37052987b815412b71f70643a6d2
2020-06-27 15:55:57 +01:00
Huji Lee b523b71fc3 Standardize the order of options in dropdown filters for search
Any should always be the first choice. Other/None should always be
the last choice. The rest of the choices come in between and should
be sorted alphabetically.

Also capitalize the first letter of "None" for filtering logs down
to those in which no action taken. This makes the options uniform.

Bug: T255533
Change-Id: Id106bbc352531437af95a303b7dcf32e44383f95
2020-06-25 19:01:16 +00:00
Daimona Eaytoy 3d06e2d165 Hard-deprecate a few methods in the AbuseFilter class
All usages fixed, see https://codesearch.wmflabs.org/search/?q=AbuseFilter%3A%3A(generate%7CgetEditVars)&i=nope&files=&repos=

Depends-On: I49c635efbe46820e6340f64504f3bc417b78dde3
Change-Id: I45d11b2b015f4abf1ec9cedd14355f9c1c049bba
2020-06-23 14:48:42 +02:00
Ostrzyciel 99e2766f26 Update PageSaveComplete hook to use EditResult
Bug: T254074
Change-Id: I3922d9a92242c9a6469058a2a2c2a95891e9e429
2020-06-23 14:25:38 +02:00
Umherirrender c55a12993d Handle null from VariableGenerator::getVars in AbuseFilterViewExamine
Bug: T255633
Change-Id: I94ce4303d9250b84181b1cb230e680c2351d887b
2020-06-17 19:42:58 +02:00
DannyS712 4b35336638 Update hooks to use PageSaveComplete
Extension requires MW 1.35+, always available

Bug: T250566
Change-Id: I60cf3cc42db989d8ccb0d06d3cf9eae8a85784ac
2020-06-16 04:18:39 +00:00
Daimona Eaytoy 6a268e7339 Drop duplicate index wiki_timestamp
For MySQL it was renamed Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae, but
the old index isn't being deleted, hence creating a duplicate.

Change-Id: I09b9f64759f6a897c393caa77458d63995d5713b
2020-06-12 19:34:41 +02:00
Umherirrender 6e2d7931a5 Pass function name to database functions
Useful for logging

Change-Id: Ib6b3ad814dd24d31aab37be486ff32d3f57c7905
2020-06-07 01:54:41 +02:00
jenkins-bot 1f72bc838c Merge "Do not abuse RCDatabaseLogEntry" 2020-06-05 12:08:05 +00:00
Umherirrender 82f549bed9 Do not abuse RCDatabaseLogEntry
When using RecentChange::getQueryInfo() it should be used to instance a
RecentChange class
RCDatabaseLogEntry is only useful in context of LogFormatter

This is a breaking change for the hook variable,
but "RC entry" refers more to the RecentChange class than to the
RCDatabaseLogEntry class

Change-Id: I3af1e42594f8235be815ce38e3411c762ae01092
2020-06-04 21:28:23 +00:00
libraryupgrader ef7f867f35 build: Updating mediawiki/mediawiki-phan-config to 0.10.2
Additional changes:
* Removed phan-taint-check-plugin from extra, now inherited from mediawiki-phan-config.

Change-Id: Ib63be75df4bfdbd2c5b97de5f80dbec715108c01
2020-06-02 21:33:38 +00:00
Daimona Eaytoy b9346a3eba runner: Improve blocking methods
This is necessary for future improvements.

Bug: T234164
Change-Id: Ia334cdc84ac1408ad72ffd8c87c958ae7deebb54
2020-05-28 18:42:31 +00:00
Daimona Eaytoy 61efc9ad87 runner: Move the filtered action to a class prop
Change-Id: I4d7666f5e2a5df0e0ed8d38855cbd32c8518a28f
2020-05-28 16:17:34 +00:00
Daimona Eaytoy 8d435ee463 Make the form on Special:AbuseLog collapsible
The form is now collapsed by default, as that seems to be the most
common way to do that.

Bug: T252584
Change-Id: Ie3fa3d2858519e6bc03854a12f90f76a684e7648
2020-05-14 18:56:53 +00:00
DannyS712 4844bfe26a Use ParserOptions::newFromUser() instead of relying on global $wgUser
Bug: T246861
Change-Id: Ie304f962c8d1e23c897e87471990db85a7d909de
2020-05-11 14:27:36 +00:00
Reedy 7a2373f7d8 Remove some SQLite specific files
Bug: T251613
Change-Id: Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae
2020-05-04 17:50:18 +00:00
jenkins-bot b118fd50dc Merge "Improve var dumping in /details, /examine and /tools" 2020-04-29 20:00:54 +00:00
DannyS712 1b65bd1862 Remove a remaining use of Revision objects
Remove use of Title::getFirstRevision and Revision::getUserText

Bug: T249393
Bug: T250579
Change-Id: I0f77b124a0c7de1dec6baf4c997e0997ecdd55f8
2020-04-23 18:39:20 +00:00
Daimona Eaytoy 1d6b9f6617 Add new methods for checking DUNDEFINED recursively, use them
The problem is explained at T250570#6068702; basically, the previous
check didn't account for DUNDEFINED nested deep inside arrays.

Bug: T250570
Change-Id: Iacee2db54ca00108de6339bb3dae70af7e2eeb56
2020-04-19 13:58:14 +02:00
Aaron Schulz 6375287964 Cleanup unique keys parameter to IDatabase::replace() calls
Bug: T248147
Change-Id: Ic355dece67eda323e5c6129ef4d1275c1235fb9a
2020-04-17 15:10:27 -07:00
Daimona Eaytoy 4c98aecf4d Improve var dumping in /details, /examine and /tools
Using var_export for better visual effect, especially for arrays.
The result from /tools is much clearer and the 'wrong syntax' message is
a bit more explicative than before.

Bug: T190653
Bug: T239972
Change-Id: I79a17305c7f19f7900f896f895e9365bb5f2fd58
2020-03-28 17:35:43 +01:00
DannyS712 25fd11f7ac Use lowercase for primitive type 'string'
Change-Id: I65d322ffbed399fddb1fbf3cb25272e812171a46
2020-03-20 16:00:05 +00:00
ArtBaltai 096fe88993 Replace usage of deprecated Page in favor of WikiPage/Article
Use Article instead object/Page
delete deprecated unused method

Bug: T239975
Change-Id: Ic3c8e3bee66c63177f51621bf727029a87b51105
2020-03-19 04:03:15 +03:00
Daimona Eaytoy cd1a8efb90 Minor fixes for the updateVarDumps script
- Increase batch size to 500
- Add an option to print progress markers
- Fix some bad logic which caused some JSONified data to be stored in
the text table without checking (and respecting) old_flags. This caused
some errors on the beta cluster.

Additionally, add a return typehint to AbuseFilter::loadVarDump to make
sure that errors are caught asap. Not only there's no apparent way that
loadVarDump can return an array, but most code is already using the
result as a VariableHolder, unconditionally. This is probably another
leftover from the past.

Bug: T213006
Bug: T246539
Change-Id: Iaebd28badb70d27693fa809cad4db956881e3e5e
2020-03-03 18:31:52 +00:00
jenkins-bot 76f8cb0ced Merge "Add fixOldLogEntries to update.php" 2020-02-28 18:35:01 +00:00
jenkins-bot 7e66b01cf3 Merge "Factor out get(Local|Global)Filters methods" 2020-02-27 20:33:56 +00:00
Daimona Eaytoy c9a4146f2c Add fixOldLogEntries to update.php
Already done in WMF production, see T228655.

Bug: T208931
Change-Id: Id477387d1607e263a6bf054060dc6dd440e88467
2020-02-27 18:19:50 +00:00
jenkins-bot 08c0a3f482 Merge "ViewEdit: add af_id to the row" 2020-02-26 16:53:35 +00:00
jenkins-bot b28e9e8c1f Merge "Start using new format for var dumps" 2020-02-26 16:51:02 +00:00
jenkins-bot 22adab7159 Merge "Stop using the Revision class" 2020-02-26 16:47:37 +00:00
Daimona Eaytoy f454e60e83 Start using new format for var dumps
Migrating old log entries is I22cf698c5be77506727cbd227c67e037a5d89b5c.

Bug: T213006
Change-Id: I3242acd5c5163a941f584d6119e3ad3b3cad8c29
2020-02-26 16:03:38 +00:00
jenkins-bot 8de3ecdb89 Merge "Replace usage of deprecated Page in favor of WikiPage/Article" 2020-02-26 15:49:19 +00:00
Daimona Eaytoy 518c176754 Stop using the Revision class
Change-Id: Ie257c9b1ea94dcadce59f4541d5947465262bd75
2020-02-26 15:39:12 +00:00
Daimona Eaytoy fe28aff82a ViewEdit: add af_id to the row
A PHP notice is sporadically emitted in production, e.g. reqId XlVEMgpAMNAAA6zMVhQAAACV

Change-Id: Ie42d00c6520aa31daf127c5df9515a3ab01d986f
2020-02-26 15:27:54 +00:00
Daimona Eaytoy b9a1e86245 Remove old number syntax
Bug: T212730
Change-Id: I7573da1683efc83b5002b8948c97dd7f6658a488
2020-02-25 23:38:19 +00:00
Daimona Eaytoy 1bac110205 Remove dependency on $wgRestrictionTypes
This was used to dynamically generate *_restriction_* variables.
However, it had two big problems:
 - We only have i18n for 'create', 'move', 'edit', and 'upload' (the
 default value of the global); other restrictions would show missing
 messages in various pages.
 - We had to access the global state in various points.

This change also makes some code in AbuseFilterVariableHolder simpler,
and also allows us to make AbuseFilterTest a unit test.

Change-Id: I321ad6e07f8243200af67a581b6e485970efd3ce
2020-02-25 23:17:54 +00:00
jenkins-bot e802e6556d Merge "Add string typehint to $summary in onParserOutputStashForEdit" 2020-02-25 22:56:27 +00:00
ArtBaltai 22925c5344 Replace usage of deprecated Page in favor of WikiPage/Article
Complete WikiPage/Article split and deprecate Page interface
Using actual WikiPage/Article contract

Bug: T239975
Change-Id: I343c3ca2e30715656950cab49c6470061c72b9a0
2020-02-26 01:03:49 +03:00
Ammar Abdulhamid 323fe4666c Draw suppression reason from Revdelete-reason-dropdown-suppress
Bug: T245990
Change-Id: Ic5adcf4e6693cb5b4c849156e54d97cf35b70dee
2020-02-24 14:06:43 +01:00
Daimona Eaytoy aa15267c79 Add string typehint to $summary in onParserOutputStashForEdit
Bug: T245928
Depends-On: I8fa287f335e90a59ac18365e7401a5cf703130a3
Change-Id: Ia075e4bdf3a3f011a181c8026ff1cdb8e186f096
2020-02-22 19:27:05 +00:00
jenkins-bot 76a1be97a4 Merge "Add site name and language variables" 2020-02-10 19:06:01 +00:00
jenkins-bot b6ca1402d0 Merge "Rename addStaticVars and related hook" 2020-02-10 19:03:34 +00:00
Daimona Eaytoy 0d2cab0deb Validate imported data
At the moment there's no validation for import data, so it's totally
possible to insert rubbish in the field, and the code will produce other
rubbish. For instance, it's not so uncommon to see lots of PHP notices
on logstash for ViewEdit code trying to access members of the imported
data as if it were an object.

Change-Id: If9d783f0f9242d3d1bc297572471e62f51ee0e40
2020-02-10 18:41:36 +00:00
Daimona Eaytoy 57415d8a31 Fix PHPNotice caused by missing row fields
Follow-up Ie9aae938cca06e38a7a834a3f74f3e8735ab01ee.

Some fields are actually necessary when the filter isn't saved. This
would cause PHP notices when showing the editor again.

Change-Id: I2b9e0f04b3e8ad4eea8e334e16ee422bb40f0eb5
2020-02-09 13:36:36 +00:00
Daimona Eaytoy d9ae71f578 Add site name and language variables
In T43172 it was told that adding the site name could increase the risk of
attracting more spam, but I don't see how this variable could cause that.

Bug: T240948
Bug: T97933
Change-Id: I1d2aeabaf008ac06798b8d7e4af7d61ae1702776
2020-02-09 14:32:02 +01:00
Daimona Eaytoy 661a77f0eb Rename addStaticVars and related hook
This code was introduced with Iba59fe8d190dd338ecc8cfd682205bce33c9738b
and is unused since then. The name should highlight that those variables are not
supposed to be "static", i.e. immutable. Examples are: timestamp, spam
blacklist, site name, site language. These are not immutable, but rather
"generic", and they're known even without an ongoing action.

Also add an RC row param and update docs.

Change-Id: I402f04585e9154059fc413e527e39dcb8e6b3d7c
2020-02-09 14:29:08 +01:00
jenkins-bot 391bbee53c Merge "Fix some edge cases in ViewEdit" 2020-02-08 16:36:19 +00:00
jenkins-bot c0b58d7699 Merge "Factor out variables-related methods" 2020-02-08 14:42:13 +00:00
Daimona Eaytoy 0834f37e42 Fix some edge cases in ViewEdit
Follow-up Iabd0ae5b18571f8cad44ef2d86bcf2519e7f95ba.

This patch:
 - Moves some save-related code to a separate method
 - Reduces conditionals nesting
 - Fixes an edge case where the content of the form would be
 wiped in case the token didn't match.
 - Adds another (basic) selenium test
 - Standardizes return types
 - Moves data load outside of buildFilterEditor

Change-Id: I89444b59f04c495c9ab59244151c8ed5d38cf0fe
2020-02-08 15:35:46 +01:00
jenkins-bot 430058f2c0 Merge "Avoid keeping superfluous row properties" 2020-02-07 21:26:34 +00:00
jenkins-bot 02cd866f53 Merge "Refactor data load in ViewEdit" 2020-02-07 21:26:32 +00:00
Daimona Eaytoy 3f83e57ad7 Factor out variables-related methods
This is another step needed to reduce the size of the gigantic
AbuseFilter and AbuseFilterHooks classes. It also makes many methods
non-static, for more testability.

Note, this layout is still not final. We should somehow merge the
functionality of VariableGenerator and AFComputedVariable, for which
I already have plans.

Change-Id: I366d598b69ad866496b7cb0059e0835c02e54041
2020-02-07 20:27:26 +00:00
Daimona Eaytoy 1686042a91 Move variable generators to new classes
RunVariableGenerator is for generating variables based on the current
action;
RowVariableGenerator is for RC entries;
VariableGenerator is the generic one.

This patch only moves the methods to the new classes, to keep the diff
easier to read, and facilitate conflict resolution. These classes will
then be revamped in I366d598b69ad866496b7cb0059e0835c02e54041.

Note that these classes are now namespaced.

One method, AbuseFilter::getEditVars, was renamed to
AbuseFilterVariableGenerator::generateEditVars, because it would
otherwise conflict with an incompatible method in RunVariableGenerator.

Change-Id: Iff412e5492873d4fae55402939a51609e64d55a8
2020-02-07 19:44:31 +00:00
Daimona Eaytoy 6b7be78534 Use RCDatabaseLogEntry as wrapper in get*VarsFromRCRow
This provides various shortcuts for user, target, comment, etc.,
avoiding direct access to the row, and thus a dependency on the
schema.

Change-Id: I250f94e0ac6cade33441a31ae8a27093a4d937a0
2020-02-07 19:19:10 +00:00
Daimona Eaytoy 472d1221bd tests: Increase and rebalance code coverage
Also fix a couple of broken tests in Consequences:
 - For createaccount, $user->addToDatabase must be called before
   testForAccountCreation, or it will throw a CannotCreateActorException.
 - In testThrottleLimit, also set wgAbuseFilterEmergencyDisableThreshold
   to avoid relying on the local config.

Bug: T201193
Change-Id: If1a50b0a729e4d554485f2e2225d5877510966b6
2020-02-07 18:32:17 +00:00
libraryupgrader a14ec744f7 build: Updating composer dependencies
* mediawiki/minus-x: 0.3.2 → 1.0.0
* mediawiki/mediawiki-phan-config: 0.9.0 → 0.9.1

Change-Id: I119f4d56cce674302f34e938e598e6cc6bf28dc0
2020-01-28 17:51:38 +00:00
Ammar Abdulhamid 641aeebbcf Replace deprecated IP class with IPUtils
Bug: T242556
Change-Id: If8e9034885726b673d1500fa8b538b5302e66165
2020-01-24 18:27:26 +01:00
Daimona Eaytoy 102789f62a Avoid keeping superfluous row properties
Most of them are overwritten either in ViewEdit::loadRequest or
AbuseFilter::saveFilter. af_hit_count and af_throttled are actually
relevant for the old version, so list them explicitly. And also add
default af_group and af_global, which are later read, for import action.

Depends-On: Iabd0ae5b18571f8cad44ef2d86bcf2519e7f95ba
Change-Id: Ie9aae938cca06e38a7a834a3f74f3e8735ab01ee
2020-01-23 12:50:03 +00:00
James D. Forrester bdef1200f8 Follow-up 87459ec: When no registration date is recorded, use 2008-01-15
Before the phan upgrade, this was silently choking on null as so falling
back to age since 1970-01-01 (~50 years); since the upgrade, the code is
breaking filters by responding with 0. The approximation of using 2008's
Wikipedia Day is less wrong and more fun (credit to Roan for making this
suggestion).

Bug: T243469
Change-Id: Ibc25ab09ecd0bf0b2292425c2768b1dc911b9974
2020-01-22 15:38:09 -08:00
jenkins-bot 5db1032618 Merge "Simplify throttling code" 2020-01-22 17:19:52 +00:00
jenkins-bot 81fd6af030 Merge "Actually record all filters in total_filters" 2020-01-22 17:19:50 +00:00
jenkins-bot 70d31da673 Merge "Stop using deprecated stuff with easy replacements" 2020-01-22 16:44:57 +00:00
Daimona Eaytoy 53b9f38888 Refactor data load in ViewEdit
Instead of having a single loadRequest method (which could end up
loading from the DB...), split it in a DB-only method and a request-only
one. Simplify the logic used to show the filter editor. Show the page
without changes or warnings if the user lost editing rights in the
meanwhile. Avoid two static properties, and pass them in when relevant
instead. Bonus: optimize a query to sort by afh_id instead of afh_timestamp to avoid filesort.

This will allow a subsequent patch to clean the $row object in
loadRequest.

Change-Id: Iabd0ae5b18571f8cad44ef2d86bcf2519e7f95ba
2020-01-21 14:15:41 +01:00
Daimona Eaytoy e9fe252def Fix remaining PHPCS issues
Mainly, add visibility modifiers on constants.

Change-Id: I41e8e2d691b2bad6ea6f244d54517d37d7783181
2020-01-21 12:36:37 +00:00
libraryupgrader 1d911b8187 build: Updating dependencies
composer:
* mediawiki/mediawiki-codesniffer: 28.0.0 → 29.0.0
  The following sniffs are failing and were disabled:
  * MediaWiki.Commenting.FunctionComment.MissingParamTag
  * MediaWiki.Commenting.FunctionComment.ParamNameNoMatch

npm:
* eslint-config-wikimedia: 0.13.1 → 0.15.0
* grunt-stylelint: 0.11.1 → 0.13.0
* stylelint-config-wikimedia: 0.6.0 → 0.8.0

Additional changes:
* Remove direct "stylelint" dependency in favor of "grunt-stylelint".
* Also sorted "composer fix" command to run phpcbf last.
* Removing manual reportUnusedDisableDirectives for eslint.

Change-Id: I8f73202db1333fbc36ccf556b3bb05b1e8c279cb
2020-01-21 07:38:54 +00:00
Daimona Eaytoy d8cb1a33a0 Factor out get(Local|Global)Filters methods
This is a preparatory step for T234427 (although not strictly related),
and in the future it will enable us not to use the DB in several tests.

Change-Id: Id069f6e74f9c4df43b3a602d4224473d5ca68ed1
2020-01-20 17:13:06 +01:00
Daimona Eaytoy 87459ec679 build: Upgrade phan
Depends-On: I6d538ce3ca7fd2d495c2bafbab7cc279da69db1c
Change-Id: Ic8c3a01a5c37fdf461f4fd5598e597eb9c9073d3
2020-01-19 18:48:51 +00:00
Daimona Eaytoy 44ea3aa7f4 Fix generation of HTML vars, simplify tests
-new_html: also strip the "Transclusion limit" comment if present, and
anyway take it into account (as well as a "</div>"), which right now
prevent the PP limit report from being stripped as well.
-new_text: trim extra whitespace on the right, which is created when
stripping the aforementioned comments.

Also simplify the test for getEditVars, make it not blindly copy what
AFComputedVariable does.

Extra: kill a temporary variable.

These changes are partly taken from
I96785c6c5fdf381c21d5f8930ee12e706abb7f3f.

Change-Id: I2b4c84a3d9d0d17ce229088197b75781d5181b4f
2020-01-12 17:44:02 +00:00
Daimona Eaytoy 10c2fe7151 Stop using deprecated stuff with easy replacements
This patch is mostly replacing Revision::* constants,
Wikimedia\(restore|suppress)Warnings, and wfWikiId.

Change-Id: I13544cc3e12955a9376ccce3c120e2cee1f2ee2e
2020-01-08 14:59:30 +01:00
jenkins-bot f0e4c22b53 Merge "Simplify a query in AFComputedVariable" 2020-01-07 19:30:20 +00:00
Daimona Eaytoy c54e2fc180 Simplify throttling code
Change-Id: I54ebdf0bc61d5628d1755b75232a934358b112ff
2020-01-07 17:52:16 +01:00
jenkins-bot eef2760d7b Merge "Use explicit variarg for VariableHolder functions" 2019-12-27 10:24:31 +00:00
jenkins-bot 8fea62529b Merge "Fix AbuseFilterCachingParser violating return type constraint" 2019-12-27 10:04:57 +00:00
jenkins-bot a46c0e7359 Merge "Restore the ability to filter content model changes" 2019-12-27 10:02:50 +00:00
jenkins-bot 5c9fe8bd9b Merge "Always evaluate the offset when retrieving array elements" 2019-12-27 09:58:50 +00:00
Daimona Eaytoy 8ad4ecd31d Always evaluate the offset when retrieving array elements
Even if the array is DUNDEFINED, we need to check the offset to ensure
that it's valid.

Bug: T237351
Change-Id: Ibfa360c4ae1d80abe14d9fdf66991b76cb5954df
2019-12-23 16:04:45 +00:00
jenkins-bot d43756a7f4 Merge "i18n: Rename msg key for abusefilter-view-oldwarning" 2019-12-23 12:16:57 +00:00
jenkins-bot db3b4703c5 Merge "Don't use mFilter in ViewTestBatch" 2019-12-23 12:11:52 +00:00
Daimona Eaytoy b3e0529d55 Log deprecated vars in the cached phase in the new parser
For the new parser, xhgui shows that AbuseFilterParser::getVarValue is
taking up a lot of time; in turn, most of the time spent inside
getVarValue is used to log the use of deprecated variables. Hence, given
that:
 - We should keep the new parser performant
 - There are tons of deprecated variables out there and they likely
 won't be replaced
 - Having gazillions of debugLog entries doesn't help

log them only in the cached phase.

Bug: T234427
Change-Id: I2bfc692c829c3cbe889e5076f5205e2c99097087
2019-12-16 13:54:58 +01:00
Daimona Eaytoy a7dd20b040 Don't use mFilter in ViewTestBatch
In other View* classes, AbuseFilterView::mFilter contains the ID of a
filter, e.g. the filter being edited in ViewEdit. In ViewTestBatch,
however, it is a string containing some filter text. Hence, use a new
private property instead (without the legacy "m" prefix).

Change-Id: Ib22ce238aff4ca5ed57ba725ee9bff7f8c3d153b
2019-12-16 12:17:49 +01:00
Daimona Eaytoy b814c0827a i18n: Rename msg key for abusefilter-view-oldwarning
Thinking about it again, all messages on ViewEdit start with
abusefilter-edit. Also add a reference to the other message to
facilitate translations.

Follow-up: I3717d06d4a757684fe6622961391ae06b5bd3c38
Bug: T235590
Change-Id: I4cbaa2e92d22296f55a4b5ef0c633fe959fe9ea3
2019-12-16 10:56:30 +00:00
DannyS712 12efe4a0ad ApiAbuseLogPrivateDetails: private-details should be privatedetails
Bug: T240812
Change-Id: I263e3a57a48ab6a58e4c7f2181a914d9800a2fc5
2019-12-16 03:25:15 +00:00
Daimona Eaytoy f382304aae Add a base class for parser transition
Change-Id: I31282b8632c332b6d46a6bb4a42f57ac0d005b5f
2019-12-15 13:29:56 +00:00
Daimona Eaytoy c432f058fd Restore the ability to filter content model changes
Follows-up I3fb7b36ab38ca1544889a4c233b8ffdfc6c80936

Bug: T240485
Change-Id: I2e8337e6f505932a18a5bb5a0d97b9d6bc3f42c8
2019-12-11 19:42:45 +01:00
Daimona Eaytoy 20c6810039 Use explicit variarg for VariableHolder functions
This is easier to read and to document, and it also allows typehints.

Change-Id: Ibd6642aa26e25a785faadf5139e64ea884ff4de2
2019-12-10 19:28:07 +00:00
Daimona Eaytoy 6be070e5a2 Strengthen the check for null edits
Even if the Content objects are different, the normalized text contents
may be identical.
Also, stop misattributing null edits by adding the last revision of the
page as afl_rev_id.

Bug: T240115
Change-Id: I3fb7b36ab38ca1544889a4c233b8ffdfc6c80936
2019-12-10 17:03:49 +01:00
Daimona Eaytoy d5ab147dcf Fix AbuseFilterCachingParser violating return type constraint
This is identical to I8a3c31e7385283d95b4712d457784016239a0b3b, except
for the array append case.

Bug: T236870
Change-Id: Iac033ba467232f6ff110d575920e968759ce0e15
2019-12-04 18:27:46 +00:00
jenkins-bot 357dcdce5c Merge "SECURITY: Unbreak blocks shorter than one hour" 2019-12-04 18:17:58 +00:00
Daimona Eaytoy 759cb38bf5 SECURITY: Unbreak blocks shorter than one hour
Bug: T238768
Change-Id: I8820a6e2953255f409ff8ddc2b41dcef2f338e37
2019-12-04 18:46:40 +01:00
jenkins-bot 6959bc3a89 Merge "Really throw for too many params" 2019-12-03 19:18:32 +00:00
jenkins-bot 156b9b7f26 Merge "Forbid assignments where the LHS is a built-in identifier" 2019-12-03 19:18:05 +00:00
DannyS712 e42a40bc06 ApiQueryAbuseFilters: Return abfstartid as an integer
Bug: T239528
Change-Id: Iee4d885c9b7fe1ee255ba9c0ac9e7e8f99938ef8
2019-12-01 14:21:31 +00:00
Daimona Eaytoy 07572da2fe Really throw for too many params
Bug: T230803
Change-Id: I4e68bb7220f1151bb32b2be859f6cffc55888a30
2019-11-30 10:57:16 +00:00
Daimona Eaytoy 2ddd79fd98 Forbid assignments where the LHS is a built-in identifier
And not just a built-in variable.

Bug: T237130
Bug: T237216
Change-Id: Ie1d86dc324993efcb863be23697732e6aa1dac10
2019-11-28 14:40:38 +00:00
Daimona Eaytoy b44c9da561 Use af_deleted as secondary sorting for af_enabled
Otherwise deleted and disabled filters would be mixed. Needs dependency
in core, otherwise we'd use af_deleted as secondary sort for every other
sortable field.

Bug: T191694
Depends-On: I0e695f96f18c7a9229753b1225dd473feb936a31
Change-Id: I979849e66bdcc158b7a3d0793ee3196e20db37b6
2019-11-22 16:23:46 +00:00
jenkins-bot a8c50150d6 Merge "Convert static arrays to constants" 2019-11-22 13:39:39 +00:00
jenkins-bot 2d2e524dca Merge "Tokenizer: don't strip backslashes from \x" 2019-11-22 13:36:49 +00:00
jenkins-bot 9a7027fe64 Merge "SECURITY: Require view-private or modify for the evalexpression API" 2019-11-21 15:54:46 +00:00
Daimona Eaytoy cee8e14cf1 SECURITY: Require view-private or modify for the evalexpression API
This is consistent with the "anti-DoS" measures on other API modules.
Although this may not be a serious DoS vector, it makes sense to
restrict this module. Moreover, it's also consistent with
Special:AbuseFilter/tools (which is the corresponding web interface),
which requires the same user rights.

Bug: T238451
Change-Id: Id09fd57195d71884674ac0470f137ca30c56e13c
2019-11-21 16:33:04 +01:00
Daimona Eaytoy b3e58067ac Set the utf-8 flag for var dumps in the text table
This is not retroactive; that will be handled as part of T213006.

Bug: T34478
Change-Id: I2c532da71719a9ace1279bbf67d6e6e30e9a986c
2019-11-16 16:00:45 +00:00
Daimona Eaytoy c03f0a3b08 Convert static arrays to constants
Beloved PHP7!

Change-Id: Id5170662f7c5ceacfc0ac8d90787f2c92fd93464
2019-11-16 16:32:36 +01:00
Daimona Eaytoy c73381b6db Tokenizer: don't strip backslashes from \x
Bug: T238475
Change-Id: I8c2ea6ad369946df93440eece60d456dc1a3fd7a
2019-11-16 16:21:39 +01:00
Martin Urbanec 5fd861365f SECURITY: Make sure provided filter id match provided history ID in history view
AbuseFilterViewEdit does privilege checks based on filter ID,
and displays what is hidden under given history ID, but doesn't
make sure those two IDs actually belong to one filter.

That means user can easily change filter ID to a public
filter and view old versions of nowadays private filters.

Bug: T237887
Change-Id: Ic12790bd33982473f77551bde9599ed083a3e1f1
2019-11-14 15:53:14 -06:00
jenkins-bot 80f4742416 Merge "When viewing old filter revisions, show abusefilter-view-oldwarning to users who cannot edit the filter" 2019-11-12 18:59:28 +00:00
Daimona Eaytoy 98bcad25c3 Also parse numbers with the new syntax and hard-deprecate the old one
This will allow people to switch their filters to the new syntax. The
deprecation warning is now more exhaustive, and the info() warning is
kept to ensure that everything proceeds smoothly.
The regex v2 has also been fixed to:
 - Consume all the digits/letters on the right (*)
 - Have named groups
 - Be created dynamically with other constants

(*) The previous version of v2 could complete the match and leave
digits/letters on the right when encountering numbers with the old
syntax, hence dropping support too early. We also cannot use a word
boundary (\b) because that would prevent matching numbers with trailing
dots (e.g. "5.").

Bug: T212730
Change-Id: Ibf6ac571f6b5c09149d69a19c38240ce6b024dff
2019-11-12 11:52:38 +00:00
Daimona Eaytoy a77a59b962 Hard-deprecate empty operands
This bumps the level to WARN, and makes it very clear that people should
fix the affected filters. It also removes the calling method, which was
mostly meant for debugging purposes, and changes the type to 'op_type'
to avoid conflicting with type:mediawiki in logstash.

Bug: T156096
Change-Id: Ie73f1604e8ed82bc2e1be9fc90fa065be37889a3
2019-11-12 11:39:25 +00:00
DannyS712 338341d097 When viewing old filter revisions, show abusefilter-view-oldwarning to users who cannot edit the filter
Currently, `abusefilter-edit-oldwarning` is shown to all users, but not all users are able to edit the filters, and thus the warning about editing isn't applicable to them.

Bug: T235590
Change-Id: I3717d06d4a757684fe6622961391ae06b5bd3c38
2019-11-12 11:36:44 +00:00
Daimona Eaytoy f7ac35d5c6 Hard-deprecate too many params
Bug: T230803
Change-Id: Icec8bcb8ab23956654857acc8b3d235889f587a9
2019-11-10 12:59:33 +00:00
jenkins-bot 91bc961712 Merge "Check for 0-like floats passed to the modulo operator" 2019-11-10 11:51:28 +00:00
Daimona Eaytoy c0f8374624 Check for 0-like floats passed to the modulo operator
That throws an error in PHP.

Bug: T237459
Change-Id: Ia0b29d6a8b9f4aac6b5b72ce8f2f45afb03f4c99
2019-11-10 11:22:04 +00:00
jenkins-bot 7ff4b95aec Merge "Expand the list of types that can be cast to int" 2019-11-10 11:00:36 +00:00
jenkins-bot 398500121a Merge "Fix conditionals examples in i18n messages" 2019-11-10 10:41:39 +00:00
Daimona Eaytoy 585d6cdb24 Make to sure to report division by zero when the LHS is undefined
Bug: T234339
Change-Id: I1575ec013c1e7e321a8f13f40804ebc5ab076268
2019-11-08 14:08:52 +00:00
Daimona Eaytoy 1abaff1aac Better handling of keywords and functions
Always run the keyword/function handler, even if there are DUNDEFINED
arguments, so that the handler can perform further validation on the
input and report any error to the user. However, replace DUNDEFINED with
DNULL before running the handler, to avoid special-casing DUNDEFINED in
every handler. If any argument was a DUNDEFINED, we will return
DUNDEFINED anyway.

Also centralize the keyword handling logic to a new method, like it
happens for functions.

Bug: T234339
Change-Id: I875cb77418a39790e91fe5867c49917bfe406ed4
2019-11-08 15:07:20 +01:00
Daimona Eaytoy e98799a00a Centralize the code for calling keywords
This allows sharing the code between cachingparser and the old parser
(for DRY-ness), and even when the old parser will be killed, having the
logic outside of the generic parse method seems saner.

This copies what I446a307e5395ea8cc8ec5ca5d5390b074bea2f24 did for
functions.

Change-Id: Ie6290243a6c78661510a9b4cb713d6e7b2778248
2019-11-08 15:02:17 +01:00
Daimona Eaytoy b7c7ae168d Explicitly forbid negative indexes in arrays
This emits its own error because:
1- It's clearer to understand
2- It's easier to find where we're dealing with negative offsets, if
we'll ever want to allow that.

Note that trying to use a negative index already results in a hard PHP
error being thrown.

Bug: T237219
Change-Id: Ib11eaaca5e21f740269141c75e62bac48093e8d0
2019-11-08 05:55:56 +00:00
Daimona Eaytoy a7b28369ea Expand the list of types that can be cast to int
Bug: T237624
Change-Id: I2220cb8a8ec998a433a4469d7e0591ec0b4f2b12
2019-11-07 15:14:17 +01:00
Daimona Eaytoy cb15400f97 Fix conditionals examples in i18n messages
Bug: T237131
Change-Id: I68ca3906c64f3da43c7a4985c16f1ab031caebb5
2019-11-02 11:32:05 +01:00
jenkins-bot 5562aade87 Merge "Use PHP regexps instead of SQL to filter on Special:AbuseFilter" 2019-11-01 00:52:28 +00:00
Daimona Eaytoy 7bc70d116e Use PHP regexps instead of SQL to filter on Special:AbuseFilter
As the code comment says, and as it was suggested in
Iafe54285384bc28b3e8812b495166f2682d4571c, we were validating the
provided regexp as PCRE, but using it in SQL, which only supports POSIX.
Furthermore, we won't have to worry about cross-DBMS compat anymore.

Bug: T193068
Change-Id: If6d8717795b6c1dcf619a23363eb6144902cfaed
2019-11-01 11:26:17 +11:00
Petr Pchelko 915b9a1538 Remove usages of deprecated User methods
Bug: T220191
Change-Id: I54e20870a32ff98b41a98495694ff563c4c4c5ca
2019-10-30 12:51:01 +00:00
Daimona Eaytoy 03b3a555ba SECURITY: Check visibility for each version in ViewDiff
Instead of checking if the filter is currently hidden, check the
visibility for each version and, if the user cannot see private filters,
only show the diff if none of the revision is hidden.
Also avoid showing a "diff" link if the user cannot see it.

Bug: T104807
Change-Id: Ie23e8234ae550273bf3f6f9c5ac45b7fc54eec2a
2019-10-28 15:32:00 -05:00
Daimona Eaytoy 3a9eac9ad5 Unbreak filter edit form
In Ib7427e15f673a575738489476e604c387f449ddd, I thought that $parameters could've only been null if $action wasn't
enabled, but actually, they're null even if the action is just not set.
Which is true for all actions when creating a new filter, and all
non-set actions when editing an existing one.

Hence, revert the part that touched ViewEdit.

Also add a selenium test to ensure that warn parameters are visible.

Bug: T236286
Change-Id: I8150baa077208eb1fc54ebc1d8415a243d0f3bd3
2019-10-23 18:50:44 +02:00
Thalia 63eb7eafb7 Use AbstractBlock setters and getters instead of deprecated properties
Change-Id: I01728f919254a9435f051af3fc390eb80ca8d17e
2019-10-20 00:35:00 +01:00
Daimona Eaytoy b9e4475985 build: Upgrade mediawiki-phan-config to 0.8.0
This is to verify that our CI is able to handle the new version.

Bug: T235049
Change-Id: Ib7427e15f673a575738489476e604c387f449ddd
2019-10-09 19:12:51 +02:00
jenkins-bot feae26116a Merge "Remove disabled variables deprecation" 2019-10-04 20:07:10 +00:00
jenkins-bot c6ee722273 Merge "Remove AFPData::dup" 2019-10-04 19:42:52 +00:00
jenkins-bot 9ab13cf24b Merge "Replace array_map with foreach" 2019-10-04 19:42:49 +00:00
Daimona Eaytoy c7fa503e9b Remove AFPData::dup
The method, which simply duplicates an AFPData instance, is only used
when casting types, to return a different instance when the object
already has the desired type.
However, nothing is assuming that, so we can just return the original
instance and save some time.

Bug: T234427
Change-Id: Id8067b418a00260ceead35f234e55268390699ab
2019-10-04 19:15:08 +00:00
Daimona Eaytoy 328dbc99c7 Remove disabled variables deprecation
I just realized that the parser is already throwing if it finds a
disabled variable. Hence, all calls to getVar with a disabled var are
from old entries and the like, and we don't care.

Bug: T234048
Change-Id: I39429d286575df91108a4119177a0d3aef181d0b
2019-10-04 15:03:08 +02:00
Daimona Eaytoy 703835e835 Drop HHVM support
Change-Id: Ib7ccb4f68278ba8ca009e9d18e9d8b127f799cde
2019-10-03 12:27:18 +00:00
Daimona Eaytoy 337771f83b Replace array_map with foreach
This is a micro-optimization, but IMHO it's necessary. The AF parser
code is executed for every active filter, for every
edit/move/deletion/accountcreation. In PHP, foreach is usually faster
than array_map. Especially in the case of variadic functions potentially
taking hundreds of strings, foreach will consume less time.

Bug: T234427
Change-Id: I1beedf419a6637a9a3dd668635645df950ceda21
2019-10-02 11:29:19 +00:00
Daimona Eaytoy 142ad5faf8 Actually record all filters in total_filters
Change-Id: If6d15423e0a96c31666690e4fe8c7aeb439f82e8
2019-09-29 11:02:29 +02:00
Krinkle a532874ee2 Update StringUtils::isValidRegex() call to isValidPCRERegex()
This follows-up 8587576655 (AF) and efbfa0a727 (core). The
method was recently introduced within the 1.34 cycle but
renamed following late CR feedback.

Change-Id: I9986deb080791c6266c6c60cc91022266ad9b5e5
2019-09-28 19:12:11 +00:00
jenkins-bot 952dfa0bb4 Merge "Hard-deprecate requesting disabled variables" 2019-09-28 18:25:24 +00:00
Daimona Eaytoy 0ae24d5489 Hard-deprecate requesting disabled variables
This also includes the filter ID. If the filter ID is not available, it
means that the user is using stuff like /tools, and they'll immediately
see the error.

Bug: T234048
Change-Id: I44a37d98c80df910b0c466fbd464e69042770c0c
2019-09-28 17:57:02 +00:00
Daimona Eaytoy 2385b3a537 Simplify a query in AFComputedVariable
Change-Id: I18596fc500bc2dcc7fdfa60bc21e85a6bd875589
2019-09-27 18:55:10 +02:00
jenkins-bot 0e30c1c34e Merge "Add new schemas for splitting afl_filter" 2019-09-27 15:41:06 +00:00
Daimona Eaytoy 0119108ee7 Fix params to ParserOutputStashForEdit
$summary and $user are always guaranteed to be passed, and $user is
guaranteed to be a User object. Hence, update the hook handler to
reflect that.

Change-Id: I3a7fcb074b460b77210de5a6bad43f500aff3249
2019-09-23 23:33:51 +02:00
Daimona Eaytoy 9a6dd1307c Add new schemas for splitting afl_filter
It'd be great if we could get this included in 1.34.

Bug: T220791
Change-Id: I62d429d0eb6a7adc51cc37fe18f878077f85a006
2019-09-22 16:04:45 +00:00
Daimona Eaytoy e2570a4c2b Actually provide a StatsdDataFactory to the parser
Follows-up Ib934be34a953166fe1b94cfe8ed216afe3b906ca

Bug: T156095
Change-Id: Ia8df84cf7c43071f304ce729b811dfd5aa96b951
2019-09-19 19:06:14 +02:00
Daimona Eaytoy e7926114ff SECURITY: Avoid info leak in SpecialAbuseLog
Deleted/suppressed usernames and summaries leak through AbuseLog.
Temporarily hide all non-public revision from AbuseLog, until we can
properly fix the issue.

Bug: T224203
Change-Id: If3d3256404d0f3dbde171831937d1a816b3e2734
2019-09-19 17:46:12 +02:00
jenkins-bot 9c786ca776 Merge "Use StringUtils::isValidRegex" 2019-09-19 08:03:39 +00:00
Daimona Eaytoy 4c8be4d374 Add profiling points throughout the code for the CachingParser switch
Bug: T156095
Change-Id: Ib934be34a953166fe1b94cfe8ed216afe3b906ca
2019-09-18 10:02:55 +00:00
DannyS712 6699237b86 Show link to test filter to all users who can use it
Bug: T232962
Change-Id: I67357975a7064991e47c60b70c487d12bdf6b7b4
2019-09-15 22:03:56 +00:00
jenkins-bot 8f4711c8ca Merge "Prevent blocked users from using /revert" 2019-09-15 12:07:43 +00:00
jenkins-bot 7add89b252 Merge "Don't show the form for restoring autopromotion to unprivileged users" 2019-09-15 11:26:19 +00:00
Daimona Eaytoy 127fd4ac3c Prevent blocked users from using /revert
Bug: T232916
Change-Id: I67e464f3182e74129186f7e803d05105a0f2ec23
2019-09-15 11:21:18 +00:00
jenkins-bot 48713c824b Merge "Throw AFPUserVisibleExceptions for empty operands in CachingParser" 2019-09-15 08:36:39 +00:00
Daimona Eaytoy a4e25c1ac9 Throw AFPUserVisibleExceptions for empty operands in CachingParser
Instead of TypeErrors. Basically, only empty parenthesis had to be
fixed.

Bug: T156096
Change-Id: I019615c7bfaa179c2184b5d3ea2c6b5da91366e3
2019-09-14 18:35:40 +00:00
Daimona Eaytoy 5267082c85 Better logging for unset variables
We have many log entries, so we need some more debug data.

Bug: T230256
Change-Id: I0e9638c1ffe537ea6cfd6886ff32ef447fdacc28
2019-09-14 16:49:55 +00:00
Daimona Eaytoy fe395bd96b Use dieBlocked instead of directly using apierror-blocked
This allows us to:
 - Defer handling of the block to the main module
 - Choose the right message depending on the block type
 - Avoid directly using the apierror-blocked message, which could change
 in the future.

Change-Id: If2e32bd2ccf5e314aa51203afd1522b8481377e0
Follows-up: I35f2c6e701a24dccb6e26e3f3c578fd44f68127d
2019-09-14 10:18:01 +02:00
jenkins-bot f8ee9fb959 Merge "Prohibit sitewide blocked users from restoring autopromotion" 2019-09-14 02:59:41 +00:00
jenkins-bot 45d7bd5971 Merge "CachingParser: ensure to catch errors inside short-circuited blocks" 2019-09-14 01:56:35 +00:00
jenkins-bot b8ad85cac7 Merge "Annotate the AST with var names before caching the AST" 2019-09-14 01:03:53 +00:00
Daimona Eaytoy 6e9a9a3bc2 CachingParser: ensure to catch errors inside short-circuited blocks
This is similar to the old parser: when discarding a node, actually
evaluate it if short-circuit is not allowed.
Add a whole lot of tests for all possible exceptions.
Move the logic to extract a message from an AFPUserVisibleException away
from the parser, to keep unit tests working.

Bug: T232498
Change-Id: I31ee4e255c6a87dd693b9bcd582539fdf57acd45
2019-09-13 21:13:15 +00:00
Daimona Eaytoy 004ccfdb5c Annotate the AST with var names before caching the AST
This implements T230982#5475400, and it should speed up the CachingParser by roughly 40%.

Bug: T230982
Change-Id: I803cc58637d50eb90e57decf243f5ca78075d63d
2019-09-13 19:43:50 +00:00
DannyS712 467fba75a0 Prohibit sitewide blocked users from restoring autopromotion
Bug: T232884
Change-Id: I35f2c6e701a24dccb6e26e3f3c578fd44f68127d
2019-09-13 18:32:55 +00:00
Daimona Eaytoy ed2bc7badf Don't show the form for restoring autopromotion to unprivileged users
Bug: T232881
Change-Id: I80c34c823f505c81e20f83ccf5c5a99e8e69b626
2019-09-13 20:31:17 +02:00
jenkins-bot cfad7d6f14 Merge "Actually return errors for action=edit API" 2019-09-10 19:59:03 +00:00
Bartosz Dziewoński 82b6f191d4 Actually return errors for action=edit API
Setting 'apiHookResult' results in a "successful" response; if we want
to report an error, we need to use ApiMessage. We already were doing
this for action=upload. Now our action=edit API responses will be
consistent with MediaWiki and other extensions, and will be able to
take advantage of errorformat=html.

Since this breaks compatibility anyway, also remove some redundant
backwards-compatibility values from the output.

To avoid user interface regressions in VisualEditor, the changes
I3b9c4fef (in VE) and I106dbd3c (in MediaWiki) should be merged first.

Before:
    {
        "edit": {
            "code": "abusefilter-disallowed",
            "message": {
                "key": "abusefilter-disallowed",
                "params": [ ... ]
            },
            "abusefilter": { ... },
            "info": "Hit AbuseFilter: Test filter disallow",
            "warning": "This action has been automatically identified ...",
            "result": "Failure"
        }
    }

After:
    {
        "errors": [
            {
                "code": "abusefilter-disallowed",
                "data": {
                    "abusefilter": { ... },
                },
                "module": "edit",
                "*": "This action has been automatically identified ..."
            }
        ],
        "*": "See http://localhost:3080/w/api.php for API usage. ..."
    }

For comparison, a 'readonly' error:
    {
        "errors": [
            {
                "code": "readonly",
                "data": {
                    "readonlyreason": "foo bar"
                },
                "module": "main",
                "*": "The wiki is currently in read-only mode."
            }
        ],
        "*": "See http://localhost:3080/w/api.php for API usage. ..."
    }

Bug: T229539
Depends-On: I106dbd3cbdbf7082b1d1f1c1106ece6b19c22a86
Depends-On: I3b9c4fefc0869ef7999c21cef754434febd852ec
Change-Id: I5424de387cbbcc9c85026b8cfeaf01635eee34a0
2019-09-09 20:15:19 +02:00
Daimona Eaytoy 173bd089b3 Remove script for blockautopromote entries
It was executed on WMF wikis, and since they were the only affected
wikis we can remove the script.
Also remove a temporary back-compat check in the log formatter.

Bug: T231131
Change-Id: I534acd9c86894eb1bdd96331e9fa85afc7502f88
2019-09-09 13:56:56 +02:00
Daimona Eaytoy 7917354716 Remove redundant logic from special pages
SpecialPage::setHeaders already handles page title, robot policy and
articleRelated. Moreover, avoid having different messages for the H1
title on the special page and the description shown elsewhere, just like
the base SpecialPage class suggests doing. The deleted messages have
been moved to the default message used by SpecialPage::getDescription.

Change-Id: Iab6beaf64b142e30469afd798c569ef40182153e
2019-09-08 12:30:01 +02:00
Daimona Eaytoy 8587576655 Use StringUtils::isValidRegex
Depends-On: I257a096319f1ec13441e9f745dcd22545fdd5cc6
Change-Id: Iafe54285384bc28b3e8812b495166f2682d4571c
2019-09-07 18:13:27 +00:00
Daimona Eaytoy 7b06be0204 Allow dangling commas in variargs
This is because there are many filters using this feature. Moreover, it
could make it a little easier to add new arguments, just like dangling
commas in PHP arrays do.
Also re-align the CachingParser code of doLevelFunctions to the one in
the old Parser.

Bug: T153251
Change-Id: Ie4325159f47310788da57415a5e36e62aa4efad0
2019-09-07 11:19:14 +02:00
jenkins-bot 5be19f6f65 Merge "Add a 'strict' option to VariableHolder::getVar" 2019-09-05 19:23:23 +00:00
Daimona Eaytoy 489da0d229 Add a 'strict' option to VariableHolder::getVar
This will help mitigating problems like T230256 by enforcing that the
requested variables must exist. For now, it will only log bad usages,
thus providing a way to identify affected filters and fix them.

Bug: T230256
Change-Id: I7a61916576e444a56f0e07da7b6e5033346226bd
2019-09-04 18:19:23 +00:00
Daimona Eaytoy 13b1e880f2 Hotfix other DUNDEFINED casts to bool
These were spotted on testwiki with wmf.21.

Change-Id: Ic4d67a2b83aedfeb574fa1363a9fc618b2862f95
2019-09-04 18:06:22 +00:00
jenkins-bot f2ae634831 Merge "Fix filter validation in ViewEdit" 2019-09-04 13:34:33 +00:00
Daimona Eaytoy 15e9f34115 Fix filter validation in ViewEdit
Currently it's impossible to create new filters!

Bug: T231985
Change-Id: I92a7739fe9defae6b3d74381792340c7125d9086
2019-09-04 14:04:45 +02:00
Daimona Eaytoy ce8539e2a5 Move parser tests back to /unit
Using `new LanguageEn()` involved a global, so use a MockObject instead.
Also fix LoggerFactory usage in Tokenizer to use DI instead.

Change-Id: I94d03f9459ab6444e239386eb96a0c2434bfe3dc
2019-09-03 13:23:11 +00:00
jenkins-bot 3d319edba9 Merge "Upgrade phan-config to 0.7.1" 2019-09-02 12:46:29 +00:00
jenkins-bot 6441e71ebe Merge "Also catch Error in the hacky workaround for bad rows" 2019-09-02 12:46:27 +00:00
jenkins-bot 2ed5be29e1 Merge "Use permissions accessors" 2019-09-02 10:55:46 +00:00
Daimona Eaytoy 393e47c5a7 Upgrade phan-config to 0.7.1
Change-Id: I859d81eda8601da91602b27a223b6d6d59ecf563
2019-09-01 09:42:26 +00:00
Daimona Eaytoy 2a956bc81a Also catch Error in the hacky workaround for bad rows
PHP7 throws an Error, not a BadMethodCallException. We don't want to
clog the logs with fatals, now that PHP7 is closer.

Bug: T187153
Change-Id: I5a9e581ee0418ae41dd911de02a64d18e4670cd4
2019-08-31 20:42:41 +02:00
jenkins-bot 25d63aa639 Merge "Add new number syntax as experimental" 2019-08-31 17:12:10 +00:00
jenkins-bot 134c75dd5e Merge "Remove AbuseFilter::saveFilter dependency on AbuseFilterViewEdit" 2019-08-31 17:06:00 +00:00
jenkins-bot fc3349df1f Merge "Fix param validation in ViewEdit" 2019-08-30 13:37:21 +00:00
Daimona Eaytoy 933b791ef3 Fix param validation in ViewEdit
We didn't check if the provided ID was valid. While editing an existing
filter (or creating a new one), we check the ID in SpecialAbuseFilter,
so it's guaranteed to get an integer in ViewEdit, and the case of a
non-existing filter is handled later, in buildFilterEditor.
But for links like Special:AbuseFilter/history/foobarbaz/item/1 (where
"foobarbaz" should be the filter ID), no validation was performed. This
caused a useless query to be carried out on the abuse_filter_history table (which would likely return false), then accessing properties of a non-object ('$row->afh_id'), and we ended up showing filter 1. This was spotted because we actually got notices in production.

Bug: T231632
Change-Id: I6436c7d2df8c1f0fc971f4a4079dac9118aa8209
2019-08-30 08:59:49 +00:00
Daimona Eaytoy fdb71b5868 Make AbuseFilterVariableHolder::$mVars public again
+fixme comment per T231542#5451567.

Bug: T231542
Change-Id: If8b7b0568568df93548f12ccdc85fa174ec3558e
2019-08-29 18:51:43 +02:00
Daimona Eaytoy eb91595d8a Use row->afl_action instead of $vars
There's not need to use the variableholder in this hacky (albeit common)
whay, since the row already holds the action.
Note, this doesn't guarantee that the next two lines won't fail - I'd
need to see the actual var dump (T231542#5450720) to determine exactly
why this is failing.

Bug: T231542
Change-Id: I2112b046d00e06b575d15ab3d7da57484fd9cbbd
2019-08-29 13:48:31 +02:00
Daimona Eaytoy d51ca862c6 Move parser tests to /unit
IMHO these can be considered unit tests; they were already fast, but now
they're executed in an instant.
This requires several changes: 1 - delay retrieving messages in
AFPUserVisibleException, to avoid having to deal with i18n whenever we
want to test exceptions; 2 - Use some DI for Parser and Tokenizer.
Equivset-dependend tests are also moved to a new class, thus helping to
fix the AF part of T189560.

Change-Id: If4585bf9bb696857005cf40a0d6985c36ac7e7a8
2019-08-28 16:36:37 +00:00
Daimona Eaytoy 540edc5174 Remove AbuseFilter::saveFilter dependency on AbuseFilterViewEdit
This dependency is wrong, and removing it will also allow creating an
edit API.

Bug: T213037
Depends-On: Id8412e2b8a4e873fd4821ecc1a3c95710be9a870
Change-Id: If8e745a3227cea5093ea3fd8f5b201adedaba3ae
2019-08-27 16:26:18 +00:00
Daimona Eaytoy 8e166f10d6 Refactor and speed up non-parser tests
Some of these are transformed into real unit tests, while the
AbuseFilterSaveTest class is refactored to avoid using the DB and to use
a lot more of mocks and DI.

Depends-On: I22743557e162fd23b3b4e52951a649d8c21109c8
Change-Id: Id8412e2b8a4e873fd4821ecc1a3c95710be9a870
2019-08-27 16:24:27 +00:00
Daimona Eaytoy 87713008d5 Use permissions accessors
There are lots of calls to $user->isAllowed which could be simplified
using available accessors like canEdit(). So simplify those calls and
avoid duplication.

Note that using canEdit also fixes a bug which affected blocked users:
we used to show e.g. the import link, and not to display as disabled
several text fields, while blocked users cannot actually edit filters.

Depends-On: I22743557e162fd23b3b4e52951a649d8c21109c8
Change-Id: I62779e940949ef49018a9c6d901bb6e10aa81da8
2019-08-27 13:21:55 +02:00
Daimona Eaytoy c469fb4b76 Mostly remove $wgUser
There are lots of cases where we can inject a User object without
additional efforts. Now $wgUser is only used inside AFComputedVariable,
which is a little bit harder to handle because some instances of that
class are serialized in the DB, and thus we cannot easily change the
constructor until T213006 is resolved.

This partly copies what Ia474f02dfeee8c7d067ee7e555c08cbfef08f6a6 tried
to do, but adopting a different approach for various can*() methods:
they're now static methods in the AbuseFilter class, so future callers
don't need to instantiate an AbuseFilterView class. This also allows to
re-use those methods in an API module for editing filters (T213037).

Bug: T213037
Bug: T159299
Change-Id: I22743557e162fd23b3b4e52951a649d8c21109c8
2019-08-27 13:20:37 +02:00
Daimona Eaytoy 71730f7d44 Warn if a function has been given too many parameters
While this is not as important as throwing for too few parameters, IMHO
it's still important to fail in this case. Mostly because if a function
receives too many parameters, chances are that who wrote the filter
didn't do that intendedly, and thus there may be a hidden bug.
Bonus: fix a few docblocks.

Bug: T230803
Change-Id: Iac2931f17b50ace8c8f4c2faa44b3f54ca134c54
2019-08-26 20:29:49 +02:00
Daimona Eaytoy 4d86758a49 Add new number syntax as experimental
For now it will only report successful parse. Next step is formally
deprecating the old one (escalated to warning), then removing it in
favour of the new one (in another MW version).

Bug: T212730
Change-Id: I5dd11fd67d8e57d1d0c52ddfa026920ebfc5ee13
2019-08-26 08:15:55 +00:00
jenkins-bot 89524790d5 Merge "Add a hook to determine whether the current action should be filtered" 2019-08-25 18:45:07 +00:00
jenkins-bot ff2f6ee26f Merge "Add a new class for the CachingParser's AST" 2019-08-25 18:00:24 +00:00
Daimona Eaytoy d515af0ae6 Add a new class for the CachingParser's AST
This allows a little bit more of abstraction: we can store other data in the
tree, without having to store it in a specific node (e.g. the variables map,
which is still unused). It also adds a few typehints, and specializes
the return value of eval'ing the AST: previously, it was the one of
evalNode, which wasn't guaranteed to be an AFPData. Now we have this
guarantee. Last but not least, we can now measure runtime metrics for
evalTree, which doesn't recurse.
Bonus: fix a check in the old parser, which used the wrong variable when
reporting outofbounds errors.

Change-Id: Iff806793b1d968e9bb6220f1459f3d0ac587c7da
2019-08-25 17:29:16 +00:00
jenkins-bot 6196801178 Merge "Log more empty operands" 2019-08-24 20:53:01 +00:00
Daimona Eaytoy 2d031d0bee Log more empty operands
And fix a couple of minor bugs.

Bug: T156096
Depends-On: I3b85087677607573f4fa68681735dc35348dcd87
Change-Id: Ia4c713a1d45827f6a8bc5566a8d8835c49f8108a
2019-08-24 19:59:53 +00:00
Daimona Eaytoy 7f554734e6 Don't hardcode blockautopromote duration
As explained on phab, and add a script to fix broken entries.

Bug: T231131
Change-Id: I95d70acb936b5ca987af8f237d236fe47b663919
2019-08-24 11:40:11 +02:00
Huji Lee 1ddb65021b Add links to AbuseFilter logs on Special:Undelete
Depends-On: I671a0479e877e6c37606b688064cb9c893717709
Bug: T231055
Change-Id: Iebf832c513c6a4e954db0ba2633dd8ba6f27b412
2019-08-23 14:56:43 +00:00
Daimona Eaytoy bf61414f88 Don't show empty "Tools:" section in ViewEdit
After having removed the export link in
I72f46247f4323fb5bfe7fa74f332076dbd346187, we don't have any tool to
show for new filters. So avoid outputting an empty section.

Change-Id: Ia07bccdbadb7b874397135bc3f7468d6e0b9eb13
2019-08-21 17:32:43 +02:00
jenkins-bot 47838715fa Merge "Allow if without else" 2019-08-20 20:12:19 +00:00
jenkins-bot 5e605aaa62 Merge "Even better handling of DUNDEFINED" 2019-08-20 20:00:52 +00:00
jenkins-bot bf8ccccade Merge "Fix a bug in the return value of the CachingParser" 2019-08-20 19:58:38 +00:00
Daimona Eaytoy af7744781f Allow if without else
Bug: T230727
Depends-On: I8e7f7710b8cb37ada8531b631456a3ce7b27ee45
Change-Id: I3b85087677607573f4fa68681735dc35348dcd87
2019-08-20 19:36:14 +00:00
Daimona Eaytoy 963221ad6d Even better handling of DUNDEFINED
Ensure that the variable isn't set before marking it as DUNDEFINED:
that's only for when we cannot use a default, but if the variable is set
we already have one. Most notably, this fixes conditionals handling: right
now, if you have a conditional with an assignment in both
branches, the variable will be undefined. That's obviously wrong, so
it's fixed in this patch.
Plus: catch only AFPExceptions in a test to avoid unintentionally
catching the assert exception; simplify some assignments using wfSetVar.

Depends-On: I446a307e5395ea8cc8ec5ca5d5390b074bea2f24
Change-Id: I8e7f7710b8cb37ada8531b631456a3ce7b27ee45
2019-08-20 19:17:30 +00:00
Daimona Eaytoy fa76405ea7 Fix a bug in the return value of the CachingParser
This has always been wrong, and remained unnoticed. Also added a
typehint for added safety.

Change-Id: I8a3c31e7385283d95b4712d457784016239a0b3b
2019-08-20 20:54:19 +02:00
jenkins-bot a8e2071351 Merge "Better handling of function params in CachingParser" 2019-08-20 18:46:01 +00:00
jenkins-bot 8527a10774 Merge "Restyle edit box dimensions" 2019-08-20 16:33:16 +00:00
Daimona Eaytoy aa867bd370 Better handling of function params in CachingParser
This patch includes various fixes to how func arguments are handled in
CachingParser:
- Add a comment about a future improvement of checkSyntax, which we
  could limit to try building the AST.
- Having enough args for each function is now also checked when
  building the AST. This allows implementing the previous point without
  stopping to report notenoughargs at syntaxcheck-time (otherwise it'd be
  a runtime error). And it also ensure that we check for the params count
  inside skipped branches, e.g. inside if/else: these were already only
  discovered at runtime in CachingParser. The old parser is not affected
  by this change, because when checking syntax it will always execute
  all branches, and at runtime it will skip braces altogether.
- Fix arg count for CachingParser, which previously added a bogus param
  in case of a function called without parameters. This was fixed for
  the other parser in I484fe2994292970276150d2e417801453339e540, and I
  just ported the updated fix. Also note that the CachingParser was
  already failing for e.g. `count()`, but instead of complaining about
  missing arguments, it failed hard when trying to pass NULL to
  evalNode.
- Fixed some tests not to use setExpectedException, which caused the
  previous point to remain unnoticed: calling that method prevents the
  loop from continuing, and thus only the AbuseFilterParser part was
  being executed. The new implementation checks the exception ID and is
  thus more future-proof if the i18n message changes.
- Fixed some function names in error reporting for the old parser.
- The arg count is now checked outside of the function handlers, thus
  it's no more necessary to call checkEnoughArguments at the beginning
  of each handler. This also produces clearer error messages in case of
  aliases (e.g. set/set_var).
- Check the args count even if some of the args are DUNDEFINED. This is
  much easier now that the check is outside of the handler. This will
  make syntax check fail for e.g. `contains_any(added_lines)`.

Bug: T156095
Change-Id: I446a307e5395ea8cc8ec5ca5d5390b074bea2f24
2019-08-20 15:32:02 +00:00
jenkins-bot 7addec7b4a Merge "Make some other AFPData methods non-static" 2019-08-20 14:16:16 +00:00
jenkins-bot 1f45336157 Merge "Move keywords handlers to the Parser" 2019-08-20 14:16:10 +00:00
jenkins-bot f18d0814e2 Merge "Make several AFPData functions non-static" 2019-08-20 14:06:02 +00:00
jenkins-bot f1ab591d27 Merge "Avoid implicit casts from DUNDEFINED to something else" 2019-08-20 13:04:48 +00:00
jenkins-bot ea01809f5e Merge "Add the filter ID to empty operand logging" 2019-08-20 13:01:14 +00:00
jenkins-bot d32b03ca10 Merge "Increase cache hits for CachingParser" 2019-08-20 12:50:31 +00:00
jenkins-bot d0b30c2534 Merge "Make parser aware of the filter it is parsing" 2019-08-20 12:50:26 +00:00
jenkins-bot 1bfd182a2e Merge "Fix object to array cast, typehint array params" 2019-08-20 12:49:09 +00:00
jenkins-bot 4bebd22e3f Merge "Add test for multiple conditions inside conditionals" 2019-08-19 14:18:10 +00:00
Daimona Eaytoy e4b1df1521 Fix object to array cast, typehint array params
This was broken in I34c040dbeb3ab01158fb3db22496def6ccaf72d9. I thought
the members of that object were always arrays, but I was wrong.
Plus typehint a few array parameters and make a couple of methods
private since they're only used in this class.

Bug: T230639
Change-Id: I0c51359769c4b3054f95755a96e7e0a2d8e5bf15
2019-08-17 17:04:34 +00:00
Daimona Eaytoy b235e1040a Restyle edit box dimensions
Now it's always wider, and so is the "notes" field. Moreover, the
fallback textarea has the exact same size. Plus removed a parameter
which only made it hard to write a CSS rule for the textarea. Since the
textarea is generated by the same code, and we're always using it for
the same thing (filter syntax, regardless of the final goal), make it
always use the same name.

Bug: T230591
Change-Id: Ibb308e80d954c0e81aa09249c38c39572f157948
2019-08-17 18:53:13 +02:00
jenkins-bot 61bb3ff3e8 Merge "Various changes for blockautopromote" 2019-08-14 23:59:08 +00:00
Daimona Eaytoy 27578d7ba0 Various changes for blockautopromote
Problems fixed:
 - Don't hardcode duration in the message
 - Move duration to a constant
 - Fix wrong parameter order for AbuseFilter::blockAutopromote
 - Log a warning if we cannot block autopromotion
 - Remove the $performer parameter, as it should only and always be the
 filter user.

Bug: T230296
Change-Id: Ice9e4b21033c430cf1fd34182c63ca64ad2f5d3e
2019-08-14 18:50:43 +02:00
Daimona Eaytoy d715f6d2c0 Increase cache hits for CachingParser
If $parser->parse returns a falsey value (=null), that's because the
filter doesn't have any statement. But that's not a valid reason not to
cache the filter. Hence, return whatever parse() is returning inside the
callback, so that the result is always cached.

Change-Id: Ib6b0e72d882dc484456a3be6bbc74da36ef48bf7
2019-08-13 18:03:13 +02:00
Daimona Eaytoy d58b5930f8 Add the filter ID to empty operand logging
To make debugging a lot easier.

Bug: T156096
Bug: T153251
Change-Id: I1f905c6e1a524a745240b05709ef9d1dfc3c23a1
2019-08-13 15:22:55 +00:00
Daimona Eaytoy 1197eb6b41 Make parser aware of the filter it is parsing
This information will mostly be used for debugging purposes.

Change-Id: Ia1bcc2acc22aba97d855382b5b173ac3d5f2c54b
2019-08-13 15:22:38 +00:00
Santhosh Thottingal 1176e3a465
Fix the warning about permission name changes
Change-Id: I16463550328eb19d33270d8677404e012e5c80df
2019-08-13 14:40:17 +05:30
Daimona Eaytoy 7edf12dbc6 Add a hook to determine whether the current action should be filtered
Bug: T229252
Change-Id: I126013b6c516eebe7273fb85f2c5681844e757be
2019-08-13 10:31:29 +02:00
Daimona Eaytoy 430ba818d0 Add test for multiple conditions inside conditionals
The regression itself was fixed in
I980aec3481a52ecc35f1811a366014a5581a7cdb, so this patch only adds a
test for it.
Also remove a comment about CachingParser failures: we don't want to
encourage people to remove it from tests anymore.

Bug: T152281
Change-Id: I3ad49050ea49bf45d3226878e091da3c8dbefdb1
2019-08-12 18:18:05 +02:00
Daimona Eaytoy 4b0911ee01 Make some other AFPData methods non-static
Change-Id: I22ea337a36f911c57d3dadb9a3c45fc2c8b7c628
2019-08-12 14:40:51 +02:00
Daimona Eaytoy 3f171dc0a5 Move keywords handlers to the Parser
Just like we do for functions, it doesn't really make sense to have
keywords separately, in AFPData.

Change-Id: I208a9b1ce2bd12038e9fbcc515c48d604ec80eb8
2019-08-12 14:29:56 +02:00
Daimona Eaytoy 2fdf091eb9 Make several AFPData functions non-static
The keywords-related ones will be handled in a subsequent patch.

Change-Id: Ifcfad438023ef136dc6f2cd5529e867df9b23789
2019-08-12 14:12:16 +02:00
Daimona Eaytoy 1fe3647268 Avoid implicit casts from DUNDEFINED to something else
This patch keeps the current behaviour for everything (since DUNDEFINED
was always casted to boolean false), but handles the cast at a higher
level instead of relying on what AFPData::castTypes will do. This way
it's easier to spot places where we may get DUNDEFINED, and decide how
to handle them one by one.

Change-Id: I1070e15ea03c7dd4a4231b87afbc42240a558581
2019-08-12 11:18:15 +02:00
jenkins-bot 3748c41e79 Merge "Remove outdated comment, add a new one" 2019-08-12 08:26:30 +00:00
jenkins-bot eff4e208ca Merge "Don't show export link for new filters" 2019-08-11 17:02:56 +00:00
Daimona Eaytoy 200905f1cf Remove outdated comment, add a new one
As explained in T230093#5408119.

Bug: T230295
Change-Id: Ic0beaf9d126d14fbb7efbd2d0ec55e10c0fbcc99
2019-08-11 14:35:38 +00:00
Daimona Eaytoy c8b5b9321c Don't show export link for new filters
Bug: T230163
Change-Id: I72f46247f4323fb5bfe7fa74f332076dbd346187
2019-08-11 12:21:36 +02:00
Daimona Eaytoy 69ad23da98 Ban variable variables
As explained on phab, it's not worth the effort of keeping this feature.

Bug: T229947
Change-Id: Ic6067cab8e1ede98545e704888c99e2ed9a004e4
2019-08-11 01:47:35 +00:00
Daimona Eaytoy ff40204ef1 Gracefully handle blockautopromote failures
Instead of returning a successful message, return null and log a
warning. Also, make autopromoteBlockKey public + internal and use it
from Hooks instead of duplicating the logic.
Follow-up: I03feb05218789a3b73a31c9a94216daafcb7c145

Change-Id: I8ce96d1bd0239003f8ee6a45f412b9502d542a18
2019-08-10 15:18:30 +02:00
Daimona Eaytoy c34181e548 Add a new API module to retrieve private details from AbuseLog
Follow-up of Iaca492371f48fecf543268c179a651841ed12c3f. This patch adds
the new module, plus some technical changes to private details-related
methods and globals.

Bug: T210329
Depends-On: I613dbadb8f75c8c4116a362607563a436a73d321
Change-Id: I3c45b74c36c191083df184ed57416067a75f6591
2019-08-09 21:10:28 +00:00
Daimona Eaytoy c7ccb68058 Use "privatedetails" instead of "private" where needed
To keep a clear distinction between "private details" (i.e. user's ip)
and "private filters" (i.e. not publicly viewable). This patch renames
rights, i18n keys and methods names.
The patch for renaming globals and rights in WMF config is
I7e6b3d4453403edb6aa602587374b4ff5b6d625f.

Bug: T211004
Change-Id: I613dbadb8f75c8c4116a362607563a436a73d321
2019-08-09 21:10:22 +00:00
jenkins-bot 8ee442234f Merge "Move "block-autopromote" key from $wgMainStash to 'db-replicated'" 2019-08-07 23:07:45 +00:00
jenkins-bot 1fa5eef94c Merge "Overhaul Blockautopromote action" 2019-08-07 23:03:08 +00:00
Aaron Schulz 9e44f1a9e9 Move "block-autopromote" key from $wgMainStash to 'db-replicated'
Keep the key mutation methods in the AbuseFilter class

Bug: T227376
Change-Id: I03feb05218789a3b73a31c9a94216daafcb7c145
2019-08-07 01:09:13 +00:00
jenkins-bot 097b2da0ed Merge "Add a parent class for special pages" 2019-08-06 18:32:29 +00:00
Daimona Eaytoy 483dab1732 Add a parent class for special pages
This allows us to extract yet another static method from the AbuseFilter
class. This class should be expanded in the future, and an example use
case could be Ia5fd4f0b35fcabf045a7b49fa40fa85b72c92544.

Depends-On: I7c0170167b508132cd16e566c654a6c98dd683e9
Change-Id: I1bb45e47c3b42c01388b99778ce833e4e44419e1
2019-08-06 14:17:38 +00:00
Daimona Eaytoy 2ed6272bb2 Partly handle set and set_var in shortcircuit
This is more complicated than the := operator, because the var name
could be a complicated expression, and we have to handle a function
call. This patch only covers the case where the variable name is a
literal, which is enough for WMF production.

Bug: T214674
Change-Id: I6c0f8e95663919a0235b5ccf0c88ad0a539315a7
2019-08-06 16:14:34 +02:00
Daimona Eaytoy 2bdb44d58b Overhaul Blockautopromote action
As for all mostly unused consequences, blockautopromote has a couple of
major problems: first, it blocked the status for a random time between 3
and 7 days, which to me makes no sense at all (is it some sort of
casino?), and this patch fixes it to 5 days. Second, nothing was logged,
not the blocking nor the unblocking. Here I'm adding a LogHandler for
two new sub-actions of 'rights' to keep track of both action.

Bug: T49412
Change-Id: If48a48f5b8baaf9e77c0826466f5d03bb7f691d0
2019-08-05 22:27:49 -04:00
Daimona Eaytoy afeff7c222 Really avoid DEMPTY leak
Follow-up I7831f3ed9f7c0656e0e8f77ded049c20eca682ba, really avoid the leak. My addition was pointless because we need DUNDEFINED, not DEMPTY, and I spent way too much time trying to understand what was still wrong.
Still have to get used to these new names...

Change-Id: I332967f6fb00b67fd355547b19638c95ffa5bba7
2019-08-04 22:02:13 +00:00
jenkins-bot 44e427601f Merge "Avoid DEMPTY leak" 2019-08-04 21:16:22 +00:00
Daimona Eaytoy 6ef2cf523b Profiling: don't count time for operations shared with the edit
Parsing wikitext and retrieving its links are operations which we share
with the edit, so that if a filter does that, it won't be done later
upon saving.
Thus, add a static variable to subtract such time and avoid to erroneously log as slow any filter using those variables.

Bug: T219092
Depends-On: I24fbd41ac188a9cf6a7d3ca33dce349aedc9faa6
Change-Id: I7c0170167b508132cd16e566c654a6c98dd683e9
2019-08-04 20:12:10 +00:00
jenkins-bot 4347331692 Merge "Reset all filter profiling data at once" 2019-08-04 18:53:57 +00:00
jenkins-bot 19182606c1 Merge "Merge global profiling keys" 2019-08-04 18:40:14 +00:00
Daimona Eaytoy f977e858ab Avoid DEMPTY leak
As shown in the coverage reports [0], some empty operand logging lines are covered, but no test should have empty operands. I see one of the cause is skipOverBraces keeping $result as is, even if DEMPTY, so turn it into a DUNDEFINED.

[0] - https://doc.wikimedia.org/cover-extensions/AbuseFilter/includes/parser/AbuseFilterParser.php.html

Change-Id: I7831f3ed9f7c0656e0e8f77ded049c20eca682ba
2019-08-04 18:25:04 +00:00
Daimona Eaytoy 2742f24bca Reset all filter profiling data at once
Instead of scattering the process all over the code (and doing it
together with checking if the key already exists).
Wrap the logic in new methods for better readability.

Depends-On: Ib12e072a245fcad93c6c6bd452041f3441f68bb7
Change-Id: I24fbd41ac188a9cf6a7d3ca33dce349aedc9faa6
2019-08-04 18:03:14 +00:00
rarohde d022377578 Merge global profiling keys
The last step of the profiling overhaul. See T53294 for the original description by Dragons flight.

Note: Here I'm adding a FixMe for a problem which already exists in the code
and the child patch will fix it.

Bug: T53294
Depends-On: I2d8c8f8278073a9420e3eb373fb89a655925618a
Change-Id: Ib12e072a245fcad93c6c6bd452041f3441f68bb7
2019-08-04 17:59:58 +00:00
jenkins-bot 879ff59fe2 Merge "Split hook handlers related to filtering" 2019-08-04 17:55:51 +00:00
jenkins-bot be4b1c5bc1 Merge "Fix @deprecated since version" 2019-08-04 17:50:47 +00:00
jenkins-bot e733872e13 Merge "Allow accessing offsets of built-in variables" 2019-08-04 17:48:46 +00:00
jenkins-bot 790cd38fb0 Merge "Further deprecation for empty conditions" 2019-08-04 17:26:40 +00:00
Daimona Eaytoy 7d192cb1f2 Split hook handlers related to filtering
This adds a new get(Type)Vars method for every action type. The goal is
to 1-have shorter methods, which is always good; 2-try to make this code
a bit more testable.
I left as a todo moving all these methods to a separate class, the idea
being to make them non-static and thus easier to be tested.

Depends-On: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
Change-Id: I6de2dd27a8f972b3f74c730a1516639f8c622166
2019-08-04 17:21:29 +00:00
Daimona Eaytoy 517919fca8 Allow accessing offsets of built-in variables
I5ec4ab44c4e88aaf18c0d7b73355d27050beeda7 almost fixed this bug, but we
also have to make it possible to access builtin variables as arrays.
This will only make sense for a few variables (e.g. added_lines and
removed_lines), but I don't think we should validate it when checking
syntax.

Bug: T198531
Change-Id: I417e1b8d4802bbfccd091ce5c7617659cfd1e4ea
2019-08-04 17:14:44 +00:00
jenkins-bot 534a282f7d Merge "Clarify "filter" field in SpecialAbuseLog and ApiQueryAbuseLog" 2019-08-04 17:13:47 +00:00
jenkins-bot 5fccb5a67b Merge "Change parameter order for newVariableHolderForEdit" 2019-08-04 17:07:27 +00:00
Daimona Eaytoy 71e3719e12 Clarify "filter" field in SpecialAbuseLog and ApiQueryAbuseLog
The "filter" fields can also accept a list of filters, and also global filters, so make it clear in the UI and in messages.

Change-Id: Ib258716d8e6792fd496938ebb4e8a2565d6370b7
2019-08-04 16:55:05 +00:00
Daimona Eaytoy c635f41697 Change parameter order for newVariableHolderForEdit
Make the old text non-optional, and typehint the old content.

Change-Id: I91be3c028e891d32fa8a61ed541336c85f8a9dfb
2019-08-04 16:48:21 +00:00
Daimona Eaytoy 4f8811bc3b Update cache key version for data in stashed edits
In I1dc3be6da1cc9e03bc47e8f8c867089ad0100f6f we added fields to the array.
Update the version to avoid PHP errors while upgrading the wikis, for edits
stashed before the upgrade, and saved afterwards.

Change-Id: I5489b556b1b0e9cb2af862dbfa0621909a5e355d
2019-08-04 16:36:23 +00:00
jenkins-bot c0b6267022 Merge "Use milliseconds for time profiling" 2019-08-04 16:12:59 +00:00
jenkins-bot fea7686724 Merge "Hide profiling for disabled filters" 2019-08-04 16:12:56 +00:00
jenkins-bot f7fd6a6daf Merge "Move per-filter matches profiling to per-filter data" 2019-08-04 16:07:58 +00:00
jenkins-bot d940ef63cd Merge "Specialize empty AFPData types" 2019-08-04 15:52:34 +00:00
Daimona Eaytoy e8866fee88 Fix @deprecated since version
The Runner was introduced in 1.34.

Change-Id: I9239705ef5628821b3ce6dbc27ac282cfc93e1e6
2019-08-04 15:39:54 +00:00
Daimona Eaytoy 5f4491f9aa Further deprecation for empty conditions
Start deprecating "empty" logic operators, and now that we have DEMPTY, simplify handling of empty function arguments introduced in Ica3e49f5b00595a95513d9683732e490aa7aae17.

Bug: T156096
Change-Id: Ied6b385e8690b6cc6e69afcf614389f737ab95bd
2019-08-04 15:33:49 +00:00
Daimona Eaytoy 9049be3609 Specialize empty AFPData types
As described in T156096#5389655.

Change-Id: Ifbf95a6b72a280cd77db6affbd8d642499bbfedc
2019-08-04 15:26:57 +00:00
Daimona Eaytoy c8ebb4956c Hide profiling for disabled filters
As data could be "old" and it may have no meaning.
Also remove a superfluous isset(), as $row->af_hidden is always set.

Depends-On: I2d8c8f8278073a9420e3eb373fb89a655925618a
Change-Id: I072363706c61f272c4c3691de4078e2a19148424
2019-08-03 23:28:42 +00:00
Daimona Eaytoy c3db63714e Use milliseconds for time profiling
Instead of seconds, and round the average condition at 1dp instead of 0.
Split from child patch by Dragons flight.

Depends-On: I2d8c8f8278073a9420e3eb373fb89a655925618a
Change-Id: I339aed5f8c1d49714e7927ce49286f9ce6c839f5
2019-08-03 23:24:46 +00:00
Daimona Eaytoy 0b7902fe6e Move per-filter matches profiling to per-filter data
They're currently stored separately, so move matches count together with
other per-filter data to keep it consistent. This also removes a
parameter from filterMatchesKey, as it's not needed anymore.
Split from child patch by Dragons flight.

Bug: T53294
Depends-On: I8f47beb73cfc1b63c4b3c809fc6d65a1e66ee334
Change-Id: I2d8c8f8278073a9420e3eb373fb89a655925618a
2019-08-03 23:22:20 +00:00
jenkins-bot 3b2b85b60d Merge "Store per-filter profiling in a single key" 2019-08-03 22:43:32 +00:00
Daimona Eaytoy 33bfe97d8c Move non-decimal numbers deprecation logging
Bug: T212730
Change-Id: Idb833c60541873bfe9c2b225009bd32e4a48cd60
2019-08-03 16:57:24 +00:00
Daimona Eaytoy d04a0d3afc Store per-filter profiling in a single key
Instead of having three keys, one for total actions, one for time and
one for conditions. This has several benefits: first, it avoids race
conditions which could happen having different keys. Second, it's much
more performant. Third, the code is also clearer to understand,
and more uniform with the one for global stats.
Split from child patch by Dragons flight.

Bug: T53294
Depends-On: I1dc3be6da1cc9e03bc47e8f8c867089ad0100f6f
Change-Id: I8f47beb73cfc1b63c4b3c809fc6d65a1e66ee334
2019-08-03 11:39:27 +00:00
Daimona Eaytoy a85e1ccc59 Make AbuseFilterParser::$funcCache non-static
Change-Id: I312efe3ce4d1f06e697aa4564aeec1bacbaf97d3
2019-08-03 09:19:49 +00:00
jenkins-bot 0e00654b7d Merge "Save profiling data and vars in cache when running filters" 2019-08-02 23:28:03 +00:00
Daimona Eaytoy 4acb266e90 Save profiling data and vars in cache when running filters
This is the proper solution to replace
Ia8e38ba25d1989fe71714d2b76891c4587921466, using a class member and an
additional method. Plus, change checkFilter not to accept a prefix, but a boolean indicating if the filter is global (as that's how it's used currently).

This change also fixes an issue which caused profiling data for local
filters to be mixed with profiling data for global filters with the same
ID.

Depends-On: Iafc142d2e5ba7aa0fb0d3265fa05cace27679738
Change-Id: I1dc3be6da1cc9e03bc47e8f8c867089ad0100f6f
2019-08-02 22:54:30 +00:00
Daimona Eaytoy 09d0254172 Better handling of DNONE
This patch includes:
 * Making it possible to access offsets of a DNONE (returning a DNONE)
 * Initializing user-defined variables as DNONE inside short-circuited branches
 * Make DNONE propagate with other operators
 * Make DNONE count as false for logic operators
 * Remove a now-outaded bit in doLevelAtom. In case of shortcircuit,
   $result is now DNONE instead of DNULL, and thus it's possible to
   access offsets of it. Performance++!
 * Don't allow modifying or adding an element of a DNONE as if it were an
    array (to avoid inconsistencies)

This re-applies Id85c673337fa90a3782fd22eb9690cd996967111 with several fixes.

NOTE: Haven't tested locally, although I'm pretty confident thanks to
the amount of tests added.

Bug: T214674
Bug: T228677
Change-Id: I5ec4ab44c4e88aaf18c0d7b73355d27050beeda7
2019-08-02 21:05:08 +00:00
jenkins-bot e3e157361d Merge "Revert "Initialize user-defined variables during shortcircuit"" 2019-07-29 23:30:50 +00:00
Daimona Eaytoy 13cdb86dd2 Revert "Initialize user-defined variables during shortcircuit"
Reason for revert: T214674#5374806

This reverts commit 56e6117afd.

Bug: T214674
Change-Id: Iccce248d2693cd9877a740b74e72a577e730435e
2019-07-29 23:06:23 +00:00
jenkins-bot dfa0109ba8 Merge "Rename old/new-(wiki)?text i18n keys" 2019-07-25 08:35:26 +00:00