Commit graph

5098 commits

Author SHA1 Message Date
Daimona Eaytoy d58b5930f8 Add the filter ID to empty operand logging
To make debugging a lot easier.

Bug: T156096
Bug: T153251
Change-Id: I1f905c6e1a524a745240b05709ef9d1dfc3c23a1
2019-08-13 15:22:55 +00:00
Daimona Eaytoy 1197eb6b41 Make parser aware of the filter it is parsing
This information will mostly be used for debugging purposes.

Change-Id: Ia1bcc2acc22aba97d855382b5b173ac3d5f2c54b
2019-08-13 15:22:38 +00:00
Daimona Eaytoy f71d731b61 Rearrange config to provide better experience
In order to bundle AbuseFilter, we have to provide an optimal
configuration for sysadmins. This patch aims to provide such
configuration, and of course it's open to changes.
Changed values:
*Grant abusefilter-view-private to admins. This is mostly for
redundancy-safety (they already have -modify)
*Grant modify-restricted to admins (see T200032)
*Disable the 'degroup' action. This is really dangerous, not ideal to be
enabled by default.
*Grant the abusefilter-revert right to sysops. This is similar to
assigning the modify-restricted right in its rationale.
*Add abusefilter-log-private to viewrestrictedlogs grant. This right
should go together with abusefilter-view-private. (T226987)
*Remove abusefilter-revert from the rollback grant, as that right is
for reverting AF actions and has nothing to do with rollbacks.

This mostly copies the config of WMF production: degrouping is already
disabled there, and modify-restricted should
be assigned to sysops on WMF wikis as well. The assignment of the
abusefilter-view-private and abusefiliter-revert rights is handled in
I09cb462312c5bbc71ee3a8f0fbd91357908130d4 (parent patch) to keep the
status quo.

NOTE: this patch should be *deployed* together with the config patch to
avoid discontinuities. Thus, please ONLY provide a virtual +2 using a
+1, and I'll coordinate the +2 with a deployer.

Bug: T191740
Bug: T200032
Bug: T226987
Depends-On: I09cb462312c5bbc71ee3a8f0fbd91357908130d4
Change-Id: I58f215b8aa3444602532098a3c9dfd25409f29eb
2019-08-13 09:42:42 +00:00
Santhosh Thottingal 1176e3a465
Fix the warning about permission name changes
Change-Id: I16463550328eb19d33270d8677404e012e5c80df
2019-08-13 14:40:17 +05:30
Daimona Eaytoy 7edf12dbc6 Add a hook to determine whether the current action should be filtered
Bug: T229252
Change-Id: I126013b6c516eebe7273fb85f2c5681844e757be
2019-08-13 10:31:29 +02:00
Translation updater bot 2b30cf5823 Localisation updates from https://translatewiki.net.
Change-Id: Ic44ca7de7020952876bf449079da7c7be47bafe1
2019-08-13 09:15:09 +02:00
Daimona Eaytoy 430ba818d0 Add test for multiple conditions inside conditionals
The regression itself was fixed in
I980aec3481a52ecc35f1811a366014a5581a7cdb, so this patch only adds a
test for it.
Also remove a comment about CachingParser failures: we don't want to
encourage people to remove it from tests anymore.

Bug: T152281
Change-Id: I3ad49050ea49bf45d3226878e091da3c8dbefdb1
2019-08-12 18:18:05 +02:00
Daimona Eaytoy 4b0911ee01 Make some other AFPData methods non-static
Change-Id: I22ea337a36f911c57d3dadb9a3c45fc2c8b7c628
2019-08-12 14:40:51 +02:00
Daimona Eaytoy 3f171dc0a5 Move keywords handlers to the Parser
Just like we do for functions, it doesn't really make sense to have
keywords separately, in AFPData.

Change-Id: I208a9b1ce2bd12038e9fbcc515c48d604ec80eb8
2019-08-12 14:29:56 +02:00
Daimona Eaytoy 2fdf091eb9 Make several AFPData functions non-static
The keywords-related ones will be handled in a subsequent patch.

Change-Id: Ifcfad438023ef136dc6f2cd5529e867df9b23789
2019-08-12 14:12:16 +02:00
Daimona Eaytoy 1fe3647268 Avoid implicit casts from DUNDEFINED to something else
This patch keeps the current behaviour for everything (since DUNDEFINED
was always casted to boolean false), but handles the cast at a higher
level instead of relying on what AFPData::castTypes will do. This way
it's easier to spot places where we may get DUNDEFINED, and decide how
to handle them one by one.

Change-Id: I1070e15ea03c7dd4a4231b87afbc42240a558581
2019-08-12 11:18:15 +02:00
jenkins-bot 3748c41e79 Merge "Remove outdated comment, add a new one" 2019-08-12 08:26:30 +00:00
Translation updater bot 172bacaa57 Localisation updates from https://translatewiki.net.
Change-Id: I46ea14d9e804181d0e102ac7b7a01cd9a7df1b08
2019-08-12 09:04:39 +02:00
jenkins-bot eff4e208ca Merge "Don't show export link for new filters" 2019-08-11 17:02:56 +00:00
Daimona Eaytoy 200905f1cf Remove outdated comment, add a new one
As explained in T230093#5408119.

Bug: T230295
Change-Id: Ic0beaf9d126d14fbb7efbd2d0ec55e10c0fbcc99
2019-08-11 14:35:38 +00:00
Daimona Eaytoy c8b5b9321c Don't show export link for new filters
Bug: T230163
Change-Id: I72f46247f4323fb5bfe7fa74f332076dbd346187
2019-08-11 12:21:36 +02:00
Daimona Eaytoy 69ad23da98 Ban variable variables
As explained on phab, it's not worth the effort of keeping this feature.

Bug: T229947
Change-Id: Ic6067cab8e1ede98545e704888c99e2ed9a004e4
2019-08-11 01:47:35 +00:00
Translation updater bot 64d580e038 Localisation updates from https://translatewiki.net.
Change-Id: I9dbb969c021751423e72b5c21a86b2d8b613c83c
2019-08-10 22:47:14 +02:00
Daimona Eaytoy ff40204ef1 Gracefully handle blockautopromote failures
Instead of returning a successful message, return null and log a
warning. Also, make autopromoteBlockKey public + internal and use it
from Hooks instead of duplicating the logic.
Follow-up: I03feb05218789a3b73a31c9a94216daafcb7c145

Change-Id: I8ce96d1bd0239003f8ee6a45f412b9502d542a18
2019-08-10 15:18:30 +02:00
Daimona Eaytoy c34181e548 Add a new API module to retrieve private details from AbuseLog
Follow-up of Iaca492371f48fecf543268c179a651841ed12c3f. This patch adds
the new module, plus some technical changes to private details-related
methods and globals.

Bug: T210329
Depends-On: I613dbadb8f75c8c4116a362607563a436a73d321
Change-Id: I3c45b74c36c191083df184ed57416067a75f6591
2019-08-09 21:10:28 +00:00
Daimona Eaytoy c7ccb68058 Use "privatedetails" instead of "private" where needed
To keep a clear distinction between "private details" (i.e. user's ip)
and "private filters" (i.e. not publicly viewable). This patch renames
rights, i18n keys and methods names.
The patch for renaming globals and rights in WMF config is
I7e6b3d4453403edb6aa602587374b4ff5b6d625f.

Bug: T211004
Change-Id: I613dbadb8f75c8c4116a362607563a436a73d321
2019-08-09 21:10:22 +00:00
Translation updater bot 20d688be04 Localisation updates from https://translatewiki.net.
Change-Id: I8f22802c98de49cb2e4004f4d8098e5b695e4acd
2019-08-09 09:05:56 +02:00
Translation updater bot 0ed690fc74 Localisation updates from https://translatewiki.net.
Change-Id: Ia72ff6afe4a6e836f53eecf906f909c5f0df51b6
2019-08-08 09:30:59 +02:00
jenkins-bot 8ee442234f Merge "Move "block-autopromote" key from $wgMainStash to 'db-replicated'" 2019-08-07 23:07:45 +00:00
jenkins-bot 1fa5eef94c Merge "Overhaul Blockautopromote action" 2019-08-07 23:03:08 +00:00
Translation updater bot d9d728c0b7 Localisation updates from https://translatewiki.net.
Change-Id: I3c082017cfb71e9125e89c6a106b88628f06ebf0
2019-08-07 09:32:53 +02:00
Aaron Schulz 9e44f1a9e9 Move "block-autopromote" key from $wgMainStash to 'db-replicated'
Keep the key mutation methods in the AbuseFilter class

Bug: T227376
Change-Id: I03feb05218789a3b73a31c9a94216daafcb7c145
2019-08-07 01:09:13 +00:00
jenkins-bot 097b2da0ed Merge "Add a parent class for special pages" 2019-08-06 18:32:29 +00:00
jenkins-bot 5a067f7237 Merge "Add tests for empty operand logging" 2019-08-06 17:38:31 +00:00
Daimona Eaytoy b91db1d7be Add tests for empty operand logging
Follow-up 5f4491f9aa.

Change-Id: I80ca8c3c75f7de23cf9ab16aa66a240e9981c395
2019-08-06 17:17:27 +00:00
Daimona Eaytoy 483dab1732 Add a parent class for special pages
This allows us to extract yet another static method from the AbuseFilter
class. This class should be expanded in the future, and an example use
case could be Ia5fd4f0b35fcabf045a7b49fa40fa85b72c92544.

Depends-On: I7c0170167b508132cd16e566c654a6c98dd683e9
Change-Id: I1bb45e47c3b42c01388b99778ce833e4e44419e1
2019-08-06 14:17:38 +00:00
Daimona Eaytoy 2ed6272bb2 Partly handle set and set_var in shortcircuit
This is more complicated than the := operator, because the var name
could be a complicated expression, and we have to handle a function
call. This patch only covers the case where the variable name is a
literal, which is enough for WMF production.

Bug: T214674
Change-Id: I6c0f8e95663919a0235b5ccf0c88ad0a539315a7
2019-08-06 16:14:34 +02:00
Translation updater bot 333bbaaeb3 Localisation updates from https://translatewiki.net.
Change-Id: I911ea882307c02f8d2837881491f5855a802a6c2
2019-08-06 09:07:52 +02:00
Daimona Eaytoy 2bdb44d58b Overhaul Blockautopromote action
As for all mostly unused consequences, blockautopromote has a couple of
major problems: first, it blocked the status for a random time between 3
and 7 days, which to me makes no sense at all (is it some sort of
casino?), and this patch fixes it to 5 days. Second, nothing was logged,
not the blocking nor the unblocking. Here I'm adding a LogHandler for
two new sub-actions of 'rights' to keep track of both action.

Bug: T49412
Change-Id: If48a48f5b8baaf9e77c0826466f5d03bb7f691d0
2019-08-05 22:27:49 -04:00
jenkins-bot 64244f1867 Merge "Really avoid DEMPTY leak" 2019-08-05 16:22:39 +00:00
Translation updater bot 7dd5045896 Localisation updates from https://translatewiki.net.
Change-Id: I91e7ff2b4c3195ad9e6cf26e23088b682e73d337
2019-08-05 09:38:58 +02:00
Daimona Eaytoy afeff7c222 Really avoid DEMPTY leak
Follow-up I7831f3ed9f7c0656e0e8f77ded049c20eca682ba, really avoid the leak. My addition was pointless because we need DUNDEFINED, not DEMPTY, and I spent way too much time trying to understand what was still wrong.
Still have to get used to these new names...

Change-Id: I332967f6fb00b67fd355547b19638c95ffa5bba7
2019-08-04 22:02:13 +00:00
jenkins-bot 44e427601f Merge "Avoid DEMPTY leak" 2019-08-04 21:16:22 +00:00
Daimona Eaytoy 6ef2cf523b Profiling: don't count time for operations shared with the edit
Parsing wikitext and retrieving its links are operations which we share
with the edit, so that if a filter does that, it won't be done later
upon saving.
Thus, add a static variable to subtract such time and avoid to erroneously log as slow any filter using those variables.

Bug: T219092
Depends-On: I24fbd41ac188a9cf6a7d3ca33dce349aedc9faa6
Change-Id: I7c0170167b508132cd16e566c654a6c98dd683e9
2019-08-04 20:12:10 +00:00
jenkins-bot 4347331692 Merge "Reset all filter profiling data at once" 2019-08-04 18:53:57 +00:00
jenkins-bot 19182606c1 Merge "Merge global profiling keys" 2019-08-04 18:40:14 +00:00
Daimona Eaytoy f977e858ab Avoid DEMPTY leak
As shown in the coverage reports [0], some empty operand logging lines are covered, but no test should have empty operands. I see one of the cause is skipOverBraces keeping $result as is, even if DEMPTY, so turn it into a DUNDEFINED.

[0] - https://doc.wikimedia.org/cover-extensions/AbuseFilter/includes/parser/AbuseFilterParser.php.html

Change-Id: I7831f3ed9f7c0656e0e8f77ded049c20eca682ba
2019-08-04 18:25:04 +00:00
Daimona Eaytoy 2742f24bca Reset all filter profiling data at once
Instead of scattering the process all over the code (and doing it
together with checking if the key already exists).
Wrap the logic in new methods for better readability.

Depends-On: Ib12e072a245fcad93c6c6bd452041f3441f68bb7
Change-Id: I24fbd41ac188a9cf6a7d3ca33dce349aedc9faa6
2019-08-04 18:03:14 +00:00
rarohde d022377578 Merge global profiling keys
The last step of the profiling overhaul. See T53294 for the original description by Dragons flight.

Note: Here I'm adding a FixMe for a problem which already exists in the code
and the child patch will fix it.

Bug: T53294
Depends-On: I2d8c8f8278073a9420e3eb373fb89a655925618a
Change-Id: Ib12e072a245fcad93c6c6bd452041f3441f68bb7
2019-08-04 17:59:58 +00:00
jenkins-bot 879ff59fe2 Merge "Split hook handlers related to filtering" 2019-08-04 17:55:51 +00:00
jenkins-bot be4b1c5bc1 Merge "Fix @deprecated since version" 2019-08-04 17:50:47 +00:00
jenkins-bot e733872e13 Merge "Allow accessing offsets of built-in variables" 2019-08-04 17:48:46 +00:00
jenkins-bot 790cd38fb0 Merge "Further deprecation for empty conditions" 2019-08-04 17:26:40 +00:00
Daimona Eaytoy 7d192cb1f2 Split hook handlers related to filtering
This adds a new get(Type)Vars method for every action type. The goal is
to 1-have shorter methods, which is always good; 2-try to make this code
a bit more testable.
I left as a todo moving all these methods to a separate class, the idea
being to make them non-static and thus easier to be tested.

Depends-On: I2eab2e50356eeb5224446ee2d0df9c787ae95b80
Change-Id: I6de2dd27a8f972b3f74c730a1516639f8c622166
2019-08-04 17:21:29 +00:00
Daimona Eaytoy 517919fca8 Allow accessing offsets of built-in variables
I5ec4ab44c4e88aaf18c0d7b73355d27050beeda7 almost fixed this bug, but we
also have to make it possible to access builtin variables as arrays.
This will only make sense for a few variables (e.g. added_lines and
removed_lines), but I don't think we should validate it when checking
syntax.

Bug: T198531
Change-Id: I417e1b8d4802bbfccd091ce5c7617659cfd1e4ea
2019-08-04 17:14:44 +00:00