This fixes the abuse_filter_log patch-afl_change_deleted_patrolled
not being applied. The patch is provided for (and should work with) all
the supported DBMS.
Additionally, fix the base table files, which would report
afl_patrolled_by as 'NULL', whereas on the WMF cluster it's 'NOT NULL
DEFAULT 0'. The schema patch takes care of converting that column as
well.
Note that this schema change needs not be applied on the WMF cluster, as
that's already up-to-date.
Finally, note that this patch must be backported to 1.33 and 1.34 (and
it might be fairly hard due to the recent schema changes on the
abuse_filter_log table).
Bug: T240895
Change-Id: Ibdbc9b50c25b9e871ebdeae93a54d10877b585f8
The <pre> element is now hidden with CSS, and is only shown after the
user clicks the "Eval" button.
Moreover, make the button primary and progressive, as to indicate that
it activates the primary function of that page.
Bug: T253492
Change-Id: I300ce6ec0a84ea73025a5af9173024df7c291e03
We have many topnav links, and future patches may add others (e.g.
Ia5fd4f0b35fcabf045a7b49fa40fa85b72c92544). The "import" feature is
probably the less used, and is also pretty similar to creating a new
filter.
Thus, remove its link from the topbar and move it to a button next to
the "Create a new filter" button.
Note that the old message is reusable, and thus it should be moved on
translatewiki after merge.
Change-Id: I52042d62b2bab7e4a1e9bbc027e7de5addec8157
While checking a filter, if a variable is not set (e.g. added_lines for
an account creation), the VariableHolder will return a DNULL, rather
than a DUNDEFINED. This means that some filters will resume working, and
the WMF servers will stop getting AF warnings at a rate of 4 millions per
day. This also requires adjusting some tests to reflect the new
behaviour (which is actually the OLD behaviour, that filters had until
last year when we introduced the DUNDEFINED data type). It also requires
adjusting a check in the old parser, but that's not really relevant
because the plan is to remove the old parser before 1.36 is released
(see I0e75f334c7e0dfc1239f2e5f5f7d7452b0bbf29e).
Bug: T230256
Change-Id: I4d06303047397674c1edbfc32628f1bc83ac3340
Rather than always using 5 days, the length (in days) can be configured by setting
`AbuseFilterBlockAutopromoteDuration` to the desired length.
Bug: T231756
Change-Id: I996e08a9099ab59657fe511ec2934d26edfa5c7b
This is an intermediate step for better "diff" links
on abuse log. With this first change, only links
to existing revisions are shown.
Change-Id: Ib420d46fd34dc38d8c7fd3d511a905738e49db0b
For history action, the link would be already added by
HistoryPageToolLinks hook, so it should not be duplicated by this hook.
See images on https://phabricator.wikimedia.org/T261087#6430172
Change-Id: Ia8dd5be49d3ffb48f298ea287e0b2f98c3052015
This shouldn't happen before the script has been tested thoroughly on
WMF wikis with --dry-run.
Bug: T213006
Change-Id: I51425c85bd6932a5c60eb870b02195aae1c24117
This can be different from the User set inside the $context object, as
seen e.g. in Wikibase jobs. Given that the hook provides a $user param,
it makes more sense to use that, rather than extracting it from the
ContextSource kitchen sink.
Bug: T258717
Change-Id: Ib5961068d3df6ae2bfc3f9c6a7b9e555d248b332
Some AbuseLog entries from 2009 are missing the 'timestamp' parameter
used to compute the old wikitext of the page. This was only used for a
short amount of time before
https://phabricator.wikimedia.org/rEABFd1d27eede6536067c5180b2515ea937d71525d4d.
Nowadays, it's causing a fatal error when we try to migrate the affected
entries, see T246539#6388362.
Since we only have a Title available, we cannot rebuild what the old
wikitext would look like, so a placeholder text is used (this should
hopefully be clearer than showing an empty string).
Bug: T246539
Change-Id: I5230f2fdc84da121728a5a75da458f1a4ef1ecd3
ParserOptions::setTidy() was already a no-op in MW 1.35, and
AbuseFilter already requires MW >= 1.35 in extension.json.
ParserOptions::setTidy() was deprecated in MW 1.35 and will be removed
in a future release.
Bug: T198214
Change-Id: I269e829cf1f33e233bfcf7f95388e041180c2556
These render string arguments with potentially sensitive information
that we don't want to store in debug logs. Use the standard
'exception' field instead per T233342, letting the central
logic be in charge of creating 'exception.trace' with the normalised
trace rendering and filtering logic we have there.
Bug: T233342
Change-Id: I4620f36229fd5076b4370d20149c890030bf4c64
Any should always be the first choice. Other/None should always be
the last choice. The rest of the choices come in between and should
be sorted alphabetically.
Also capitalize the first letter of "None" for filtering logs down
to those in which no action taken. This makes the options uniform.
Bug: T255533
Change-Id: Id106bbc352531437af95a303b7dcf32e44383f95
For MySQL it was renamed Ic1252efe9f96743d9402fa31a7b2dca1f57ff6ae, but
the old index isn't being deleted, hence creating a duplicate.
Change-Id: I09b9f64759f6a897c393caa77458d63995d5713b
When using RecentChange::getQueryInfo() it should be used to instance a
RecentChange class
RCDatabaseLogEntry is only useful in context of LogFormatter
This is a breaking change for the hook variable,
but "RC entry" refers more to the RecentChange class than to the
RCDatabaseLogEntry class
Change-Id: I3af1e42594f8235be815ce38e3411c762ae01092
Additional changes:
* Removed phan-taint-check-plugin from extra, now inherited from mediawiki-phan-config.
Change-Id: Ib63be75df4bfdbd2c5b97de5f80dbec715108c01
The form is now collapsed by default, as that seems to be the most
common way to do that.
Bug: T252584
Change-Id: Ie3fa3d2858519e6bc03854a12f90f76a684e7648
The problem is explained at T250570#6068702; basically, the previous
check didn't account for DUNDEFINED nested deep inside arrays.
Bug: T250570
Change-Id: Iacee2db54ca00108de6339bb3dae70af7e2eeb56
Using var_export for better visual effect, especially for arrays.
The result from /tools is much clearer and the 'wrong syntax' message is
a bit more explicative than before.
Bug: T190653
Bug: T239972
Change-Id: I79a17305c7f19f7900f896f895e9365bb5f2fd58
- Increase batch size to 500
- Add an option to print progress markers
- Fix some bad logic which caused some JSONified data to be stored in
the text table without checking (and respecting) old_flags. This caused
some errors on the beta cluster.
Additionally, add a return typehint to AbuseFilter::loadVarDump to make
sure that errors are caught asap. Not only there's no apparent way that
loadVarDump can return an array, but most code is already using the
result as a VariableHolder, unconditionally. This is probably another
leftover from the past.
Bug: T213006
Bug: T246539
Change-Id: Iaebd28badb70d27693fa809cad4db956881e3e5e
This was used to dynamically generate *_restriction_* variables.
However, it had two big problems:
- We only have i18n for 'create', 'move', 'edit', and 'upload' (the
default value of the global); other restrictions would show missing
messages in various pages.
- We had to access the global state in various points.
This change also makes some code in AbuseFilterVariableHolder simpler,
and also allows us to make AbuseFilterTest a unit test.
Change-Id: I321ad6e07f8243200af67a581b6e485970efd3ce
Complete WikiPage/Article split and deprecate Page interface
Using actual WikiPage/Article contract
Bug: T239975
Change-Id: I343c3ca2e30715656950cab49c6470061c72b9a0
At the moment there's no validation for import data, so it's totally
possible to insert rubbish in the field, and the code will produce other
rubbish. For instance, it's not so uncommon to see lots of PHP notices
on logstash for ViewEdit code trying to access members of the imported
data as if it were an object.
Change-Id: If9d783f0f9242d3d1bc297572471e62f51ee0e40
Follow-up Ie9aae938cca06e38a7a834a3f74f3e8735ab01ee.
Some fields are actually necessary when the filter isn't saved. This
would cause PHP notices when showing the editor again.
Change-Id: I2b9e0f04b3e8ad4eea8e334e16ee422bb40f0eb5
In T43172 it was told that adding the site name could increase the risk of
attracting more spam, but I don't see how this variable could cause that.
Bug: T240948
Bug: T97933
Change-Id: I1d2aeabaf008ac06798b8d7e4af7d61ae1702776
This code was introduced with Iba59fe8d190dd338ecc8cfd682205bce33c9738b
and is unused since then. The name should highlight that those variables are not
supposed to be "static", i.e. immutable. Examples are: timestamp, spam
blacklist, site name, site language. These are not immutable, but rather
"generic", and they're known even without an ongoing action.
Also add an RC row param and update docs.
Change-Id: I402f04585e9154059fc413e527e39dcb8e6b3d7c
Follow-up Iabd0ae5b18571f8cad44ef2d86bcf2519e7f95ba.
This patch:
- Moves some save-related code to a separate method
- Reduces conditionals nesting
- Fixes an edge case where the content of the form would be
wiped in case the token didn't match.
- Adds another (basic) selenium test
- Standardizes return types
- Moves data load outside of buildFilterEditor
Change-Id: I89444b59f04c495c9ab59244151c8ed5d38cf0fe
This is another step needed to reduce the size of the gigantic
AbuseFilter and AbuseFilterHooks classes. It also makes many methods
non-static, for more testability.
Note, this layout is still not final. We should somehow merge the
functionality of VariableGenerator and AFComputedVariable, for which
I already have plans.
Change-Id: I366d598b69ad866496b7cb0059e0835c02e54041
RunVariableGenerator is for generating variables based on the current
action;
RowVariableGenerator is for RC entries;
VariableGenerator is the generic one.
This patch only moves the methods to the new classes, to keep the diff
easier to read, and facilitate conflict resolution. These classes will
then be revamped in I366d598b69ad866496b7cb0059e0835c02e54041.
Note that these classes are now namespaced.
One method, AbuseFilter::getEditVars, was renamed to
AbuseFilterVariableGenerator::generateEditVars, because it would
otherwise conflict with an incompatible method in RunVariableGenerator.
Change-Id: Iff412e5492873d4fae55402939a51609e64d55a8
This provides various shortcuts for user, target, comment, etc.,
avoiding direct access to the row, and thus a dependency on the
schema.
Change-Id: I250f94e0ac6cade33441a31ae8a27093a4d937a0
Also fix a couple of broken tests in Consequences:
- For createaccount, $user->addToDatabase must be called before
testForAccountCreation, or it will throw a CannotCreateActorException.
- In testThrottleLimit, also set wgAbuseFilterEmergencyDisableThreshold
to avoid relying on the local config.
Bug: T201193
Change-Id: If1a50b0a729e4d554485f2e2225d5877510966b6
Most of them are overwritten either in ViewEdit::loadRequest or
AbuseFilter::saveFilter. af_hit_count and af_throttled are actually
relevant for the old version, so list them explicitly. And also add
default af_group and af_global, which are later read, for import action.
Depends-On: Iabd0ae5b18571f8cad44ef2d86bcf2519e7f95ba
Change-Id: Ie9aae938cca06e38a7a834a3f74f3e8735ab01ee
Before the phan upgrade, this was silently choking on null as so falling
back to age since 1970-01-01 (~50 years); since the upgrade, the code is
breaking filters by responding with 0. The approximation of using 2008's
Wikipedia Day is less wrong and more fun (credit to Roan for making this
suggestion).
Bug: T243469
Change-Id: Ibc25ab09ecd0bf0b2292425c2768b1dc911b9974
Instead of having a single loadRequest method (which could end up
loading from the DB...), split it in a DB-only method and a request-only
one. Simplify the logic used to show the filter editor. Show the page
without changes or warnings if the user lost editing rights in the
meanwhile. Avoid two static properties, and pass them in when relevant
instead. Bonus: optimize a query to sort by afh_id instead of afh_timestamp to avoid filesort.
This will allow a subsequent patch to clean the $row object in
loadRequest.
Change-Id: Iabd0ae5b18571f8cad44ef2d86bcf2519e7f95ba
composer:
* mediawiki/mediawiki-codesniffer: 28.0.0 → 29.0.0
The following sniffs are failing and were disabled:
* MediaWiki.Commenting.FunctionComment.MissingParamTag
* MediaWiki.Commenting.FunctionComment.ParamNameNoMatch
npm:
* eslint-config-wikimedia: 0.13.1 → 0.15.0
* grunt-stylelint: 0.11.1 → 0.13.0
* stylelint-config-wikimedia: 0.6.0 → 0.8.0
Additional changes:
* Remove direct "stylelint" dependency in favor of "grunt-stylelint".
* Also sorted "composer fix" command to run phpcbf last.
* Removing manual reportUnusedDisableDirectives for eslint.
Change-Id: I8f73202db1333fbc36ccf556b3bb05b1e8c279cb
This is a preparatory step for T234427 (although not strictly related),
and in the future it will enable us not to use the DB in several tests.
Change-Id: Id069f6e74f9c4df43b3a602d4224473d5ca68ed1
-new_html: also strip the "Transclusion limit" comment if present, and
anyway take it into account (as well as a "</div>"), which right now
prevent the PP limit report from being stripped as well.
-new_text: trim extra whitespace on the right, which is created when
stripping the aforementioned comments.
Also simplify the test for getEditVars, make it not blindly copy what
AFComputedVariable does.
Extra: kill a temporary variable.
These changes are partly taken from
I96785c6c5fdf381c21d5f8930ee12e706abb7f3f.
Change-Id: I2b4c84a3d9d0d17ce229088197b75781d5181b4f
This patch is mostly replacing Revision::* constants,
Wikimedia\(restore|suppress)Warnings, and wfWikiId.
Change-Id: I13544cc3e12955a9376ccce3c120e2cee1f2ee2e
Even if the array is DUNDEFINED, we need to check the offset to ensure
that it's valid.
Bug: T237351
Change-Id: Ibfa360c4ae1d80abe14d9fdf66991b76cb5954df
For the new parser, xhgui shows that AbuseFilterParser::getVarValue is
taking up a lot of time; in turn, most of the time spent inside
getVarValue is used to log the use of deprecated variables. Hence, given
that:
- We should keep the new parser performant
- There are tons of deprecated variables out there and they likely
won't be replaced
- Having gazillions of debugLog entries doesn't help
log them only in the cached phase.
Bug: T234427
Change-Id: I2bfc692c829c3cbe889e5076f5205e2c99097087
In other View* classes, AbuseFilterView::mFilter contains the ID of a
filter, e.g. the filter being edited in ViewEdit. In ViewTestBatch,
however, it is a string containing some filter text. Hence, use a new
private property instead (without the legacy "m" prefix).
Change-Id: Ib22ce238aff4ca5ed57ba725ee9bff7f8c3d153b
Thinking about it again, all messages on ViewEdit start with
abusefilter-edit. Also add a reference to the other message to
facilitate translations.
Follow-up: I3717d06d4a757684fe6622961391ae06b5bd3c38
Bug: T235590
Change-Id: I4cbaa2e92d22296f55a4b5ef0c633fe959fe9ea3
Even if the Content objects are different, the normalized text contents
may be identical.
Also, stop misattributing null edits by adding the last revision of the
page as afl_rev_id.
Bug: T240115
Change-Id: I3fb7b36ab38ca1544889a4c233b8ffdfc6c80936
This is identical to I8a3c31e7385283d95b4712d457784016239a0b3b, except
for the array append case.
Bug: T236870
Change-Id: Iac033ba467232f6ff110d575920e968759ce0e15
Otherwise deleted and disabled filters would be mixed. Needs dependency
in core, otherwise we'd use af_deleted as secondary sort for every other
sortable field.
Bug: T191694
Depends-On: I0e695f96f18c7a9229753b1225dd473feb936a31
Change-Id: I979849e66bdcc158b7a3d0793ee3196e20db37b6
This is consistent with the "anti-DoS" measures on other API modules.
Although this may not be a serious DoS vector, it makes sense to
restrict this module. Moreover, it's also consistent with
Special:AbuseFilter/tools (which is the corresponding web interface),
which requires the same user rights.
Bug: T238451
Change-Id: Id09fd57195d71884674ac0470f137ca30c56e13c
AbuseFilterViewEdit does privilege checks based on filter ID,
and displays what is hidden under given history ID, but doesn't
make sure those two IDs actually belong to one filter.
That means user can easily change filter ID to a public
filter and view old versions of nowadays private filters.
Bug: T237887
Change-Id: Ic12790bd33982473f77551bde9599ed083a3e1f1
This will allow people to switch their filters to the new syntax. The
deprecation warning is now more exhaustive, and the info() warning is
kept to ensure that everything proceeds smoothly.
The regex v2 has also been fixed to:
- Consume all the digits/letters on the right (*)
- Have named groups
- Be created dynamically with other constants
(*) The previous version of v2 could complete the match and leave
digits/letters on the right when encountering numbers with the old
syntax, hence dropping support too early. We also cannot use a word
boundary (\b) because that would prevent matching numbers with trailing
dots (e.g. "5.").
Bug: T212730
Change-Id: Ibf6ac571f6b5c09149d69a19c38240ce6b024dff
This bumps the level to WARN, and makes it very clear that people should
fix the affected filters. It also removes the calling method, which was
mostly meant for debugging purposes, and changes the type to 'op_type'
to avoid conflicting with type:mediawiki in logstash.
Bug: T156096
Change-Id: Ie73f1604e8ed82bc2e1be9fc90fa065be37889a3
Currently, `abusefilter-edit-oldwarning` is shown to all users, but not all users are able to edit the filters, and thus the warning about editing isn't applicable to them.
Bug: T235590
Change-Id: I3717d06d4a757684fe6622961391ae06b5bd3c38
Always run the keyword/function handler, even if there are DUNDEFINED
arguments, so that the handler can perform further validation on the
input and report any error to the user. However, replace DUNDEFINED with
DNULL before running the handler, to avoid special-casing DUNDEFINED in
every handler. If any argument was a DUNDEFINED, we will return
DUNDEFINED anyway.
Also centralize the keyword handling logic to a new method, like it
happens for functions.
Bug: T234339
Change-Id: I875cb77418a39790e91fe5867c49917bfe406ed4
This allows sharing the code between cachingparser and the old parser
(for DRY-ness), and even when the old parser will be killed, having the
logic outside of the generic parse method seems saner.
This copies what I446a307e5395ea8cc8ec5ca5d5390b074bea2f24 did for
functions.
Change-Id: Ie6290243a6c78661510a9b4cb713d6e7b2778248
This emits its own error because:
1- It's clearer to understand
2- It's easier to find where we're dealing with negative offsets, if
we'll ever want to allow that.
Note that trying to use a negative index already results in a hard PHP
error being thrown.
Bug: T237219
Change-Id: Ib11eaaca5e21f740269141c75e62bac48093e8d0
As the code comment says, and as it was suggested in
Iafe54285384bc28b3e8812b495166f2682d4571c, we were validating the
provided regexp as PCRE, but using it in SQL, which only supports POSIX.
Furthermore, we won't have to worry about cross-DBMS compat anymore.
Bug: T193068
Change-Id: If6d8717795b6c1dcf619a23363eb6144902cfaed
Instead of checking if the filter is currently hidden, check the
visibility for each version and, if the user cannot see private filters,
only show the diff if none of the revision is hidden.
Also avoid showing a "diff" link if the user cannot see it.
Bug: T104807
Change-Id: Ie23e8234ae550273bf3f6f9c5ac45b7fc54eec2a
In Ib7427e15f673a575738489476e604c387f449ddd, I thought that $parameters could've only been null if $action wasn't
enabled, but actually, they're null even if the action is just not set.
Which is true for all actions when creating a new filter, and all
non-set actions when editing an existing one.
Hence, revert the part that touched ViewEdit.
Also add a selenium test to ensure that warn parameters are visible.
Bug: T236286
Change-Id: I8150baa077208eb1fc54ebc1d8415a243d0f3bd3
The method, which simply duplicates an AFPData instance, is only used
when casting types, to return a different instance when the object
already has the desired type.
However, nothing is assuming that, so we can just return the original
instance and save some time.
Bug: T234427
Change-Id: Id8067b418a00260ceead35f234e55268390699ab
I just realized that the parser is already throwing if it finds a
disabled variable. Hence, all calls to getVar with a disabled var are
from old entries and the like, and we don't care.
Bug: T234048
Change-Id: I39429d286575df91108a4119177a0d3aef181d0b
This is a micro-optimization, but IMHO it's necessary. The AF parser
code is executed for every active filter, for every
edit/move/deletion/accountcreation. In PHP, foreach is usually faster
than array_map. Especially in the case of variadic functions potentially
taking hundreds of strings, foreach will consume less time.
Bug: T234427
Change-Id: I1beedf419a6637a9a3dd668635645df950ceda21
This follows-up 8587576655 (AF) and efbfa0a727 (core). The
method was recently introduced within the 1.34 cycle but
renamed following late CR feedback.
Change-Id: I9986deb080791c6266c6c60cc91022266ad9b5e5
This also includes the filter ID. If the filter ID is not available, it
means that the user is using stuff like /tools, and they'll immediately
see the error.
Bug: T234048
Change-Id: I44a37d98c80df910b0c466fbd464e69042770c0c
$summary and $user are always guaranteed to be passed, and $user is
guaranteed to be a User object. Hence, update the hook handler to
reflect that.
Change-Id: I3a7fcb074b460b77210de5a6bad43f500aff3249
Deleted/suppressed usernames and summaries leak through AbuseLog.
Temporarily hide all non-public revision from AbuseLog, until we can
properly fix the issue.
Bug: T224203
Change-Id: If3d3256404d0f3dbde171831937d1a816b3e2734
This allows us to:
- Defer handling of the block to the main module
- Choose the right message depending on the block type
- Avoid directly using the apierror-blocked message, which could change
in the future.
Change-Id: If2e32bd2ccf5e314aa51203afd1522b8481377e0
Follows-up: I35f2c6e701a24dccb6e26e3f3c578fd44f68127d
This is similar to the old parser: when discarding a node, actually
evaluate it if short-circuit is not allowed.
Add a whole lot of tests for all possible exceptions.
Move the logic to extract a message from an AFPUserVisibleException away
from the parser, to keep unit tests working.
Bug: T232498
Change-Id: I31ee4e255c6a87dd693b9bcd582539fdf57acd45
This implements T230982#5475400, and it should speed up the CachingParser by roughly 40%.
Bug: T230982
Change-Id: I803cc58637d50eb90e57decf243f5ca78075d63d
