Why:
* Protected variables were introduced to support temporary accounts
so that temporary users could be filtered based on their IP address.
* Filters that use protected variables are protected in order to
preserve privacy. This can't be undone.
* The current workflow for protecting filters is to have a 'protect'
checkbox that can be checked for any filter, even one without
protected variables, to the discretion of the editor. This has
led to mistakes (see T37765).
What:
* Do not show the 'protect' checkbox on the form by default. Instead,
show it when saving a filter with protected variables, after form
submission. The user must check it to save the filter.
* Still show the checkbox in a disabled state with a warning message
if a filter is already protected, so that the editor can easily see
it is protected.
Bug: T377765
Change-Id: Ie5c94ac1399860ccdca4482508dd37ff07309764
- Clarify that to use protected variables in a filter, they must be
enabled and will cause the filter to be considered protected.
Bug: T377553
Change-Id: I69b879f12cfe76e6fff0080dd93024d6bd29159d
- abusefilter-blocked-domains-intro: link to Special:Log/abusefilterblockeddomainhit
- log-description-abusefilterblockeddomainhit: link to Special:BlockedExternalDomains
Bug: T376506
Change-Id: If21c6e2de8b9d524d5299487f58a09d2a8d53720
To hint to translators that gender can be used,
and to avoid warnings on translatewiki about
missing parameters.
Change-Id: Ie9523527d1ce138f978145ddaa565137a7b34ab1
Like CheckUser, AbuseFilter should also log when specific protected
logs are viewed.
- Add support for debouncing logs to reduce log spam
- Log when AbuseFilterViewExamine with protected variables available
is accessed
- Log when SpecialAbuseLog with protected variables available is
accessed
- Log when QueryAbuseLog with protected variables available is accessed
Bug: T365743
Change-Id: If31a71ea5c7e2dd7c5d26ad37dc474787a7d5b1a
Similar to how CheckUser logs access to IP information about temporary
accounts, AbuseFilter needs to log whenever protected variables are
accessed.
- Implement ProtectedVarsAccessLogger which handles access logging
- Log whenever a user changes their ability to access protected
variables via Special:Preferences
Bug: T371798
Change-Id: Ic7024d9c5f369eb33c4198a59638de9a1d58b04b
Users need to enable a preference before gaining access to the IPs
from `user_unnamed_ip`, a protected variable.
- Add a preference that the user can check to toggle their access
- Check for the preference and the view right for logs that reveal
protected variables on:
+ AbuseFilterViewExamine
+ SpecialAbuseLog
+ QueryAbuseLog
Bug: T371798
Change-Id: I5363380d999118982b216585ea73ee4274a6eac1
A filter using a protected variable can be loaded via filter id
using testing tools even though the user might not have the right
to view protected variables. This can potentially leak PII and as
such, testing tools should check for the right before allowing
protected filters to be seen.
- Unload a filter asap if it uses protected variables and the
requestor doesn't have viewing rights. This:
+ disallows loading of existing protected filters on page load
+ disallows testing against rules that use protected variables
+ disallows subsequent requests for protected filters (via API)
There is a known bug (see T369620) where no user feedback is
provided if an API request for a filter returns no result (typically
when no filter matches the requested id). This commit adds another
pathway to that bug (the filter exists but is protected and not
returned by the API) but does not update this UI/UX.
Bug: T364834
Change-Id: I6a572790edd743596d70c9c4a2ee52b4561e25f3
Some features restrict access when filters are private. These features
should treat protected filters similarly.
If the user doesn't have view rights for protected filters:
- Disallow viewing of logs generated by protected filters
- Disallow querying of matches against protected filters
Bug: T363906
Change-Id: Id84bd4ca7c8e0419fccc3ad83afff35067c9bf70
The UI/UX for acknowledging a filter will be protected/is protected
could be clearer. The checkbox implemented currently doesn't make
it clear that the acknowledgement is mandatory and filters that are
already protected allow for the checkbox to be unchecked even though
that doesn't reflect that the filter cannot be unprotected.
- Update copy for the protected filter acknowledgement to make it clear
that it's a mandatory acknowledgement, not an optional one
- Update copy for the error that shows when a filter that should be
protected doesn't have the acknowledgement checked
- When a filter is already protected, disable the acknowledgement
checkbox to indicate this is not mutable
Bug: T364485
Change-Id: I667fcca4511dff1ac3ca69930c5b5e5eb5001787
Attempting to access a diff in the history of a protected filter when
the user doesn't have the right to view protected variables results
in the 'abusefilter-history-error-protected' error and requires copy.
- Add copy for the user-facing 'abusefilter-history-error-protected'
error
Bug: T364465
Change-Id: I0e9afae90c43bd3f792f1330ea865c0d56a023d1
- Add a basic checkbox on the filter edit page that must be checked if a
filter uses a protected variable to ensure that the user is aware that
their filter will also become protected
Bug: T364485
Change-Id: I7c7652f7d1a81223229b839ff7eee5da4af74c8a
Some exposed variables (eg. `user_ip`) used in filters are sensitive
and need to only be available to restricted groups of users.
Back-end changes:
- Add `AbuseFilterProtectedVariables` which defines what variables are
protected by the new right `abusefilter-access-protected-vars`
- Add the concept of a `protected` variable, the use of which will
denote the entire filter as protected via a flag on `af_hidden`
New UX features:
- Display changes to the protected status of filters on history and diff
pages
- Check for protected variables and the right to see them in filter
validation and don't allow a filter to be saved if it uses a variable
that the user doesn't have access to
- Check for the right to view protected variables before allowing access
and edits to existing filters that use them
Bug: T364465
Bug: T363906
Change-Id: I828bbb4015e87040f69a8e10c7888273c4f24dd3
After temporary accounts are enabled, filters that rely on an ip
in the `user_name` will fail (eg. `ip_in_range` and `ip_in_ranges`).
To keep these filters working:
- Expose the IP through another variable, `user_unnamed_ip`, that can be
used instead of `user_name`.
- The variable is scoped to only reveal the IPs of temporary accounts
and un-logged in users.
- Wikis that don't have temporary accounts enabled will be able to see
this variable but it won't provide information that `user_name`
wasn't already providing
- Introduce the concept of transforming variable values before writing
to the blob store and after retrieval, as IPs need to be deleted from
the logs eventually and can't be stored as-is in the amend-only blob
store
Bug: T357772
Change-Id: I8c11e06ccb9e78b9a991e033fe43f5dded8f7bb2
This solves two issues described in bug T360909:
* Usage of unsafe characters that have to be
manually reviewed in translations.
* Incorect display of some functions and
operators in RTL UI languages.
It also reduces the translators' need to copy
those operators and functions, which are always
identical to English.
Finally, this patch adds those consistently to all
the messages. Some messages didn't mention them
for an unspecified reason, and now they are mentioned
everywhere.
Bug: T360909
Change-Id: I3283c91b6b1d5fe9b48b1477cd454d9def3a7ded
Make the order of the messages that describe
operators and functions in the en.json file
identical to their order in
KeywordManager::BUILDER_VALUES, which is also
their order in the actual UI of the filter editor.
This only reorders the mesages in the en.json file.
It's not supposed to change anything in
the end users' experience, but it will change
the order in which translators on translatewiki.net
see them.
This is a cleanup step towards removing
the explicit operators from the messages,
as suggested in T360909, and this reordering
is hopefully useful even without that change,
for general consistency.
Comments about particular messages:
* abusefilter-edit-builder-vars-timestamp-expanded
is moved to the very end because, despite its key,
it's not actually used in the filter builder.
* old-text, old-html, and minor-edit are moved towards
the end because they are outdated. They are listed
separately from BUILDER_VALUES and they are not used
in the filter builder UI, but they are used in the logs
of previous actions. This patch adds a code comment
for the benefit of developers who touch that code
in the future.
Bug: T360909
Change-Id: I86ecdca5a6173b9068d5e968e69c57c74a379888
Why:
* An AbuseFilter variable is needed that allows filters to determine
what type the current user is. That is, whether the user is an
IP address, temporary account, named user or external user.
* Currently filters implement this by inspecting the value in
the 'user_name' variable, but this is likely to break when
temporary accounts are enabled as IPs would be hidden.
* Giving a dedicated variable that indicates the type of the user
allows filters to work out this information without having to
know the specific username of the user before performing the
check.
What:
* Add the 'user_type' variable which is lazily computed. It can have
the value 'named', 'temp', 'ip' or 'external' depending on the
type of the user. If the user does not match any of these, then
the value is 'unknown'.
* Replace call to deprecated User::newFromIdentity with a use of the
UserFactory service that is dependency injected.
* Add and update tests to ensure consistent test coverage.
Bug: T357615
Change-Id: Ifffa891879e7e49d2430a0330116b34c5a03049d
This feature never worked very well, and the original wish
https://w.wiki/7ZsE didn't ask for a 2010 editor solution, anyway.
Rather than have AbuseFilterBlockedExternalDomainsNotification linger in
an unstable state, we remove the code entirely.
Bug: T347435
Follow-Up: I7eae55f12da9ee58be5786bfc153e549b09598e7
Change-Id: I88e87c4e0a2968b892394461b1227f4d15938e8e
This field gets added automatically when using the special page form but
is only shown to admins and other people who have access. It's not private
information (users can find it in history) but this is to avoid making
these admins an easy target for harassment (Talking to PM of moderation
team he agreed this is a good compromise).
Bug: T341626
Change-Id: I8410f39db54b96981b05de8e064fed65df30ef2f
- Mentions filter number and name in the title
- Distinguishes between viewing and editing
Bug: T353106
Change-Id: Idda9854a78937033b168603810154b48288c3f4c
This leverages the new BlockedExternalDomains system that is now part of
AbuseFilter. It notifies editors in realtime if a link they add is
blocked. See https://w.wiki/7ZsF for more information.
BlockedExternalDomains is slated to have its own API tantamount to the
action=spamblacklist endpoint, after which case this code will need to be
updated. In the meantime, it's meant to serve as a minimal viable product
for the CWS 2023 wish <https://w.wiki/7ZsE> for wikitext users.
The new $wgAbuseFilterBlockedExternalDomainsNotification configuration
setting controls the availability of this feature.
A similar feature for VisaulEditor is tracked at T276857
Bug: T347435
Change-Id: I7eae55f12da9ee58be5786bfc153e549b09598e7
These two messages appear at the top of pages such as
https://en.wikipedia.org/wiki/Special:AbuseFilter/history/50/diff/prev/27425
The words "Back to" are not really necessary there. It's clear enough
to simply write "Filter editor" and "Filter history", without showing
"Back to" next to each other.
Change-Id: I44672ffd49ba6c24dbf49f9c7de3bbd72f154dbf
This is basically copy paste of SpamBlacklist logging with the added
extra bit of what triggered the hit.
Bug: T337431
Change-Id: Ieb9e3ca615af88ab56735b56e24c80c42a68d478
This makes raw page editing safer, and potentially enables opening up
access to less restricted user groups.
Bug: T337431
Change-Id: I14f21003a551f34b6e524e9b229613e79b0e5a70
It is behind a feature flag. Improvements on it can happen in follow
ups. The patch is already quite massive.
Bug: T337431
Bug: T279275
Change-Id: I3df949c4d41ce65bb4afa013da9c691ac05fc760
Temporary users get the same block expiry as anonymous (IP) users,
since d42b7335d5.
Update the checkbox label to reflect this.
Change-Id: Ibf60936d9c746d857fc4354552d71e1efdd52066
Added support for ip_in_ranges which allow multiple ranges to be
checked at the same time. If the IP is in any of the ranges, the
function returns true.
Bug: T305017
Change-Id: Ic75c87ecd4cacf47ce2ff1b04173405230ff81d0
- Use a /64 range for IPv6 instead of /16.
- Fix a curious and serious bug for IPv6, where grouping by range
would only use the first (!) number of the IP address, due to the
'v6-' prefix returned by IP::toHex.
- Fail hard if the identifier is unknown -- it's not something that's
supposed to happen.
- Include the type name in each identifier, instead of prefixing all
type names to all identifiers. This makes it easier to understand the
parts of the key.
- Test the whole lot.
Bug: T211101
Change-Id: I54c4209f2f0d5a4c5e7b81bed240ca3e28a2ded7
List which actions were disabled, or explicitly say that no actions were
disabled if that's the case. Also avoid the word "throttle" in messages
as it may be hard to translate. Also don't suggest optimizations to the
filter conditions -- unoptimized rules have nothing to do with a filter
being throttled.
Bug: T200036
Change-Id: Id989fb185453d068b7685241ee49189a2df67b5f
This commit adds a class AFPSyntaxChecker which can statically analyze
a filter code to detect the following errors:
- unbound variables (which comes in two modes: conservative and liberal,
default to conservative)
- unused variables (disabled by default for compatibilty)
- assignment on built-in identifiers
- function application's arity mismatch
- function application's invalid function name
- non-string literal in the first argument of set / set_var
The existing parser and evaluator are modified as follows:
- The new (caching) evaluator no longer needs to perform variable
hoisting at runtime.
- Note that for array assignment, this changes the semantics.
- The new parser is more lenient, reducing parsing errors.
The static analyzer will catch these errors instead, allowing us
to give a much better error message and reduces the complexity of
the parser.
* The parser now allows function name to be any identifier.
* The parser now allows arity mismatch to occur.
* The parser now allows the first argument of set to be any expression.
Concretely, obvious changes that users will see are:
1. a := [1]; false & (a[] := 2); a[0] === 1
would evaluate to true, while it used to evaluate to the undefined value
due to hoisting
2. f(1)
will now error with 'f is not a valid function' as opposed to
'Unexpected "T_BRACE"'
3. length
will now error with 'Illegal use of built-in identifier "length"'
as opposed to 'Expected a ('
Appendix: conservative and liberal mode
The conservative mode is completely compatible with the current evaluator.
That is,
false & (a := 1); a
will not deem `a` as unbound, though this is actually undesirable because
`a` would then be bound to the troublesome undefined value.
The liberal mode rejects the above pattern by deeming `a` as unbound.
However, it also rejects
true & (a := 1); a
even though (a := 1) is always executed. Since there are several filters
in Wikimedia projects that rely on this behavior, we default the mode
to conservative for now.
Note that even the liberal mode doesn't really respect lexical scope
appeared in some other programming languages (see also T234690).
For instance:
(if true then (a := 1) else (a := 2) end); a
would be accepted by the liberal checker, even though under lexical scope,
`a` would be unbound. However, it is unlikely that lexical scope
will be suitable for the filter language, as most filters in
Wikimedia projects that have user-defined variable do violate lexical scope.
Bug: T260903
Bug: T238709
Bug: T237610
Bug: T234690
Bug: T231536
Change-Id: Ic6d030503e554933f8d220c6f87b680505918ae2
