A filter using a protected variable can be loaded via filter id
using testing tools even though the user might not have the right
to view protected variables. This can potentially leak PII and as
such, testing tools should check for the right before allowing
protected filters to be seen.
- Unload a filter asap if it uses protected variables and the
requestor doesn't have viewing rights. This:
+ disallows loading of existing protected filters on page load
+ disallows testing against rules that use protected variables
+ disallows subsequent requests for protected filters (via API)
There is a known bug (see T369620) where no user feedback is
provided if an API request for a filter returns no result (typically
when no filter matches the requested id). This commit adds another
pathway to that bug (the filter exists but is protected and not
returned by the API) but does not update this UI/UX.
Bug: T364834
Change-Id: I6a572790edd743596d70c9c4a2ee52b4561e25f3
Search is restricted to users with the right to view private variables
but not necessarily the right to view protected variables. Users who
don't have the right to view protected variables shouldn't be able to
search against protected variables, as this might leak the PII.
- Filter out filters using protected variables in search results
if the user doesn't have the right to view protected variables
Bug: T367390
Change-Id: I7412112c9cc676f29d706b116b779bc17183a952
Alternative approach to fixing the regression proposed by
Daimona in I78d3a2cd7bada962d7ef9b0f2c39d898bf8987ce.
Bug: T368203
Change-Id: I637367c3b3850f7988d890379fef7f4753159953
- Rename `$hidden` to `$privacyLevel` in Flags::__construct for
consistency with other places.
- Rename `shouldProtectFilter` and simplify its return value to always
be an array, since that's how it's currently used. Rename a variable
that is assigned the return value of this method.
- Add a missing message key to a list of dynamic message keys.
- Rename a property from 'hidden' to 'privacy' in FilterStoreTest for
consistency. Add a test for removing the protected flag.
- Update old comment referencing `filterHidden`; the method was removed
in I40b8c8452d9df.
- Use ISQLPlatform::bitAnd() instead of manual SQL in
AbuseFilterHistoryPager.
- Update mysterious reference to "formatRow" in SpecialAbuseLog.
- Update other references to the very same method in two other places,
this time credited as "SpecialAbuseLog".
- Add type hints to a few methods; this not only helps with type safety,
but it also allows PHPUnit to automatically use the proper type in
mocks.
Change-Id: Ib0167d993b761271c1e5311808435a616b6576fe
`canSeeLogDetails` should also be checked when a filter is protected, as
it is the base right for being able to see abuselog entries. With this
in mind, check that immediately at the beginning of the method, instead
of repeating calls. Also merge the conditionals, and return early when a
permission check fails.
Move a test up so that it comes immediately after its data provider, and
add test cases for a few combinations of rights.
Change-Id: Ic3cf58f43803bef8bf2d65566434baff145b3fd5
Why:
* The wgCheckUserLogAdditionalRights global is shortly being
removed as no-longer necessary after T324907 removed the need to
generate the action text on insert time. As such, the action
text can be generated on view and therefore respect the rights
that the viewing user has.
* This means that the config can be removed, as users with the
'abusefilter-view' right will be able to see the action text
associated with the entry because the log formatter will check
for the right on view.
What:
* Remove the modification of wgCheckUserLogAdditionalRights in
AbuseLogger::insertLocalLogEntries.
Bug: T346022
Change-Id: I7504e4729fde5e51f6a795fc3e60d735152b2eea
Why:
- The previous attempt to integrate AbuseFilter with ConfirmEdit set
a flag on the request object
(I110a5f5321649dcf85993a0c209ab70b9886057c) didn't work in WMF
production because in WMF, we load ConfirmEdit first, followed by
AbuseFilter. Therefore any flag set in an AbuseFilter hook is ignored
by ConfirmEdit
What:
- Remove implementation of ConfirmEditTriggersCaptchaHook, as this does
not work when AbuseFilter is loaded after ConfirmEdit.
- Repurpose onConfirmEditTriggersCaptcha to handle non-edit actions only
- Implement the EditFilterMergedContent hook and call SimpleCaptcha's
public confirmEditMerged method if CaptchaConsequence has specified
that a CAPTCHA should be displayed, and if the CAPTCHA has not already
been solved
Soft-Depends-On: Idc47bdae8007da938f31e1c0f33e9be4813f41d7
Bug: T20110
Change-Id: I7dd3a7c41606dcf5123518c2d3d0f4355f5edfd3
MediaWiki has never used this path for running the maintenance
scripts, only the class name provided in the other parameter.
Providing the parameter is no longer needed in MediaWiki 1.43.
Bug: T367918
Change-Id: Ie098fa124039cc1122135cae72c74579d43dc04f
- Allow testing "move_redir" page moves.
- Allow testing "create2" and "byemail" account creations.
- Add remarks in the code that "autocreate" account creations
cannot be tested since they are not in the recent changes.
Change-Id: Idd38327df1477e1cba4396003a6c0f23cb75d276
In 1904cf8, "($1)" were appended to the messages, but
the argument was not substituted in the var dump table,
showing literal ($1). Substitute <code>var_name</code>
to keep previous experience.
However, many translations have not been updated yet.
If the variable name was indicated by the message
argument, it would often be missing. Therefore, make
sure the placeholder is always present.
Bug: T360909
Change-Id: I1e4a97210c891c375b0f14c0891c2d25a0a389d1
Some features restrict access when filters are private. These features
should treat protected filters similarly.
If the user doesn't have view rights for protected filters:
- Disallow viewing of logs generated by protected filters
- Disallow querying of matches against protected filters
Bug: T363906
Change-Id: Id84bd4ca7c8e0419fccc3ad83afff35067c9bf70
Changes to the use statements done automatically via script
Addition of missing use statement done manually
Change-Id: I48fcc02c61d423c9c5111ae545634fdc5c5cc710
In some cases, evaluation short-circuits when getting a list of
used variables resulting in an incomplete array of variables. This
subsequently causes issues when using those arrays for validation
checks (eg. if protected variables are used).
- Force full evaluation by setting `mAllowShort` to false
Bug: T364485
Change-Id: Idf2112d9ebf63846cde3ce9b8a8ade0ed909505d
First patch in a series of dropping the old columns.
Wikis now need to run the maintenance script (e.g., via
update.php) prior to serving this commit.
Wikimedia wikis are already on SCHEMA_COMPAT_NEW stage.
Bug: T188180
Change-Id: I86ec2b816eed17b62bf02bfd085570f132011b3e
Any filter using protected variables must be marked as protected via
the a checkbox on the filter edit form. This checkbox should not be
visible to users without the right to use protected variables.
Bug: T364485
Change-Id: If2c4b8f50f447e951d820798f181839d10501aa3
The UI/UX for acknowledging a filter will be protected/is protected
could be clearer. The checkbox implemented currently doesn't make
it clear that the acknowledgement is mandatory and filters that are
already protected allow for the checkbox to be unchecked even though
that doesn't reflect that the filter cannot be unprotected.
- Update copy for the protected filter acknowledgement to make it clear
that it's a mandatory acknowledgement, not an optional one
- Update copy for the error that shows when a filter that should be
protected doesn't have the acknowledgement checked
- When a filter is already protected, disable the acknowledgement
checkbox to indicate this is not mutable
Bug: T364485
Change-Id: I667fcca4511dff1ac3ca69930c5b5e5eb5001787
When a user without the right to see protected filters visits
Special:AbuseFilter/history/<ID>, show the permission error
message for protected filters.
Before this commit, the error message for hidden filters is
used instead, even if the filter is not hidden.
Bug: T364465
Change-Id: If2573fe256a7e29e8184feaf2f0622659706fd56
- Add a basic checkbox on the filter edit page that must be checked if a
filter uses a protected variable to ensure that the user is aware that
their filter will also become protected
Bug: T364485
Change-Id: I7c7652f7d1a81223229b839ff7eee5da4af74c8a
Some exposed variables (eg. `user_ip`) used in filters are sensitive
and need to only be available to restricted groups of users.
Back-end changes:
- Add `AbuseFilterProtectedVariables` which defines what variables are
protected by the new right `abusefilter-access-protected-vars`
- Add the concept of a `protected` variable, the use of which will
denote the entire filter as protected via a flag on `af_hidden`
New UX features:
- Display changes to the protected status of filters on history and diff
pages
- Check for protected variables and the right to see them in filter
validation and don't allow a filter to be saved if it uses a variable
that the user doesn't have access to
- Check for the right to view protected variables before allowing access
and edits to existing filters that use them
Bug: T364465
Bug: T363906
Change-Id: I828bbb4015e87040f69a8e10c7888273c4f24dd3
Protected variables will cause the filter using them to become
protected as well. `af_hidden` can be used to track this flag,
as it is a TINYINT and can be converted into a bitmask with no
schema changes.
This is not a backwards-compatible change, as now all checks must
check the `hidden` flag specifically or otherwise will be cast to
true if any flag is set.
To support this change:
- "hidden" is considered a flag set in the `af_hidden`. This is a
change in concept with no need for updates to the column values,
as there is currently only one flag in the bitmask.
- `Flag`s store the bitmask as well as the state of single flags
and can return either.
- Any checks against the `af_hidden` value no longer check a
boolean value and instead now check the `hidden` flag value.
Bug: T363906
Change-Id: I358205cb1119cf1e4004892c37e36e0c0a864f37
After temporary accounts are enabled, filters that rely on an ip
in the `user_name` will fail (eg. `ip_in_range` and `ip_in_ranges`).
To keep these filters working:
- Expose the IP through another variable, `user_unnamed_ip`, that can be
used instead of `user_name`.
- The variable is scoped to only reveal the IPs of temporary accounts
and un-logged in users.
- Wikis that don't have temporary accounts enabled will be able to see
this variable but it won't provide information that `user_name`
wasn't already providing
- Introduce the concept of transforming variable values before writing
to the blob store and after retrieval, as IPs need to be deleted from
the logs eventually and can't be stored as-is in the amend-only blob
store
Bug: T357772
Change-Id: I8c11e06ccb9e78b9a991e033fe43f5dded8f7bb2
This removes the last usages of the problematic open/closeElement
from this codebase.
One actual issue gets fixed: Some of the <th> floated around without
a <tr>. That's technically invalid. Luckily the browsers are flexible
and show it correctly. Visually nothing changes.
Similarly <th> should be wrapped in a <thead>. This wasn't done
before.
Change-Id: Ia45096670888173e49f9c25e72f429f0961b75ae
This issue exists ever since this code was added in 2009. Note how
this element is invisible anyway. The non-breaking space is never
seen. The purpose of this element is to act as a container for a
debug result that will be put into this container via JavaScript.
I confirmed this still works fine without the placeholder character
being there.
The problem here is that this HTML entity is double escaped because
of the element() function. That would need to be a rawElement() call
or we can just remove it.
Change-Id: Id560f392be4cc2106a7ac224309c8b605bec3f6c
Why:
- We want AbuseFilter to able to require a CAPTCHA if an action
matches conditions in an AbuseFilter
What:
- Implement the ConfirmEditTriggersCaptcha hook, and check to see if
the CaptchaConsequence set a global flag that indicates if we
should show a CAPTCHA
Depends-On: Ie87e3d850541c7dc44aaeb6b30489a32a0c8cc60
Bug: T20110
Change-Id: I110a5f5321649dcf85993a0c209ab70b9886057c
This solves two issues described in bug T360909:
* Usage of unsafe characters that have to be
manually reviewed in translations.
* Incorect display of some functions and
operators in RTL UI languages.
It also reduces the translators' need to copy
those operators and functions, which are always
identical to English.
Finally, this patch adds those consistently to all
the messages. Some messages didn't mention them
for an unspecified reason, and now they are mentioned
everywhere.
Bug: T360909
Change-Id: I3283c91b6b1d5fe9b48b1477cd454d9def3a7ded
A custom API error code and data similar to those used when an edit is
blocked by a normal AbuseFilter filter were accidentally added when
the feature was introduced. They should not be there, as the blocked
domains feature is not a normal AbuseFilter filter.
Hopefully nobody is relying on the format of this API response yet.
This commit changes the action=edit response for this case from:
{
"error": {
"code": "abusefilter-disallowed",
"info": "The text you wanted to publish was blocked by our filter. The following domain is blocked from being added: example.edu",
"abusefilter": {
"id": "blockeddomain",
"description": "blockeddomain",
"actions": "disallow"
}
}
}
to:
{
"error": {
"code": "abusefilter-blocked-domains-attempted",
"info": "The text you wanted to publish was blocked by our filter. The following domain is blocked from being added: example.edu"
}
}
Change-Id: I61ccc8f44b63e5cd0f11b1fe9a00ff60104a6249
Instead of having separate methods for each variable,
have one method which can work not only with "_links",
but with any array of strings.
Change-Id: I05f1b1cbd15f283b314c72259f183f7788e4e214
It is a common pattern to avoid SELECT * and use the fields used by
the application to avoid loading to much data into memory and maybe use
performance benefits when fields are covered by index.
Change-Id: I08a399f1b6a66442317b151be5386c9d2485f1fb
Constructing a Status like this does not make sense (and I want
to deprecate it in I0675e557bb93a1c990fa923c50b9f6ee8a9836c8),
because the parameters are ignored by most Status methods:
$error = Message::newFromSpecifier( 'abusefilter-blocked-domains-attempted' );
$status = Status::newFatal( $error, 'blockeddomain', 'blockeddomain' );
But it worked here, because FilteredActionsHandler::getApiStatus()
used a deprecated method that allowed inspecting them.
I feel like this code in BlockedDomainFilter has been added to make
the tests pass without thinking about what it actually does, which is
to output a bunch of useless incorrect data in API errors.
I'm not sure if we can remove that now without breaking API
compatibility, so add the useless data in FilteredActionsHandler
instead, closer to where it's output.
Change-Id: Ic12241bd3029bc1b0e7a0023689a2be35ccd30a8
From Status class documentation:
> The recommended pattern for Status objects is to return a Status object
> unconditionally, i.e. both on success and on failure -- so that the
> developer of the calling code is reminded that the function can fail, and
> so that a lack of error-handling will be explicit.
Change-Id: Ie6a55e297a35374fbdef880dd40e65f5cd00b6bf
The static method has already been migrated.
Also rewrite the test cases to avoid non-static provider (T337144).
Change-Id: Ibf98539f442e1ba8a9e9eb510784d40778123f17
MediaWiki core change Icb8822def9ce56f42ff52a8e469bb08d61d576c6
improved the type hints for OutputPage::addWikiMsg(), resulting in
two new errors:
* AbuseFilterViewEdit.php: False positive, update suppression
to include new error code.
* SpecialAbuseLog.php: Genuine bug, the return value of
Status::getErrors() can't be used directly as a message key.
I have another change pending that introduces a nicer way
to do this: Ibc4ce11594cf36ce7b2495d2636ee080d3443b04,
but in the meantime, make do with the available getters.
Change-Id: Iee0e87496e27a5261adccb977361b3ccf4c9ee2c
InsertQueryBuilder does not ignore insert of no rows,
adding some conditions to avoid calling the query builder
Change-Id: I1752b90cc3a7ec3a7f9ee32a1873bf8c82b6e02e
Introduced in 2019 with 4c8dac4dc6. Redundant since 2020 with
commit c6c62e2c8f in MediaWiki core.
Bug: T139216
Change-Id: I51e9fc3899cf5505917d7899a395350dd86f5c0b
Make the order of the messages that describe
operators and functions in the en.json file
identical to their order in
KeywordManager::BUILDER_VALUES, which is also
their order in the actual UI of the filter editor.
This only reorders the mesages in the en.json file.
It's not supposed to change anything in
the end users' experience, but it will change
the order in which translators on translatewiki.net
see them.
This is a cleanup step towards removing
the explicit operators from the messages,
as suggested in T360909, and this reordering
is hopefully useful even without that change,
for general consistency.
Comments about particular messages:
* abusefilter-edit-builder-vars-timestamp-expanded
is moved to the very end because, despite its key,
it's not actually used in the filter builder.
* old-text, old-html, and minor-edit are moved towards
the end because they are outdated. They are listed
separately from BUILDER_VALUES and they are not used
in the filter builder UI, but they are used in the logs
of previous actions. This patch adds a code comment
for the benefit of developers who touch that code
in the future.
Bug: T360909
Change-Id: I86ecdca5a6173b9068d5e968e69c57c74a379888
And more db clean ups:
- Use QueryBuilders
- Stop relying on actor migration to simplify query building
- Using expression builder in one case.
- Change the default actor migration stage to read new and write both.
Bug: T354194
Depends-On: I7c116cab0c748707d9a9fd17feeffe26e7d188ec
Depends-On: I74002911749335f4323a03fb430d02f936771b7e
Change-Id: Id84d1db7a2991f3cccc2f4f1502ba77643ddef24
The following sniffs are failing and were disabled:
* MediaWiki.Commenting.FunctionComment.MissingDocumentationPublic
Change-Id: I6075c76d53a899aac56af027f9a956a6b9e6a667
Why:
* The AbuseLogger::insertLocalLogEntries calls the CheckUser
extension to insert the abuse filter log entries, so that
checkusers can see these log entries in the results.
* However, these logs can be inserted on GET requests (such as
when a filter matched a autocreation of an account). This means
that the TransactionProfiler warns about writes.
* Updating the writes to occur on POSTSEND and also silencing the
expectations about not performing write queries should prevent
the logstash spam being caused by the TransactionProfiler related
to this code.
What:
* Make the call to Hooks::updateCheckUserData occur on POSTSEND
instead of PRESEND.
* Silence the TransactionProfiler for 'EXPECTATION_REPLICAS_ONLY'
only for the call to Hooks::updateCheckUserData.
Bug: T359648
Change-Id: I08e1674ff4dca386c046374c77dd31b4b29bb41e
Why:
* An AbuseFilter variable is needed that allows filters to determine
what type the current user is. That is, whether the user is an
IP address, temporary account, named user or external user.
* Currently filters implement this by inspecting the value in
the 'user_name' variable, but this is likely to break when
temporary accounts are enabled as IPs would be hidden.
* Giving a dedicated variable that indicates the type of the user
allows filters to work out this information without having to
know the specific username of the user before performing the
check.
What:
* Add the 'user_type' variable which is lazily computed. It can have
the value 'named', 'temp', 'ip' or 'external' depending on the
type of the user. If the user does not match any of these, then
the value is 'unknown'.
* Replace call to deprecated User::newFromIdentity with a use of the
UserFactory service that is dependency injected.
* Add and update tests to ensure consistent test coverage.
Bug: T357615
Change-Id: Ifffa891879e7e49d2430a0330116b34c5a03049d
This patch solves a pending TODO which is to remove the ::factory()
method from the AutoPromoteGroupsHandler class. If the cache instance
is injected, we'll use it otherwise we'll default to a HashBagOStuff.
Bug: T358346
Change-Id: I2bc414da8733431d1d11025e954282fc7c73aa80
Why:
* AbuseFilter can send AbuseFilter logs to CheckUser if they are
not being sent to Special:RecentChanges.
* However, if this action is indirectly causing the creation of
an account (such as through temporary account auto-creation),
the log entry is sent to CheckUser before the temporary account
actually exists in the 'user' table.
* This causes a CannotCreateActorException, as the performer does
not exist on the wiki just yet and therefore cannot have an
actor ID until the temporary account is created.
* This exception can happen if the AbuseFilter filter only creates
a log entry and does not prevent the edit, so would not be
necessarily fixed by T334623.
* Sending the logs to CheckUser on PRESEND avoids this, as the
user will exist by the time that PRESEND is run but still allows
any failures to cause an exception which can be seen by the user.
What:
* Wrap the call to Hooks::updateCheckUserData in AbuseLogger
::insertLocalLogEntries in a DeferredUpdate which is set to run
on PRESEND.
Bug: T358632
Change-Id: Ia615fce3e26b88d5457ecc01231044b326b79973
