Commit graph

448 commits

Author SHA1 Message Date
Daimona Eaytoy f49d4e5caa Emit debug logs when filtering without title
We have two situations where we try to execute filters without a title.
However, the code doesn't handle it correctly: some points expect $title
to actually be a Title object, and we also pass it around using a hook
which explicitly says it always pass a Title. This patch adds two debug
points to help understand why we end up with null titles, so that we can
fix it upstream.

Bug: T144265
Change-Id: I35bfc483a0c69a5cbd38eae8ba299189955fa1ec
2018-12-13 20:34:21 +00:00
daniel 688eccea47 Expose text from all slots to AbuseFilter
This is a first step towards MCR support in AbuseFilter. The textual
representation of all slots is concatenated. Since AbuseFilter uses
getTextForSearchIndex to determine the textual representation of
content, blind concatenation should not break any assumptions
made by AbsueFilter rules: this naive approach is no worse than
AbuseFilters handling of non-textual content in general, and should
work fine for textual content.

Bug: T209291
Change-Id: Ic141085cad2e11bfe106fe83dafcb35ac31206ba
2018-12-05 09:24:08 -08:00
jenkins-bot 1dd8f41d0d Merge "Use the updated TitleMove hook to filter move actions" 2018-12-04 19:32:04 +00:00
Daimona Eaytoy 206bdc1f6a Use the updated TitleMove hook to filter move actions
For several reasons:
*We're not really checking permissions (and the hook previously used is
meant to be used in such case)
*We'll show a cleaner error message (i.e. without the "You do not have
permission..." part)
*Filtering will happen closer to the actual move

Bug: T208907
Depends-On: I4733724075b7514e9db59e7be772d9409aa9da87
Change-Id: If88f736a446247f8b4b13c055c641d56f544d1ea
2018-12-04 18:58:04 +01:00
jenkins-bot 23a7aa69a5 Merge "Fix regex group counting for get_matches" 2018-12-04 13:58:06 +00:00
jenkins-bot bb289862ff Merge "Remove code for old global variables" 2018-12-04 06:27:32 +00:00
Huji Lee b523194032 SECURITY: Remove private information from the API results
Later, we will add a new POST request which will allow retrieving
the private details; it will have a mandatory "reason" parameter,
and will result in a log entry in the private details access log,
just like the web interface.

Bug: T210329
Change-Id: Iaca492371f48fecf543268c179a651841ed12c3f
Signed-off-by: sbassett <sbassett@wikimedia.org>
2018-12-03 23:11:32 +00:00
Daimona Eaytoy 7ca0941d1f Remove code for old global variables
Those two global config variables were removed more than 2 years ago, in
I790d39c2849922d7daf7479f298cd90cf30af129. Nothing else in the code
references them, so we can just remove the warning.

Change-Id: I427d06a80131447ea64064f45e84349f93e72cca
2018-12-02 16:24:09 +01:00
Daimona Eaytoy 6aff37fb52 Further clarify docs for emergency disable
This is a follow-up to Ic3bc6e36506973b19a9b1bcecbc1a5080faed2ec. I
believe it's important to specify how many recent actions we're looking
at, and I also think it's not nice to rely on a variable amount of
actions to determine whether a filter should be throttled. Also, require
a $group parameter in filterUsedKey (we always pass one, and there's no
reason not to).

Change-Id: I0384d3f1913ead593f605248950606c81c8f8542
2018-11-28 19:29:15 +01:00
Daimona Eaytoy 1f2b7474ed Clarify code and docs for automatic throttling
For the docs part: make it clear how things work there. For the code
part, these are mostly style changes: shorter variable names, no
unnecessary parameters, make the method private, use clearer variable
names.

Change-Id: Ic3bc6e36506973b19a9b1bcecbc1a5080faed2ec
2018-11-26 16:51:10 +01:00
Daimona Eaytoy 4480c9493a Remove wgParser and wgRequest
As part of the deprecation process of non-config globals.

Change-Id: Ia84ddc20adbfda72347cf256601050b055b87ecf
2018-11-19 13:40:58 +01:00
jenkins-bot 0d58f78030 Merge "Revert "Revert "Add typehinting for every object-only parameter""" 2018-11-18 16:27:27 +00:00
jenkins-bot 6541d7c5cc Merge "Check that the user block is sitewide when determining permissions" 2018-11-15 17:26:21 +00:00
Daimona Eaytoy 346063eec0 Check that the user block is sitewide when determining permissions
And bump MediaWiki version.

Bug: T208621
Change-Id: Icfcf09c5d7c7498711cb000c3bb16480270efb9c
2018-11-15 17:59:22 +01:00
Daimona Eaytoy badde6ba75 Revert "Revert "Add typehinting for every object-only parameter""
This reverts commit 1ed75b4ae0.
Fixed the one which caused errors, by making articleFromTitle
only use WikiPage, instead of silently mixing WikiPage and Article.

Note for reviewers: this patch is identical to the one which was
previously +2ed, which was mostly correct. To see the actual change,
diff AFComputedVariable with 1..current.

Change-Id: I6747eaed861af6c40a3b1610aebcc1174296e9ed
2018-11-15 10:09:16 +01:00
jenkins-bot 213c2aa011 Merge "Change throttle selector to restore old functionality, overall improvement" 2018-11-15 00:58:11 +00:00
Daimona Eaytoy d3a8491c3f Change throttle selector to restore old functionality, overall improvement
Long (sigh) explanation in T203587#4569698. Also, simplified the way
TagMultiselect are generated, this one and the one for change tags.
This new selector is back-compat both with the old textarea and the OOUI
checkboxMultiselect; actually, this one is //fully// compatible with the
old textarea.
Add validation for throttle parameters and unit tests for validation
(split from I976c95658cddb2585910b6f8a5f047aadc4e4d47).
Added a trim when retrieving throttle identifier to allow syntax like
'ip, user'.
Improved the message shown on history.
Re-added the maintenance script to clean DB.

As I wrote in the task, a review by two other people would be great, at
least for the maintenance script (it could potentially break the DB).

Bug: T203587
Bug: T203336
Bug: T203584
Bug: T203585
Depends-On: I3b2e763bd8835207dc5df1db43d3e1881e6961c3
Change-Id: I7831dbb0bab55807392ac1f7915d6cb0cb713593
2018-11-14 12:51:36 +01:00
Brad Jorsch f6349e7a32 Update tests that fail with comment/actor migration
* AbuseFilterConsequencesTest is somehow leaving blocks behind. Mark
  ipblocks as being used to avoid that.
* AFComputedVariable::getLastPageAuthors() uses indeterminate order for
  multiple revisions with the same timestamp. Fall back to rev_id
  ordering like MySQL accidentally did before.
* AbuseFilterTest tries to create revisions attributed to users that
  don't exist. Switch to interwiki usernames.

Change-Id: I30f7cdcc3875f3f7af116c1e41e88f62ab9e91d0
2018-11-09 17:03:36 -05:00
jenkins-bot 58018ac7cc Merge "Use log channel 'AbuseFilter' instead of 'AbuseFilter<Suffix>'" 2018-11-08 14:32:58 +00:00
Timo Tijhof e7c0d5f238 Use log channel 'AbuseFilter' instead of 'AbuseFilter<Suffix>'
The channels are a fairly low-level primitive. Having multiple
in production for the same extension I think makes the logs
difficult to navigate and easy to miss things.

For the purpose of grouping, we have normalized_message instead,
which works by using the Monolog template string capabilities,
this is enabled in WMF Logstash (and in Beta).

Change-Id: I10b1fd2d9bdfe518089c053d77fef568170ecb65
2018-11-07 20:21:10 +00:00
Daimona Eaytoy 6658a24554 Remove typehint to avoid fatal error
Temporarily remove the typehint, as it causes some fatals. This doesn't
solve the underlying problem, for which we should first investigate with
I35bfc483a0c69a5cbd38eae8ba299189955fa1ec.

Bug: T208144
Change-Id: I0fdda51010243690ff3806c16d4e203c9ccd8e0a
2018-11-07 11:23:50 +01:00
Daimona Eaytoy 16475c0266 Fix regex group counting for get_matches
Adding the * as character to match after parentheses, since it may be
used with backtrack verbs (e.g. (*FAIL), (*SKIP)). I guess this is a
very, very rare use case, but since the fix is easy, let's include it.
Also, added a ToDo since we should probably find a better way to count
capturing groups, although I cannot figure out any.

Change-Id: Idcb303b4740530af9d3f009414d35d68f59effd0
2018-11-01 11:52:33 +01:00
C. Scott Ananian b73786df5c Replace deprecated OutputPage::parse/parseInline()
The OutputPage::parse/parseInline() methods emit untidy output and
are often used with the wrong user interface/content language
selection.  Replace with new methods added in 1.33 which are
tidy and consistent.

Bug: T198214
Depends-On: Ica79c2acbc542ef37f971c0be2582ae771a23bd0
Change-Id: Iec8071f4e50f169356e4f68ccb746c55f1606ea6
2018-10-26 13:33:20 -04:00
jenkins-bot c8d85e27b8 Merge "Use proper cache key construction for throttle, rules, and autoblock keys" 2018-10-24 10:10:51 +00:00
Daimona Eaytoy 103dfa3b66 Remove info leak
Oversighted/deleted edits and log actions were entirely accessible to
non-oversighters via AbuseFilter/examine for RC, and via AbuseFilter/test.
Now, we take into account the revision/log visibility and user permissions to
determine what to show.
Other changes in this patch:
*Show the examine link if and only if the user can examine the given row
*If a revision is hidden but the user can see it, don't hide its elements in
 ChangesList (only leave them striked/greyed)
*Make APIs better understand revision visibility.
*Make a clear distinction between deleted and suppressed edits/log
entries.

Co-authored with rxy <git@rxy.jp>

Bug: T207085
Change-Id: Icfa48e366a7e5e3abd5d2155ecfddfc09b378088
2018-10-23 10:53:39 +00:00
Aaron Schulz 5071c6574a Use proper cache key construction for throttle, rules, and autoblock keys
Change-Id: I72ab39048f955d4262fae81141cf97243e5cd184
2018-10-21 00:42:08 -07:00
jenkins-bot 97602b8a68 Merge "Remove useless array_filter" 2018-10-19 10:14:56 +00:00
jenkins-bot 7e151f5edc Merge "Unbreak short circuit for arrays" 2018-10-18 04:04:31 +00:00
jenkins-bot eb1303c8cd Merge "Revert "Add typehinting for every object-only parameter"" 2018-10-17 03:09:55 +00:00
Jforrester 1ed75b4ae0 Revert "Add typehinting for every object-only parameter"
This reverts commit 69d7669069.

Reason for revert: Causing UBN train blocker

Bug: T207220
Change-Id: I3445d9b3065149e2beb149e10fbbf5502b480f57
2018-10-17 01:22:23 +00:00
jenkins-bot 7a8a2fa3e1 Merge "Add typehinting for every object-only parameter" 2018-10-16 02:48:39 +00:00
Daimona Eaytoy 69d7669069 Add typehinting for every object-only parameter
This patch covers every object-only parameter, adding a typehint for it
to avoid errors.

Change-Id: Iebf700621b9dbff78c3bd8f3c136ed15ef4b8d4b
2018-10-15 09:56:09 +02:00
jenkins-bot fea08f45b8 Merge "Avoid useless error message for regexfailure exception" 2018-10-14 13:47:37 +00:00
Matěj Suchánek a3cc3dff75 Remove some $wgUser usage
Bug: T159299
Change-Id: I1613e2bb0c551cbadc0c57351fc40bd9e21abf52
Depends-On: I35adef06dfc799cddeddfa6c5eed53b8b1bb7282
Depends-On: Id19a6d883ac6e0cc9c26c923486bca0e414ecaa7
2018-10-14 11:24:52 +02:00
se4598 9d12e1b353 Allow selecting custom disallow message
You can now select a custom message to be displayed for disallowing a edit
the same way as for warn mode. This can be the same or a totally different
message.

This also solves the usecase, when a edit filter is set to warn AND disallow,
to be able to show the user a custom message, but the generic is shown
on the second try (disallow). Now it can be only set to disallow.

Bug: T27086
Change-Id: Ic1de03a6944c43a346fa317ee0a217551f0d284a
2018-10-11 10:35:01 +02:00
Daimona Eaytoy eafb4f56c7 Avoid useless error message for regexfailure exception
Users writing filters probably don't care about preg_match or whatever
happens in PHP. Also, it's not that useful to see "unspecified error".

Change-Id: I014742fa6f678126f55ac5ccff38e44b2c5a7d15
2018-10-08 19:19:01 +02:00
Daimona Eaytoy e60dacbbea Fix code comments
Fixed some comments adding explanations, fixing syntax, and parameter types
for docblocks. Also fixed some whitespace mess, and added a missing use
statement.

Change-Id: I3547c90bdaa2cab5443e8bf0c63b217fe6ba663f
2018-10-03 16:45:03 +02:00
Daimona Eaytoy d9d5af3890 Unbreak short circuit for arrays
This problem have been making filters potentially fail silently since
2009. Also add tests for arrays to make sure that no problems arise
when short circuit is used.

Bug: T204841
Change-Id: Ie4e2e06498c1202ba73afcc5d164a72427abbca5
2018-10-03 16:44:10 +02:00
jenkins-bot 3efc69960c Merge "Fix database schema for PostgreSQL" 2018-10-01 15:43:29 +00:00
Umherirrender 45e6fa932d Fix caller name in AbuseFilterHooks::fetchAllTags
Seeing {closure} in the logs as caller is not helpful

Change-Id: Id3bf5c7fd810d48dc04a167692b336b3ccba2eb4
2018-09-30 14:08:06 +02:00
Umherirrender 4fdd1bbf20 Fix caller name in AFComputedVariable::getLastPageAuthors
Seeing {closure} in the logs as caller is not helpful

Change-Id: I23ee52609510f8efefba8c1ee466d491f468f494
2018-09-30 14:06:04 +02:00
Matěj Suchánek db50bef21e Fix database schema for PostgreSQL
Bug: T62639
Change-Id: I5ddb781a2971677410f4cb96e5fc5964e53c862a
2018-09-29 12:12:52 +02:00
Daimona Eaytoy 50d5137880 Remove useless array_filter
Not only it's useless, but also removes the namespace if it's 0, thus
causing the query to only add a WHERE on rc_title, but the index is on
rc_title AND rc_namespace, so the query has bad performance.

Bug: T204228
Change-Id: I33694cfeddbc4eaf39e3e840b207dba433188834
2018-09-24 14:34:53 +02:00
jenkins-bot eae59db542 Merge "Fix the block options on ViewEdit" 2018-09-20 11:25:00 +00:00
Daimona Eaytoy 9144dbf4a1 Remove unused parameter
Nothing uses it, plus it wouldn't work anyway: AbuseFilterParser
constructor only uses $vars if it's instanceof
AbuseFilterVariableHolder.

Change-Id: Idbf53f6058148e9f0e73beb949e1c028a81663ce
2018-09-19 19:58:30 +02:00
jenkins-bot a813140e44 Merge "Unbreak /examine for old log entries" 2018-09-16 12:00:34 +00:00
Daimona Eaytoy fc867a1c5c Allow testing account autocreation
Bug: T204231
Change-Id: If566cfdeb4cdbb78833077da09aeef33754f88d3
2018-09-14 13:09:07 +02:00
Daimona Eaytoy 31729b044e Unbreak /examine for old log entries
For the moment, this is a simple workaround to get them back working.
Ideally we'd also need a maintenance script to update var dumps as I
wrote in the task, but it needs more thinking (see Phab).

Bug: T204236
Change-Id: Ia20a2eb495557f46f789467a96e654ec6cd3f355
2018-09-13 18:42:47 +02:00
Matěj Suchánek 6eb5d9766b Use correct variable in AbuseFilter::addLogEntries
The data was inserted to the foreign database, so the id needs
to be fetch from that one.

Change-Id: I8eef8d74fc924829447e31f4445154b01b92aa7a
2018-09-13 11:57:55 +02:00
jenkins-bot a0a4755c59 Merge "Remove unused method from parser" 2018-09-09 12:32:56 +00:00