Commit graph

284 commits

Author SHA1 Message Date
Reedy 7316825155 Replace MimeMagic::singleton() call
Change-Id: Ib01a23b909cb3e19c6d5ea9a8cf03d06ff8dab99
2017-11-27 01:46:00 +00:00
Matěj Suchánek a7ac1f2b3f Normalize IPv6 addresses in Special:AbuseFilter/examine
This is re-application of I0b4a5468c.

Change-Id: Ifad191f460717e70203f1740103f3af77b21b03d
2017-11-17 16:24:42 +01:00
jenkins-bot 94c7a1ea82 Merge "Add get_matches function" 2017-11-13 16:52:13 +00:00
Daimona Eaytoy 4e20c933f4 Add get_matches function
Added the get_matches function to store a regex match.

Bug: T179957
Change-Id: I19366ebcaa4d0f007dd675a61c91457dde57f604
2017-11-13 17:32:45 +01:00
Matěj Suchánek f29c053378 Remove unused parameter from the message
This parameter was never documented. It was just
a leftover from migrating to the current message API
in Id69a9d603. Note that the same message pair further up
in this file does not use it.

Change-Id: I38caa1611d78b6cb182861c8f5d731b27379f62a
2017-11-11 13:35:21 +01:00
Matěj Suchánek 957598261c Format filter IDs as numbers
Follows up I2bd833c35.

Change-Id: I3f2241fe791e9918750ef06805f9c4e5d52ee3b4
2017-11-11 12:38:12 +01:00
jenkins-bot f7fe5834dc Merge "AbuseFilter block range should not exceed $wgBlockCIDRLimit" 2017-11-05 05:36:10 +00:00
jenkins-bot b7b04cb830 Merge "Avoid Call Stack warning when there is an error in the regex pattern" 2017-11-03 20:01:30 +00:00
Huji Lee 7b7be07957 AbuseFilter block range should not exceed $wgBlockCIDRLimit
This patch introduces a config variable for the range block sizes.
It changes the default IPv6 block size from /16 to /19 using the
same reasoning as  Ia25e156fd8234519c4d74f1d41d93f94a313ce14

Using a config var (as opposed to hardcoded range size) allows
future changes proposed in T179454 to make the range size vary
for different IPs, based on the actual subnet they belong to.

Bug: T179455
Bug: T179456
Change-Id: I8dfa17f553a7af524f0a11c0fd51c48773e27be5
2017-11-02 12:08:49 +00:00
Huji Lee 0f7f4ad917 Add missing documentation for protected functions
Bug: T178007
Change-Id: Ia1ae78b30b889b7a8965354ae0a404bf9a520917
2017-11-02 03:20:34 +00:00
Huji Lee ace1822575 Filter ID should always go through formatNum()
Change-Id: I2bd833c35128b3c39c7882321747837184095bef
2017-10-31 22:11:55 -04:00
jenkins-bot 689b7abaff Merge "Remove back-compat code paths" 2017-10-31 16:12:40 +00:00
jenkins-bot f5536f964e Merge "Update for deprecation of selectFields() methods" 2017-10-31 16:08:46 +00:00
Brad Jorsch 6071e7a43f Remove back-compat code paths
Per T178092, AbuseFilter now maintains compatibility with older versions
of MediaWiki using release branches. Thus, various back-compat code
paths may be removed from the master branch.

Change-Id: Ia1b5eade30d7486e3b1b386b15a7db4e5c8cfead
2017-10-31 09:37:54 -04:00
Brad Jorsch 7ed9160ea9 Update for deprecation of selectFields() methods
Various selectFields() methods were deprecated in MediaWiki core change
Idcfd1556, replaced with getQueryInfo() methods.

Change-Id: If75d2e76c2f166bc40a544dd502da43171ce1e7b
Depends-On: Idcfd15568489d9f03a7ba4460e96610d33bc4089
2017-10-31 09:27:27 -04:00
Max Semenik 32598c18bb Drop FastStringSearch support
This PHP extension doesn't have any speed benefits over our minimum
required PHP and so isn't used in WMF production anymore.

Change-Id: I4883643908f765eee5db6b3ca88eed179264e93f
2017-10-26 00:51:07 +00:00
David Barratt 5335b6c811 Use Equivset library intead of AntiSpoof
Use the new equivset library instead of AntiSpoof.

Bug: T175413
Change-Id: I439387deeba99543e194c210953ac73ff98bc5b7
Depends-On: I977d3498b2084a426e2ab4d85c000d1b9dcfe824
2017-10-21 21:55:18 -07:00
jenkins-bot 83d67e4a12 Merge "Restore accidentally deleted code" 2017-10-22 04:09:59 +00:00
jenkins-bot 82ce683000 Merge "Pass LinkRenderer instance to AbuseFilter::addNavigationLinks" 2017-10-22 04:09:59 +00:00
jenkins-bot 610cfa8662 Merge "Normalize IPv6 addresses in Special:AbuseFilter/test" 2017-10-22 04:09:14 +00:00
Matěj Suchánek 5839b7c412 Restore accidentally deleted code
I have noticed I accidentally dropped this code during
Iec237b288 (apologize!).

Change-Id: Ifbd53575ddf8dd1014de19afea395d0eb537c6ee
2017-10-21 08:09:36 +02:00
Simeon Dahl 4c0636bc0a Cleanup, added spaces
Added spaces in some classes so it follow the same style as the others.
Edit is pure cleanup.

Change-Id: If5d5e6e4e99eed83aa69dfb4a224fbcc7c077d43
2017-10-19 09:40:22 +02:00
jenkins-bot 22e68a61ca Merge "Add slow filters debug data to the logs." 2017-10-12 23:11:50 +00:00
Dayllan Maza 3e1c5b9099 Add slow filters debug data to the logs.
When $wgAbuseFilterRuntimeProfile is true, all filters taking
longer than $wgAbuseFilterRuntimeLimit will be logged for
later analysis

Bug: T174205
Change-Id: Id81833afa8421476a6cee47eb3393acdb3a38d65
2017-10-12 17:41:49 -04:00
melos 9eb99b1dc9 Avoid Call Stack warning when there is an error in the regex pattern
When you are in Specia:AbuseFilter/test and you test a filter
against RC if the filter has a regex with a rlike condition
you receive a Call Stack warning for a wrong regex.

Bug: T177744

Change-Id: I2bc62b5709d2863eb355a249610b3e80fab55448
2017-10-09 10:48:20 +02:00
Umherirrender a2ebd0c70a Improve some parameter docs
Change-Id: Ibac10a20243a4eedd826485d56eddd5234da6fec
2017-10-07 00:54:58 +02:00
Dayllan Maza 2bc8873c30 Add ccnorm_contains_any function
Normalize and search a string for multiple substrings

Bug: T65242
Change-Id: I4034c0054a6849babbf2d96ea13dc97d3660d5b4
2017-10-06 11:32:45 -04:00
Matěj Suchánek 08899056d2 Normalize IPv6 addresses in Special:AbuseFilter/test
Forcing user namespace will handle IPv6 correctly as well
as possible "User:" prefix supported by the former code.

Bug: T176045
Change-Id: I0b4a5468ca44799cade0b0774d749e05d4ff5865
2017-09-29 21:02:52 +02:00
libraryupgrader 781f6c2554 build: Updating mediawiki/mediawiki-codesniffer to 13.0.0
The following sniffs are failing and were disabled:
* MediaWiki.VariableAnalysis.ForbiddenGlobalVariables.ForbiddenGlobal$wgTitle

Change-Id: I7163cd8f97a7d2fe5b4410245a72eb416302f4f6
2017-09-24 05:19:01 +00:00
jenkins-bot f00de10b24 Merge "Fix confusing warning message on throtthled filters" 2017-09-14 21:10:47 +00:00
Dayllan Maza 539884f428 Fix confusing warning message on throtthled filters
A confusing warning message was displayed when filters
have af_throttled = true. That message was replaced with a
new one reflecting the behavior that is actually ocurring and
how to solve it

Bug: T54525
Change-Id: I5c6e434249d5c9649eb2d7c5b16b9ecb1f530c8a
2017-09-10 01:26:24 -03:00
Antoine Musso 3efdc518da Select proper db fields in AbuseFilterExaminePager
It queried the 'recentchanges' table with the fields from the 'revision'
table.

Bug: T175338
Change-Id: Ib232e49e3f59285c7f66cdb48c51fe4c97b53e15
2017-09-08 14:41:19 +02:00
Chad Horohoe 3f4ba191f6 Avoid CommentStore errors by selecting correct set of columns
Change-Id: I7d83a45fbf86dc0144d50e8b6258b97f1b0c7188
2017-09-07 17:21:50 -07:00
jenkins-bot f238148fec Merge "Use CommentStore to access core comment fields when available" 2017-09-05 19:02:26 +00:00
Matěj Suchánek d59e62a7b2 Get LinkRenderer instance from special page
Just use SpecialPage::getLinkRenderer().

Change-Id: I7c6e839ed8005e666e7c3c1c08dada8aaadbd28f
2017-09-02 19:04:20 +02:00
Brad Jorsch cfa98448c3 Use CommentStore to access core comment fields when available
See core change Ic3a434c0.

Bug: T166732
Change-Id: I130510381ef97d8a2d29686843e2710cb5f72195
2017-08-30 13:27:46 -04:00
Max Semenik 0686f99653 DB_SLAVE -> DB_REPLICA
Change-Id: I371e092b3cd7f0af3770bf4e64b01a630e23ff92
2017-08-29 19:51:39 -07:00
jenkins-bot 13b60c5e35 Merge "Support delete action in retrospective testing" 2017-08-29 18:02:55 +00:00
jenkins-bot d95b5dcb76 Merge "Improve queries for testing on recent changes" 2017-08-29 18:02:54 +00:00
Matěj Suchánek 2c9af794fd Support delete action in retrospective testing
Bug: T170576
Change-Id: Ied9b6446ea5edccd902d6a88b2601b545d4a6910
2017-08-29 17:48:58 +00:00
jenkins-bot 8dc5c6427c Merge "Add runtime metrics to statsd" 2017-08-24 21:08:14 +00:00
Dayllan Maza c07294cc9c Add runtime metrics to statsd
Metrics per edit:
    - Execution time of all filters
    - Number of filters executed
    - Number of conditions executed

Due to the current structure of abuse filter there was not
a clean way to include filter actions and abuselog creation
as part of the runtime metrics.

Bug: T161059
Change-Id: I6208b620453863133c6623aa419775f63c7d3eb1
2017-08-24 15:58:52 -04:00
Matěj Suchánek 174be6dc9c Restore specific submit message on Special:AbuseLog
This line was dropped in Ifad07bac45eadc4ed377967aa4ab3e3b9e14aa8b.

Bug: T173995
Change-Id: I84982620abbda38c86159b3aa7ed408c44d447ad
2017-08-24 09:46:27 +02:00
Matěj Suchánek 1e5a5539b2 Remove unused message parameters
It should have been deleted in I02ba4ce31b6aca5b7324114093f8ece143abc295
but accidentally survived.

Change-Id: Icdbe8fb0154513d643905f7f8bd6391780cd44a7
2017-08-20 14:48:51 +02:00
Matěj Suchánek 748978fda5 Pass LinkRenderer instance to AbuseFilter::addNavigationLinks
Change-Id: I1de60955344d06e5b55f4d2cfdfa8f2d13df831a
2017-08-20 14:38:40 +02:00
Matěj Suchánek efaae31263 Improve queries for testing on recent changes
- Use rc_source with values that we know we support. In
  particular, this drops categorization changes.
- Filter on rc_log_type and rc_log_action (which itself
  may be shared across types).
- Use the same query on both Special:AbuseFilter/test
  and Special:AbuseFilter/examine.

Bug: T170574
Change-Id: I79b903b4424d3c15095a1e0491d35f6e005db0b8
2017-08-20 14:00:39 +02:00
Rendann 8ff80d0134 Convert Special:AbuseFilter/import to OOjs
Based on https://gerrit.wikimedia.org/r/#/c/370150/
Bug: T132284

Change-Id: Iab2a24926c6b547e34f516149ff94953fb0da634
2017-08-18 13:35:30 +00:00
jenkins-bot 3318bf1570 Merge "Remove manual implementation of AntiSpoof::equivString" 2017-08-14 23:11:45 +00:00
Dayllan Maza 22a7f6e912 Remove manual implementation of AntiSpoof::equivString
Cleaned ccnorm method to use AntiSpoof::normalizeString instead going
after AntiSpoof extension files and doing a manual implementation of it.
Also removed composer requirement for AntiSpoof extension.

Bug: T172766
Depends-On: I731733671b650b6bb2f480c41c4f6f2d2f5c62e8
Change-Id: Ib38ba0b06918e81e8af03032eef95e3942773bc1
2017-08-14 16:25:25 -04:00
jenkins-bot 391966dec4 Merge "Fix the order of conditions when choosing the correct method" 2017-08-14 20:04:42 +00:00
Rendann 51924f3a92 Convert Special:AbuseLog to OOjs
Bug: T132284
Change-Id: Ifad07bac45eadc4ed377967aa4ab3e3b9e14aa8b
2017-08-14 17:19:28 +02:00
matejsuchanek 3b29498f21 Migrate abusefilter/modify log
Bug: T32553
Change-Id: I7e33d1064329124755c77ffe6efbd5d572f43cb9
2017-08-12 10:00:42 +02:00
Matěj Suchánek 79c25e5583 Improve documentation of methods
Change-Id: I231d6a3e095324ae1509fdf028bcc352148a0dbe
2017-08-11 23:26:29 +00:00
Kunal Mehta 72427ec974 build: Updating mediawiki/mediawiki-codesniffer to 0.11.0
Change-Id: I26adebde9fcb2894804ab705402d05d9de745cb1
2017-08-10 22:57:33 -04:00
jenkins-bot 3e9efb90f7 Merge "Require MediaWiki 1.29" 2017-08-10 18:06:06 +00:00
NoRandom 389995916c Allow searching for visible-only/hidden-only/both on abuselog entries
Bug: T153065
Change-Id: I44ca86ff8564328ae932ccad9675298b686dc6ab
2017-08-10 11:46:57 +02:00
Matěj Suchánek 55c27a8f6b Require MediaWiki 1.29
After I544cdfa75c7472f2d98b2561bc6f6f9c2d2ad639 (dieWithError
and checkUserRightsAny), this is the oldest MediaWiki version
AbuseFilter can be run on.

AbortMove was removed from MediaWiki in 1.25, UploadVerifyFile
is only relevant for 1.27 and older.

(Replaces I1e962217c3b20d901a5742cf76339a3f488a6e97.)

Change-Id: Iec237b2887f72b115fdcef78d2d7a944ba82c784
2017-08-10 11:01:34 +02:00
jenkins-bot 7b67f903e0 Merge "Completely disable Special:AbuseFilter/tools for non-maintaners" 2017-08-09 17:12:35 +00:00
Umherirrender eaa1e9fb25 Improve some parameter docs
Change-Id: I03fa9b58c72bcd28985c5a3467b82d8b98f3a0fc
2017-08-08 15:05:06 +02:00
Matěj Suchánek 020517dbe2 Completely disable Special:AbuseFilter/tools for non-maintaners
As described in the task, anyone can view the page but do nothing
with it. It doesn't make sense, it should either work or be disabled.
I chose the latter.

Bug: T124096
Change-Id: I7271cd5a0d00d5fdba28ce3974c50ea5dfd0ad9b
2017-08-08 10:38:25 +02:00
Max Semenik b67cb42c09 Fix some deprecated function usage
Change-Id: I544cdfa75c7472f2d98b2561bc6f6f9c2d2ad639
2017-08-07 16:35:21 -07:00
Max Semenik de7389e19a Fix class and function name case
Change-Id: I55ed3b26ee457863372ec063b7c3ff27bc849b8b
2017-08-07 16:22:48 -07:00
Max Semenik 7e62f23242 Rm unused locals
Change-Id: Ieb77c089645b858290a15c5804de636c5a7b53cc
2017-08-07 16:19:45 -07:00
Max Semenik a6c87eca7a Get rid of deprecated manual profiling
Change-Id: Iefbfb4b9937ed265156fb1b644889fd9e2b29a2e
2017-08-07 16:14:31 -07:00
Max Semenik 2f250127b4 Normalize file layout
Aka move all code into includes/.

Change-Id: I21f7b80bb6df04abbed6bfccb94f92100dc8f071
2017-08-07 16:11:38 -07:00
Kunal Mehta 3a58875637 build: Updating mediawiki/mediawiki-codesniffer to 0.10.1
The following sniffs are failing and were disabled:
* MediaWiki.Commenting.FunctionComment.MissingDocumentationProtected
* MediaWiki.Commenting.FunctionComment.MissingDocumentationPublic
* MediaWiki.Commenting.FunctionComment.MissingParamComment
* MediaWiki.Commenting.FunctionComment.MissingParamName
* MediaWiki.Commenting.FunctionComment.MissingParamTag
* MediaWiki.Commenting.FunctionComment.MissingReturn
* MediaWiki.Commenting.FunctionComment.ParamNameNoMatch
* MediaWiki.Commenting.FunctionComment.WrongStyle

The following sniffs now pass and were enabled:
* MediaWiki.Commenting.FunctionComment

Change-Id: I0874c547ef2bc8a7c3fa4ca72738aa3320f2bdbe
2017-07-23 00:03:40 -07:00
Matěj Suchánek 014f0adda1 Fix the order of conditions when choosing the correct method
It seems that move actions have rc_this_oldid set. rc_log_type
should have higher priority.

Bug: T170586
Change-Id: I9bb4ea599f12ef01fad823ac3232330966b0d281
2017-07-13 17:31:19 +02:00
Umherirrender 1a58507870 build: Updating mediawiki/mediawiki-codesniffer to 0.10.0
Change-Id: I5f37c45d748d5f0da21aceaef32cc89367e312ff
2017-07-08 20:49:30 +02:00
Matěj Suchánek a1aa09d31c Cache global filter names
AbuseLog fetches the names in an additional query for each line but the
result is always same.

Change-Id: Ie12f909727ae82ab60781ccedfdf1b241c105388
2017-06-17 20:50:18 +02:00
Umherirrender a063e33ee8 Use short array syntax
Done by phpcbf over composer fix

Change-Id: I53fd1fc8d056b9b60194d2d630852cfca37aadea
2017-06-15 17:02:57 +02:00
Umherirrender d4b50be650 Update mediawiki/codesniffer to 0.7.2
Also move --ignore to phpcs.xml for use by phpcbf
Short array syntax will be used in follow ups
to keep this patch set small

Change-Id: Ib91f3768cc7cdccdc26a4d5200178ceb8e61e098
2017-06-13 11:59:10 +02:00
Tim Starling 525b6c064c Fix usage of $db->nextSequenceValue()
The return value from the method is only suitable for passing to
$db->insert(). To get the inserted ID, you need to call $db->insertId()
even if $db->nextSequenceValue() returned non-null.

Change-Id: Id5a0df17c77445e9f29564a55fb850c3ecad2630
2017-05-11 10:03:49 +10:00
jenkins-bot 6b70225281 Merge "Only add 'sysop' group to filter user if not in it" 2017-05-09 16:46:08 +00:00
jenkins-bot 7f02b8f0cc Merge "Remove usage of RequestContext::getStats()" 2017-04-26 20:51:14 +00:00
Florian Schmidt 0c663b50e9 Remove usage of RequestContext::getStats()
Bug: T156810
Change-Id: I795080cf431d396337a1ba6fa42e82ffb1b9c145
2017-04-26 20:38:21 +00:00
Jcrespo 848a9ff6f5 Add FORCE INDEX to AFComputedVariable::{closure}
We add FORCE INDEX to revision because probably we have hit a MariaDB
bug that can potentially create an outage on pages with thousands of
revisions due to extreme resource usage by this query when using the
wrong index page_user_timestamp, instead of page_timestamp.

This is considered to be a hack, and once we are in the clear, I promise
to review this an try to get a saner execution path (both in MySQL and
in PHP.

Bug: T116557
Change-Id: I41853da5c0e1a15efad5594eff0cee62be1ad9a4
2017-04-24 19:34:41 +00:00
Kunal Mehta 1392aa05d6 Only add 'sysop' group to filter user if not in it
This should avoid repeated, unnecessary calls to User::addGroup().

Bug: T163032
Change-Id: I010e6eb45c4fbf984a3b5e5df2671117ca4e0136
2017-04-17 10:34:47 -07:00
Timo Tijhof f95e5dd086 Update getStashKey() to use $cache parameter
Follows-up e4ac1ef. A lot of this class gets its own cache object
in key-generation methods because of legacy compat with public
methods and because of how wfMemcKey used to be.

However where possible, we should encourage passing $cache
so that makeKey() can be used on the same instance that uses
the key.

This is a no-op since it's exactly the same object.

Change-Id: Ib3c31110176659a9175679eb716369e7f0a1d3b2
2017-03-09 14:08:04 -08:00
Matěj Suchánek e4ac1efb7a Remove unused argument from a private method
Change-Id: I2f34bb04a90c2535fbeb0e515714aa4a71779f50
2017-03-02 16:34:23 +00:00
Huji Lee 630ef7f2ce Don't use wfDiff() in AbuseFilter
Bug: T158850
Change-Id: Ib5bd4eacc3dd26dc2abdf4eedce66ed228b326d8
2017-02-28 00:29:33 +00:00
Matěj Suchánek de1c2af0e1 Unify global filters decoding across AbuseFilter
Change-Id: I3c97c24b5d0e4818b36e233a8bb3b205cbb2c7d0
2017-01-02 12:14:02 +00:00
Matěj Suchánek 9379a0c767 Add GENDER to the message logentry-abusefilter-hit
As far as I can see, this parameter is already supported.

Bug: T153607
Change-Id: If3eed341ef2f2e7bbad103c2738a7eae59886faa
2016-12-26 09:32:31 +00:00
Aaron Schulz 2d57141600 Move AbuseFilterVariableHolder up to /includes
This class is not for parsing logic or tree nodes

Change-Id: I07a499cc972c30fc249ec4de3250900a3b703443
2016-12-18 17:13:06 -08:00
Aaron Schulz 9b1021b055 Move various classes to their own files
Change-Id: I5d418b3fa27aa6e04b9a680922e5eab2439ffb20
2016-12-17 11:40:10 -08:00