Commit graph

233 commits

Author SHA1 Message Date
Marius Hoch 42bd0d84f4 AbuseFilter: Change format of database logging/ performance
AF is setting several lazy load variables for the currently editing user.
To do this it's passing along the user name extracted from a user object
and generating a new user object later from that name which is of course
pointless. With this patch I'll pass user objects directly to prevent that.
On top of that I've deprecated a method in AFComputedVariable::compute which
was redundant as there is a more generic one which can solve that task
just fine.

Furthermore I've changed the logging behaviour from serializing the whole
AbuseFilterVariableHolder object to only store the variables. That has two
major advantages:
 * The amount of data that needs to be saved on a filter hit is reduced
   to about 1/10 of what the old version needed.
 * This is much more forward compatible as the old way of saving this
   relied on the class structure to stay the same while this is a simple
   array containing the vars.

On top of that we now only log variables already set by the time
a filter is hit. On top of the obvious performance increasement
that makes it easier for the user to spot the relevant data.

Another thing this change alters is the way the AbuseFilter internally
works with AbuseFilterVariableHolder objects. Right now we use one for
testing the filter(s) and later we use another one to compute the same
data again in case a filter was hit (for logging)!

This is not thoroughly tested yet, but way more sane than what we're
currently doing!

Change-Id: Ib15e7501bff32a54afe2d103ef5aedb950e58ef6
2013-02-28 22:35:22 +01:00
Marius Hoch fdae51ec07 Fix default for $wgAbuseFilterAnonBlockDuration
$wgAbuseFilterAnonBlockDuration didn't default to the actual
value of $wgAbuseFilterBlockDuration but to the default value
of that (which is indefinite). Fixed that (untested)

Change-Id: I26a929bfba997b80445a108e212030fe7faa6428
2013-02-23 00:20:29 +01:00
Hoo man e817f73f13 Merge "Check against $wgAbuseFilterRestrictedActions instead of hardcoded list for block actions" 2013-02-19 13:09:51 +00:00
Hoo man d858cfca35 Merge "Check that $title is defined and is a Title object" 2013-02-19 12:41:39 +00:00
jeblad 53e230c5c7 Check that $title is defined and is a Title object
During testing the context does not always contain a valid
Title object. In those cases AbuseFilter will fail hard.
This changeset makes the filter survive some of those
failures.

Change-Id: I0b2247432619ddf15cc17ed41b4b7a6a11e910e0
2013-02-19 13:39:47 +01:00
Hoo man 750f594b0f Merge "Create variable to set length of IP blocks" 2013-02-19 12:30:53 +00:00
Reedy d729a0ee02 Bug 43011 - AbuseFilter: Invalid argument supplied for foreach
Change-Id: I83f17b1747dd46998ffab2b6f7f867ab24c665a1
2013-02-11 22:30:59 +00:00
Kunal Mehta b8f53e27c3 Create variable to set length of IP blocks
This creates a $wgAbuseFilterAnonBlockDuration, which
allows for IP blocks to have a different length.

For backwards compatability this is default set to
$wgAbuseFilterBlockDuration.

Change-Id: Ibfd5c9639317150442f745a5759f3c34b38de274
2013-02-08 04:59:54 -06:00
Kunal Mehta acb123b148 Check against $wgAbuseFilterRestrictedActions instead of hardcoded list for block actions
Currently $wgAbuseFilterDisallowGlobalLocalBlocks checks against a
hardcoded list for "block actions", which means that extensions
like GlobalBlocking cannot add actions to it.

Change-Id: I6ac5125782cf0029447948d6d0080103700e397c
2013-02-08 02:17:17 -06:00
daniel c1f8d749a7 Allow special pages as context for abuse filter.
The title in the context provided for an edit may be a special page
in some cases, e.g. when an extension uses a special page for creating
new content, and the final name of the content is not yet known
when the EditFilterMergedContent hook is called.

This change will allow special pages to be used in this context.

Change-Id: I5551bff69e3b617a437bd043fdc893dbdcd9f09f
2013-01-22 14:08:50 +01:00
Marius Hoch c4e78e9713 (bug 18174) Added Older and Newer change link to Abusefilter diff
On top of that I changed /history to no longer show the
changes link if it's the first revision of the filter.

As I was on it I fixed the diff selection for diffs like
history/1/diff/2/next where change number 2 was compared
to the last change of the filter instead of the next one.
This probably went unnoticed because the /next action isn't
linked anywhere within the software itself.

Change-Id: Ic28bb696a4dbbb75e0bfcae1843ccefab1a9508a
2013-01-18 10:42:22 +01:00
Hoo man 24427e6d76 Merge changes I72e1a6dd,Ibb9d4c9a
* changes:
  Use Status object to report filter results.
  (bug 42064) AbuseFilter + EditFilterMergedContent
2013-01-17 17:43:32 +00:00
daniel 6522f70fb8 Use Status object to report filter results.
This changes the AbuseFilter to use Status objects for internal
error handling. This allows for more flexibility in passing error
messages back from hooks and avoids double escaping.

Change-Id: I72e1a6dd7dee19f889fc13b60456e9bfebd5e22b
2013-01-17 18:34:19 +01:00
daniel f3788c4f0c (bug 42064) AbuseFilter + EditFilterMergedContent
This makes AbuseFilter use EditFilterMergedContent if support for
the ContentHandler infrastructure is present. This means living
without some nice bits of context, because EditFilterMergedContent
doesn't provide an EditPage object.

This requires core change I99a19c93 to work correctly.

Change-Id: Ibb9d4c9a36b8a199213958b920902e8006c71fe8
2013-01-17 11:18:28 +01:00
Kunal Mehta 772a9fa619 (bug 28362) add conditional article_views variable
If $wgDisableCounters is set, using the variable will result
in a syntax error for an undefined variable.

Change-Id: I136484356506c87ac0e2ede088787e854a0a2fa9
2013-01-16 17:03:47 +01:00
Hoo man a885db279d Merge "Let AbuseFilter work with 1.19" 2013-01-09 01:42:29 +00:00
CSteipp b8b799630b Merge "Properly integrate $wgAbuseFilterValidGroups" 2013-01-07 23:01:48 +00:00
csteipp 586598b5b2 Let AbuseFilter work with 1.19
This patch allows the current AbuseFilter to work with 1.19, by only
using MWTimestamp if MediaWiki is version 1.20 or later.

Api still needs fixing, but this will get filters running.

Change-Id: I7c9eaf777d529ce15dfd85761784f9ad1443f2bf
2013-01-07 14:53:57 -08:00
Matthias Mullie 0c174aec42 Properly integrate $wgAbuseFilterValidGroups
The concept of different AbuseFilter groups had not properly been integrated, but should be now:
- The total number of matches is now group-specific
- .. Which will also fix the emergency shutdown calculations
- And a portion of incorrect code (involving $logged_local_filters) has been fixed (action != group)

Change-Id: I091199a9d74aee47dcb3d8942394a28e0ffd3234
2013-01-07 14:42:51 +01:00
CSteipp 452b918203 Merge "Add hooks to AF that allow altering User and Title specific vars" 2013-01-03 20:38:59 +00:00
techman224 192f2e250e Hidden the test subpage in the navlinks for those without modify rights
The link should not be shown to users that don't have the right to use it,
like with the tools and import pages

Change-Id: Ie48e61a7de61ab095d621845466aea56b7445f5a
2012-12-30 15:23:36 -06:00
Marius Hoch 9fa6ef5a95 Add hooks to AF that allow altering User and Title specific vars
Added two hooks to the AbuseFilter, one that allows altering of the
variables created for a specific user and one for title specific
variables. This a much cleaner way to add title or user specific
variables than using the current general-purpose hooks.

Change-Id: I7374846346dd220b5d0633c11c134030f98bcf00
2012-12-28 00:40:48 +01:00
Krenair c7a617f24d (bug 43105) Don't send HTML to the IRC feeds
Change-Id: Idcbcf1f194f2fc1bdd10fefab570849a72ed30e7
2012-12-22 19:05:51 +00:00
Hoo man 253812a2f4 Merge "(bug 22548) add boolean user_blocked variable" 2012-12-18 19:09:32 +00:00
Kunal Mehta f97df950d7 Add option to have private filters notify UDP/RC (default: disabled)
Change-Id: I7e54f1da9f01634791316e613e79b2b7740f9760
2012-12-16 17:09:30 +01:00
Kunal Mehta 2b01494cfc (bug 22548) add boolean user_blocked variable
Change-Id: I8b61ce42338596a38e411d6023c8f7ff963641a1
2012-12-14 04:32:37 -06:00
Ori Livneh 90791cb24e Fix class name
Change-Id: If300aad2c3acf4d007e80a11158419c315c800e4
2012-12-13 15:59:54 -08:00
Kunal Mehta 578bc116e0 Don't send private filter hits to IRC/UDP
Change-Id: I433aa0d92f7bf805ea27eaf235909065e4d478a7
2012-12-13 04:29:16 -06:00
jeblad ae2c2f7e67 (Bug 42064) Change getText to getContent in AbuseFilter
Patchset 2: Handle more content types
Patchset 3: Change cover letter
Patchset 4: Make it b/c
Patchset 5: Refactor and use previous audience
Patchset 6: Fixed/ minor changes

Change-Id: Ib1d2f9803bc95bb9efd445e3778126d5c3090a71
2012-11-25 18:09:47 +01:00
Hoo man d2568a7d43 Merge "fix php notices" 2012-11-02 19:57:52 +00:00
Kaldari 6e3c9e4acf Merge "Bug 40672 - Abuse filter: Increase 5% limit to allow filtering for very short posts" 2012-10-25 18:22:52 +00:00
Matthias Mullie 1706ca0832 Bug 40672 - Abuse filter: Increase 5% limit to allow filtering for very short posts
This patchset will make it possible for other extensions to tap into abusefilter with a custom group, and set different tresholds per group.
See https://gerrit.wikimedia.org/r/#/c/29569/

Change-Id: I21d31bdf28e26f3c830652efc08a247db9f7a86c
2012-10-25 20:21:19 +02:00
raymond 7347c5d728 Add existing page/file restrictions
Already shown with the code but without text:
i.e. https://commons.wikimedia.org/w/index.php?title=Special:AbuseFilter/examine/82786829&testfilter=

PS2: Add message documentation

Change-Id: Ibb23bf556e8702dcd592e7a9b0ff3c851f83380c
2012-10-25 18:02:57 +02:00
Siebrand Mazeland 176227e721 Maintenance for AbuseFilter extension.
* Replace deprecated methods.
* Remove no longer needed function fnmatch().
* Remove superfluous newlines.
* Remove unused and redundant local variables and globals.
* Deglobalization.
* Update documentation.
* Fix incorrect return values or add FIXMEs when in doubt.
* Escape output in a few places where needed.
* Remove unneeded MEDIAWIKI constant checks.
* Fix various JSHint/JSLint issues.

Patch Set 11: Merged https://gerrit.wikimedia.org/r/24701 into
this one per Siebrand's request

Change-Id: I02ba4ce31b6aca5b7324114093f8ece143abc295
2012-10-09 22:26:45 +02:00
Matthias Mullie 457c38cc3a fix php notices
Change-Id: I44ec2181ac7f2aea775f17c162838d52524084f8
2012-09-27 12:15:22 +02:00
Siebrand b7ca16003a Merge "(bug 20272) AbuseFilter does not handle autocreation of accounts." 2012-09-23 18:21:26 +00:00
Krenair 4804699e70 (bug 30324#c9) Remove 'autoblock' log params for blocks
Change-Id: I86bd5fa949a06eed95708d7f1948b684251788bd
2012-09-21 16:26:02 +01:00
Werdna d34b3659d9 Merge "(bug 30324) Don't falsely claim to angry-autoblock in log entries, don't stop talk page access." 2012-09-16 13:14:15 +00:00
Krenair c3bd782ce5 (bug 30324) Don't falsely claim to angry-autoblock in log entries, don't stop talk page access.
Also autoblock.

Change-Id: I0563944752f2c5b536b6d05d74c0814183afe9de
2012-09-09 14:12:13 +01:00
raymond dabc66a883 Replace non standard 'tt' with 'code' for better HTML5 conformity
See http://www.w3.org/wiki/HTML/Elements/tt

Change-Id: Ia2ffb09c002bc3aededfb6bcb8a430da32b52cde
2012-09-07 10:50:13 +02:00
csteipp 1973ea6714 Add Global Rules
* Update rules list view to show global rules toggle, global rules
* Update rule processing to get global rules from memcache, if no
rule exists, get them from the central database and store them in
memcache
* Delete global rule key whenever global rules are updated
* Add filtering for log by wiki on the central database, updated
table definitions to add index on afl_wiki
* Add global $wgAbuseFilterDisallowGlobalLocalBlocks so local wikis
can prevent global rules from locally blocking, removing or revoking
permissions.

* patchset 13: Include recommendations from Tim. Add db updates to
LoadExtensionSchemaUpdates hook.
* patchset 14: forgot to add new files

Change-Id: Id69a9d603f9679f838e8691c651a3e9d8461b422
2012-08-27 03:30:07 +00:00
csteipp 56ca593a71 Add global throttling to AbuseFilter
Adds/checks keys to central memcache instance on throttle actions
to allow for global throttling of events.

Change-Id: Ic4ace6c4aa07990671fe75fbbc1b019a8395fc46
2012-08-16 16:01:19 -07:00
Reedy 268f976a12 Warning: Invalid argument supplied for foreach() in /usr/local/apache/common-local/php-1.20wmf8/extensions/AbuseFilter/AbuseFilter.class.php on line 1852
Change-Id: I9a00f341be161a8b33ff7a482a4f32f5b98b8716
2012-07-24 17:59:50 +00:00
Andrew Garrett 31dba52a94 AbuseFilter: Fix bug in 53aea9c where two log entries would be produced, one of them broken.
Change-Id: I075dce25756683ab4c04edab5f43ba451376ff6d
2012-07-24 10:55:58 -07:00
Reedy fad1fe4c41 Call to a member function getVars() on a non-object in SpecialAbuseLog.php on line 284
Seen on wmf fatal log

Change-Id: I50eb4f9d1b19c8c39ede07e1149baf475cccaaaf
2012-07-24 14:50:01 +01:00
Andrew Garrett 53aea9c0ce AbuseFilter: Resolve bug 18374, bug 28633.
* Store the revision ID associated with a log entry
 if the action is successful.
* Expose this as a diff link in the UI.
* Implicitly hide log entries if their
 corresponding revisions are also hidden.
* Includes scope for expanding to log entries if desired.

Change-Id: Ie2d43dd1bacf14289fdf0492bb22267590ee649d
2012-07-11 10:16:59 -04:00
Reedy 3da992692c Add a few bits of method documentation
De-indent a line

Tidy up method returns

Change-Id: I7aa93072c80a16eb27b7f7ac3d2f030ea63ebf60
2012-06-21 14:32:05 +01:00
Catrope c2869a28fd Merge "Abuse Filter: Allow filters to be split into "groups" for the purposes of operating them on different types of input." 2012-05-14 18:47:12 +00:00
Andrew Garrett dc207d0cbd Abuse Filter: Allow filters to be split into "groups" for the purposes of operating them on different types of input.
The purpose of this change is to allow AFTv5 developers to run a separate list of filters against article feedback actions without issues of cross-contamination and bumping up against the condition limit.

Change-Id: I758795f01eaf3ff56c5720d660cd989ef95764a7
2012-05-12 12:53:32 +10:00
Szymon Świerkosz 36257344f1 (bug 20272) AbuseFilter does not handle autocreation of accounts.
This change depends on I92f57fc2c3189c42157478de14a8d48045a253b5.

This change provides a handler for AbortAutoAccount hook and adds new
action to the filter: autocreateaccount. Every time the AbortAutoAccount
is invoked the filter is executed. This may create some issues with
users which are affected by a filter, because it may be triggered on
every page view. The AbuseFilter relies on CentralAuth session
blacklisting - for each session the filter will be triggered only
once and then the autocreationg of account will not be attempted.

I don't know why AbortNewAccount hook takes as $message argument
a text of message, however AbortAutoAccount takes a name of the
message. This makes impossible to produce a user friendly message
why account creation is not allowed.

Change-Id: Ie3a7ee9210fd884d214ad3132a502a00332c3138
2012-05-10 10:08:48 +02:00