Commit graph

233 commits

Author SHA1 Message Date
Raimond Spekking 3e003e0856 * Move parentheses into tt styled section. From a typographical point of view standard parentheses around tt styled text look ugly. 2009-10-09 16:36:56 +00:00
Jack Phoenix e2a16fe755 AbuseFilter: big commit - coding style & spacing cleanup; also marked SpecialPages' construct() and execute() functions as public and capitalized some SQL keywords 2009-10-07 13:57:06 +00:00
Andrew Garrett 6fc1b1ca79 Don't show navigation links (import, tools) where the user has no rights to view them. Fixes bug 19714. 2009-07-17 16:59:14 +00:00
Andrew Garrett 5c08223f77 Fix bug 18176, hard-coded JS strings needed localisation. 2009-07-17 13:52:40 +00:00
Andrew Garrett fb9d1978cf Change disabled to readonly for textboxes on uneditable filters 2009-07-03 13:54:08 +00:00
Andrew Garrett 3c51c81c93 Add condition limit profiling as well as time profiling to the abuse filter (bug 19256) 2009-07-03 13:46:51 +00:00
Andrew Garrett f176a96fa7 (bug 18659) Allow setting AbuseFilter block duration with $wgAbuseFilterBlockDuration 2009-06-03 15:52:53 +00:00
Andrew Garrett 4a0ce9e562 (bug 18642) Supply filter ID as a parameter to the AbuseFilter warning messages 2009-06-03 15:41:45 +00:00
Andrew Garrett 3b7d6574e6 (bug 18095) Trigger AbuseFilter on file upload. Presently, the only metadata is SHA1, but we'll see what we can do for other stuff on request. 2009-06-03 15:10:44 +00:00
Andrew Garrett 48bfcc35ee Various code quality fixes for AbuseFilter suggested by Tim Starling in a private email, including bugfixes, memory safeguards, performance improvements, removal of redundant code, consolidation of similar functionaality. 2009-05-26 13:08:15 +00:00
Tim Starling da372fdec0 Reverted r49855, r49656, r49401, r49399, r49397. The language converter cannot be used outside the parser at present without generating a large number of bugs, due to global lifetime state variables, inappropriate $wgParser references, etc. Some refactoring needs to be done before it can be used in this way. 2009-05-26 07:46:29 +00:00
Siebrand Mazeland e454269bfc Proper casing for getDBkey() 2009-05-24 08:33:57 +00:00
Andrew Garrett 32aedf1644 (bug 18077) PostgreSQL compatibility issues in AbuseFilter, patch contributed by Brad Jorsch 2009-04-24 03:27:14 +00:00
Andrew Garrett 1a0fc0fea9 Add import/export interface for filters so that filters can be copied across wikis 2009-04-23 04:23:56 +00:00
Aaron Schulz 865d597620 profiling fix 2009-04-17 08:08:35 +00:00
Philip Tzou 28202160b8 Add a new function named 'convert()', allow user to convert a string to specified variant in Abuse Filter. With the support of LanguageConverter which updated on r49397. 2009-04-11 10:59:38 +00:00
Andrew Garrett 7c2a7a2fe0 Support for variable setting with the set_var function, and multiple expressions separated by semicolons (;). In evaluation, the result of the LAST expression will be the return value. 2009-04-01 06:53:18 +00:00
Andrew Garrett 3f62707206 String manipulation functions substr, str_replace and strpos for AbuseFilter 2009-04-01 05:05:23 +00:00
Andrew Garrett 7c3f048fff Usability work, making abuse filter forms readonly (rather than just lacking a submit button and rejecting submissions) for users without permission. 2009-04-01 04:34:21 +00:00
Andrew Garrett 57d1a0b16e Add timestamp variable 2009-04-01 03:59:58 +00:00
Andrew Garrett 186fc67373 Crackdown on filter evaluation by users unable to modify filters. This is a DoS vector which I thought I'd already plugged. 2009-03-31 15:13:26 +00:00
Andrew Garrett 14b850f891 Implementation of global filters, including a major i18n change for abusefilter-log-detailedentry message (rename and split). Needs further testing before deployment 2009-03-30 06:12:12 +00:00
Andrew Garrett 920f04d4fa Fix batch-testing (follow-up to r48855) 2009-03-26 04:41:05 +00:00
Andrew Garrett 3458bf1a7a (bug 18165) Keep the same parser object between filter runs 2009-03-26 04:09:07 +00:00
Andrew Garrett c597c1915f Add contains_any function, for searching a single haystack for multiple needles. Implemented with FSS with a fallback to a for loop, so it should be really fast. 2009-03-26 02:03:32 +00:00
Andrew Garrett cf6f2899f6 Follow-up to r48674. 2009-03-22 10:34:54 +00:00
Andrew Garrett f672c4cee3 Fix batch testing interface, there was old code using the old variable API 2009-03-22 02:46:05 +00:00
Andrew Garrett 4e6754e082 Prevent leaking of filters through diffs 2009-03-22 02:12:51 +00:00
Andrew Garrett 31c6e1d519 Fix bug in testing interface introduced by rsomething 2009-03-19 06:18:41 +00:00
Andrew Garrett 82c958e434 Document contains, ternery operator and conditionals 2009-03-19 05:11:55 +00:00
Andrew Garrett 966b020e6b Document rmspecials, rmwhitespace, ip_in_range 2009-03-19 05:06:39 +00:00
Andrew Garrett f234bcf66d Add basic filter profiling to AbuseFilter -- display on the edit filter page the average time taken to run the filter. Currently sampling at 1/50 2009-03-19 02:40:48 +00:00
Andrew Garrett ce47bca4a8 Load link vars from DB on edit. Includes some tracking of context, so it knows whether to check the DB or the parser. 2009-03-19 02:05:58 +00:00
Andrew Garrett 7b3abbc7c3 Report createaccounts as 'user creating own accounts', rather than as 'IP creating user' 2009-03-18 23:24:10 +00:00
Victor Vasiliev d400dc6763 Fix an XSS bug in AbuseFilter 2009-03-18 19:57:25 +00:00
Andrew Garrett 34cb78e156 Increase threshold for resetting counters. 2009-03-18 04:50:55 +00:00
Andrew Garrett 0ce853e093 Prevent leaking of hidden filters through history interface. 2009-03-17 13:18:33 +00:00
Andrew Garrett 00b468285a Duuuuuh 2009-03-16 23:47:10 +00:00
Andrew Garrett c7f2cb2488 Fix boundary condition for prevention of double-warnings 2009-03-16 23:42:04 +00:00
Andrew Garrett 357c4f030e Workaround for empty $wgTitle in API calls. 2009-03-16 23:02:18 +00:00
Andrew Garrett 0e070fac7f Fix problems with prevention of double warnings 2009-03-13 08:02:05 +00:00
Andrew Garrett 5493b2a690 Add diffs to AbuseFilter. Includes a related pare-down of history, under the assumption that diffs will take up the slack 2009-03-12 05:04:39 +00:00
Andrew Garrett dd0cf2a650 Fix logging where no skin is given 2009-03-11 08:03:25 +00:00
Raimond Spekking 960c9d7463 Use the long time existing pipeList :-) 2009-03-11 07:49:56 +00:00
Andrew Garrett fe5141c412 Add a normal log for filter changes, mostly just a pointer back to the real log. 2009-03-11 07:12:42 +00:00
Andrew Garrett 3a55ecbc85 Link together abusefilter pages with a navigation interface at the top. Add a better intro to the abusefilter home page. 2009-03-11 05:55:06 +00:00
Andrew Garrett 4f2b9a97a4 A whole lotta abusefilter profiling 2009-03-10 22:56:51 +00:00
Andrew Garrett 26752e1d74 Appropriate handling of invalid titles and moves with suppressed redirects. 2009-03-09 13:22:58 +00:00
Siebrand Mazeland 5ebcee6089 * add GENDER support for 'abusefilter-reautoconfirm-done' and 'abusefilter-reautoconfirm-none'
* update message 'abusefilter-reautoconfirm-none'
2009-03-08 18:32:59 +00:00
Andrew Garrett 55b417f517 Add rcount function, same as count except it takes a regex as the needle 2009-03-07 01:26:42 +00:00
Andrew Garrett e60dee6cac Add an interface for extensions to add variables into the variable list (only for ones generated for filtering, for now). Includes an implementation in the TorBlock extension 2009-03-05 02:43:05 +00:00
Andrew Garrett be7dcd9b3b (bug 17674) Add minor_edit variable to AbuseFilter 2009-03-04 02:04:48 +00:00
Andrew Garrett 03e12503a3 (bug 17664) Handle section-editing correctly by switching from EditFilter to EditFilterMerged.
Also, fix a weird bug where the Abuse Filter was diffing the parsed text, not the wikitext
2009-03-04 01:56:21 +00:00
Andrew Garrett f94f42b506 Store Abuse Filter variable dumps to external storage instead of leaving in afl_var_dump. afl_var_dump needs to be left as a BLOB for the moment for backwards-compatibility. 2009-02-27 03:06:19 +00:00
Andrew Garrett 787672cdf7 Don't pointlessly add lots of whitespace to the end of rules. One newline is enough 2009-02-26 12:22:26 +00:00
Andrew Garrett 92698e95ba Improve AbuseFilter performance by implementing lazy initialisation of computed variables.
This has been done by replacing simple associative arrays with an AbuseFilterVariableHolder, which recognises helper classes called AFComputedVariables.
Computation may occur during the abuse filter analysis, or later when testing and reviewing filters.
2009-02-26 12:15:14 +00:00
Andrew Garrett 98a94ff5d9 Fix Abuse Filter after-the-fact var generation -- was using request IP address in place of IPs for anonymous changes 2009-02-25 04:31:53 +00:00
Andrew Garrett 20c9542fe3 Another silly reautoconfirm bug -- nothing was being returned. 2009-02-19 00:18:31 +00:00
Andrew Garrett 1c87a7ba69 Fix some silly bugs in reautoconfirmation 2009-02-18 19:34:36 +00:00
Andrew Garrett d3d4333f42 Fix silly fatal 2009-02-13 18:30:34 +00:00
Andrew Garrett 64154e283d Correctly deal with emergency disabling, hit counts, etc for filters with 'throttle' and 'warn' actions. 2009-02-13 01:40:57 +00:00
Andrew Garrett 35e61feeb6 Abuse Filter Parser updates
* Deprecate parseTokens in favour of a parse-as-you-go approach, faster and uses less memory.
* Display variables in lower_case so they aren't SHOUTING_AT_PEOPLE.
* Tell people if they try to use variables that don't exist, rather than silently returning NULL.
2009-02-11 20:00:33 +00:00
Tim Starling c8b0007232 * Break long lines. If I'm going to review this code, I need to be able to read it.
* Write array literals with one item per line. This makes diffs which add or remove items far easier to interpret, and makes merging such changes feasible. And it looks nicer too.
* Use line breaks to show the logical structure of your code. This enhances readability. Bring similar elements in a list into alignment, in order to reveal the differences between those elements at a glance.
* Removed a fun game of spot-the-difference in AbuseFilterHistoryPager::getQueryInfo(). If I want fun games I'll play UFO:AI.
* Moved some oddly placed assignments (in expressions) to their own statements: such assignments reduce readbility.
2009-02-07 09:34:11 +00:00
Andrew Garrett 53edcafc69 Reduce exception on encountering an unrecognised action to a debug log message. 2009-02-07 02:26:56 +00:00
Andrew Garrett bcf5381eec Fix up throttling -- was allowing one extra action. 2009-02-03 23:44:47 +00:00
Andrew Garrett 0918642869 Fix tester for titles which don't seem to work properly. 2009-02-03 18:48:16 +00:00
Andrew Garrett 3d13a2dc92 Fix double-prefixing. 2009-02-03 00:15:12 +00:00
Andrew Garrett 2e41b7d9d7 Fix abuse filter blocking *everything*. 2009-02-02 23:30:48 +00:00
Andrew Garrett 9eca9a7857 E_NOTICE 2009-02-02 17:57:06 +00:00
Andrew Garrett 6f91720eac Allow af_public_comments to be wikitext.
Relatedly, add parseInline function to OutputPage, to avoid duplicating the same awful regex in wfMsgExt.
2009-01-31 01:59:13 +00:00
Andrew Garrett 61ad9a0aca ADDED_LINKS was completely broken. Seems to have been comparing links in the old text to.... links in the old text! 2009-01-30 23:31:31 +00:00
Andrew Garrett b68f53bef6 Check syntax before evaluating an expression 2009-01-30 23:23:52 +00:00
Alexandre Emsenhuber 65cec0be93 * Added new lines for source readability
* Change null to an empty string, so that Xml::element() doesn't generate a opening element only, which results in a XHTML error
2009-01-30 15:40:59 +00:00
Andrew Garrett f16a825ab2 Filter out NewPP limit report from NEW_HTML variable. It's just not really necessary, and that stuff could be included as a separate variable in any case. 2009-01-29 23:46:19 +00:00
Andrew Garrett 2d24fcced0 Add a few missing messages, and fix a syntax error from r46557. 2009-01-29 23:36:09 +00:00
Andrew Garrett 783c69c3ba Clarify rlike v. like in builder menu. 2009-01-29 23:28:59 +00:00
Andrew Garrett 3e90ea3eea Don't show the 'disallow' action message if we're already showing other messages. 2009-01-29 23:24:24 +00:00
Andrew Garrett f9c9c07ccf Abuse Filter changes designed for testing filters against real data:
* Add searching and filtering functionality to the existing 'test' interface.
* Add an 'examine' interface designed for testing filters against a previous change, selectable through the search interface in either the 'test' or the 'examine' view.
* Minor enabling change in ChangesList core, to allow subclassing.
2009-01-29 22:44:31 +00:00
Andrew Garrett cb5eca50fd Add Special:AbuseFilter/test, which allows (trusted for now, due to DoS potential) users to enter a filter, and have it checked against the last 100 RecentChanges items while-u-wait.
Some related cleanup to change tagging in ChangesList.
2009-01-28 23:54:41 +00:00
Andrew Garrett 8a54e6ba00 Branch merge of change-tagging branch with trunk
-- Introduce tagging of individual changes (revisions, logs, and on recentchanges). The tags are customisable, and currently settable by the Abuse Filter and by the TorBlock extension. The tags can be styled on the various pages on which they appear.
-- Introduces a schema change, three new tables (valid_tag, change_tag, and tag_summary).
2009-01-28 19:08:18 +00:00
Andrew Garrett c472ec7ff0 Fix RECENT_CONTRIBUTORS query -- apparently options need to be uppercased. 2009-01-28 06:29:59 +00:00
Andrew Garrett c04e354053 Add some missing vars in the builder. 2009-01-28 02:56:47 +00:00
Andrew Garrett 36b78662b0 Refactor, pretty up and spread localisation of filter consequences (in logs, history, filter summary, etc) 2009-01-28 01:26:38 +00:00
Andrew Garrett 050575a5af Allow users with the appropriate permission to mass-revert all actions (like blocking, deautoconfirming and degrouping) taken by a filter, with an appropriate summary. 2009-01-28 00:10:35 +00:00
Andrew Garrett 9775ea2787 Soften emergency disable feature, by making it only block restricted actions, rather than disabling the whole filter. 2009-01-27 20:18:58 +00:00
Raimond Spekking 53887bb338 * Add $wgAbuseFilterStyleVersion to SpecialAbuseLog::execute
* Remove one dupe call of addExtensionStyle from SpecialAbuseFilter::execute
* Bump $wgAbuseFilterStyleVersion per CSS changes in r46325
* Add a space between drop down and submit button
2009-01-27 17:32:30 +00:00
Andrew Garrett 6131d42c5d Don't try to increment the hit counters if no filters were matched, waste of a DB query. 2009-01-27 05:22:24 +00:00
Andrew Garrett 54ee8685d3 Remove stray 'return' which was breaking hit counters. 2009-01-27 05:07:11 +00:00
Andrew Garrett 5861a0fed1 Add OLD_TEXT and NEW_TEXT variables, refactor variable storage, and clean up the AbuseLog interface just a little. 2009-01-27 01:31:42 +00:00
Andrew Garrett 11ab345814 Localise Abuse Filter exceptions. 2009-01-26 23:32:46 +00:00
Andrew Garrett 83d4b5a91f Bold and apply a subtle background colour to the changed fields in Abuse Filter history -- will make things much easier to track. 2009-01-26 22:31:02 +00:00
Andrew Garrett 4adfc225f1 If no groups are being removed, then don't add a LogPage entry for the 'degroup' action! 2009-01-26 18:52:41 +00:00
Andrew Garrett 9916d54b81 Strict standards. 2009-01-25 05:12:05 +00:00
Andrew Garrett 819986be43 Put filter-builterator on tools page. 2009-01-23 22:49:13 +00:00
Andrew Garrett 8ddd002c14 Remove dependency on change-tagging being branch-merged to trunk for now, and fix a few miscellaneous related bugs that came up in final testing. 2009-01-23 19:23:44 +00:00
Andrew Garrett 53179c675f Apply changes from change-tagging branch. I will remove all of the stuff actually related to change tagging in a moment, to avoid trunk changes on Wikimedia sites. 2009-01-23 19:23:19 +00:00
Andrew Garrett 83a741ee24 Do initialiseRange into Block::insert.
Fix related fatal in Abuse Filter.
2008-12-15 12:10:41 +00:00
Andrew Garrett 4f374b561d Abuse Filter:
* Remove some old debugging code.
2008-11-15 01:19:20 +00:00
Siebrand Mazeland 30b422c330 Consistent casing for wfGetDB(), getDB(), and getDBKey() 2008-11-06 22:38:42 +00:00
Andrew Garrett fe00c5e019 Minor bugs in abuse filter, plus new ALL_LINKS/ADDED_LINKS/REMOVED_LINKS 2008-10-24 08:58:32 +00:00
Andrew Garrett 0d52f1261a Abuse Filter:
* Consolidate details on the last edit.
* Allow specification of a warning message to display instead of the standard warning (was already implemented in code, but there was no UI for it.
2008-10-21 14:11:03 +00:00
Andrew Garrett 56065d6a59 Re-commit re-autoconfirmation, with fixes for permissions 2008-09-29 13:30:11 +00:00
Andrew Garrett a374d100ff Revert r41145. Restoring autoconfirmed should really have access control, eh? 2008-09-22 14:07:04 +00:00
Andrew Garrett 74d4fa198f Add a hacky tool for restoring autoconfirmed status when it's blocked by a filter. Don't expect it to get much use, so I haven't put much effort in. It gets the job done, though. 2008-09-22 09:16:45 +00:00
Andrew Garrett 8c66e81715 Make some obscure options actually useful, and mark some as obsolete. 2008-09-22 06:28:07 +00:00
Andrew Garrett 432d579ed6 Remove some debugging code 2008-09-21 13:17:11 +00:00
Andrew Garrett bcca5492af Don't log throttles, it should only log when the throttle is actually hit, not when it's incremented. 2008-09-21 13:08:10 +00:00
Andrew Garrett 00b46cef8b Minor fix for last commit, and add DeleteQueue support. 2008-09-18 13:33:39 +00:00
Andrew Garrett 74097d106e Add rangeblocks, a bit of a nuclear option for the abuse filter. Not intended for Wikimedia use 2008-09-18 13:01:50 +00:00
Andrew Garrett 4fd43cdea9 Add basic functionality to delete filters (not really a deletion, just moved to another list) 2008-09-05 14:27:18 +00:00
Andrew Garrett 016da694a6 Abuse Filter:
* Reimplement tools, starting with a spiffy AJAX expression tester. Didn't bother implementing as non-AJAX, as it's just an extra.
* Change exchange protocol between native parser and PHP app. Resulted in 2-3-fold performance boost.
* Fix minor bug from last night.
2008-08-04 14:27:48 +00:00
Andrew Garrett 4c57a8cf4d Abuse Filter:
* Make with -O3 instead of -g, to improve performance.
* Add AJAX filter syntax checking.
2008-08-04 12:15:14 +00:00
Andrew Garrett 529d7b4277 Abuse Filter:
* Introduce a syntax check on-save.
* Introduce filter history (requires DB schema change).
* Deprecate now non-working /tools interface (will come back).
2008-08-03 14:04:26 +00:00
Andrew Garrett cb65839e40 Fix more deprecated pass-by-reference syntax 2008-08-02 13:51:29 +00:00
Andrew Garrett 4b53ea93ec Fix deadlocking with shell out to native parser 2008-08-02 11:10:42 +00:00
Andrew Garrett 75b5c18f3a AbuseFilter:
* Add a bunch of profiling, so we can figure out where performance bottlenecks are.
* Cache the in-object token cache in memcached, bringing per-condition run times from 4ms to a cooler 296us
2008-07-18 08:30:25 +00:00
Andrew Garrett f665c88bc5 AbuseFilter:
* Improve normalisation: use AntiSpoof.
* Allow callers to prevent the use of a condition counter. It's kinda annoying in batch processes.
* Improved caching of tokenisation - reduces average time to process a condition from 4ms right down to 200us
* Some new modifiers, caching of modifier data.
2008-07-18 02:18:58 +00:00
Andrew Garrett 34a4bbe10a Abuse filter updates:
* Allow matching on the changes made in an edit (added/removed lines, etc)
* Improve display of variables in details view - standardise the table size, add white-space: pre, font-family: monospace.
* Improve modifiers: add htmlencode/htmldecode/urlencode/urldecode.
* Add 'abuse filter tools', currently consisting of a testing ground for the modifiers. In future, may include facilities for testing filters against user contributions, recent changes, or other sources.
2008-07-17 13:40:45 +00:00
Andrew Garrett f34000f1fb Abuse filter updates:
* Disallow more than 100 condition-checks per action.
* Disallow filters which are misbehaving (more than 5 matches, constituting more than 5% of recent actions)
* Display statistics for all filters (percentage of times the condition limit is reached, and percentage of times some filter is matched), and individual filters (percentage of times it's matched).
2008-07-17 02:43:45 +00:00
Andrew Garrett b142b81dd7 Include size delta in vars for editing 2008-07-15 08:46:17 +00:00
Andrew Garrett ef2dac1a7c Add log entries for actions done by the extension. Also, properly scramble conflicting accounts by resetting email too 2008-07-09 07:02:13 +00:00
Niklas Laxström 10652d4d79 * Added missing desc message
* Do not transform messages that are later parsed
* Split paragrahps to shorter lines, svn is bad at tracking long lines
2008-06-29 20:10:42 +00:00
Andrew Garrett 796dab30dd Clean up code a little, tell users which filters they've matched, and don't display the 'disallowed' text if that's already being done by another action. 2008-06-29 14:00:39 +00:00
Andrew Garrett cef3040152 Also allow spaces, newlines, tabs to be ignored when processing rules 2008-06-28 04:11:21 +00:00
Andrew Garrett aebb17233e Add specialratio to modifiers 2008-06-27 11:33:27 +00:00
Andrew Garrett 9a2ddd9aba Fix bug in OR condition parsing 2008-06-27 11:13:07 +00:00
Andrew Garrett 44da7a3dce Change primary key of abuse_filter_action from afa_filter to afa_filter,afa_consequence 2008-06-27 09:49:26 +00:00
Andrew Garrett 2cc6bdfa5e Add install.php, installer which adds the blocker account if it doesn't exist, adds the tables 2008-06-27 09:38:54 +00:00
Andrew Garrett bf1009091b Add contains to the list of operators 2008-06-27 09:11:59 +00:00
Andrew Garrett 554f8ff412 Add contains operator 2008-06-27 08:50:03 +00:00
Andrew Garrett 1c69a9c569 Remove some ugly debugging code 2008-06-27 08:19:49 +00:00
Andrew Garrett 3bc50d4f95 Change multi-table select to LEFT OUTER JOIN, so actions without consequences show up on the filter page' 2008-06-27 07:55:13 +00:00
Andrew Garrett 1f5f95131d Add AbuseFilter extension 2008-06-27 06:18:51 +00:00