In the common case where no banned links were found, cache this
information to skip the checks on save.
Change-Id: I5f936622bc62d9fc905edaa2a69f52388c047d10
* Have SpamBlacklist::doLogging() actually run
* Bump schema ID so userId property is an integer
* Don't try logging URLs that were unable to be parsed
* Make sure path/query/fragment are always strings
Bug: T115119
Change-Id: Ia81037e8939dd547f00e79c169fa84ca0a7b917e
Follows-up 5910bfd7ba.
* Remove one-off $domain variable.
* Rename $urlChanges to highlight that urlChanges is a log of changes,
not a list of changes to be applied.
* Rename doEventLogging() to isLoggingEnabled().
Change-Id: Idbd6551502362422beea4d86b912128a43e9c96b
If enabled, changes in URLs on a page will be logged to the
"ExternalLinkChange" schema. To avoid extra lookups, the diff of URLs is
calculated during the filter step of the SpamBlacklist, and stored in
the SpamBlacklist instance state until the post-save hook is called, and
then they are queued to go to EventLogging.
Bug: T115119
Change-Id: I9a5378dca5ab473961f9fe8f7a6d929dc6d32bba
* This works via plugging into ApiStashEdit.
* The query is relatively slow per performance.wikimedia.org/xenon/svgs/daily/2016-02-15.index.svgz.
Change-Id: I0ad5289324b5482db7e2276f58fc1ac140250d47
When a regex detects a URL on the blacklist, use an expanded regex
that matches the full line to get the URL to log.
Bug: 55356
Change-Id: I6dfbc1b70f9a305e76664ac28ccb90fe1594f342
This resulted in doubling the appserver-memcached traffic across the
Wikimedia cluster.
This reverts commit 32b546a223.
Change-Id: I03e96a1bb223360e62d47f98a505cc5b26e5aadf
Add API action spamblacklist, accepting parameter url, that returns the
parts of the URLs that match the spam blacklist, if any.
Bug: 54441
Change-Id: Ia6aea8b463fc63f951224520b8cc5abf185c5c74
This changes SpamBlacklist to make use of the new, ContentHandler
aware hooks.
This change also includes some refactoring and cleanup which made
the migration to the new hooks easier.
Change-Id: I21e9cc8479f2b95fb53c502f6e279c8a1ea378a5
The default blacklist was no longer being used, because the line
$wgSpamBlacklistFiles =& $wgBlacklistSettings['spam']['files']
initialized $wgBlacklistSettings['spam']['files'] to null,
and BaseBlacklist::__construct then overrode $this->files
with null.
Change-Id: I22448bfb87eef6dd86b61362f3eb6bb2198a10b6
Adds a logBlacklistHit function function, which is callend whenver a new url
matches the blacklist. A new log type of "spamblacklist" is created, and is only
viewable by people with the "spamblacklistlog" userright. This is given to sysops
by default.
By default this is disabled, and is controlled by $wgLogSpamBlacklistHits.
Bug: 1542
Change-Id: I7b1ee2b3bb02b693d695bf66d157e2c33526c919
Currently if a rule says "go.gle" and http://www.google.com/ is being
added, "http://www.google" is displayed as the blocked part. This
doesn't look quite nice. Now it just displays "google".
Change-Id: I0851c00d38129a8e9910c65100998eb3f1e5b2c2
provide all blocked URLs").
SpamBlacklist extension to provide all matched URLs to
spamPageWithContent() rather than just one. Performance
hit negligible and zero for all edits that don't hit the
SpamBlacklist (99.999%+).
DEPENDENT ON OTHER HALF OF FIX (now in core):
https://gerrit.wikimedia.org/r/3740
Change-Id: Ia951d5795c5cedb6c3876be89f8a08f110004102
* SpamBlacklist: code is weird but I'm pretty sure this needs HTTP
* ContributionTracking: expand return URL to current protocol. Use HTTP in the test suite (PROTO_CURRENT makes no sense in tests since they run from the command line)
* GlobalUsage: remove URL expansion, not needed after r95651
* CentralNotice: expand URL because it gets fed to window.location indirectly via JS
* OpenSearhXml: use canonical URLs in XML output
* MobileFrontend: expand a URL that's used in a Location: header
Bug comment: Set PCRE_MULTILINE on spamblacklist regexes. $ on spam blacklist regex should match the end of the url (not of the text) so it can be used to match only the mainpage. Since the candidate urls are already joined with a new-line separator, it's just setting PCRE_MULTILINE on the regex.
