Commit graph

171 commits

Author SHA1 Message Date
Matthias Mullie 0dee3d3587 Pass title along to signature parser
* Parser generates signature to compare against
* Signature can be overwritten per wiki, in NS_MEDIAWIKI
* Such overwritten default can be different depending on
  page the signature is on[1]
* Our comparison signature generation was page-agnostic
  (always from Title::newMainPage)
* Signatures didn't match up on own talk pages, where
  default signature is different

Also added 2 new tests cases & improved tests by also
setting the page

1: https://en.wikipedia.org/w/index.php?title=MediaWiki%3ASignature&diff=176507985&oldid=176229132

Bug: T78424
Change-Id: Ice151d4d16236a5d1556ef62805b61310c7beb85
2015-02-17 10:24:12 -08:00
Chad Horohoe 4a09f730e3 Remove obvious function-level profiling
Change-Id: I498d79d99ab6016cadf502f4c606a7a911470250
2015-02-10 14:32:58 -08:00
jenkins-bot d3bfc5a5b9 Merge "Improve signature detection" 2015-02-06 01:21:27 +00:00
Matthias Mullie 58e1c765d6 Improve signature detection
Previously, there were a couple of hacks in play.
It was also not picking up ~~~ (signature without timestamp)
And it relied an a nasty regular expression which, although
based on Parser, may some day get out of date.
And it relied heavily on a specific signature format, which
isn't guaranteed (it's an i18n msg)

This patch changes the approach: it will use a very simple
regex to match links, and will send those through Parser to
generate the signature anew. My reasoning is that that should
be exactly the same as what Echo just received (should've
also gone through parser)

Biggest discomfort of this approach is that it's much stricter.
It should still match whatever it generated from a ~~~ or ~~~~,
but no longer the e.g. not-real signatures we were doing in
our tests. Also had to update our tests, because signatures
change depending on anon. So I had to generate all the users.
And fix some of the signature formats used in the tests.

Bug: T75426
Bug: T87852
Bug: T75366
Bug: T78424
Change-Id: Ibeff36397129fdd5d376f3668a23a45f9a014525
2015-02-05 20:45:19 +01:00
jenkins-bot 028388aa6c Merge "Bug: Fails signature detection with extra long signature" 2015-01-31 01:13:43 +00:00
Erik Bernhardson 438a7d5bb4 Prevent undefined index warnings
In some languages the \w+ does not match the characters used
when translating UTC and the regular expression attempting to
match the timezone fails. Testing in prod wikis where this fails
such as ne.wikipedia.org shows it still works, it just generates
a more generic regular expression.

Since the overall process still works acceptably on the wikis outputting
warnings this patch just adds a guard to prevent the warning and does
not attempt to fix the underlying issue.

Bug: T76558
Change-Id: If8e1ddd2d642b042cc24c51d5ba5aa8b34bc9552
2014-12-09 01:43:03 +00:00
Erik Bernhardson 2578c37d6c Bug: Fails signature detection with extra long signature
There were two different circumstances that could trigger echo's signature
detection to fail: multibyte characters in signature, and signatures near
$wgMaxSigChars limit that expanded past the limit due to wfEscapeWikiText().

This patch adjusts to use mb_substr to appropriatly handle the multibyte
characters, and adds a couple extra charactesr to $wgMaxSigChars to allow
for wfEscapeWikiText().  This isn't perfect, but a stricter implementation
would require much more work than i think we should spend here.

Bug: 73426
Change-Id: Ic51c2bc2a08600f188db13a9a0537f1321c9a655
2014-12-03 11:33:39 -08:00
Erik Bernhardson 347d68807f Support joins via EchoBatchRowIterator
Change-Id: Ib52e431edc2d05c4d033163bd64e28e127cad5f7
2014-11-24 23:23:09 +00:00
Erik Bernhardson 4549a312cc Bug: parenthesis are allowed in namespaces
Bug: 73366
Change-Id: I7304046cfa7eda6356b2c61e716d76330cf1ee56
2014-11-13 11:52:06 -08:00
jenkins-bot 2946919a03 Merge "Limit Echo mentions to 20 users" 2014-10-27 15:40:38 +00:00
jenkins-bot ffe55d81b4 Merge "I-2. Change the default number of Flow Messages in the flyout" 2014-10-27 14:50:19 +00:00
jenkins-bot c5559da8cb Merge "Detect signature using Title class" 2014-10-23 21:25:37 +00:00
jenkins-bot d495389a1c Merge "Use MailAddress::newFromUser()" 2014-10-20 17:25:11 +00:00
Erik Bernhardson b552201829 I-2. Change the default number of Flow Messages in the flyout
Previously this was loading 25 unread notifications.  If less than 25 were
found it would add in 3 read notifications so it wasn't empty.

This patch changes things around to always return 25 notifications if the
user has at least that many.  Instead of backfilling a static count of 3
we backfill 25 - $count with previously read notifications.

Change-Id: I723316486216d7b9dacfcc9765c1a8212e973518
2014-10-15 10:08:00 -07:00
Erik Bernhardson 44ec374632 Limit Echo mentions to 20 users
Bug: 50082
Change-Id: I7eb0f41fc9b0e3a396383a1adaecf51dc883217f
2014-10-07 22:27:20 +00:00
jenkins-bot ce51ecb8f0 Merge "Add reverse chronological ordering to unread notifications" 2014-10-07 18:01:48 +00:00
Erik Bernhardson 8da75c3a50 Detect signature using Title class
Currently echo attempts to find a signature by looking for a series of
strings starting with what it thinks are the current aliases of NS_USER
and NS_USER_TALK.  This has shown to be error prone, see the linked bug
for how a change to ru.wikipedia.org/wiki/Mediawiki:Signature broke
mention notifications.

Patch switches things arround to pull wikilinks out of the text and run
them through the Title class.  The results of this parsing are checked
for NS_USER and NS_USER_TALK, giving a much stronger guarantee of finding
translated namespaces.

Bug: 71353
Change-Id: Ib0d0f4e068339d2fd28761087c05f5a1acb3c1fc
2014-09-30 19:06:16 -07:00
Erik Bernhardson 0cfe5ca269 Don't fatal if a notification fails to initialize
Log info to Echo debug log group instead.

Bug: 71489
Change-Id: I4a7c669ef55936e3c683d268041d828b54a5a9b2
2014-10-01 01:11:30 +00:00
bsitu 371abf6430 Add reverse chronological ordering to unread notifications
Has a performance impact, so this depends on deploying the job
that deletes older notifications.

Bug: 69919
Change-Id: Ia485c853d1b04c3c85e25e6a12f5060a046e9b11
2014-09-29 23:14:42 +00:00
bsitu 55ef794d74 Add job to keep user notifications in reasonable volume
Change-Id: I4d4fa4c987a1732e5e29536a7669e28c34d4ab18
2014-09-29 23:10:18 +00:00
Kunal Mehta d196cbf94e Use MailAddress::newFromUser()
Change-Id: I7c6ce9aade27dae66732274979d8baa9eeb0b17c
2014-09-21 18:25:14 -07:00
Erik Bernhardson 0f36db727e Hygiene: Convert value to constant and clarify outlier use case
Change-Id: I7a82fa99b41362368a9f58cfb84241cca97bfb15
2014-09-11 20:07:29 -07:00
Erik Bernhardson 97417af20c Mark notifications as read if they fail rendering
The only issue is that the badge is not up to date till
you refresh the page, I think that's fine for now

Change-Id: I585b4cc185bf859ddb06829df75309ff3d56d8b8
2014-09-08 14:22:10 -07:00
Nemo bis 940ea72ab5 Remove trailing whitespace, followup 0b1275b9e3
Change-Id: I1d48f9c3c0af68496bf0672c76202bf00063cb34
2014-08-28 12:06:06 +03:00
Erik Bernhardson 57130aadf1 Hygiene: Move master/slave code to MWEchoDbFactory
Change-Id: I1b36bab2126b12359b00c5623885117ef17d0768
2014-08-22 14:01:57 -07:00
Matthias Mullie 411f0151dc efLogServerSideEvent is deprecated
Change-Id: I3fb95ff95dd9371d55285e64a37b32cec35b029a
2014-08-21 15:19:27 +02:00
jenkins-bot abb7655297 Merge "Add support to retrieve unread + read notifcations for section" 2014-08-20 17:37:51 +00:00
bsitu 8e2de3f02b Add support to retrieve unread + read notifcations for section
Change-Id: Ife3750400315f545f5f3e54ac9847f56c2efe9be
2014-08-19 16:54:48 -07:00
jenkins-bot 6ae565e1e8 Merge "[5] Code cleanup and miscellneous fix" 2014-08-19 22:20:14 +00:00
jenkins-bot 5f27ea2c8d Merge "Add waitForSlave() wrapper function to Echo Database wrapper" 2014-08-19 20:16:18 +00:00
jenkins-bot e11614bc85 Merge "[4] Add revision local cache holder object" 2014-08-19 17:58:28 +00:00
bsitu 25d6d71a61 [5] Code cleanup and miscellneous fix
* cache function result in local variables

* Update the logic of generating notification date header

Change-Id: I04c3ed853076f17c819da8f27bfdb169e99b2a3a
2014-08-19 17:58:07 +00:00
jenkins-bot 51dda5473c Merge "[2] Add title local cache holder objects" 2014-08-19 17:57:22 +00:00
bsitu 55fb42b681 [4] Add revision local cache holder object
Change-Id: I85f68b3ca269c711a8a177e013bd8eba38833444
2014-08-19 10:28:38 -07:00
bsitu f781a594d4 Add waitForSlave() wrapper function to Echo Database wrapper
Plus a couple of comments update

Change-Id: I1678cd1cfde88806a15b280532ecfc6cf2672e38
2014-08-18 15:00:46 -07:00
bsitu 1270da19e8 [2] Add title local cache holder objects
Core titleCache doesn't do what I expect it to do, issuing
Title::newFromId( 1 ) mulitple times would issue multiple
idential queries to the database. It doesn't return what's
already in the cache.

The goal of this patch is to batch load titles via newFromIDs,
and save the number of mysql queries

Change-Id: I8fe767ac2669e67bdf7d17eecccfc0dcb6b5fc7d
2014-08-18 13:52:40 -07:00
jenkins-bot 7fac1c3ee8 Merge "Use batch queries for users-watching-title" 2014-08-18 08:33:13 +00:00
jenkins-bot 124cbbd036 Merge "Auto mark notification as read if it doesn't have target page" 2014-08-15 18:35:27 +00:00
Erik Bernhardson 1667e25854 Use batch queries for users-watching-title
The new locateUsersWatchingTitle implementation could end up returning
thousands of users, currently on enwiki there are 25 titles with more
than 10k subscribed users and aprox 550 titles with more than 1k subscribed
users.

This switches the user collection to an iterator based implementation so that
we no longer need to have the entire users list at any one time.

Change-Id: I3d3fa9328f348bb48682d3658622952ce82d3925
2014-08-15 10:44:55 -07:00
bsitu 1cc0f83829 Move 'notification count refresh' logic out of mapper
Change-Id: I4c10b66e537dea5f9c5bc7ccba8f558cafd103d7
2014-08-15 16:02:23 +02:00
bsitu 61fea56641 Create TargetPage data if specified by EchoEvent
Change-Id: I71039eb03b4b7e617ce03d515a6d51c4f3666ab8
2014-08-15 00:16:41 +00:00
bsitu 7be9d68219 Auto mark notification as read if it doesn't have target page
The special page will now not auto mark a notification as read if
it has a target url. Currently no notifications have target urls
but this will be changed in a later patch...

Change-Id: I9bd71d59391189d5d761ab5f1c84af0bc3554be0
2014-08-15 00:11:57 +00:00
bsitu cac4050990 Mark notification as read upon visiting a page
Change-Id: I84493fbf742acd90645d484d42f049796a5e48ee
2014-08-14 14:10:55 -07:00
jenkins-bot f001382328 Merge "Add TargetPage model and mapper to notifications" 2014-08-14 20:18:44 +00:00
bsitu ff0784e7da Add support to mark all as read for "sections"
This also updates the way how mark all as read works

Change-Id: Ifb7b1b7b7feb4a5af65c79bb16b91a5a9c70166c
2014-08-13 17:16:19 -07:00
bsitu a7fbb53584 Add method for fetching unread notifications
* This will be used for the unread notifications in message overlay

* This will be used to "mark all as read" based on sections ( event types ),
  since we can't do updateJoin.  We could set a max update limit like 3000,
  and use this to do "mark all as read", this would handle majority "mark all as read"
  use cases. This would also prevent updating big amount of data on a web request

Change-Id: I2a9103a73d0aa91a52d5c0233e946a0ef979f96d
2014-08-13 13:10:38 -07:00
bsitu 029668423d Add TargetPage model and mapper to notifications
This will be used for marking a notificaiton as read when
a user visits a target page. The new table should keep the
volume as low as possible for fast data loopup. records
should be removed from the table once it's marked as read.

Change-Id: I605cbc79adfc12d22bd889c5bb513d05c479fe6e
2014-08-13 00:48:07 +00:00
bsitu 4bf88aa741 Hygiene: Add abstract model and mapper for notification
* Shared function can be put in the abstract class and this also enforces some interface methods 

* Initialize a default dbFactory when it's not passed to the mapper

Change-Id: I1033dafaa90a1f683fbe9ad69bed04f4844e357b
2014-08-13 00:11:15 +00:00
Kunal Mehta 631d0252f7 Fix Echo breaking
Bug: 66992
Change-Id: I7add48c55e04ef4ea5d0f537c6a1a1ba959f628e
2014-08-08 21:35:58 +01:00
bsitu 427b7e2c38 Add support for splitting notifications into alert & message
Change-Id: I8eeeeb9a7a1539a258bc42584274897f9e7dc775
2014-08-05 14:50:54 -07:00