Commit graph

107 commits

Author SHA1 Message Date
Bartosz Dziewoński c1f4668806 Change CommentParser and ImmutableRange to use offsets in codepoints instead of bytes
The PHP DOM extension measures lengths and offsets in Unicode codepoints.
Our PHP code used UTF-8 bytes, causing some offsets to be slightly off.
Now it mostly uses Unicode codepoints as well (we're forced to use bytes
in a few places, because preg_match returns offsets in bytes).

In practice, this had no visible effect to the user. It caused the
markers `<span data-mw-comment-end="..."></span>` to be placed at
the end of their container instead of the correct position when the
timestamp contained multibyte characters (e.g. "ź" in Polish); but
the correct position is usually at the end of the container anyway.

In the test cases, the only difference is placing these markers before
a trailing line break inside `<p>...</p>` tags rather than before it.

The patch also accidentally fixes another bug, where element nodes
with no children (mostly <img>) were incorrectly excluded when calling
cloneContents(), because they were treated as if they were text nodes.

Change-Id: Iccdccf1078598f4b62cab96225e9c85a4c0e93ee
2021-09-27 19:04:16 +00:00
jenkins-bot abd6c2fedd Merge "Enhance Echo user talk edit and mention notifications" 2021-09-24 02:17:57 +00:00
Bartosz Dziewoński 435b0c65c7 Enhance Echo user talk edit and mention notifications
If the user talk edit or mention coincides with exactly one new comment:
* Change the primary link to be a direct link to the comment
* Add a text snippet to notifications that don't already include one
  (user talk edits that are not new sections).

This is done for all such notifications, regardless of whether anyone
has topic subscriptions enabled.

Bug: T281590
Bug: T253082
Change-Id: I98fbca8e57845cd7c82ad533c393db953e4e5643
2021-09-20 15:05:42 +02:00
Bartosz Dziewoński a6a547f2b2 Add some tests covering ThreadItem::getHTML() and related methods
* ThreadItem::getText
* CommentItem::getBodyText (used when generating notifications)
* ThreadItem::getHTML (may soon be used in API)
* CommentItem::getBodyHTML (may soon be used in API)
* ImmutableRange::cloneContents (the common implementation for all
  of the above)

The outputs are only lightly reviewed. This is mostly meant to
document the current behavior rather than the expected behavior,
to avoid making unintentional changes while refactoring.

Change-Id: I14471ee4969aa3d0b5577d9de2a6d4462fab4d09
2021-08-24 07:54:09 +02:00
Bartosz Dziewoński 47510a22f3 EventDispatcher: Fix ignoring level 3+ headings
The code (prior to d25825a754) assumed
that level 3+ headings would always follow a level 2 heading or the
placeholder heading, but we don't generate a placeholder heading if
there are no comments in section zero.

Add more tests to confirm that comments under level 3+ headings (that
are not sub-headings of level 2), and level 1 headings, are ignored
when generating notifications, and do not mess with normal headings.

Bug: T288775
Change-Id: Ic57b56752a4797cb01234f66e0ed7b849752bd70
2021-08-16 15:42:06 +00:00
jenkins-bot 56f4dbbf86 Merge "Test cases for interactions with events generated by base Echo" 2021-08-03 23:18:46 +00:00
jenkins-bot d14f31fd08 Merge "Improve notifications for comments posted in close succession" 2021-08-03 23:17:55 +00:00
jenkins-bot 352a4d0555 Merge "Test cases for comments posted in close succession" 2021-08-03 20:44:02 +00:00
jenkins-bot 1f4706a308 Merge "Recognize links to add a new topic that use Special:NewSection" 2021-08-02 17:42:30 +00:00
Bartosz Dziewoński 80704b6e80 Test cases for interactions with events generated by base Echo
Adding test cases in a separate commit to make it easier to review how
the test results change after I98fbca8e.

* For mentions, the 'mentioned-users' extra parameter is copied to our
  event (which is then used to avoid duplicate notifications).
* For user talk page edit, nothing special happens right now (we use
  the target page title to avoid duplicate notifications, but this is
  not apparent from the test case, since page titles are not present).

Bug: T281590
Bug: T253082
Change-Id: I153e7735f63f1e2643ed881281d807313cd699c3
2021-08-01 12:27:33 +02:00
Bartosz Dziewoński a5099739a6 Improve notifications for comments posted in close succession
In case 4 and case 6, no notifications are expected. In all other
cases we now get the expected notifications.

Bug: T285528
Change-Id: I9e813bb3a053bc1232783f9eae1ad75672b4fa7e
2021-08-01 12:27:33 +02:00
Bartosz Dziewoński 78cb03c471 Test cases for comments posted in close succession
Adding test cases in a separate commit to make it easier to review how
the test results change.

As expected, in every case, no notifications are generated right now.

Bug: T285528
Change-Id: I25308754112c521d2db8c54ef0c82373456d9e31
2021-08-01 12:27:33 +02:00
Bartosz Dziewoński d0e4aeaecb Fix notifications when new comment is under subheading
The user interface only allows you to subscribe to level 2 headings.
But we would generate events for whatever heading was the closest,
If it was e.g. level 3, no one would receive that notification.

Now we generate events for the closest level 2 heading, or we don't
generate the event at all if there isn't one (if the only headings are
of level 3 and below, or level 1, or if the comment is added before
the first heading on the page).

Bug: T286736
Change-Id: Iae99853070e353ab81c9cc29ef1d53c877adfc66
2021-07-24 05:28:10 +02:00
Bartosz Dziewoński 801b57b0f4 Add PHPUnit integration tests for EventDispatcher
Bug: T286608
Change-Id: I711483be80d455f4439e96d37844ee4552619a92
2021-07-24 05:28:04 +02:00
Bartosz Dziewoński 4ebf05d802 Recognize links to add a new topic that use Special:NewSection
Bug: T277371
Change-Id: I40a13d8bf87bcd3ecea1427b444b6b7b621213c4
2021-07-22 22:25:11 +02:00
Bartosz Dziewoński 4ebe4b29bb Only show [subscribe] links on sections that contain at least one comment
Bug: T285796
Change-Id: I48264d55464fa6bce56b47fc1075f5d348101676
2021-07-13 02:35:15 +02:00
Bartosz Dziewoński 9c8d709b8a Use placeholder localisation messages in CommentFormatter tests
Otherwise they will fail whenever translations are updated
(and they are failing right now).

Change-Id: I849c57b86d36fb6c7739cc31a74df741e08462f4
2021-06-02 21:46:36 +02:00
Ed Sanders 6a24ceaeca Subscribe/unsubscribe with plain text links
Bug: T279149
Bug: T279151
Change-Id: Ie7d46ea2e8d458fcdad4f91bb89ba038969f6b62
2021-06-01 20:37:47 +02:00
jenkins-bot 90528f2b49 Merge "Fix adding comments in lists containing <dt> tags" 2021-05-25 19:19:49 +00:00
Bartosz Dziewoński ffd680ee7f Fix adding comments in lists containing <dt> tags
The issue occurred when replying to a comment consisting of multiple
list items, starting with a <dt> (instead of the expected <dd>), so
that the comment is considered to be unindented.

Modifier tried to add the reply directly inside the list (<dl>) rather
than inside the last list item (<dt>), which caused it to be confused
about indentation levels and try to un-indent more times than there
were indentations.

The simplest solution, given the existing code, is to add the reply
outside the list instead, in a new list. This results in a "list gap"
(<dl><dt>...</dt><dd>...</dd></dl><dl><dd>...</dd></dl>), but I think
it's acceptable for this rare case.

There are separate tests cases for old Parser and for Parsoid HTML,
because they parse the original wikitext differently (with the old
Parser producing HTML with a list gap too).

Bug: T279445
Change-Id: Ie0ee960e7090cf051ee547b480c980e9530eda51
2021-04-21 16:00:07 +02:00
Bartosz Dziewoński 5103e651be Add tests for CommentFormatter::postprocessTopicSubscription
Change-Id: Ief9648b8805fadcc170c54b627eb669cc8b907b6
2021-04-21 11:57:25 +00:00
Bartosz Dziewoński a3f665e816 Remove <header> tags around headings for compat with MobileFrontend
We added it because the initial designs for the subscribe action were
much easier to implement like this, and topic "containers" (T269950)
would have required it.

However, the latest design of the subscribe action will not need it
(T279149), and topic containers are still very far away, so let's
remove it for now.

Bug: T280433
Change-Id: I21a23e9bea43f24d265750926fbd62b99038d3f1
2021-04-19 17:47:43 +02:00
Ed Sanders eb7e682d2f Topic subscription front end
Bug: T276996
Change-Id: Ifb62c04c2a0ea8399749b22021d6a1c5a079bf94
2021-04-06 23:28:28 +02:00
Bartosz Dziewoński 42ce942c86 Introduce comment "names" to identify comments across revisions/pages
The existing comment IDs can't be used to find the same comment on
a different revision or page (when it's transcluded), because they
depend on the comment's parent and its position on the page.

Comment names depend only on the author and timestamp. The trade-off
is that they can't distinguish comments posted within the same minute,
or in the same edit, so we will still need the IDs sometimes.

Prefer using comment names when replying, if they're not ambiguous.
This fixes T273413 and T275821.

Heading names depend on the author and timestamp of the oldest comment.
This way we don't have to detect changes to the heading text, but we
can't distinguish headings without any comments.

Bug: T274685
Bug: T273413
Bug: T275821
Change-Id: Id85c50ba38d1e532cec106708c077b908a3fcd49
2021-03-23 16:08:42 +00:00
Ed Sanders c4de603ef9 Give comments IDs so they can be scrolled to with hash links
Bug: T265268
Change-Id: Idb985ed38bdb74e23cb7840899a61dc919f05f6f
2021-03-20 15:43:23 +00:00
Ed Sanders 4a0802065c Make IDs (to be used as URL hashes) wikitext safe
* Use hyphens instead of pipes a separators
* Use underscores for spaces in usernames

Change-Id: I6efd9739fc73e45002e50e64c43ce0de1c2f1239
2021-03-18 20:45:21 +01:00
Ed Sanders ece8ff69ff Change dt- class prefix to ext-discussiontools-
Longer, but follows the style guide and less likely to conflict.

We need to account for init classes in the cache being around for
a while.

Change-Id: I738bc93393850db320fdbda2b003ca8ac40556da
2021-03-13 14:42:39 +00:00
Bartosz Dziewoński 44f2209abf Trim signatures when added in an empty existing node, too
Add unit tests for appendSignature().

Bug: T276612
Change-Id: Ic44c52f4d54492e092f9396c626380e2637b6f0f
2021-03-08 23:38:46 +00:00
Ed Sanders ccc19d8df2 Add 'href' to reply links for better compatibility with skins
Change-Id: Id948d576bbe5a6d43c4f8a06cdb2cd8ad19be193
2021-03-09 00:04:53 +01:00
Bartosz Dziewoński 5a07139249 CommentFormatterTest: Avoid re-serializing the HTML
The code we're testing already produces a string of serialized HTML,
no need to parse and re-serialize it.

Also, we recently learned that the precise format matters here
(T274709), and now this test *actually* covers the fix for that bug.
Follow-up to 5b26e9664b.

As a downside, this test might now spuriously fail if the format of
the output of Parsoid's XMLSerializer changes. Hopefully that won't
happen too often.

Change-Id: I69b514f545e47dcb437fb39a83edb8e2f19ed99b
2021-03-01 21:30:28 +01:00
jenkins-bot 0eb37a87df Merge "Don't detect comments within quotes" 2021-02-28 22:56:20 +00:00
Bartosz Dziewoński 024a978ffd Don't detect comments within quotes
Bug: T275881
Change-Id: I8f7a4279837bd95ebf5b604ff350c0a3f29c2c05
2021-02-28 22:49:48 +00:00
Bartosz Dziewoński efe95494a8 Improve signature detection to handle formatting on the timestamp
Now it detect signatures generated by en.wp's {{Undated}} template,
and signatures of people who do weird stuff to the timestamps.

Bug: T275938
Change-Id: I27b07f6786ca5433a3c02a5fe68e4716d41401bb
2021-02-27 02:33:30 +01:00
Bartosz Dziewoński af082908a5 Improve merging multiple comments on one paragraph
The horrendous 11-line if() condition did not correctly handle
signatures wrapped in inline formatting markup, like <small>.

Instead, implement this logic in the code for skipping to the end
of a paragraph, which didn't exist yet when that condition was
added, but seems like a much better place to check this now.

Bug: T275934
Change-Id: I5cccff889b5e15b5f8fde0538bf4bccb22e762cf
2021-02-27 02:21:36 +01:00
Bartosz Dziewoński 9fafe72fc7 Broken test cases for comments with double signatures
Bug: T275934
Change-Id: I00edc63b723053e7ac7d26ce5fc1dde4b824380d
2021-02-27 00:12:42 +01:00
Bartosz Dziewoński 5b26e9664b CommentFormatter: Fix inserting placeholder heading marker
This code expected $container->firstChild to be a
<div class="mw-parser-output">, but that element is not present
when we're running on HTML to be saved in parser cache.

We ended up inserting the marker inside whatever node was the
first on the page, and if it was a <style> element, both our
marker and the styles would be lost when serializing, like in
6c7a0ca9a2.

When we're running on final HTML, the marker will now be outside
of <div class="mw-parser-output">, but that seems to be fine. Only
early versions of I4e60fdbc098c1a74757d6e60fec6bcf8e5db37c1 had
problems with that (see comments on patchset 41), but it works now.

The added test case also covers the fix for T274709.

Bug: T275440
Change-Id: I38d45dd8686919be51e1d307ded12b0afe185eb5
2021-02-24 20:32:48 +01:00
jenkins-bot d5b2389ffa Merge "Fix replying outside wrappers for partially indented comments" 2021-02-11 00:09:48 +00:00
jenkins-bot 89dd44bb35 Merge "More test cases for comment wrappers (multiple siblings)" 2021-02-09 18:07:14 +00:00
Bartosz Dziewoński 9d2b35828d Fix replying outside wrappers for partially indented comments
Top-level comments that start or end with a list (inconsistent
indentation) would not have triggered the logic for detecting
wrappers.

Bug: T273692
Change-Id: Idcb4eed73e391f5f86eca2eb05cb3cea0d86f30a
2021-02-08 22:18:37 +01:00
Ed Sanders d05109b24d Truncate user generated part of IDs to 80 characters
This ensures that IDs fit in a 255 character database field.

Bug: T273658
Change-Id: I3cfe4fce6a865b4343f0f01121cd696aa5f98b22
2021-02-03 15:04:58 +00:00
Bartosz Dziewoński 0f4db60e72 More test cases for comment wrappers (multiple siblings)
For a moment I doubted if we handle this case correctly, but in fact
I didn't botch that code *this* badly.

Change-Id: I5a9d142e4bd97ac40aa388bb43b65ab1286e3f18
2021-02-03 00:26:52 +01:00
Ed Sanders 47aea0b160 Use tabs for indentation in JSON test files
Change-Id: I1d8f8b33b19bcff249ad08dfe687f87f5e5bf9bf
2021-01-27 00:25:15 +00:00
Bartosz Dziewoński c781b127c9 Handle category links at ends of comments affecting indentation
* Ignore rendering-transparent nodes between discussion comments.
* Improve isRenderingTransparentNode() so that <link> nodes
  representing TemplateStyles are not considered transparent,
  otherwise this would undo ae920b831f.
  Using a regexp from Parsoid.

Bug: T272746
Change-Id: I0b3c3251156ba6c4826abf5ba44ea93f80ebc01d
2021-01-26 04:55:03 +01:00
Bartosz Dziewoński 8f42c74985 Fix skipping to the end of paragraph, now it considers nested tags
Add yet another tree walking utility: CommentUtils::linearWalk().
Unlike TreeWalker, it allows handling the beginnings and ends of nodes
separately – kind of like parsing a XML token stream, or kind of like
VisualEditor's linear model.

(Add unit tests for this utility. The simple.html test case is copied
from [VisualEditor/VisualEditor]/demos/ve/pages/simple.html.)

Use this utility to stop skipping when we reach either a closing or
opening block node tag. Previously we'd skip over such tags inside
nested "transparent" nodes (like <a>, <del>, or apparently <font>).

Bug: T271385
Change-Id: I201a942eb3a56335e84d94e150ec2c33f8b4f4e0
2021-01-18 18:20:20 +00:00
Bartosz Dziewoński 0da00d44be Broken test cases for a big mess of <font> tags
Bug: T271385
Change-Id: I26b08923583593f40a372ef6614524f69781a87a
2021-01-18 18:20:13 +00:00
Ed Sanders 8b71a2b5dc Load site config data in CommentFormatter tests
This fixes missing reply links in arwiki test output.

Change-Id: I24d3b8371a8343c4445c716fadf0692be0924eed
2021-01-08 23:03:33 +00:00
jenkins-bot 8a6bb8efd0 Merge "Ignore outdent templates at the beginning of comments" 2021-01-04 21:48:27 +00:00
jenkins-bot fa9d729728 Merge "Change which nodes are ignored at the beginning of comments again" 2021-01-04 21:47:40 +00:00
Bartosz Dziewoński 6e37a172ae Fix detecting decorative comment frames with whitespace
As a result of 0fc71f60cd, "empty" text
nodes (containing only whitespace) at the end of the comment may be
inside the comment's range, and trying to ignore them caused the
ranges not to match and the frame not to be detected.

Now the code works whether they're inside the comment's range or not.

Add a test case for wrapped discussion comments with HTML comments and
with whitespace.

Bug: T250126
Bug: T268407
Change-Id: I2217ff5a635fd1c9c9e803f46795b1bfb3d17535
2021-01-04 20:31:33 +01:00
Bartosz Dziewoński efccc28b5d Add test case for trailing void tags (<br>)
Bug: T266288
Change-Id: I9385a5b6804fa199327f7af2cfd8275f30727f66
2021-01-04 19:22:18 +00:00