For a moment I doubted if we handle this case correctly, but in fact
I didn't botch that code *this* badly.
Change-Id: I5a9d142e4bd97ac40aa388bb43b65ab1286e3f18
Documentation:
https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests_for_extensions#Two_types_of_tests
We can do this because the tested methods do not depend on any globals
or on MediaWiki being installed.
In addition to being the new hotness, MediaWikiUnitTestCase allows the
test classes that use it instead of MediaWikiTestCase to start up much
faster. In my testing, running this test case individually now takes
0.35s, compared to 1.1s before.
Try:
* With new code:
time php tests/phpunit/phpunit.php extensions/DiscussionTools/tests/phpunit/unit/CommentUtilsTest.php
* With old code:
time php tests/phpunit/phpunit.php extensions/DiscussionTools/tests/phpunit/CommentUtilsTest.php
Change-Id: I771b1f3d101a394ee869e42547d9ae7839397752
* Ignore rendering-transparent nodes between discussion comments.
* Improve isRenderingTransparentNode() so that <link> nodes
representing TemplateStyles are not considered transparent,
otherwise this would undo ae920b831f.
Using a regexp from Parsoid.
Bug: T272746
Change-Id: I0b3c3251156ba6c4826abf5ba44ea93f80ebc01d
Add yet another tree walking utility: CommentUtils::linearWalk().
Unlike TreeWalker, it allows handling the beginnings and ends of nodes
separately – kind of like parsing a XML token stream, or kind of like
VisualEditor's linear model.
(Add unit tests for this utility. The simple.html test case is copied
from [VisualEditor/VisualEditor]/demos/ve/pages/simple.html.)
Use this utility to stop skipping when we reach either a closing or
opening block node tag. Previously we'd skip over such tags inside
nested "transparent" nodes (like <a>, <del>, or apparently <font>).
Bug: T271385
Change-Id: I201a942eb3a56335e84d94e150ec2c33f8b4f4e0
As a result of 0fc71f60cd, "empty" text
nodes (containing only whitespace) at the end of the comment may be
inside the comment's range, and trying to ignore them caused the
ranges not to match and the frame not to be detected.
Now the code works whether they're inside the comment's range or not.
Add a test case for wrapped discussion comments with HTML comments and
with whitespace.
Bug: T250126
Bug: T268407
Change-Id: I2217ff5a635fd1c9c9e803f46795b1bfb3d17535
While working on T270009, I noticed that <style> and <link> nodes
are treated differently, which seemed weird. Rewrite this again,
hopefully this is the last time.
The changed test cases also involve <area> and <input> nodes,
and the new results make more sense to me.
Bug: T264116
Change-Id: I3af90c84768a4b3dc53446927f4dba6f72175a2f
We've recently decided that we want to "extend" comments until
the end of the paragraph (e36dc8e78a,
d0ae6c4e44).
However, we still had this special case that did the opposite: it
ensured that if a comment ended in the middle of a text node, the
comment would not be extended to the end of the node. Remove it.
Note the change in the test file signatures-funny-formattedreply.html,
which actually covered this case specifically.
Change-Id: Id1384bb0c6e1a5f0c70f55efcb4caa240f230f07
The end marker is skipped forward until an open or close
block tag is reached. In tree traversal terms this means
moving either to the next sibling, or the parent (to skip
over close tags).
Bug: T256033
Change-Id: Iaa2c588698790d576ac4f9ecc126f58a082ef6b3
The general rule is that comments start after their preceding
thread item, but when that is a heading we should skip past
the entire <h[1-6]> node to avoid making section edit links
part of the first comment.
Bug: T267988
Change-Id: Ia7f1b27e0a69a9aab7c7da743bf8549479304096
As CommentFormatter no longer needs HTMLFormatter, remove
the inheritance and make addReplyLinks a static method.
Testing locally this is marginally slower, going from 2.55s
to 2.9s for the CommentFormatterTest case.
Bug: T266317
Bug: T267973
Change-Id: If69749cae678a1647a138d782a32032189f55cec
After recent changes allowing ThreadItems to have IDs, they can now
also have warnings about duplicate IDs.
Bug: T267035
Change-Id: If3edfe34e6e29741e29fac8946a3c88badc4ab7f
Use the same logic for marking ranges in the document, and ensure
that the heading range does not include section edit links or
section numberings.
Change-Id: I782caafc34fee2a822b0a17b24dd6b9528202eca
We avoided fixing these because it causes changes in just about all of
the test data, which is annoying when reviewing or blaming changes.
But the previous several commits also caused changes in just about all
of the test data, so we might as well do this too.
Change-Id: I83b64d83b6f12c04dc06c0cadff7cdd89417e137
To avoid old threads re-appearing on popular pages when someone
uses a vague title (e.g. dozens of threads titled "question" on
[[Wikipedia:Help desk]]: https://w.wiki/fbN), include the oldest
timestamp in the thread (i.e. date the thread was started) in the
heading ID.
Bug: T264478
Change-Id: If918bfd5e025248923d1939bc86916697ead95a0
Sequential numbers aren't great because they change when an earlier
comment is archived. Parent comment/heading IDs should change less
often.
This also makes much more sense for disambiguating subsections,
e.g. a dozen identical ===Votes=== sections for a dozen proposals.
Bug: T264478
Change-Id: I466454984fd919ebef35f2b37ddb5d86dc842996
Our threads now also contain all replies to their sub-threads.
This is similar to how sections work in MediaWiki, where the parent
section also contains the content of all the lower-level sections.
We're going to need this for notifications about replies in a thread.
Bug: T264478
Change-Id: I241fc58e2088a7555942824b0f184ed21e3a8b6f
Previously, only comments could have IDs, because we only needed IDs
for replying. But we might also use them for notifications soon.
Bug: T264478
Change-Id: I1bcad02bf17ab54bc5028a959543c10f0430836b
I haven't really reviewed the outputs, but at least a) they don't
crash b) they will fail if the output suddenly changes (which could
cause problems due to caching).
Bug: T252555
Change-Id: I1bbcbc5dd17ce1e24b3622062f5e8df4baf5f389
Also, add tests covering this and the previous bug fixes in this code
(T259818, T261706).
Note that the test data added in tests/cases/ doesn't exactly match
the entire configuration of the wiki, only the parts we want to cover.
This is unlike the data in tests/data/, which was literally copied
from the relevant wikis, and which is used as input for other tests.
Bug: T265500
Change-Id: I29a59a5952f6dc9fb5910434bb6bcc9dcdaa01a9
The wikitext parser outputs `<p><br></p>` for empty paragraphs, so we
need to ignore `<br>` tags when searching for an "interesting" node
that marks the beginning of a comment. Otherwise the empty paragraphs
mess up the detection of indentation levels.
Bug: T264116
Change-Id: I84a97ab577baa7336b78935ccdc48041ecfc231a
The tests pass either way because this trivial difference is ignored,
but it's annoying when trying to update changed tests.
Change-Id: I7ad1acad43e30a63843cfda4d0d08ff663ef3252
* Export parser data (date format, digits, timezone names, and
messages for weekday/month names) converted to language variants
* Update the parsers to try matching using every variant, in case
the page is displayed in non-default variant (and to avoid
problems with incomplete variant conversion)
Bug: T259818
Change-Id: I04d73992cd31ce06fa79f87df0c0a53d7efc3c58
This is primarily to cover the handling of localised digits,
which previously wasn't being tested, leading to T261706.
Bug: T261706
Change-Id: I9de7f01f77e767e9048c85604b559af4bca0de91
* Remove 'wgMetaNamespace' and 'wgMetaNamespaceTalk', the same data
exists in 'wgFormattedNamespaces'.
* Rename 'wgContentLang' to 'wgContentLanguage', to match its real
name in JS config. MediaWiki doesn't use 'wgContentLang' anywhere,
although the related PHP global is called $wgContLang.
* Document how I made these files, previously only mentioned in the
commit message of e9c401e3aa.
Change-Id: I67f962812c155aedf41154e0d837e7feb5af972d
The PHP code incorrectly assumed that the digits are single-byte in
UTF-8, which is never the case (except for 0-9).
The JS code worked correctly because it uses UTF-16 strings, so the
bug would only affect non-BMP digits there. This was noted in a TODO
comment, but we overlooked it when reimplementing in PHP.
Instead of a string of 10 characters, use an array of 10
single-character strings.
Bug: T261706
Change-Id: Ic5421382474c88f003424799c53ff473d99cce92
When a comment ended before the end of a paragraph, the next
comment would begin right there in the middle of the paragraph.
This could result in the detected indentation level of that
comment being incorrect, and replies being inserted in wrong
places, as seen in the 'signatures-funny' test case.
The code moved to the parser was previously repeated twice in
addListItem() and addReplyLink(), which should have been a hint
that something isn't quite right.
Also, fix the code guarding against overlapping signatures,
now that signatures may not be at the end of a comment.
Bug: T260855
Change-Id: Ic26a87642f8a15d5de2f7073d4d8176b299c7f94
Expand the 'signatures-funny' test case with more examples, which
don't behave correctly.
Follow-up commits I04a8ea09401e06f2d4bb1f226f17eb528b29ed95 and
Ic26a87642f8a15d5de2f7073d4d8176b299c7f94 fix them.
Bug: T255738
Change-Id: I0fdd8bdf11b497ffeed37c37953c5730f6e4f3b7
Previously, parser would output offsets that don't exist in their
containers, because we were pretending that entities are parts of
their neighboring text nodes.
Turns out it's much easier to do it right when going backwards.
Change-Id: I9bccca2d403f1a976ae517449989170cdd99721e
Follow-up to ccd9e411d2.
* Fix variable name in CommentTestCase::overwriteHtmlFile()
* Overwrite before assertions, because they abort execution if they fail
Change-Id: I5bba016ba93f9dd1994325ae82c3105ba11cf033
The latter results in lots of extra HTML entity encoding.
The former is built by the Parsing team and appears to result
in no unexpected changes elsewhere in the document.
As Parsoid's selser relies on HTML fragments being byte-for-byte
equal, these changes were resulting in wikitext normalisations
in untouched parts of the document ("dirty diffs").
Bug: T259855
Change-Id: Ib3cb605911e690ec3e8c2f9df25fd1a2e2849d7e
When adding a reply, we take a node at the end of the previous comment,
compare that comment's indentation level to the expected indentation level
of the reply, and add (or remove) that number of wrapper lists.
The existing code did not consider that comments may have lists within
them, and so the indentation of that node may not match the indentation
of the comment.
Bug: T252702
Change-Id: Icc5ff19783d2b213bff99f283cb0599a8b5c1ab4
Previously we preferred that, but used '*' (<ul><li>) when the parent
comment or the previous reply also used it.
Bug: T252708
Change-Id: I3abf606da6693905764f1be745fad999fdf57fbe
This is similar to the code we already have in JS tests, but instead
of printing to the console where you have to copy-paste from, it just
overwrites the files.
Also, update all of the expected results by this method.
Changes in the expected outputs:
* In JSON files, the "warnings" are now always in the same place
regardless of the type of the warning.
* In all HTML files, self-closing tags now include the trailing slash,
some characters are no longer encoded as entities when not necessary,
and attributes may be single-quoted when that makes them shorter.
* In Parsoid HTML files, the header is no longer terribly mangled.
Other notes:
* CommentParserTest.php: Change the output of serializeComments()
to be in similar order as in JS, to reduce the diffs in this commit
and because it's a better order for humans.
* modifier.test.js: Remove some hacks that were working around small
inconsistencies between the previous expected outputs and the actual
outputs.
Change-Id: I9f764640dae823321c0ac35898fa4db03f1ca364
* Remove the existing approach for detecting signatures that only
worked in source mode; remove autoSignWikitext()
* Use the same approach for auto-signing in source mode as we have
already used in visual
* In both modes, detect whether the user has already typed a signature
at the end of their comment in the modifier, and if so, don't add a
signature
* Add test cases for the detection
Bug: T255738
Change-Id: I791d3035cb1ffc33ce3966d4617a25d08700c35b
* Pass rootNode to the constructor
* Rename getters to match CommentItem/HeadingItem/ThreadItem
value classes.
* Always build the thread tree so CommentItem's always have
and ID and replies/parent.
Change-Id: I508be9534de59016ff806e3d84edcbb1c76cb0c6
Rather than the <body> node, we were passing <body>'s first child.
Current implementation of CommentParser::getComments() doesn't fail
the tests in spite of this because the XPath query incorrectly returns
results relative to the document's real root node, but these tests
would start failing after I2441f33e6e7bad753ac830d277e6a2e81ee8c93d.
Follow-up to 3e6ab2c4d2.
Change-Id: Ic26e0a1ee4443987e215c5f26ef1f084ccd0b40b
The expected HTML was wrong, a '<br />' tag inside 'data-mw' was
somehow turned into '<br ></span>'. No idea how that happened.
Something must be wrong with the HTML parsing in JS tests, which
were used to generate this file.
Change-Id: I69caa68fe70e706df81e8adf29889254704f601e
I was wondering if the different approach to childIndexOf()
implemented in PHP in b8d7a75c34 would
be faster in JS as well, and yes, it is.
Our test suite now takes (on my machine):
* Chrome: 8337 ms → 7355 ms (average over 5 tries)
* Firefox: 5321 ms → 5044 ms (average over 5 tries)
Change-Id: I71963eeb92dcea9bfd59cbf01a7aa0b7de5d9cf1
When there is a wrapper element whose range matches the range of
a comment, any replies will now be added outside of that wrapper,
instead of directly after the comment (inside the wrapper).
Bug: T250126
Change-Id: I6b42c4db019ae998e91eebd324f9cbd2aa791b4f
It was useful when I was debugging those parts of the code, but now
it's usually annoying.
The warnings can still sometimes be useful for understanding how the
tool parses some discussion, though. To keep that functionality, add
displaying warnings for each comment in the debug mode.
Change-Id: I2d218a8a394f179bcc0990ff988a0567c275ccf2
