Commit graph

116 commits

Author SHA1 Message Date
jenkins-bot 485f9a4f8c Merge "Add 'id' attributes in the "wrappers" test case" 2020-10-06 22:39:42 +00:00
Bartosz Dziewoński 9775a60ace Add 'id' attributes in the "wrappers" test case
Change-Id: I40ca69586a915590e36dea45a90ab3f332931656
2020-10-02 23:47:47 +02:00
Bartosz Dziewoński ed17f640b6 Ignore other empty-ish things at the beginning of comments
Follow-up to 432a959436.

Bug: T264116
Change-Id: I0685cafab70c7e9d22f504f1a1309c9a28d6f2e1
2020-09-30 23:42:47 +02:00
jenkins-bot f7c7ba3c44 Merge "Ignore empty paragraphs at the beginning of comments" 2020-09-30 18:16:36 +00:00
jenkins-bot 5c61b23be1 Merge "Update test case to match actual output" 2020-09-30 18:16:35 +00:00
Bartosz Dziewoński 432a959436 Ignore empty paragraphs at the beginning of comments
The wikitext parser outputs `<p><br></p>` for empty paragraphs, so we
need to ignore `<br>` tags when searching for an "interesting" node
that marks the beginning of a comment. Otherwise the empty paragraphs
mess up the detection of indentation levels.

Bug: T264116
Change-Id: I84a97ab577baa7336b78935ccdc48041ecfc231a
2020-09-29 22:22:35 +02:00
Bartosz Dziewoński 385a5c1622 Update test case to match actual output
The tests pass either way because this trivial difference is ignored,
but it's annoying when trying to update changed tests.

Change-Id: I7ad1acad43e30a63843cfda4d0d08ff663ef3252
2020-09-29 22:22:35 +02:00
Ed Sanders 6b8312e610 Ignore HTML comments which are more than two lines from a reply
Bug: T264026
Change-Id: I989132d7599a7fa156dba46d87a9ed4b76322c0c
2020-09-29 11:30:03 +01:00
Bartosz Dziewoński f934e9aefd Add integration tests using pages from sr.wp
For testing our handling of language variants.

Bug: T259818
Change-Id: Id25b537fecd789640c209ff7f30e777455a3aece
2020-09-16 22:07:16 +00:00
Bartosz Dziewoński 329df8c953 Parsing discussions converted to language variants
* Export parser data (date format, digits, timezone names, and
  messages for weekday/month names) converted to language variants
* Update the parsers to try matching using every variant, in case
  the page is displayed in non-default variant (and to avoid
  problems with incomplete variant conversion)

Bug: T259818
Change-Id: I04d73992cd31ce06fa79f87df0c0a53d7efc3c58
2020-09-16 22:07:07 +00:00
Ed Sanders d4f67918b2 Skip over whitespace when looking for trailing comments
Bug: T257651
Change-Id: Icce377f1833b80bd066622d6be3e711a18c58eea
2020-09-11 15:37:09 +01:00
Bartosz Dziewoński 934872a170 Add integration tests using pages from ckb.wp
This is primarily to cover the handling of localised digits,
which previously wasn't being tested, leading to T261706.

Bug: T261706
Change-Id: I9de7f01f77e767e9048c85604b559af4bca0de91
2020-09-01 01:50:33 +02:00
Bartosz Dziewoński 084f45128c Improve and document the files in tests/data/
* Remove 'wgMetaNamespace' and 'wgMetaNamespaceTalk', the same data
  exists in 'wgFormattedNamespaces'.

* Rename 'wgContentLang' to 'wgContentLanguage', to match its real
  name in JS config. MediaWiki doesn't use 'wgContentLang' anywhere,
  although the related PHP global is called $wgContLang.

* Document how I made these files, previously only mentioned in the
  commit message of e9c401e3aa.

Change-Id: I67f962812c155aedf41154e0d837e7feb5af972d
2020-09-01 01:50:33 +02:00
Bartosz Dziewoński 2d3fe47ac1 Fix parsing localised digits in PHP discussion parser
The PHP code incorrectly assumed that the digits are single-byte in
UTF-8, which is never the case (except for 0-9).

The JS code worked correctly because it uses UTF-16 strings, so the
bug would only affect non-BMP digits there. This was noted in a TODO
comment, but we overlooked it when reimplementing in PHP.

Instead of a string of 10 characters, use an array of 10
single-character strings.

Bug: T261706
Change-Id: Ic5421382474c88f003424799c53ff473d99cce92
2020-09-01 01:50:33 +02:00
Bartosz Dziewoński e36dc8e78a Skip to the end of the paragraph in the parser, not modifier
When a comment ended before the end of a paragraph, the next
comment would begin right there in the middle of the paragraph.
This could result in the detected indentation level of that
comment being incorrect, and replies being inserted in wrong
places, as seen in the 'signatures-funny' test case.

The code moved to the parser was previously repeated twice in
addListItem() and addReplyLink(), which should have been a hint
that something isn't quite right.

Also, fix the code guarding against overlapping signatures,
now that signatures may not be at the end of a comment.

Bug: T260855
Change-Id: Ic26a87642f8a15d5de2f7073d4d8176b299c7f94
2020-08-20 19:35:55 +00:00
Bartosz Dziewoński 986840e7e8 More test cases for multiple signatures in funny places
Expand the 'signatures-funny' test case with more examples, which
don't behave correctly.

Follow-up commits I04a8ea09401e06f2d4bb1f226f17eb528b29ed95 and
Ic26a87642f8a15d5de2f7073d4d8176b299c7f94 fix them.

Bug: T255738
Change-Id: I0fdd8bdf11b497ffeed37c37953c5730f6e4f3b7
2020-08-11 20:41:32 +02:00
Bartosz Dziewoński 375bfe028e parser: Fix comment ranges when timestamp has entities
Previously, parser would output offsets that don't exist in their
containers, because we were pretending that entities are parts of
their neighboring text nodes.

Turns out it's much easier to do it right when going backwards.

Change-Id: I9bccca2d403f1a976ae517449989170cdd99721e
2020-08-11 20:41:06 +02:00
Bartosz Dziewoński f0225243e0 tests: Fix some issues with overwriting outputs from PHP tests
Follow-up to ccd9e411d2.

* Fix variable name in CommentTestCase::overwriteHtmlFile()
* Overwrite before assertions, because they abort execution if they fail

Change-Id: I5bba016ba93f9dd1994325ae82c3105ba11cf033
2020-08-11 06:45:45 +02:00
jenkins-bot 4d4722a6ab Merge "Fix indentation level when replying to comments with mixed indentation" 2020-08-10 22:27:44 +00:00
jenkins-bot 7a18cc8902 Merge "Always use ':' (<dl><dd>) for indentation of replies" 2020-08-10 22:27:42 +00:00
Ed Sanders 7b2448d2f0 Use DOMCompat::getOuterHTML instead of doc->saveHTML()
The latter results in lots of extra HTML entity encoding.

The former is built by the Parsing team and appears to result
in no unexpected changes elsewhere in the document.

As Parsoid's selser relies on HTML fragments being byte-for-byte
equal, these changes were resulting in wikitext normalisations
in untouched parts of the document ("dirty diffs").

Bug: T259855
Change-Id: Ib3cb605911e690ec3e8c2f9df25fd1a2e2849d7e
2020-08-07 21:31:38 +02:00
Bartosz Dziewoński 31b26a5bec Fix indentation level when replying to comments with mixed indentation
When adding a reply, we take a node at the end of the previous comment,
compare that comment's indentation level to the expected indentation level
of the reply, and add (or remove) that number of wrapper lists.

The existing code did not consider that comments may have lists within
them, and so the indentation of that node may not match the indentation
of the comment.

Bug: T252702
Change-Id: Icc5ff19783d2b213bff99f283cb0599a8b5c1ab4
2020-08-06 01:25:33 +02:00
Bartosz Dziewoński a4ffdd37de Always use ':' (<dl><dd>) for indentation of replies
Previously we preferred that, but used '*' (<ul><li>) when the parent
comment or the previous reply also used it.

Bug: T252708
Change-Id: I3abf606da6693905764f1be745fad999fdf57fbe
2020-08-04 23:37:00 +02:00
Bartosz Dziewoński ccd9e411d2 Allow updating the expected results when running PHP tests
This is similar to the code we already have in JS tests, but instead
of printing to the console where you have to copy-paste from, it just
overwrites the files.

Also, update all of the expected results by this method.

Changes in the expected outputs:
* In JSON files, the "warnings" are now always in the same place
  regardless of the type of the warning.
* In all HTML files, self-closing tags now include the trailing slash,
  some characters are no longer encoded as entities when not necessary,
  and attributes may be single-quoted when that makes them shorter.
* In Parsoid HTML files, the header is no longer terribly mangled.

Other notes:
* CommentParserTest.php: Change the output of serializeComments()
  to be in similar order as in JS, to reduce the diffs in this commit
  and because it's a better order for humans.
* modifier.test.js: Remove some hacks that were working around small
  inconsistencies between the previous expected outputs and the actual
  outputs.

Change-Id: I9f764640dae823321c0ac35898fa4db03f1ca364
2020-08-04 03:05:28 +02:00
Bartosz Dziewoński c8d06f6bd6 Add a test case for replies to top-level comments with mixed indentation
The bug was accidentally fixed in 569db3603c.

Bug: T252702
Change-Id: I0020ac41adcf111b8496d3d4bce65740faf9e7ef
2020-07-30 01:46:45 +02:00
Bartosz Dziewoński 31e371e944 Better handle HTML comments following replies
Bug: T257651
Change-Id: I07e995beca4f031be062958ff7d75727afa8e606
2020-07-23 18:18:21 +02:00
jenkins-bot 765a1d27bc Merge "Improve detecting template-generated multi-line comments" 2020-07-22 15:04:00 +00:00
jenkins-bot 889de1bcdf Merge "Improve detecting typed signatures" 2020-07-22 01:43:40 +00:00
Bartosz Dziewoński 80e52e1155 Improve detecting typed signatures
* Remove the existing approach for detecting signatures that only
  worked in source mode; remove autoSignWikitext()

* Use the same approach for auto-signing in source mode as we have
  already used in visual

* In both modes, detect whether the user has already typed a signature
  at the end of their comment in the modifier, and if so, don't add a
  signature

* Add test cases for the detection

Bug: T255738
Change-Id: I791d3035cb1ffc33ce3966d4617a25d08700c35b
2020-07-22 00:00:53 +02:00
Bartosz Dziewoński 569db3603c Improve detecting template-generated multi-line comments
Bug: T252058
Change-Id: Ic010b8aeff9b177031184f02f92fcdea5280dc36
2020-07-21 22:26:45 +01:00
Ed Sanders a2431fe006 Refactor CommentParser
* Pass rootNode to the constructor
* Rename getters to match CommentItem/HeadingItem/ThreadItem
  value classes.
* Always build the thread tree so CommentItem's always have
  and ID and replies/parent.

Change-Id: I508be9534de59016ff806e3d84edcbb1c76cb0c6
2020-07-20 23:38:10 +01:00
Ed Sanders a4636d39fc Move #getTranscludedFrom from parser to ThreadItem
Also requires moving getTitleFromUrl to CommentUtils

Change-Id: I9cb83a3fdd456eba66899433b866ce7a7f00eeb5
2020-07-20 15:56:48 +01:00
Ed Sanders 7ae5bbf384 Move #getAuthors from parser to ThreadItem
Change-Id: I16e513000e5366b3044b17a99da07d8d0f47a61f
2020-07-20 15:13:59 +01:00
Ed Sanders b32f991913 Documentation fixes
Change-Id: I2c7ccecbf8a50bd4d658b0f17f4a21fe90a3c399
2020-07-20 13:34:08 +01:00
Bartosz Dziewoński 08b467bf9f tests: Fix wrong $rootNode in some tests using CommentParser::getComments()
Rather than the <body> node, we were passing <body>'s first child.
Current implementation of CommentParser::getComments() doesn't fail
the tests in spite of this because the XPath query incorrectly returns
results relative to the document's real root node, but these tests
would start failing after I2441f33e6e7bad753ac830d277e6a2e81ee8c93d.

Follow-up to 3e6ab2c4d2.

Change-Id: Ic26e0a1ee4443987e215c5f26ef1f084ccd0b40b
2020-07-15 16:40:30 +00:00
Ed Sanders ed70d49285 CommentParser.php: Fix URL parsing
Change-Id: I406fd98b308dd4d975ea974f2369737a7052b556
2020-07-01 17:06:02 +01:00
Ed Sanders 6459e7dc82 Move wikitext modifiers to modifier.js
Re-create methods in PHP.

Change-Id: Iae6117b65e3b8f50ecc68e1e3ea17c8359bdcb06
2020-07-01 17:06:02 +01:00
Ed Sanders d75a340026 CommentModifierTest: Use DOMCompat::get/setInnerHTML to match JS code
Change-Id: Idd057ff1a5028b377903ff3798ca2bce22535337
2020-06-27 13:13:27 +00:00
Ed Sanders 3e6ab2c4d2 PHP: Use DOMUtils::parseHTML
Change-Id: Ifed0ab99b3da9f8b35ca815ada45f804a8756c1b
2020-06-26 20:06:47 +01:00
Ed Sanders d5376e28fc Improve ThreadItem documentation
Change-Id: Ia266fc22b02af0edbb32f356b4e0d92fe3a4da5f
2020-06-26 14:56:19 +02:00
Ed Sanders 811f8bdf02 Allow non-lists to be passed to unwrapList
Bug: T256292
Change-Id: I036e0fdf3dde51c33f64abb7df142e26ebe66554
2020-06-24 19:19:06 +01:00
Ed Sanders 306faba93d Tests: childNodes[0] -> firstChild
Change-Id: Iae53012f289552d80dad907bfb54a8b5d44cb484
2020-06-12 19:46:08 +00:00
Ed Sanders 7be0cc3209 Create ThreadItem classes
Change-Id: Id2c5324d74eccb1209ccb76768c557722c6d9400
2020-06-12 20:35:59 +01:00
Ed Sanders 62c0080850 Fix whitespace handling in unwrapList
Update 'referenceNode' before modifying the DOM.

Bug: T254308
Change-Id: I76af898e238c1e6d3db96d3e6156e80e55c87820
2020-06-03 13:54:36 +01:00
Ed Sanders 0d14fcea6a wt->visual: Don't unwrap template lists
Bug: T253150
Change-Id: I1584d9834e29c38edf4234f2f022c1c48bfd485f
2020-06-01 22:32:23 +01:00
jenkins-bot 431ba1c87f Merge "Make signature prefix customisable" 2020-06-01 17:18:04 +00:00
Ed Sanders dada23e0f4 Make signature prefix customisable
Bug: T249861
Change-Id: I506a0c1b803275f5505fdf8ee29cc535814d7742
2020-06-01 18:55:04 +02:00
Bartosz Dziewoński 43a7e82ed0 Tests covering fr.wp unsigned comment templates
Bug: T252058
Bug: T252059
Change-Id: Ica7426055e73562a98b99d6ea06e257df77b7493
2020-05-28 00:13:07 +02:00
jenkins-bot d8a6362361 Merge "Fix failing test case for PHP modifier" 2020-05-26 03:08:48 +00:00
jenkins-bot 9f2ead06eb Merge "Insert replies outside of decorative comment frames" 2020-05-25 19:30:28 +00:00