Commit graph

135 commits

Author SHA1 Message Date
jenkins-bot 889de1bcdf Merge "Improve detecting typed signatures" 2020-07-22 01:43:40 +00:00
Bartosz Dziewoński 80e52e1155 Improve detecting typed signatures
* Remove the existing approach for detecting signatures that only
  worked in source mode; remove autoSignWikitext()

* Use the same approach for auto-signing in source mode as we have
  already used in visual

* In both modes, detect whether the user has already typed a signature
  at the end of their comment in the modifier, and if so, don't add a
  signature

* Add test cases for the detection

Bug: T255738
Change-Id: I791d3035cb1ffc33ce3966d4617a25d08700c35b
2020-07-22 00:00:53 +02:00
Ed Sanders a2431fe006 Refactor CommentParser
* Pass rootNode to the constructor
* Rename getters to match CommentItem/HeadingItem/ThreadItem
  value classes.
* Always build the thread tree so CommentItem's always have
  and ID and replies/parent.

Change-Id: I508be9534de59016ff806e3d84edcbb1c76cb0c6
2020-07-20 23:38:10 +01:00
Ed Sanders a4636d39fc Move #getTranscludedFrom from parser to ThreadItem
Also requires moving getTitleFromUrl to CommentUtils

Change-Id: I9cb83a3fdd456eba66899433b866ce7a7f00eeb5
2020-07-20 15:56:48 +01:00
Ed Sanders 7ae5bbf384 Move #getAuthors from parser to ThreadItem
Change-Id: I16e513000e5366b3044b17a99da07d8d0f47a61f
2020-07-20 15:13:59 +01:00
Ed Sanders b32f991913 Documentation fixes
Change-Id: I2c7ccecbf8a50bd4d658b0f17f4a21fe90a3c399
2020-07-20 13:34:08 +01:00
Bartosz Dziewoński 08b467bf9f tests: Fix wrong $rootNode in some tests using CommentParser::getComments()
Rather than the <body> node, we were passing <body>'s first child.
Current implementation of CommentParser::getComments() doesn't fail
the tests in spite of this because the XPath query incorrectly returns
results relative to the document's real root node, but these tests
would start failing after I2441f33e6e7bad753ac830d277e6a2e81ee8c93d.

Follow-up to 3e6ab2c4d2.

Change-Id: Ic26e0a1ee4443987e215c5f26ef1f084ccd0b40b
2020-07-15 16:40:30 +00:00
Ed Sanders ed70d49285 CommentParser.php: Fix URL parsing
Change-Id: I406fd98b308dd4d975ea974f2369737a7052b556
2020-07-01 17:06:02 +01:00
Ed Sanders 6459e7dc82 Move wikitext modifiers to modifier.js
Re-create methods in PHP.

Change-Id: Iae6117b65e3b8f50ecc68e1e3ea17c8359bdcb06
2020-07-01 17:06:02 +01:00
Ed Sanders d75a340026 CommentModifierTest: Use DOMCompat::get/setInnerHTML to match JS code
Change-Id: Idd057ff1a5028b377903ff3798ca2bce22535337
2020-06-27 13:13:27 +00:00
Ed Sanders 3e6ab2c4d2 PHP: Use DOMUtils::parseHTML
Change-Id: Ifed0ab99b3da9f8b35ca815ada45f804a8756c1b
2020-06-26 20:06:47 +01:00
Ed Sanders 306faba93d Tests: childNodes[0] -> firstChild
Change-Id: Iae53012f289552d80dad907bfb54a8b5d44cb484
2020-06-12 19:46:08 +00:00
Ed Sanders 7be0cc3209 Create ThreadItem classes
Change-Id: Id2c5324d74eccb1209ccb76768c557722c6d9400
2020-06-12 20:35:59 +01:00
Ed Sanders 0d14fcea6a wt->visual: Don't unwrap template lists
Bug: T253150
Change-Id: I1584d9834e29c38edf4234f2f022c1c48bfd485f
2020-06-01 22:32:23 +01:00
jenkins-bot d8a6362361 Merge "Fix failing test case for PHP modifier" 2020-05-26 03:08:48 +00:00
Bartosz Dziewoński 72c730f6c4 Fix failing test case for PHP modifier
The expected HTML was wrong, a '<br />' tag inside 'data-mw' was
somehow turned into '<br ></span>'. No idea how that happened.
Something must be wrong with the HTML parsing in JS tests, which
were used to generate this file.

Change-Id: I69caa68fe70e706df81e8adf29889254704f601e
2020-05-25 21:09:18 +02:00
Ed Sanders b3ca37c1c5 Create ImmutableRange class in PHP
TODO: Create one in JS as well

Change-Id: I6c9dc2455afcb8d0b68674a2985c5e43dd94b6fb
2020-05-22 15:01:09 +01:00
Bartosz Dziewoński c64bb6b5b7 Add the test for getAuthors() in JS too
Change-Id: Id7dabc535b6bb688602c0d55fc3696f662cb10c7
2020-05-19 21:13:52 +02:00
Bartosz Dziewoński e12aea2f77 Add test case for unwrapParsoidSections()
Covers the bug fixed in I9133d4365a71d6db1fa58b69ae3b970166d15c1e.

Depends-On: I9133d4365a71d6db1fa58b69ae3b970166d15c1e
Bug: T252238
Change-Id: I92831696864e04384eb514ab69f14563cceafc19
2020-05-18 21:36:48 +00:00
Ed Sanders c5d1029b25 Move /cases and /data up to /tests
Theses are no longer QUnit specific.

Change-Id: I5f3cca1ff686922e0cdaaedb80858f37df04799a
2020-05-18 21:47:17 +01:00
Ed Sanders b1427163af Parser.php: Add tests for getTranscludedFrom
Requires an implementation of unwrapParsoidSections

Change-Id: I96c929b1117ba652dbd5af6a1ee37a5f9e87ed1e
2020-05-18 19:53:01 +00:00
Ed Sanders d1e58841af Rename removeListItem to removeAddedListItem and remove in PHP
This method shouldn't be required on the server. Leave comments
relating to it in addListItem so JS & PHP can be kept in sync.

Change-Id: I849fac660faf6e750272c20776f96b9250f96b1b
2020-05-18 19:25:08 +00:00
Reedy 234988155e Add leading \ to covers
Change-Id: I1ed4cd28bf630c6aae238e548410d1293a8b71f1
2020-05-15 22:08:25 +01:00
Ed Sanders e6e0b1ead9 PHP: Add missing typehints
Change-Id: I5639f8cbdae9aaa9cfa06136e19cc94f9fad10ea
2020-05-15 22:04:47 +02:00
Ed Sanders b78fb3f4c1 Move all PHP to the MediaWiki\Extension\DiscussionTools namespace
Change-Id: I654ebb3e646a6d8d62f7bd14d48805e39f836d7e
2020-05-15 21:57:13 +02:00
Ed Sanders 340572bc05 Create a Utils class in PHP
Also move htmlTrim to utils in JS.

Change-Id: Ia5356d713c1c5d521c396cc28bcd4ecc7ee5bbbb
2020-05-15 00:25:32 +01:00
Ed Sanders a3889fd400 Port modifier.js to PHP
Change-Id: I03b9e4377cb3ce6a5ca9d06e49dca9b2516f4979
2020-05-15 00:20:41 +01:00
Bartosz Dziewoński 6f32369b6a tests: Fix comparing PHP and JS ranges
In JS, strings are internally encoded as UTF-16, and properties like
.length return values in UTF-16 code units.

In PHP, strings are internally encoded as UTF-8, and we have the
option of using methods that return bytes like strlen() or UTF-8 code
units like mb_strlen().

However, the values produced by preg_match( …, PREG_OFFSET_CAPTURE )
are in bytes, and there's nothing we can do about that. So let's use
bytes throughout, mixing the two types results in meaningless numbers.

Then in the test code, we have to calculate UTF-16 code units offsets
based on the UTF-8 byte offsets.

We also have to copy the entire workaround for mw:Entity nodes… Maybe
the parser should be fixed to return the real nodes for ranges' ends
in this case.

Change-Id: I05804489d7de0d60be6e9f84e6a49a885e9fb870
2020-05-14 22:37:34 +00:00
Bartosz Dziewoński 33d69e26c9 tests: Fix different whitespace trimming in PHP and JS
Notably, JS trims the no-break space, while PHP doesn't. There are
some other differences that don't come up in our tests. What we really
want is to trim the ASCII whitespace as defined in the HTML spec.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/trim
https://www.php.net/manual/en/function.trim.php
https://infra.spec.whatwg.org/#ascii-whitespace

Change-Id: I95b8fb38878716a2fa7ec84c9f2e8065ebe77c0d
2020-05-14 21:37:26 +00:00
Bartosz Dziewoński c0002be7cd tests: Fix computing ranges in Parsoid documents
In JS tests, we load the documents via mw.template, which apparently
causes the <html>, <head> and <body> tags to disappear, resulting
in the ranges not matching in PHP tests (and the real document).

Put in a big hack that makes them match, and update the JSON files.

Change-Id: I8194752cd5f82c3716c99e76a37226af5d4a0ec1
2020-05-14 01:11:44 +02:00
Bartosz Dziewoński 95a87911eb tests: Check ranges in PHP parser tests
Comment out only the cases that still fail, so it's easier to fix them.

Change-Id: I85d205731d572c93ababa7dd66e674321969edb7
2020-05-13 23:58:37 +02:00
Bartosz Dziewoński b8d7a75c34 Fix performance of DiscussionToolsCommentParser::childIndexOf()
Profiling reveals that >87% of the run time of our test suite is spent
in this tiny method. Apparently, DOMNodeList::item() is extremely slow
(possibly it's linear time instead of constant time?).

Profiled using XDebug and KCacheGrind:
https://phabricator.wikimedia.org/F31815264

We can calculate the child's index in its parent by counting its
precending siblings instead, which turns out to be much faster.

Before:
 1. 275444ms to run DiscussionToolsCommentParserTest:testGetComments with data set #2
 2. 12668ms to run DiscussionToolsCommentParserTest:testGetComments with data set #3
 ...
After:
 1. 9545ms to run DiscussionToolsCommentParserTest:testGetComments with data set #2
 2. 5549ms to run DiscussionToolsCommentParserTest:testGetComments with data set #3
 ...

That's still kind of slow but now it's bearable to run the test suite.

Change-Id: I49155f7aa2e231a9a20bf282cf6aaa28fc902e0b
2020-05-13 02:56:39 +02:00
Ed Sanders 745101c02b PHP tests: Move some test utils to a parent class
Change-Id: I6ae5aa85f8aaa02e1b9323820a841c06c5d62b64
2020-05-12 12:33:04 +01:00
Ed Sanders a000b8c7b1 Add comment tests to PHP
TODO: Make the assertions less slow (currently ~50s)

Change-Id: I1d774e353c070484b5bae18e2ec3e3e41da68202
2020-05-12 12:33:04 +01:00
Roan Kattouw 7b7a2cd69c The Great Parser JS to PHP port of 2020!*
* Not to be confused with the Parsing Team's
"Great Parser JS to PHP port of 2019"

Gasp as OR hacks are changed to null coalescing operators.
Applaud as variable declarations are dropped.
Cheer as parameters and return values are type-hinted.
Shudder as DomNodeLists have no indexOf method.

Moving discussion parsing to the server should allow
us to implement much cleaner APIs for commenting.

Bug: T252252
Co-authored-by: Ed Sanders <esanders@wikimedia.org>
Change-Id: Ic1438d516e223db462cb227f6668e856672f538c
2020-05-12 12:33:04 +01:00