Commit graph

641 commits

Author SHA1 Message Date
Ed Sanders b32f991913 Documentation fixes
Change-Id: I2c7ccecbf8a50bd4d658b0f17f4a21fe90a3c399
2020-07-20 13:34:08 +01:00
Ed Sanders 092cfd6075 Parser: Replace findTimestamps with findTimestamp
Instead of doing a separate tree walk and finding all timestamps
separately, make it part of the getComments tree walk, and find
timestamps one at a time.

Change-Id: I47f466eaf228504faa189fd99e07493bc7f022cd
2020-07-15 21:34:22 +02:00
Bartosz Dziewoński 308c2747b0 CommentParser.php: Use tree walking instead of XPath
This is similar to what the JS version does.

The TreeWalker and NodeFilter classes are adapted from
https://github.com/Krinkle/dom-TreeWalker-polyfill
(MIT license).

This makes #getComments twice as fast on en-big-oldparser.html

Change-Id: I2441f33e6e7bad753ac830d277e6a2e81ee8c93d
2020-07-15 16:40:50 +00:00
Ed Sanders 94511c6bad Add getHeading method to CommentItem
Change-Id: I14ccdbb432e835fe74760a52f07de172df2aba27
2020-07-01 17:21:15 +01:00
Ed Sanders ed70d49285 CommentParser.php: Fix URL parsing
Change-Id: I406fd98b308dd4d975ea974f2369737a7052b556
2020-07-01 17:06:02 +01:00
Ed Sanders 6459e7dc82 Move wikitext modifiers to modifier.js
Re-create methods in PHP.

Change-Id: Iae6117b65e3b8f50ecc68e1e3ea17c8359bdcb06
2020-07-01 17:06:02 +01:00
Ed Sanders 978eae5547 Move createCommentContainer to modifier
Refactor and create PHP versions.

Change-Id: I0ae9785e5fa86bcc1ac6eef9bce4088d7507dd33
2020-07-01 17:06:02 +01:00
Ed Sanders fb0d2cbe7f Move some postReply logic to modifier.js
Re-create the function in PHP

Change-Id: I1c8b505542c3b82c10b7b258e48054bc59cb4836
2020-07-01 17:06:02 +01:00
Ed Sanders d5376e28fc Improve ThreadItem documentation
Change-Id: Ia266fc22b02af0edbb32f356b4e0d92fe3a4da5f
2020-06-26 14:56:19 +02:00
Ed Sanders 811f8bdf02 Allow non-lists to be passed to unwrapList
Bug: T256292
Change-Id: I036e0fdf3dde51c33f64abb7df142e26ebe66554
2020-06-24 19:19:06 +01:00
Ed Sanders badbd823b2 Use $wgLocaltimezone global instead of request context
Bug: T255704
Change-Id: Ic26a1d77a64f52addcb45131ea2f9cd41c7c51e2
2020-06-17 23:14:13 +02:00
James D. Forrester d6c3df31f5 Remove various phan suppressions and fix issues
Change-Id: I73b535f284566a0a8876a3198b9784b47567fac6
2020-06-12 20:35:59 +01:00
Ed Sanders 7be0cc3209 Create ThreadItem classes
Change-Id: Id2c5324d74eccb1209ccb76768c557722c6d9400
2020-06-12 20:35:59 +01:00
Ed Sanders ba295c81dd Replace 'whitelist' in documentation
Bug: T254646
Change-Id: Ie67b0f801dcd95142b6b2ba5793cb87fa1fb5932
2020-06-11 04:51:39 +00:00
Ed Sanders a4d767d97c CommentModifier.php: Remove unnecessary toLowerCase
We only need .toLowerCase() or strtolower() when doing
case insensitive comparisons.

Change-Id: I19caca50139a42d86ff20e1ee0224cd3deb0d092
2020-06-10 22:06:16 +01:00
Umherirrender 48e860916a build: Add mediawiki/mediawiki-phan-config
Replace phan-taint-check-plugin by phan, it is now included

Change-Id: I0e682a83afd30faa8967e3c586431be4ae9a29b3
2020-06-10 22:21:07 +02:00
jenkins-bot 94bbaa846f Merge "Depend on Linter" 2020-06-04 16:46:57 +00:00
jenkins-bot 203df3aabc Merge "Call the linter API directly if installed" 2020-06-04 16:24:36 +00:00
David Lynch 068ff29e88 Depend on Linter
Depends-On: I8b183d816748245722ca1a459203333614f58f32
Change-Id: If2252d54a39a9d12744ccdee08e334a211f41d46
2020-06-04 18:23:57 +02:00
Ed Sanders e9f2bb80d1 Call the linter API directly if installed
If the Linter extension is not installed, just allow the page
to be edited.

Bug: T253799
Change-Id: Id3e4c9c2d48aa7ab0b270f41d33ec19184af9a48
2020-06-04 09:46:07 -05:00
Ed Sanders 62c0080850 Fix whitespace handling in unwrapList
Update 'referenceNode' before modifying the DOM.

Bug: T254308
Change-Id: I76af898e238c1e6d3db96d3e6156e80e55c87820
2020-06-03 13:54:36 +01:00
Ed Sanders 0d14fcea6a wt->visual: Don't unwrap template lists
Bug: T253150
Change-Id: I1584d9834e29c38edf4234f2f022c1c48bfd485f
2020-06-01 22:32:23 +01:00
Ed Sanders da433037a3 Move getTranscludedFromElement to Utils
Change-Id: I8bdd757f949c013ba426150a192d71243fadf45d
2020-06-01 22:32:23 +01:00
Bartosz Dziewoński 79ae8a32c5 Support parsing when timestamp is wrapped in a link
Bug: T252059
Change-Id: Ib8952fb80503bad407e8d0fe725103a0fae12a6a
2020-05-27 22:47:17 +02:00
Bartosz Dziewoński 01b4a8f4f4 Support replying when timestamp is template-generated
* Move modifier#getFullyCoveredWrapper to utils
* Use that method to find the node where we start searching for
  template wrappers, rather than using endContainer

Bug: T252058
Change-Id: I55de58102f3468fce01290bd413a7fdc96d322d6
2020-05-27 21:16:03 +02:00
Bartosz Dziewoński 420c514091 Insert replies outside of decorative comment frames
When there is a wrapper element whose range matches the range of
a comment, any replies will now be added outside of that wrapper,
instead of directly after the comment (inside the wrapper).

Bug: T250126
Change-Id: I6b42c4db019ae998e91eebd324f9cbd2aa791b4f
2020-05-22 15:01:12 +01:00
Ed Sanders b3ca37c1c5 Create ImmutableRange class in PHP
TODO: Create one in JS as well

Change-Id: I6c9dc2455afcb8d0b68674a2985c5e43dd94b6fb
2020-05-22 15:01:09 +01:00
jenkins-bot 838dcdccad Merge "Avoid deprecated User::getOption()" 2020-05-21 16:21:36 +00:00
jenkins-bot c53cecac10 Merge "Create a user preference to store visual/source mode" 2020-05-21 16:21:34 +00:00
Bartosz Dziewoński 5675eea4d0 Avoid deprecated User::getOption()
Change-Id: Iad32c3e63116c54c47cd3aa2aaf142c5dd5404c5
2020-05-21 13:12:49 +00:00
Ed Sanders 5feb69612f Create a user preference to store visual/source mode
Bug: T250523
Depends-On: I5b3b47feeeca2085fcd283d55d7e9ceafa73be70
Change-Id: I18f5d38f9d694b12e5f983be7b3b165f135a890d
2020-05-20 22:16:19 +01:00
Bartosz Dziewoński 515af82061 Reduce duplication between PHP parser and data gen for JS parser
Also, make the handling of TranslateNumerals and digitsRegexp the same
between PHP and JS.

Change-Id: I1d81343d0b59ab3ecd59ba1c2ad99a729d983ac4
2020-05-19 20:54:44 +02:00
jenkins-bot 366aca2ccd Merge "Stop printing console warnings" 2020-05-18 22:52:30 +00:00
Bartosz Dziewoński 219339551c Stop printing console warnings
It was useful when I was debugging those parts of the code, but now
it's usually annoying.

The warnings can still sometimes be useful for understanding how the
tool parses some discussion, though. To keep that functionality, add
displaying warnings for each comment in the debug mode.

Change-Id: I2d218a8a394f179bcc0990ff988a0567c275ccf2
2020-05-18 23:37:37 +02:00
Ed Sanders 607440498e Spell check pass
Change-Id: Ia20da358297126bd52a968bd77c960f81fe82b8f
2020-05-18 21:24:14 +00:00
Bartosz Dziewoński c848d8a90e Parser tweaks
Follow-up to Ic1438d516e223db462cb227f6668e856672f538c.
Minor corrections and comment improvements in PHP parser,
and "backporting" some changes to JS parser that I like.

Change-Id: I5e54121914ec6b323e556dd133bcb71b3aefbb61
2020-05-18 19:53:26 +00:00
Ed Sanders b1427163af Parser.php: Add tests for getTranscludedFrom
Requires an implementation of unwrapParsoidSections

Change-Id: I96c929b1117ba652dbd5af6a1ee37a5f9e87ed1e
2020-05-18 19:53:01 +00:00
Ed Sanders d1e58841af Rename removeListItem to removeAddedListItem and remove in PHP
This method shouldn't be required on the server. Leave comments
relating to it in addListItem so JS & PHP can be kept in sync.

Change-Id: I849fac660faf6e750272c20776f96b9250f96b1b
2020-05-18 19:25:08 +00:00
jenkins-bot 0496e6b7e6 Merge "PHP: More missing typehints" 2020-05-18 17:54:16 +00:00
jenkins-bot a41eadfc51 Merge "Modifier: Pass document to createWikitext" 2020-05-18 17:54:15 +00:00
Ed Sanders bc437fc43f PHP: More missing typehints
Change-Id: I483c9e70b65dcd685436b4099bcfc4925c65b002
2020-05-16 16:46:25 +01:00
Ed Sanders 5e996fdfdd Modifier: Pass document to createWikitext
Change-Id: I1793e1d690835af746a4e25a50e2e0a474811e8e
2020-05-15 22:47:54 +01:00
jenkins-bot 9cc665a8bc Merge "Add missing use MWException" 2020-05-15 21:46:21 +00:00
jenkins-bot a6fcb965ea Merge "Fix return type of callable" 2020-05-15 21:46:20 +00:00
jenkins-bot db29a29306 Merge "Don't call non static functions statically" 2020-05-15 21:46:20 +00:00
Reedy ac6cd26ca0 Don't call non static functions statically
Change-Id: I2db66a8da3ab325f2bbabb37afd276d4a62077e9
2020-05-15 22:02:57 +01:00
Reedy 70c5a1e435 Fix return type of callable
Change-Id: I7e594a9f9f6f9d4737fd880e449c43b9b2cf24fb
2020-05-15 22:01:21 +01:00
Reedy c3a7ba1d13 Add missing use MWException
Change-Id: I4d00106718c0f7e32060d23aaa2bc8c74a4d6d1f
2020-05-15 22:00:30 +01:00
Reedy 08a6a83f9d Replace stObject with stdClass
Change-Id: Id380c34cf0ca85a0a9a0d044474aa228b620c78f
2020-05-15 21:57:28 +01:00
jenkins-bot 21a892d5e7 Merge "Follow-up b78fb3f4: Make sure we're using a MW User" 2020-05-15 20:38:42 +00:00
James D. Forrester d775f30791 Follow-up b78fb3f4: Make sure we're using a MW User
Change-Id: Iafd4d3b17a873c0702a64e063a1d0d7e19cf572b
2020-05-15 13:18:06 -07:00
Ed Sanders e6e0b1ead9 PHP: Add missing typehints
Change-Id: I5639f8cbdae9aaa9cfa06136e19cc94f9fad10ea
2020-05-15 22:04:47 +02:00
Ed Sanders b78fb3f4c1 Move all PHP to the MediaWiki\Extension\DiscussionTools namespace
Change-Id: I654ebb3e646a6d8d62f7bd14d48805e39f836d7e
2020-05-15 21:57:13 +02:00
Ed Sanders 340572bc05 Create a Utils class in PHP
Also move htmlTrim to utils in JS.

Change-Id: Ia5356d713c1c5d521c396cc28bcd4ecc7ee5bbbb
2020-05-15 00:25:32 +01:00
Ed Sanders a3889fd400 Port modifier.js to PHP
Change-Id: I03b9e4377cb3ce6a5ca9d06e49dca9b2516f4979
2020-05-15 00:20:41 +01:00
Bartosz Dziewoński 6f32369b6a tests: Fix comparing PHP and JS ranges
In JS, strings are internally encoded as UTF-16, and properties like
.length return values in UTF-16 code units.

In PHP, strings are internally encoded as UTF-8, and we have the
option of using methods that return bytes like strlen() or UTF-8 code
units like mb_strlen().

However, the values produced by preg_match( …, PREG_OFFSET_CAPTURE )
are in bytes, and there's nothing we can do about that. So let's use
bytes throughout, mixing the two types results in meaningless numbers.

Then in the test code, we have to calculate UTF-16 code units offsets
based on the UTF-8 byte offsets.

We also have to copy the entire workaround for mw:Entity nodes… Maybe
the parser should be fixed to return the real nodes for ranges' ends
in this case.

Change-Id: I05804489d7de0d60be6e9f84e6a49a885e9fb870
2020-05-14 22:37:34 +00:00
Bartosz Dziewoński 76289cdf73 tests: Fix failures due to CDATA handling in PHP
It appears PHP's DOM library always uses CDATA nodes for the contents of
<style> tags, even if there is no such markup in the source HTML.

Change-Id: Id04b27086c5e7a0b016a3a440b2b4895d6b13c93
2020-05-14 22:37:23 +00:00
Bartosz Dziewoński 33d69e26c9 tests: Fix different whitespace trimming in PHP and JS
Notably, JS trims the no-break space, while PHP doesn't. There are
some other differences that don't come up in our tests. What we really
want is to trim the ASCII whitespace as defined in the HTML spec.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/trim
https://www.php.net/manual/en/function.trim.php
https://infra.spec.whatwg.org/#ascii-whitespace

Change-Id: I95b8fb38878716a2fa7ec84c9f2e8065ebe77c0d
2020-05-14 21:37:26 +00:00
Ed Sanders 076c6d828a Parser: Use Element instead of Node when appropriate
Change-Id: I67015989954e61810302bf8931d05419f007ffeb
2020-05-13 14:46:36 +01:00
Bartosz Dziewoński b8d7a75c34 Fix performance of DiscussionToolsCommentParser::childIndexOf()
Profiling reveals that >87% of the run time of our test suite is spent
in this tiny method. Apparently, DOMNodeList::item() is extremely slow
(possibly it's linear time instead of constant time?).

Profiled using XDebug and KCacheGrind:
https://phabricator.wikimedia.org/F31815264

We can calculate the child's index in its parent by counting its
precending siblings instead, which turns out to be much faster.

Before:
 1. 275444ms to run DiscussionToolsCommentParserTest:testGetComments with data set #2
 2. 12668ms to run DiscussionToolsCommentParserTest:testGetComments with data set #3
 ...
After:
 1. 9545ms to run DiscussionToolsCommentParserTest:testGetComments with data set #2
 2. 5549ms to run DiscussionToolsCommentParserTest:testGetComments with data set #3
 ...

That's still kind of slow but now it's bearable to run the test suite.

Change-Id: I49155f7aa2e231a9a20bf282cf6aaa28fc902e0b
2020-05-13 02:56:39 +02:00
Roan Kattouw 7b7a2cd69c The Great Parser JS to PHP port of 2020!*
* Not to be confused with the Parsing Team's
"Great Parser JS to PHP port of 2019"

Gasp as OR hacks are changed to null coalescing operators.
Applaud as variable declarations are dropped.
Cheer as parameters and return values are type-hinted.
Shudder as DomNodeLists have no indexOf method.

Moving discussion parsing to the server should allow
us to implement much cleaner APIs for commenting.

Bug: T252252
Co-authored-by: Ed Sanders <esanders@wikimedia.org>
Change-Id: Ic1438d516e223db462cb227f6668e856672f538c
2020-05-12 12:33:04 +01:00
Ed Sanders 78aba45909 Hooks: Remove unused variable
Change-Id: I80f5dd63a2224f1c523ab1aa53da435851c2789d
2020-05-11 16:55:08 +01:00
jenkins-bot 922443f68f Merge "ConfirmEdit CAPTCHA support" 2020-04-29 12:04:44 +00:00
jenkins-bot 483aac1fca Merge "Enable on all ExtraSignaturesNamespaces" 2020-04-28 18:32:36 +00:00
Bartosz Dziewoński 7ef3bffb1e Show error on pages with fostered content to avoid page corruption in Parsoid
When the user clicks a "Reply" link on a page that is affected by the
'fostered' lint error (indicating fostered content in the HTML
representation), display an error and refuse to edit it, as Parsoid's
transformations will damage the page content.

The error message includes a link to documentation about lint errors,
and a link to the editor that will highlight the error location.

Depends-On: I723ec766d1244d117f8d624440026fe5af0d3403
Bug: T246481
Change-Id: Ic60cb58f98d10dc9b113469e5d3bbfb2d2b0564f
2020-04-21 17:40:39 +00:00
Bartosz Dziewoński 40c18508ec ConfirmEdit CAPTCHA support
Depends-On: I6605017fd31a4f96c529dd0beb69e9f4433cebc1
Bug: T247513
Change-Id: If9b3162969aa6ee67baff80aa84fa719bd3c3720
2020-04-21 18:55:38 +02:00
Ed Sanders 5565c7e4f0 Enable on all ExtraSignaturesNamespaces
For most Wikimedia wikis, this is Project (Wikipedia) and Help.

Bug: T249036
Change-Id: I535240dc5232b78f11d5003d1141ad8114f01be2
2020-04-19 14:53:40 +01:00
jenkins-bot c5e9185ae7 Merge "DiscussionToolsHooks: Only enable when viewing pages" 2020-04-14 15:29:04 +00:00
David Lynch e38aad96e4 Add override config for instrumentation rates
Bug: T250086
Change-Id: I32fbf3d6562fb0a93698a24bbd8177149a19fed3
2020-04-13 12:38:40 -05:00
Bartosz Dziewoński 3bd513374e DiscussionToolsHooks: Only enable when viewing pages
Previously we'd also run on action=history, action=info etc. (which did nothing).

Change-Id: I4bd10311a91433f357a6f5506c65364d399f3915
2020-04-01 22:48:51 +02:00
Ed Sanders bcd3fbfe9a Treat pages with __NEWSECTIONLINK__ as talk pages
Bug: T245890
Change-Id: Ibb9f6c675ce26c5a0e6e7934bd4659f5aaae6a25
2020-04-01 14:34:05 +01:00
Bartosz Dziewoński 92c3f079b8 controller: Make edit summary use the wiki's content language
Bug: T246043
Change-Id: I591e1a3ed3e7c8b68f5e203a0c77d057793fc767
2020-03-14 14:17:29 +01:00
Thiemo Kreuz d217b2b85f Replace deprecated Language class properties with constants
These are going to be removed via Id370937.

Change-Id: I87b3d13f66f0f449fead49865af9f9242717c568
2020-02-28 09:19:21 +01:00
jenkins-bot fcb935a2ba Merge "Add beta feature preference" 2020-02-21 22:00:44 +00:00
jenkins-bot cf8913d6a0 Merge "Don't use globals to access config" 2020-02-21 21:16:04 +00:00
Ed Sanders 2577d5aa28 Add beta feature preference
Bug: T245539
Change-Id: I8bb2c10255a13c291bc97c591612cadaf0e4339f
2020-02-21 15:20:48 +00:00
Ed Sanders f75137ecc1 Don't use globals to access config
Change-Id: I6180dff629af554b5430e20c2db2f6f94ef96478
2020-02-21 15:20:47 +00:00
DannyS712 6bef74693f Stop using old deprecated SpecialPageFactory static functions
Bug: T245686
Change-Id: I7501a66b0649bb3d2a928999f63a08371e1856ff
2020-02-19 23:15:32 +00:00
David Lynch 16215bbea2 Change tags method so anon edits will get tagged
Bug: T242184
Change-Id: I38baddc0febe02f6d2321be616adc018c87b5a54
2020-02-07 01:19:11 -06:00
David Lynch 2fcc23ae61 Add editor-mode changetags
Bug: T242184
Change-Id: Ib307e44ec883b01e0705c889f97e16326519f4c2
2020-02-06 20:22:27 +01:00
Bartosz Dziewoński 44eb801844 Add config option and query parameter to control loading
By default, DiscussionTools loads on all talk pages when the extension
is installed. This can now be disabled by setting the configuration
option `$wgDiscussionToolsEnable=false`.

To test DiscussionTools, one can now use the query parameter
`?dtenable=1`, which allows it to be loaded on any wikitext page
(overriding the config option).

Bug: T243621
Change-Id: I3d5a9cc9a4183fb6951f05c557b1d42735a9df7c
2020-02-04 22:06:13 +01:00
David Lynch 664b2890d7 Tag DiscussionTools edits
This sets up the tags:
* discussiontools
* discussiontools-reply
* discussiontools-edit (not yet implemented)
* discussiontools-newsection (not yet implemented)

The tags are flagged as user-addable, because otherwise they can't be
passed through to the VE API (at least, not without editing it so that
it explicitly knows about them, which seems like a strange
interdependency). It's assumed that letting users who know about the
tags add them to random changes via action=editchangetags would be
(a) the pettiest and most inconsequential vandalism possible, and
(b) unlikely to happen.

This relies upon I2c1d0f8d69bc03e5c1877c790247e165f160e966 in
VisualEditor to not also tag the edits with `visualeditor`.

Bug: T242184
Change-Id: I4e5e26afdd52279df242e1912f073b415b812c3b
2020-02-04 11:15:18 -06:00
Umherirrender 1c05fa9049 build: Add missing .phpcs.xml and make pass
Add .phpcs.xml to run codesniffer against the extension.
The entry in composer.json is already there

Change-Id: If9de7e9b1b9439dc4d5fa2ba521883d13deaa739
2020-01-13 20:48:11 +01:00
Ed Sanders e2c31a8450 Don't load DT on edit pages
Bug: T239873
Change-Id: I2aaa2f008d08dd5be9e0dfa980a68a9bfda91b44
2019-12-09 14:43:39 +00:00
Ed Sanders 7745d4a610 Require $wgLocaltimezone to be set
Change-Id: I0e09d87152bf7694c8ada935d5c37c244d70cb69
2019-10-28 22:41:10 +00:00
Ed Sanders ccb631eedc Separate DiscussionToolsData from DiscussionToolsHooks
Change-Id: Icb15cc7d118ac20bcf131f6862095b4481d1c882
2019-10-28 12:29:56 +00:00
Bartosz Dziewoński 9efe8b1dd4 Add unit tests for parser#getTimestampRegexp
Depends-On: I6c3d186de1877f73d4a4e3fec7d6d632a5d5fa83
Change-Id: Icdb44f793a8f5e56666ec635bb8b0125041b5aab
2019-10-24 23:21:29 +02:00
Bartosz Dziewoński db80e48933 Handle timestamps in daylight saving time
Add the Moment Timezone library. Add a script for managing libraries,
like in MediaWiki core.

Depends-On: I9a59a6ad01850b30327e4215f2be61b8d1c41277
Change-Id: I64bc79e7d0ccdf42b006e5a225c8aa70ea5f4e15
2019-10-22 16:33:21 +02:00
Bartosz Dziewoński b105bf7ded Detect and parse timestamps, signatures, comments and threads
Bug: T232780
Bug: T234404
Change-Id: Ie9c80121089742cfc7cd7c04d694c2e0fe8d6a98
2019-10-18 13:59:07 +02:00
Ed Sanders 53fae97723 Add an empty hook to load discussion tools on talk pages
Change-Id: Iafa24ed41c06d4bd8b9a7a684b986fdb37aa87c7
2019-10-12 17:14:28 +00:00
Ed Sanders fe0a05ad49 Add required files for an extension
Change-Id: I626e607e6ae7efc2521f9dd1da83c6fd90b0767d
2019-10-10 19:42:56 +01:00