ParserOutput::getTOCHTML() is being deprecated, and all skins are now
generating the TOC from the TOCData returned by
ParserOutput::getTOCData(). The SHOW_TOC flag was introduced in core
to determine if the TOC should be shown, but Vector-2022 *may* begin to
use other heuristics to determine whether to show the TOC (T315862).
We're conservatively going to process the TOC as long as there is
TOC present to process.
Bug: T328072
Change-Id: I38b439c6752157dbee9b09c9f5443a740dbaabf4
Change code to match the documented consensus formed on T321683:
https://www.mediawiki.org/wiki/Manual:Coding_conventions/PHP#Exception_handling
* Do not directly throw Exception, Error or MWException
* Document checked exceptions with @throws
* Do not document unchecked exceptions
For this extension, I think it makes sense to consider DOMException an
unchecked exception too (in addition to the usual LogicException and
RuntimeException).
Depends-On: Id07e301c3f20afa135e5469ee234a27354485652
Depends-On: I869af06896b9757af18488b916211c5a41a8c563
Depends-On: I42d9b7465d1406a22ef1b3f6d8de426c60c90e2c
Change-Id: Ic9d9efd031a87fa5a93143f714f0adb20f0dd956
We verify that they appear in the action=parse API output.
We should also have a test to verify that they appear in the Vector
2022 skin, but that might have to be done as a browser test.
Bug: T326677
Change-Id: I81d5ddecd3b244aeee8cba71498f4d65f8e53696
The alias stored in the language files is with underscores,
but the value is compared in CommentParser against text with spaces
Affected languages: bjn, hu, id, jv, kaa, tl, tpi, vi, war
Bug: T327021
Change-Id: I8626627d10a240973e631e24508937a9eee9fb14
Test ServiceWiring.php using tests copied from CentralAuth. Because
phpunit does not support marking a file as covered, the ServiceWiring
file is ignored for code coverage as the tests fully cover the file.
Change-Id: I7da8d74fec84a5aa9c77bc0678ad8f55b550893a
ApiDiscussionToolsPageInfo and ApiDiscussionToolsCompare in direct parsoid
or VRS modes tries to fetch HTML using VisualEditor thus stashing the
HTML gotten which we don't want, we only need it for viewing in these cases.
This seems like something that was/is already happening in RESTBase. So for
APIs in DiscussionTools that need the HTML for viewing, just get it from
parser cache and not stash it.
Bug: T323357
Change-Id: I101c1e84739a2ac1f562f2f7bdc4b8f53d9f3b23
This is a better way than a loop in the test code.
When using DISCUSSIONTOOLS_OVERWRITE_TESTS, if both desktop and mobile
cases fail, both will be updated now. Previously the first failed
assertion would stop the test after only updating one file.
Change-Id: I4ce6f45b047e02c9f00024a9e5057adcb0e28047
Just to confirm that this works correctly, because I broke it a few
times in different ways while working on T321121.
In this scenario, discussiontools_item_revisions contains rows for
the oldest and newest revisions of pages containing each comment
distinguishable by name, as well as extra rows for the newest
revisions of comments indistinguishable by name, so that they can
be looked up by ID.
Change-Id: Ic8450a6b082ed343dd633d3a43c50696b5d6d2bb
discussiontools_item_pages row itp_id=5 has incorrect
itp_oldest_revision_id. It should point to the older of the two
revisions with the same time, but it points to the newer one.
discussiontools_item_revisions row is missing.
Depends-On: I56f0e161e5438d5f77b7d53d4db7411f90f97d05
Change-Id: I61ed42515891a84729455a7a32c98276c7cacd40
We originally used 'templates' because it seemed like an obvious
choice for HTML files, and because 'packageFiles' requires extra code
to include anything that isn't a .js or .json file.
However, the templates are expected to be HTML fragments rather than
whole documents, and they are parsed in a particular way that takes a
lot of code to clean up (which we needed to do, because we use the
same test files for testing PHP code).
I tried doing it in the 'packageFiles' way, and the extra code doesn't
seem that bad in comparison after all. Moreover, the 'templates'
mechanism (when used the intended way) feels vaguely deprecated in
favor of Vue.js, and I'd rather move away from it.
This makes the tests faster too (probably mostly thanks to the removal
of the clean up code) – on my machine they go from 1800ms to 1500ms.
(Simplify linearWalk tests, as we no longer need to do weird things
with document fragments to get consistent outputs in PHP and JS.)
Change-Id: I39f9b994ce5636d70fea2e935a7c87c7d56dcb26
Many methods are covered indirectly, and using method-level
@covers filters means these are reported as not being covered.
Change-Id: I94eb3e8c48209ff0b6bfc09e18c93555bb167e8f
The implementation in Parser doesn't descend into sub-thread.
Re-use the getThreadSummary method in ThreadItem and traverse
the thread properly.
Bug: T298617
Change-Id: I318d9012eb83f37ccbe463923524ef2e9f995ced
In the future the notifications can be improved to look up
the new location of the comment, using the permalinks data.
Depends-On: Ia8a21749a8edc20f34b2a3e445278ea6922b9109
Bug: T299657
Change-Id: I5f5e7b73fb84ff0d31fb8260b24066a17da71628
We can no longer change IDs so easily, because they're stored in the
permalink database, so remove this mechanism to make sure it's not
accidentally used in the future.
Change-Id: I392ee1f49c48fc2f23d05e9a37c643438b4f2b9a
* clock, userAvatar, speechBubble were dropped before topic
containers was merged
* Load ellipsis & edit icons on mobile only
* Load bell & bellOutline for topic subscriptions button
Change-Id: I77d1336627b564be756e3ec50b4686b8f9ac72dc
Rename ThreadItem to ContentThreadItem, then create a new ThreadItem
interface containing only the methods that we'll be able to implement
using only the persistently stored data (no parsing), then create a
DatabaseThreadItem. Do the same for CommentItem and HeadingItem.
ThreadItemSet gets a similar treatment, but it's basically only for
Phan's type checking. (This is sad.)
Change-Id: I1633049befe8ec169753b82eb876459af1f63fe8
While in many cases the class will never be sub-classed, it's easier
just to always use static:: and not worry about predicting which
classes might have problems in the future.
Change-Id: I23072a1701b5acf62bb3379a877de97627d8fcf3
We were calling Title::newFromText() before setupEnv(), which meant
that the title for each test case was parsed using the default rules
for English, rather than the rules for the specified wiki.
This only makes a practical difference for tests with self-links.
Changed the only such test to demonstrate the fix.
Change-Id: I45561f1c9f0d149e2b743f0000b742bf6fc014af
Since times immemorial, and for reasons lost to history, our test code
was adding an extra <div> wrapper before parsing the HTML used for
tests. This wasn't a problem, until now, because I want to add some
tests for T303396 that need to check that the *real* wrappers present
in some test cases are handled correctly.
Changes to test cases mostly remove a leading "0/" from serialized
ranges, corresponding to removing the extra wrapper.
Change-Id: Ia50e3590538c8cd274b02d2a937ba1a3fbb4ac89
It is friendlier for static analysis tools like Phan, which can't
infer anything from the `->nodeType === …` checks, and we were already
using it in most places.
Fix newly revealed Phan failures (and one unneeded suppression).
Change-Id: Id789f05e16a210f7ba22ca7514587c392fac0741
Previously, we required a signature at the end of the comment.
This was a pretty rough heuristic that did not correctly handle
many comments that we would consider entirely properly signed
in CommentParser (e.g. comments wrapped in formatting like
<small>…</small>, comments with a post-scriptum or in parentheses,
or comments generated by various templates).
Now we process the user input using the same code that adds reply
links, and only add a signature when we detect that there really
isn't a signature (including template-generated), or if the signature
is in the wrong place and would result in the reply link showing up
in the wrong place as well (not at the end of the comment).
Bug: T278442
Bug: T268558
Bug: T278355
Bug: T291421
Bug: T282983
Change-Id: I46b6110af328ebdf93b7dfc2bd941e04391a1599
Goal:
-----
Finishing the work from Iadb7757debe000025e52770ca51ebcf24ca8ee66
by changing CommentParser::parse() to return a data object, instead of
the whole parser.
Changes:
--------
ThreadItemSet.php:
ThreadItemSet.js:
* New data class to access the results of parsing a discussion. Most
methods and properties are moved from CommentParser with no changes.
CommentParser.php:
Parser.js:
* parse() returns a new ThreadItemSet.
* Remove methods moved to ThreadItemSet.
* Placeholder headings are generated slightly differently, as we process
things in a different order.
* Grouping threads and computing IDs/names is no longer lazy. We always
needed IDs/names anyway.
* computeId() explicitly uses a ThreadItemSet to check the existing IDs
when de-duplicating.
controller.js:
* Move the code for turning some nodes annotated by CommentFormatter
into a ThreadItemSet (previously a Parser) from controller#init to
ThreadItemSet.static.newFromAnnotatedNodes, and rewrite it to handle
assigning parents/replies and recalculating legacy IDs more nicely.
* mw.dt.pageThreads is now a ThreadItemSet.
Change-Id: I49bfe019aa460651447fd383f73eafa9d7180a92
Goal:
-----
To have a method like CommentParser::parse(), which just takes a node
to parse and a title and returns plain data, so that we don't need to
keep track of the config to construct a CommentParser object (the
required config like content language is provided by services) and
we don't need to keep that object around after parsing.
Changes:
--------
CommentParser.php:
* …is now a service. Constructor only takes services as arguments.
The node and title are passed to a new parse() method.
* parse() should return plain data, but I split this part to a separate
patch for ease of review: I49bfe019aa460651447fd383f73eafa9d7180a92.
* CommentParser still cheats and accesses global state in a few places,
e.g. calling Title::makeTitleSafe or CommentUtils::getTitleFromUrl,
so we can't turn its tests into true unit tests. This work is left
for future commits.
LanguageData.php:
* …is now a service, instead of a static class.
Parser.js:
* …is not a real service, but it's changed to behave in a similar way.
Constructor takes only the required config as argument,
and node and title are instead passed to a new parse() method.
CommentParserTest.php:
parser.test.js:
* Can be simplified, now that we don't need a useless node and title
to test internal methods that don't use them.
testUtils.js:
* Can be simplified, now that we don't need to override internal
ResourceLoader stuff just to change the parser config.
Change-Id: Iadb7757debe000025e52770ca51ebcf24ca8ee66
The Data class contained utilities for two unrelated purposes.
Split each half to a separate class.
Notably, this improves the signature of the getLocalData() function.
Change-Id: Icde615fb9d483fee1f352c34909b37f8ffde8081
(suggested by PhpStorm)
composer.json:
* Document required PHP extensions
Parser.js:
* Remove incorrect param documentation
* Fix some typos in comments (missing parentheses)
CommentParser.php:
* Fix some typos in comments (missing parentheses)
ImmutableRange.php:
* Remove unused property
* Add a `throw` to indicate that code path is unreachable
SubscribedNewCommentPresentationModel.php:
* Add missing `return false`
CommentParserTest.php:
* Remove unnecessary pass-by-reference
CommentModifierTest.php:
* Remove unused variable
CommentParserTest.php:
* Don't construct Element objects directly. PHP's DOMElement allows
it, but Parsoid/Dodo's doesn't, and we use the latter for static
analysis. This generates all kinds of confusing warnings.
Change-Id: Ia9598ebea0e99830dd485296e94a9d96acc4b258
Goal: To be able to re-use or test the transformations we previously
performed in addWikitextReply() / addHtmlReply(), without requiring
a Comment object or adding the result as a reply.
Change-Id: I040c4be9b6b9bddba661f30fd0566f8850673074
These are not used for anything yet, but soon the parser will
want to know the title of the page it is parsing.
Change-Id: I02fa5d63fae78f3e92032d93bc27ac5c744faecb
The following values for configuration variables are supported:
$wgDiscussionToolsReplyIndentation = 'invisible'; (default)
$wgDiscussionToolsReplyIndentation = 'bullet';
Bug: T259864
Change-Id: Icefad79630adc6ed35687498614e6a03ede1451b
Data used for the tests assumes there are no variants for English,
and some tests fail when there are. Correct behavior with language
variants is tested using other languages.
Change-Id: I348a0ba0389c2a18644ce5e05c7f37d8f26a8c55
We do something similar in CommentItem.js with a moment object.
The object can be converted to a string when required.
Change-Id: Id7221e9201db0d89c3b771574634c878c9515ca0
The PHP DOM extension measures lengths and offsets in Unicode codepoints.
Our PHP code used UTF-8 bytes, causing some offsets to be slightly off.
Now it mostly uses Unicode codepoints as well (we're forced to use bytes
in a few places, because preg_match returns offsets in bytes).
In practice, this had no visible effect to the user. It caused the
markers `<span data-mw-comment-end="..."></span>` to be placed at
the end of their container instead of the correct position when the
timestamp contained multibyte characters (e.g. "ź" in Polish); but
the correct position is usually at the end of the container anyway.
In the test cases, the only difference is placing these markers before
a trailing line break inside `<p>...</p>` tags rather than before it.
The patch also accidentally fixes another bug, where element nodes
with no children (mostly <img>) were incorrectly excluded when calling
cloneContents(), because they were treated as if they were text nodes.
Change-Id: Iccdccf1078598f4b62cab96225e9c85a4c0e93ee
* ThreadItem::getText
* CommentItem::getBodyText (used when generating notifications)
* ThreadItem::getHTML (may soon be used in API)
* CommentItem::getBodyHTML (may soon be used in API)
* ImmutableRange::cloneContents (the common implementation for all
of the above)
The outputs are only lightly reviewed. This is mostly meant to
document the current behavior rather than the expected behavior,
to avoid making unintentional changes while refactoring.
Change-Id: I14471ee4969aa3d0b5577d9de2a6d4462fab4d09
The code (prior to d25825a754) assumed
that level 3+ headings would always follow a level 2 heading or the
placeholder heading, but we don't generate a placeholder heading if
there are no comments in section zero.
Add more tests to confirm that comments under level 3+ headings (that
are not sub-headings of level 2), and level 1 headings, are ignored
when generating notifications, and do not mess with normal headings.
Bug: T288775
Change-Id: Ic57b56752a4797cb01234f66e0ed7b849752bd70
This DOMDocument property has no effect, because we do not use
DOMDocument methods for parsing HTML, but rather DOMUtils::parseHTML()
provided by Parsoid.
Change-Id: I1d9e73e53f2d44f41cf9dcda4f06ac8647671096
Use `DOMCompat::getBody( ... )` as a nicer getter than
`->getElementsByTagName( 'body' )->item( 0 )`.
Remove overly defensive checks and redundant annotations on its
return value. Since we're dealing with HTML documents throughout,
the document body is guaranteed to exist.
We previously needed some of them to convince Phan when it thought
the body may be null, but this seems to no longer be needed.
Change-Id: If7aee7b6adbfa78269c7ba28b26a6eaa21fe935b
Adding test cases in a separate commit to make it easier to review how
the test results change after I98fbca8e.
* For mentions, the 'mentioned-users' extra parameter is copied to our
event (which is then used to avoid duplicate notifications).
* For user talk page edit, nothing special happens right now (we use
the target page title to avoid duplicate notifications, but this is
not apparent from the test case, since page titles are not present).
Bug: T281590
Bug: T253082
Change-Id: I153e7735f63f1e2643ed881281d807313cd699c3
Adding test cases in a separate commit to make it easier to review how
the test results change.
As expected, in every case, no notifications are generated right now.
Bug: T285528
Change-Id: I25308754112c521d2db8c54ef0c82373456d9e31
These changes ensure that DiscussionTools is independent of DOM
library choice, and will not break if/when Parsoid switches to an
alternate (more standards-compliant) DOM library.
We run `phan` against the Dodo standards-compliant DOM library,
so this ends up flagging uses of non-standard PHP extensions to
the DOM. These will be suppressed for now with a "Nonstandard DOM"
comment that can be grepped for, since they will eventually
will need to be rewritten or worked around.
Most frequent issues:
* Node::nodeValue and Node::textContent and Element::getAttribute()
can return null in a spec-compliant implementation. Add `?? ''` to
make spec-compliant results consistent w/ what PHP returns.
* DOMXPath doesn't accept anything except DOMDocument. These uses
should be replaced with DOMCompat::querySelectorAll() or similar
(which end up using DOMXPath under the covers for DOMDocument any way,
but are implemented more efficiently in a spec-compliant
implementation).
* A couple of times we have code like:
`while ($node->firstChild!==null) { $node = $node->firstChild; }`
and phan's analysis isn't strong enough to determine that $node is still
non-null after the while. This same issue should appear with DOMDocument
but phan doesn't complain for some reason.
One apparently legit issue:
* Node::insertBefore() is once called in a funny way which leans on
the fact that the second option is optional in PHP. This seems to be
a workaround for an ancient PHP bug, and can probably be safely
removed.
Bug: T287611
Bug: T217867
Change-Id: I3c4f41c3819770f85d68157c9f690d650b7266a3
For compatibility with Parsoid's document abstraction (Parsoid may
switch to an alternate DOM library in the future), don't explicitly
create a new document object using `new DOMDocument`; instead use
the Parsoid wrapper `DOMCompat::newDocument()`. This ensures that
the Document object created will be compatible with Parsoid.
There are a number of other subtle dependencies on the PHP `dom`
extension in DiscussionTools, like explicit `instanceof` tests; those
will be tweaked in a follow-up patch
(I3c4f41c3819770f85d68157c9f690d650b7266a3) since they do not affect
correctness so long as Parsoid is aliasing Document to a subclass of
the built-in DOMDocument. Similarly, the Phan warnings we suppress
do not cause runtime errors (because of the fixes included in
c5265341afd9efde6b54ba56dc009aab88eff83c) but phan will be happier
once the follow-up patch lands and aligns all the DOM types.
Bug: T287611
Depends-On: If0671255779571a91d3472a9d90d0f2d69dd1f7d
Change-Id: Ib98bd5b76de7a0d32a29840d1ce04379c72ef486
The user interface only allows you to subscribe to level 2 headings.
But we would generate events for whatever heading was the closest,
If it was e.g. level 3, no one would receive that notification.
Now we generate events for the closest level 2 heading, or we don't
generate the event at all if there isn't one (if the only headings are
of level 3 and below, or level 1, or if the comment is added before
the first heading on the page).
Bug: T286736
Change-Id: Iae99853070e353ab81c9cc29ef1d53c877adfc66
The existing comment IDs can't be used to find the same comment on
a different revision or page (when it's transcluded), because they
depend on the comment's parent and its position on the page.
Comment names depend only on the author and timestamp. The trade-off
is that they can't distinguish comments posted within the same minute,
or in the same edit, so we will still need the IDs sometimes.
Prefer using comment names when replying, if they're not ambiguous.
This fixes T273413 and T275821.
Heading names depend on the author and timestamp of the oldest comment.
This way we don't have to detect changes to the heading text, but we
can't distinguish headings without any comments.
Bug: T274685
Bug: T273413
Bug: T275821
Change-Id: Id85c50ba38d1e532cec106708c077b908a3fcd49
The code we're testing already produces a string of serialized HTML,
no need to parse and re-serialize it.
Also, we recently learned that the precise format matters here
(T274709), and now this test *actually* covers the fix for that bug.
Follow-up to 5b26e9664b.
As a downside, this test might now spuriously fail if the format of
the output of Parsoid's XMLSerializer changes. Hopefully that won't
happen too often.
Change-Id: I69b514f545e47dcb437fb39a83edb8e2f19ed99b
Now it detect signatures generated by en.wp's {{Undated}} template,
and signatures of people who do weird stuff to the timestamps.
Bug: T275938
Change-Id: I27b07f6786ca5433a3c02a5fe68e4716d41401bb
Documentation:
https://www.mediawiki.org/wiki/Manual:PHP_unit_testing/Writing_unit_tests_for_extensions#Two_types_of_tests
We can do this because the tested methods do not depend on any globals
or on MediaWiki being installed.
In addition to being the new hotness, MediaWikiUnitTestCase allows the
test classes that use it instead of MediaWikiTestCase to start up much
faster. In my testing, running this test case individually now takes
0.35s, compared to 1.1s before.
Try:
* With new code:
time php tests/phpunit/phpunit.php extensions/DiscussionTools/tests/phpunit/unit/CommentUtilsTest.php
* With old code:
time php tests/phpunit/phpunit.php extensions/DiscussionTools/tests/phpunit/CommentUtilsTest.php
Change-Id: I771b1f3d101a394ee869e42547d9ae7839397752
Add yet another tree walking utility: CommentUtils::linearWalk().
Unlike TreeWalker, it allows handling the beginnings and ends of nodes
separately – kind of like parsing a XML token stream, or kind of like
VisualEditor's linear model.
(Add unit tests for this utility. The simple.html test case is copied
from [VisualEditor/VisualEditor]/demos/ve/pages/simple.html.)
Use this utility to stop skipping when we reach either a closing or
opening block node tag. Previously we'd skip over such tags inside
nested "transparent" nodes (like <a>, <del>, or apparently <font>).
Bug: T271385
Change-Id: I201a942eb3a56335e84d94e150ec2c33f8b4f4e0
