Commit graph

127 commits

Author SHA1 Message Date
thiemowmde 5aa6cb0c7b Replace extremely slow parser test with fast unit tests
This parser test is a bit obscure, in my opinion. We added it in
I8c4de96 to make sure we don't get thousand separators in most
places.

We continued reworking the code since then. By now it's effectively
impossible to "accidentally" get thousand separators. The
problematic methods from the Language class are not even accessible
any more from this code.

To make the tests more robust we now use createNoOpMock (done via
the previous patch) where it matters, specifically for all Language
and Parser mocks. This proves the problematic Language methods are
never called.

Bug: T253743
Bug: T238187
Change-Id: I9bfe1f4decfaf699996da63e19473c2c0d581d9d
2023-07-28 00:32:38 +00:00
Arlo Breault 9177b50feb Sync up Cite repo with Parsoid
This now aligns with Parsoid commit 7b724ddc6c4abd44de2e1f67f64ca1d9685c6b4f

Change-Id: If24ad064d7d84fb070cb0c7976d56373e8a9db3a
2023-05-12 09:59:44 -04:00
Arlo Breault 6ed2daec12 Disabled tests to break circular dependency
html/php sections are added since otherwise it complains that the
"Test lacks html or metadata section on lines"

Change-Id: Ib1c47be09bdbe1e84b595373ad71772f2a983fc9
2023-05-10 19:47:13 -04:00
Arlo Breault 1c8a0115e9 Update parsertests with new media classes
Depends-On: Ifd4001e312a5fa4b7beaad63ba8c4e79e3201b9b
Change-Id: I80b76e2f4f538eba323f47cb2bf831016e2b2dc2
2023-04-26 13:19:58 -04:00
Arlo Breault 984aa7750c Disable tests to break circular dependency
Needed-By: Ifd4001e312a5fa4b7beaad63ba8c4e79e3201b9b
Change-Id: Ie85ee7048273023a2c51f42a333a9c1493360404
2023-04-20 16:00:43 -04:00
Subramanya Sastry fc0a239887 Document Parsoid's differing HTML for follows
* Add a file-level comment in the cite tests file.
* Document the CSS rule that hides the Parsoid HTML.

Change-Id: I27dc6d5f6ab09b67e28ce88a2e13bf2d1a13e9c0
2023-03-13 14:38:30 -05:00
Subramanya Sastry d8da2cbb28 Enable integrated testing with Parsoid
* The failing tests added to known failures are the tests
  known to fail as documented in T307741.

Bug: T307741
Change-Id: I5e5163a4bd093768d1364516ed79fb2d225ee656
2023-03-07 00:22:55 -06:00
thiemowmde c207e343a2 Add test case with conflicting dir="…" values
Just to document the current behavior.

Bug: T202593
Change-Id: I87c4118cd8ca9f860319dc1d3a25f448019339c4
2023-03-01 08:54:56 -06:00
thiemowmde 899775b5ce Remove problematic spaces from a parser test case + Rename a test file
IDEs like my PHPStorm trim spaces from the end of the line. It looks
like they are not relevant for the test and can as well be removed.

Change-Id: I54cb4fdf74dd7174450dcc552b077d388dbac749
2023-03-01 08:54:11 -06:00
Arlo Breault 1dd994a28a Re-enable test
Change-Id: I9c909e10170a0437676bdf1290cc6421073a8284
Follow-Up: If9cdabdfac26656272fcf3b4aaae0576aaed1346
Depends-On: If1e55feb86ce8b32f772e3b78bc9d29f122f4d58
2023-02-09 17:30:52 -05:00
Arlo Breault 5fa75277b7 Disable test for CI
Needed-By: If1e55feb86ce8b32f772e3b78bc9d29f122f4d58
Change-Id: If9cdabdfac26656272fcf3b4aaae0576aaed1346
2023-02-09 17:28:08 -05:00
Arlo Breault d5b5a83ae9 Sync up Cite repo with Parsoid
This now aligns with Parsoid commit b520b8092db6f092603e7244cdac5f1fc645e89a

Change-Id: I671c5831d90bbeb088f5ff2133df553142e1af53
2023-02-09 16:58:34 -05:00
Arlo Breault 0060e2b43d Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 9041d0bac69fe763112c0d8fcbd553a211a25c26

Change-Id: I34b1fdf2232e67db5326ed25ab3f4ad128e59e5d
2022-08-23 15:11:06 -04:00
Thiemo Kreuz 8fef0dd2aa Improve two error messages
Makes it easier to find the source of the error in the wikitext.

Change-Id: I648a19881210184ab1abe9b948b5efbbbdabcdc9
2022-08-20 12:23:28 +02:00
C. Scott Ananian d730d17c1a parser tests: move test which requires {{#ifeq}} into its own file
Since parser test requirements are per-file, move the smoke test which
requires `{{#ifeq}}` (from [[mw:Extension:ParserFunctions]]) into its
own file and define the requirement properly in the file header.

That avoids spurious parser test failures if developers don't have
the ParserFunctions extension installed locally.

Change-Id: Ia5ffbe0896d5033fe2da526e42bf111edbc56adf
2022-05-27 11:38:02 -04:00
Subramanya Sastry 5515de2003 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit e4305217923162568cf5b6ec08ba3c96fd5b04e3

Change-Id: I935d8641d90ec5cbcf7d29fd984085cc5338b6ec
2022-05-17 18:00:18 -05:00
C. Scott Ananian af2352e523 parser tests: Make !! config values JSON-compatible
Bug: T307720
Change-Id: Ib716c70bc47659701edfc572674b3e890e19605b
2022-05-11 21:05:55 -04:00
Subramanya Sastry 45cc963ca2 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 12d896bd5852b4a7d602fb22dd09d7fc2c5c5b64

Change-Id: I1d3183ee4afa37e0d71768cd02f03112771b82c4
2022-05-11 18:08:24 -05:00
Subramanya Sastry a478d6c3a3 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit c434df9.

Change-Id: Ib8b8593d1a303530dac56ed01335e0d635863eda
2022-04-20 19:03:18 -04:00
Subramanya Sastry 169ebf6344 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 3b5a7a318eaf8dcc5e6ed142352f1cf1a9439474

Change-Id: I32f9fcac231cdf25bd1501d7f6e022e067c8d8e1
2022-04-14 12:15:53 +05:30
Thiemo Kreuz 7b30a165e4 Use correct Sanitizer method for id/fragment escaping
Note how only the HTML5 mode behavior changes, but nothing in
legacy mode.

Also note this does not 100% fix the issue. The esample with a
non-breaking space is still broken. But it's already much better
than before.

Bug: T298278
Change-Id: Idf50dad4219ff4c594a0cc15f63cb10fdac5ffb7
2022-01-03 16:23:45 +01:00
Thiemo Kreuz 83041449a7 Add parser & unit test cases for different $wgFragmentMode's
This is only to document the current status quo and make later patches
smaller and easier to review.

Bug: T298278
Change-Id: I6c78f4d3ee32de596f2b5ee081d56eaffb1cc7bd
2022-01-03 14:14:47 +01:00
Subramanya Sastry ed59e2ac38 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 29f8e7051529ecbb62fc52bff6726a4df8bf20c2

Change-Id: I165ee24e1b78bdf181fa45430fdec1549310c359
2021-09-30 15:00:56 -05:00
C. Scott Ananian 53e8dc7b39 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 88d4620278988d121761fb440952d1d66a70ce99

Required some newline fixes to resync after "Refactor newline logic"
(change I6691c70f8e3fa3f21e2d11035bed9cdc2dc87093 /
commit 6389459b1e) was merged
this morning.

Change-Id: I64fba6cc9330a55d4e1eeb5371164b3eb4efa508
2021-07-30 11:14:38 -04:00
C. Scott Ananian 6b813c6874 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit d9a3e14dfcb422e95de7a79f0eb662fd43f9354f

Change-Id: Iae34d2107bfb47304819da7f7c715dec83da1a48
2020-08-18 16:54:33 -04:00
C. Scott Ananian a0faedb942 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 77db605163990ad851e3da0fb4fa7eca2081f379

Change-Id: I19ad9f1b6845e5557c2f5f87ac435db0ad87000d
2020-07-28 15:53:08 -04:00
C. Scott Ananian 90728bde87 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 7321ab547b7663ba86c1cfe0bc021ff1918c0970

Change-Id: I2cc88069b19e7611f23c83ca993f9caa70f786f0
2020-07-15 11:39:46 -04:00
Bartosz Dziewoński 1975cb3dcb Do not add thousands separators when formatting reference numbers
Bug: T253743
Change-Id: I8c4de963277895d7751d6bfe3c34ca6097ebe606
2020-05-28 00:08:44 +02:00
Thiemo Kreuz 7fbd5de7f5 Merge two code paths related to follow
This patch also adds a test case that was missing before. If a
follow="…" is followed by another, normal <ref>, the internal key
(a.k.a. $this->refSequence) is not incremented. This was the case
before, just not covered by any test.

Change-Id: I102d1e67a6918017acc7e4a4663b08c828d101a6
2020-05-12 10:52:08 +02:00
jenkins-bot 53cf713cdb Merge "Add test cases for impossible follow vs. rollback edge case" 2020-05-12 02:03:35 +00:00
Subramanya Sastry eb8e07c69c Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 1e72ddefc2df8d5b38d7b370d67a87b48d8a09f7

Change-Id: Ic75ac33c92997427751d037339f7eb79027356a2
2020-05-07 12:14:19 -05:00
Bartosz Dziewoński 3678215a77 Add a newline in wikitext before autogenerated reflist
Previously the reflist was added at the end of the last line of text,
which messes up paragraph wrapping (as seen in many test cases), and
generated invalid HTML when the last line was a list item (T148701).

(second try, previously reverted in 8c933d03c5)

Note this affects only pages where the <references /> tag is missing,
and the references section is auto-generated at the very end of the page.

Bug: T148701
Change-Id: Ib2101346434a4e317b5fc7379215b60c7020cb2b
2020-05-06 20:51:25 +02:00
C. Scott Ananian 376c0418d3 Update parser tests to v2 (tidy by default)
The most common cleanup required by switching to tidy output was adding
missing <p>-wrappers to the last item before <references/>.

Bug: T246285
Change-Id: I7c8a08c4e6eff7caf4539a26fae475a4133f9a0c
2020-04-01 11:11:13 -04:00
Thiemo Kreuz a00b8d990d Add test cases for impossible follow vs. rollback edge case
While working on the patch I4303642 I was worried about the line

array_pop( $this->refCallStack )

in the rollback code. Since the patch changed the position of follow
elements in the stack, an array_pop() would pop different elements.

It turns out this is impossible. Rollbacks are only done for <ref>
elements inside a <references> tag, immediatelly after reaching the
closing </references>. It's impossible to use follow="…" inside
<references>. It will not be added to the stack, and therefore not
rolled back.

Even if the edge case would be possible, the *old* code that placed
follow elements on the *other* side of the stack would have been
wrong then.

The test cases in this patch try to hit this edge case, and are
expected to not be able to do so.

Change-Id: I4380bf443db17c6214dbfa2cbda62b46db04258a
2020-04-01 09:03:19 +00:00
Awight 8c933d03c5 Revert "Add a newline in wikitext before autogenerated reflist"
This reverts commit 90697ffe43.

Change-Id: I659ce1689603fd16e378fb8d3d5bd6d1089342b2
2020-04-01 08:03:55 +00:00
Bartosz Dziewoński 90697ffe43 Add a newline in wikitext before autogenerated reflist
Previously the reflist was added at the end of the last line of text,
which messes up paragraph wrapping (as seen in many test cases), and
generated invalid HTML when the last line was a list item (T148701).

Bug: T148701
Change-Id: Ifc873fc913e717026d80d54b570c594d1073fb42
2020-03-31 19:00:51 +00:00
Thiemo Kreuz 48e2f02e20 Add two extreme follow edge cases back to parser tests
This reverts parts of the revert I3bee35f, which reverted a3d312c8.
I believe it's helpful to keep these test cases just to document how
the code currently behaves. I removed all TODO because we don't know
if and how we want to touch this again.

Bug: T240858
Change-Id: Ib91acfcb7292e5c03ce9cc4d7be782085e10aa27
2020-02-05 15:04:49 +01:00
Adam Wight f2bd6b6dcc Revert "Standardize "follow" validation"
This reverts commit a3d312c8f4.

Bug: T240858
Change-Id: I3bee35f27797a04c41c265f7e598d8383414b67a
2020-02-05 11:42:28 +01:00
Adam Wight a3d312c8f4 Standardize "follow" validation
Perform the validation in validateRef, and display a new error message for
broken "follow" refs.  This changes existing behavior, where broken folow
ref content is arbitrarily displayed at the top of the references list and
no error is rendered.

Thanks to weasely wording, the new error can later be reused for "extends"
errors.

Bug: T240858
Change-Id: I506e4dcd1151671f5302ecd99581145d979d8124
2020-01-30 17:25:42 +00:00
jenkins-bot 3c8a225052 Merge "Fix incomplete rollback producing bad footnote numbers" 2020-01-24 15:11:31 +00:00
Thiemo Kreuz 2ddc6f133b Fix incomplete rollback producing bad footnote numbers
Bug: T48140
Change-Id: I53ce5d8488d4c24d6f23f6f0e70806d7db4064e1
2020-01-24 13:02:53 +01:00
Thiemo Kreuz 816b1b0add Remove newline characters from all error messages
These create bogus output, depending on the surrounding wikitext the
<ref> tag is used in. For example, this example wikitext:

* Example.<ref name="1">a</ref> More text.

… will be rendered with the "More text" sentence wrapped on the next
line, outside of the list. However, this does *not* happen in many of
the localizations, e.g. German, because many Tanslatewiki translators
did not copied the bogus \n. Why should they.

TL;DR: These newline characters either do nothing, or destroy the output.
In both cases the proper fix is to replace them with spaces.

Some of the test cases touched in this patch demonstrate the issue.

Change-Id: I395a40637a5293eda1f477963d252ce1a215f8b2
2020-01-24 12:29:14 +01:00
Thiemo Kreuz 51d55bb8de Introduce dedicated error message for nested <ref extends=…>
This resolves another TODO. Since this is an intentional limitation in
the design of the feature, I find it pretty signigicant to give it it's
own error message.

Note that the text does not need to be perfect, just good enough for now.
We will review all error messages later via T238188.

Bug: T242141
Change-Id: Id9c863061e855350320131e81f6702c8810736f4
2020-01-23 15:00:26 +01:00
Thiemo Kreuz 9565d6e887 Resolve a TODO by covering it with a test case
It turns out this is indeed necessary. The test demonstrates why.

Change-Id: Id9c6a48f72ef8d3f0cc9a714d826418e69913b0a
2020-01-21 10:11:03 +00:00
jenkins-bot 258b23a6dd Merge "Error when reusing <ref> with conflicting "extends" attributes" 2020-01-20 13:49:41 +00:00
Adam Wight 8a58ed55dc Error when reusing <ref> with conflicting "extends" attributes
"Conflicting" here includes the case where one of two <ref> with the
same name does not have an extends attribute. The first occurence of
a name specifies if a <ref> is a top-level or a sub-reference. This can
not be changed later.

This patch changes multiple existing test cases. I checked all of them
in detail and confirmed the behavior is fine. The error reporting is
better or at least equally good in all cases.

Bug: T242141
Change-Id: Iaec306eefe5b168d496990105e297ca044a5e721
2020-01-20 13:33:52 +00:00
Adam Wight b3ea9f4ef8 Relax empty-string name validation
Allow a ref with `name=""` for backwards-compatibility.
Partially reverts I07738cce2641026dfaa92ba263ed6f9834be0944

Bug: T242437
Change-Id: Iaed2d1c41be377a4961aff39838b0965f6c00616
2020-01-20 12:40:09 +00:00
Arlo Breault 7e2bbae4c2 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 0a6c576ad6ccfc81c2bfa20757417c62e554ef56

Change-Id: Ifa5d60e362e5c530d12d3b94351aef2d1b1962cc
2020-01-17 14:51:03 -05:00
Adam Wight f3031b80b9 Fix for blank-named ref in #tag
A fun edge case where `name=""` fools both validation branches after
a references rollback, and triggered a LogicException.  Stop these
freak refs.

Bug: T242437
Change-Id: I07738cce2641026dfaa92ba263ed6f9834be0944
2020-01-17 11:19:29 +01:00
Adam Wight 1c947a808d Fix for nested #tag:references
It's possible to nest <references> by using tricky constructs like the
{{#tag function, and this breaks our rollback logic.  Try to show normal
output, otherwise show an error.

Includes regression tests.

Bug: T242437
Change-Id: I33e497cdf8508ce7ccb7f0f315c00af5eee47d0e
2020-01-15 12:44:29 +01:00