Commit graph

54 commits

Author SHA1 Message Date
jenkins-bot 3b41cfa472 Merge "Fail early on nested extends="…", if possible" 2019-12-09 10:12:54 +00:00
jenkins-bot 399a9c63bf Merge "Numbering bug: Parser test which should fail" 2019-12-09 10:06:06 +00:00
Thiemo Kreuz c5fe49ff11 Fail early on nested extends="…", if possible
This partly reverts Ied2e3f5. I haven't properly tested this before.
Rendering a bad extends (that extends a <ref> that's already extended)
not indented messes the order up and rips other extended <ref>s out of
context.

For now it might be better to stick to the previous, "magic" behavior:
Such an extends behaves like it is extending the *parent*, and is
ordered and indented as such. This is still not correct, but I feel
this is much better than rendering such a bad extends on the top level.

This patch also makes the code fail much earlier for a nested extends,
if this decision can be made already. In this case the error message is
rendered in the middle of the text (as other errors also are), not in
the <references> section.

Change-Id: I33c6a763cd6c11df09d10dfab73f955ed15e9d36
2019-12-09 10:54:52 +01:00
Adam Wight a91cf43154 Remove outdated TODOs
These edge cases are handled correctly already, I just forgot to
remove the TODOs when updating test content.

Note that there's only one TODO left, and it's to forbid a feature which
actually works!

Change-Id: I0d3a1f55f0ce943b0d034dda40e3779fbf241fe4
2019-12-09 10:25:19 +01:00
Adam Wight d8433101a7 Numbering bug: Parser test which should fail
Includes the TODO for what correct output looks like.

Bug: T237241
Change-Id: I0e60724f2c418b19e5affc24dca7f446c2b38bb3
2019-12-09 09:53:53 +01:00
Thiemo Kreuz 31bda4777b Don't indent refs with forbidden extends="…"
Change-Id: Ied2e3f56ce66d2a8ccf60df2bdbf99acad461595
2019-12-04 15:17:03 +00:00
Adam Wight 81261493c2 Show error when extending a subreference
Change-Id: Iaa47e302e5e49dfc190fde37567a3e7a2e743d67
2019-12-04 13:49:31 +01:00
Thiemo Kreuz 54333c9bd6 Stop formatting "1.2" as "1,2" in languages like German
The new extends="…" feature is using numbers like "1.2". These should be
localized in languages like Hebrew that uses other symbols for the digits.
But the "." should not change.

The existing feature when a <ref> is reused multiple times does have the
same "issue". But it seems this is intentional, because it is covered by
a test. Note this is not visible in German, because German uses custom
labels "a", "b", and so on.

This patch also improves the so called "smoke" tests and makes one cover
numbers up to "1,10" for a <ref> that is reused that often.

Bug: T239725
Change-Id: Iffcb56e1c7be09cefed9dabb1d6391eb6ad995ce
2019-12-04 09:43:04 +01:00
Adam Wight 008526b3aa Can use extends before its parent
If `extends` is encountered before the parent ref, we reserve the
sequence number and leave a placeholder to record the link between
ref name and number.  This is necessary to render a list like,
"[1] [2.1] [2]", or to use subreferencing when the parent ref is
declared in the references tag.

When a placeholder is encountered during references section rendering,
it means that the parent was never declared.

Change-Id: I611cd1d73f775908926a803fae90d039ce122ab6
2019-12-02 17:14:11 +01:00
jenkins-bot fa4410836d Merge "Split ref.number field" 2019-12-02 15:29:53 +00:00
jenkins-bot 79e7b2b474 Merge "Add test cases for duplicate <references> with same group" 2019-12-02 15:26:02 +00:00
Thiemo Kreuz 2cb7e5d438 Add test cases for duplicate <references> with same group
Change-Id: I9603e7ebf167330b1eddae1676e9234edf6557bc
2019-12-02 15:08:15 +00:00
Adam Wight 3f276388bf Split ref.number field
This was carrying the entire footnote marker, but subreferences need
to extract just the first (group ref sequence) part.  Storing number
and extendsIndex in two separate fields gives us more flexibility
during rendering, for example these might use two different symbol sets.

Change-Id: I75bd6644c336036f9e84ba91e1c35e05bc1ca7f3
2019-12-02 10:17:24 +01:00
Adam Wight 00f3be7c7f Reset extendsCount after each group
This was a bug which would affect book references, if the same group
and parent ref name combination occur twice in an article.

Change-Id: I608f58aac0cec31c8650835fc80195a87bc851d3
2019-12-02 10:17:18 +01:00
Thiemo Kreuz 13598ba11e Render nested references
Forked from Icd933fc983.

Bugs and unimplemented features are documented as TODOs in the parser test
fixtures.

Bug: T237241
Change-Id: I9427e025ea0bcf2fa24fd539a775429cc64767cc
2019-11-29 13:40:34 +01:00
jenkins-bot dbf4c56896 Merge "[Refactor] Pass validation error with StatusValue" 2019-11-27 21:23:30 +00:00
Adam Wight 22a0350d84 [Refactor] Pass validation error with StatusValue
This has clearer semantics than checking for a `false` attribute.

Change-Id: I68f777eda40f8f157deafacaed02d4bd10cbf25c
2019-11-27 18:05:19 +01:00
Thiemo Kreuz 38a38ae472 Add smoke tests for previously uncovered combinations
I noticed a possible issue related to the $this->refSequence counter
in the patch Ida9612d. Some of these counters might get messes up, but
there was never a test that checked what will happen to the *next*
reference then.

I checked the test cases in this patch with a very old version of the
codebase.

Change-Id: If6e56f727dce5d0e5e38e048e602437597248a42
2019-11-27 16:34:46 +01:00
Thiemo Kreuz 99d23ac841 Remove redundant attribute trimming
We noticed the group="…" attribute was the only one that was not
trimmed. Does this mean it was possible to have two groups "a" and
" a"? It turns out: no. This was never possible because the parser
already trims all attributes before calling this code.

I tried to come up with the worst possible test case, but it succeeds,
even with very old versions of this codebase.

I suggest to remove the extra trimming from this codebase and rely on
what the parser provides.

Note the content is special and *not* trimmed by default.

Change-Id: Idff015447d7156ba7b5c03a5c423f199a71349f2
2019-11-27 12:12:51 +00:00
Adam Wight 8453e3ecd7 Extract stack and state to a new class
Most of this state is used to manage interactions with other state,
and encapsulation allows us to hide data structures and access behind
self-explanatory function names.

The interface is still much wider than I'd like, but it can be improved in
future work.

There is one small behavior change in here: in the `follows` edge case
demonstrated by I3bdf26fd14, we prepend if the splice point cannot be
used because it has a non-numeric key.  I believe this was the original
intention of the logic, and is how the numeric case behaves.  I've verified
that when array_splice throws a warning about non-numeric key, it fails to
add anything to the original array, so the broken follows ref disappeared.

Bug: T237241
Change-Id: I091a0b71ee9aa78e841c2e328018e886a7217715
2019-11-25 14:06:32 +01:00
Subramanya Sastry 2cfb76f8b6 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 7dfc2e931a6afeb62d2a0d791cda88fd8d39c070

Change-Id: I7edd1f293530653ae1bbfe47028e585f2b46927b
2019-11-22 18:44:22 +00:00
jenkins-bot 7f4cff9523 Merge "Move bad dir="…" error reporting down to the renderer" 2019-11-22 13:46:44 +00:00
Thiemo Kreuz 8e42a6ecdf Add missing test cases for follow="…"
One of the test cases was duplicated, but a lot of the possible code
paths never had tests, including the happy code path!

I found this issue while trying to rework some of the more confusing
loops in this codebase. These changes are still part of this patch. All
loops still do the same as before, but are (I hope) more readable now.

Bug: T238187
Change-Id: I85baeadd9b149025a14c7522bcc4182339c66972
2019-11-22 11:32:28 +01:00
Thiemo Kreuz ea6cea93ed Move bad dir="…" error reporting down to the renderer
… and make the error message for bad dir="…" shorter and more to the
point.

Now I understand why the error reporting was not done when $text was
empty: the error was actually appended to $text, which messes with
everything else that also works with the $text variable! This even
includes the API. This error message was exposed via the API. That was
certainly a bug.

With this patch, all error checking for the dir="…" attribute is now
done way down, when rendering the <references> section.

Note this also fixes a bug where the dir="…" was *not* rendered when
previewing a section.

Change-Id: I4ab0cb510973ed879c606bfaa394aacc91129854
2019-11-22 10:07:28 +01:00
Thiemo Kreuz b10dd4ec27 Block de-facto empty <ref> as if it's empty
The use case we care about is this:
<ref extends="some_book"> </ref>

It doesn't make sense that works, but the following doesn't:
<ref extends="some_book"></ref>

We decided that both need to behave the same.

For consistency this patch is applying the same change to all references,
no matter if they use the extends attribute or not. This is an actual
change and might make existing wikitext render differently. However, I
would like to argue that all wikitext that was using this was broken. The
effect of a <ref> </ref> with some whitespace is that the <references>
section at the end of the article will contain – well – an empty footnote.

Bug: T237241
Change-Id: Iaee35583eabcb416b0a06849b89ebbfb0fb7fef9
2019-11-20 15:07:54 +00:00
Adam Wight 9d706047f3 Rename refines -> extends
Bug: T171581
Change-Id: I42b2d8859f2958357024cbba089715c10712f370
2019-11-12 10:19:17 +00:00
Thiemo Kreuz fe385ecc37 Block all combinations of refines="…" and follows="…"
Note it doesn't make a difference if this is behind the feature flag or
not. It should always be forbidden, and in fact is: Either the follows
attribute is unknown, or the combination is forbidden.

Bug: T236256
Change-Id: Iebbb2d1d5bab183ab0590b8a7a7f6e79d319b72c
2019-11-11 12:56:58 +01:00
Thiemo Kreuz d919615e28 Merge bookReferencingUnimplemented.txt into bookReferencing.txt
What we find critical is:
* That all tests relevant for book referencing are in a separate file.
* That unimplemented stuff is marked with TODOs.

Not having to move tests to another file allows for nice diffs.

I tried to order the tests as good as I could. E.g. have all tests with
a group="…" next to each other, followed by all with a follow="…".

Change-Id: Idc1d9e7843b341235ab3d8ebe398e01946eb1845
2019-11-11 12:50:51 +01:00
Adam Wight 0ebf86fdf3 Split out BookReferencing parser tests
Encapsulate the feature tests in dedicated files.  These are picked
up by the test runner for matching glob `tests/parser/*.txt`, as can
be shown by,

  phpunit.php --testsuite parsertests --filter=bookRef

Also adds TODO comments to some tests, documenting how the current output
will not match the fully implemented code's results.

Bug: T236256
Change-Id: Ie3e769c84856256180754aeff417da893a84b479
2019-11-08 10:02:38 +01:00
Andrew Kostka 1dcb096776 Add parser tests for refined references as rendered right now
These tests document the current status quo, and are meant to change
with every patch that makes the code for refined references more
feature complete.

Bug: T236256
Change-Id: I8c11b1decc36b86e7f7d1919cc39d0c16a200055
2019-11-08 09:38:08 +01:00
Adam Wight 5e8d48b331 Minimal support for bookreferencing tag
Allows the "refines" attribute when the feature flag is set, but doesn't
render.  This is part of our rollback strategy, so that we aren't left
with invalid wikitext in case of undeployment.

Bug: T236257
Change-Id: I936be0e62dccb46caeb84162d2c5166956fd9916
2019-10-24 12:24:36 +00:00
Fomafix 70b6a48db7 Remove parser test with mw-editsection
This change allows to change the editsection HTML by
I305e3313ca2f931a2ea9cee34194b8cb93b90b0e without failing in Jenkins.
The parser test gets restored in the new format by the follow-up change
Ibb4341b405f0d6fa6883c992c5dd3a9e594c9efc.

Change-Id: I337d7f7c0cd134a3766343565e2c30edf1d70f7e
2019-05-08 19:21:03 +02:00
Adam Wight 9347dfeb6d Test rendering of high-ascii reference names
Bug: T220196
Change-Id: I2423e0908154a9eb3ecd687945d934269255a939
2019-04-19 17:10:53 -07:00
Arlo Breault c735c9021f Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 94b8b491098f882582f372218df07f5b68f4eba1

Change-Id: Ic357d36d3643fbead6f7e1c0a03aaefb5b7e005c
2019-03-19 11:40:14 -04:00
Arlo Breault 2528762640 Update tests to match parser changes
Bug: T208070
Depends-On: I3da235cb83efa424f0cf1cf4fc7233240fcdf6b2
Change-Id: I6119b4af9632496dbda81c3a3951c55217e7c2d5
2019-03-15 18:23:43 +00:00
Subramanya Sastry c23cd59a53 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 2c0770eb3b441800b74651cf415de1edf29a5a5e

Change-Id: I0a28bbe9db3a7b8d09c14a2e6c1ad0d94c1591f6
2019-01-02 11:31:42 -06:00
Gilles Dubuc ad559d4add Fix test for img decoding="async"
Bug: T212124
Depends-On: I79de6f3b0ec5529881525d32925519b47fed5311
Change-Id: I9bd9e24453838130eeb221c2f09961e1c4c15938
2018-12-20 13:42:30 +01:00
Thiemo Kreuz 3f22189998 Fix <ref> ignoring all parameters when there are more than two
We can resolve this bug by either replacing the bogus "return false"
with the intended "return [ false, … ]". Or rely on the code a few
lines below that also bails out with a "return [ false, … ]" when to
many parameters ($cnt is not 0 then) are present. The tests prove both
solutions are equally valid.

Bug: T211576
Change-Id: Iadd55c134dede7042cfd152c69bc8f27b59d8912
2018-12-11 20:49:40 +01:00
jenkins-bot 9e981d28b6 Merge "Sanitize underscores as core does, to not create broken links" 2018-12-11 00:01:57 +00:00
Thiemo Kreuz 9a8c718c2a Add missing test cases for code in Cite::refArg
This is split from I642d38e and does nothing but adding test cases
that document the current (broken) behavior.

Bug: T211576
Change-Id: Iee313d26e7bed6deb34101e37736a1c697947905
2018-12-10 12:54:58 +01:00
Thiemo Kreuz (WMDE) 7c06347fc7 Simplify weirdly complex [\n\t ] regex
This change does have two consequences:

1. A few more whitespace characters act as separators. This should not
have any consequence in real life situations, and is mainly done to
make the code easier to read and less surprising.

2. Sequences of two or more whitespace characters previously resulted
in partly *empty* results. This was a potential source of errors. The
additional + fixes this.

Change-Id: Ib58326109c740dd0cbd05d8fddb4af2145f232fe
2018-11-21 17:33:25 +00:00
Thiemo Kreuz (WMDE) 2b34dede6c Sanitize underscores as core does, to not create broken links
Core sanitizes link targets and removes double spaces and underscores.
But the corresponding id="…" attributes are not sanitized the same
way. This results in broken links. This patch is not perfect (two
references with name="a_b" and name="a__b" will conflict), but the
best solution I can think of at the moment.

Bug: T184912
Change-Id: I9dbc916ad99269517d84c8ffb8581628d44a9f4e
2018-11-20 13:07:35 +01:00
Arlo Breault 1d687e23f3 Use the dir parameter only from the full definition of a named ref tag
Bug: T196827
Change-Id: Iaf84966e37cea730c9eca07c19a555971ffeadf3
2018-08-22 19:31:23 -04:00
Arlo Breault 97f346438c Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 129d71f5d6eab8c87a0e6591fcad4ad5e55b8da2

Change-Id: If2f540f0adf317eaa3cac7d0413c6bde8adc58e7
2018-08-17 15:45:23 -04:00
Eranroz 1ca27aa0d8 Support directionality for reference
Adding option for dir attribute in ref tags. The value must be a valid
direction ('ltr' or 'rtl', case insensitive) or the direction will be
stripped out.

The directionality of the li element is set using a css class accordingly.

Bug: T15673
Change-Id: Iff480bc8cc4f81403b310e8efecd43e29d1d4449
2018-05-02 17:27:32 +02:00
C. Scott Ananian 5433d46bb8 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 7d2a92f81ebbc0941e8fba2a136f5929406ea5e6

Change-Id: Ie7354c9c36f8532dfa36e5ab5a2a4c01fae65b69
2018-03-07 02:23:38 -05:00
C. Scott Ananian 0e8f1c961f Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 0723e5c47845ff4361b9635b591e7d386c975fdf

Change-Id: Ic5b30a88189e5a8809d0f330d8b399bdb1994c60
2017-11-21 17:25:33 -05:00
Max Semenik 351a08d1b7 Don't break when reference names contain []
Bug: T29694
Bug: T179544
Depends-On: I189bdefbc9034cf8d221a89d7158195de1c0fa6c
Change-Id: Iec3439f76ecc2a3543b30b35f8735c92b0cfb711
2017-11-15 23:23:45 +00:00
Arlo Breault 185b5c57c0 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 7ec1f8369ef2e620858b40eadb7c43f1c4fa6d3d

Change-Id: I0e65d14ace2c0420fccc407eceae0f33ae1f1e06
2017-10-04 13:19:19 -04:00
C. Scott Ananian 14459c226b Use HTML5 id attributes; remove use of deprecated Sanitizer::escapeId()
When using HTML5 ids, we need to take greater care to properly escape the
id (or derived strings) before passing them back through
Parser::recursiveTagParse().

Bug: T176170
Change-Id: I89a4f8ba24b867f2d5ccdc2bf9a4312ab9b385a9
2017-09-19 15:42:41 -04:00