Commit graph

113 commits

Author SHA1 Message Date
Thiemo Kreuz f92792f64a Fix bad localization of extended references numbers when reused
This adds a test for numbers like "1.2.0" that appear when an extended
reference (e.g. "1.2.") is reused multiple times.

The first separator is from the extended reference. We decided to never
localize it. However, the second seperator is from reusing a reference.
This was always localized. We believe this is a bug, but haven't fixed
it yet.

The test is documenting the status quo "1.2,0" with a comma. This kind
of makes sense, one could argue, because the "1.2" appears like this up
in the text, but the ",0" is a different indicator for a reuse, which
*never* occurs in the text.

Change-Id: Ie3d26bcadd8929b906bfbcac4806af2150d61f2a
2019-12-09 17:25:14 +01:00
Thiemo Kreuz 7c1849d7b0 Report both nested <ref> and <references> as an error
Before, this regular expression was looking for incomplete wikitext
like this:

<ref>unclosed
<ref>closed</ref>

With this change, wikitext like this will trigger the same error:

<ref>unclosed
<references />
incomplete</ref>

This should be much, much more rare. But I feel it's reasonable to mark
this as an error, instead of just rendering the broken inner tag in
plain text.

This patch also replaces `.*?>` with `[^>]*+>`. Both do the exact same.
Instead of doing an "ungreedy search for the first possible closing
bracket", which might cause backtracking, the new syntax consumes all
non-brackets before expecting one. This is guaranteed to never backtrack
(guaranteed by the extra +), and potentially faster because of this.

Change-Id: Ic76a52cd111b28e4522f095ce3984e3583f602c1
2019-12-09 14:26:28 +01:00
jenkins-bot aa12b53e3e Merge "Comment about annoying thing" 2019-12-09 13:13:49 +00:00
jenkins-bot 740bd24178 Merge "Rename field to "key"" 2019-12-09 12:21:18 +00:00
jenkins-bot 4a0026d9bc Merge "Split validation function depending on inReferencesGroup" 2019-12-09 12:21:17 +00:00
Adam Wight 3e728799c2 Comment about annoying thing
Change-Id: I3c7f85bd822391d4e63314c0829ea1668d30a4ce
2019-12-09 12:56:58 +01:00
Adam Wight d6c0155e4c Rename field to "key"
This is consistent with the fact that it contains $ref['key'].

Change-Id: I134dba4a2405bb44b785e9cf191adc7bdd54c0d1
2019-12-09 12:52:26 +01:00
Adam Wight 8097c4c148 Split validation function depending on inReferencesGroup
Some validation is exclusively used in a specific context, some is shared.

Change-Id: I390db1c9d4854871e25a2e74411476e4e1c0b66f
2019-12-09 12:27:52 +01:00
Adam Wight f51060eaf4 Fix footnote mark after extends numbering glitch
The visible numbering needed to be rolled back after an extends.

Bug: T237241
Change-Id: I95404515110df1fa7e3279ea499577df0ed45ddf
2019-12-09 12:06:59 +01:00
jenkins-bot 0f0356ffc1 Merge "Refine and fix "unclosed <ref> detected" regular expression" 2019-12-09 10:29:13 +00:00
jenkins-bot a8e882e39f Merge "Show "Preview" headline in user instead of content language" 2019-12-09 10:13:17 +00:00
jenkins-bot 3b41cfa472 Merge "Fail early on nested extends="…", if possible" 2019-12-09 10:12:54 +00:00
Thiemo Kreuz c5fe49ff11 Fail early on nested extends="…", if possible
This partly reverts Ied2e3f5. I haven't properly tested this before.
Rendering a bad extends (that extends a <ref> that's already extended)
not indented messes the order up and rips other extended <ref>s out of
context.

For now it might be better to stick to the previous, "magic" behavior:
Such an extends behaves like it is extending the *parent*, and is
ordered and indented as such. This is still not correct, but I feel
this is much better than rendering such a bad extends on the top level.

This patch also makes the code fail much earlier for a nested extends,
if this decision can be made already. In this case the error message is
rendered in the middle of the text (as other errors also are), not in
the <references> section.

Change-Id: I33c6a763cd6c11df09d10dfab73f955ed15e9d36
2019-12-09 10:54:52 +01:00
Thiemo Kreuz 8fdce945bd Show "Preview" headline in user instead of content language
This partly reverts Id7a4036e64920acdeccb4dfcf6bef31d0e5657ab.

The message "cite_section_preview_references" says "Preview of references".
This line is not meant to be part of the content, but an interface message.
It should use the users (interface) language, not the content language.

Change-Id: I1b1b5106266606eb0dfaa31f4abd3cee9ba92e8c
2019-12-09 10:53:07 +01:00
jenkins-bot 238ed31d2e Merge "Add fail-safe default branch to switch-case" 2019-12-09 09:52:41 +00:00
Thiemo Kreuz a7ee7c9586 Refine and fix "unclosed <ref> detected" regular expression
This simplifies as well as fixes a series of issues with this regular
expression:

* Before, the wikitext `<REF><REF>` would not trigger the error, but
`<ref><ref>` would. Parser tags are case-insensitive, but the error
check was not.

* Before, the wikitext `<ref><ref name="<">` would not trigger the error.
That's a valid name. The error check should not stop just because it
found a `<`.

* Both the old and the new code do *not* fail with the wikitext
`<ref><ref</ref>` where the inner `<ref` does not have a closing `>`. I
was thinking about changing this, but figured it might be used as a
feature.

* The old code was not able to properly understand HTML comments,
<nowiki> tags and such that contain a line break. That caused
inconsistent and confusing error reporting in some cases, but not in
others. This change *reduces* the amount of errors this code produces.

* The old code was looking for "SGML tags" with names that could be
anything, not just alphanumeric characters. This allowed for strange
edge-cases like `<ref><>><ref></>></ref>` that have not been reported,
but should be. This change *increases* the amount of errors. However,
relevant edge-cases should be extremely rare.

Note the ++ avoids backtracking, speeding up the regex.

Change-Id: I0c61a245f4f743871b4cad886ce239650af2b37c
2019-12-08 04:37:13 +00:00
Thiemo Kreuz 92312b4421 Add fail-safe default branch to switch-case
Reported as a possible code-smell at sonarcloud.io.

Change-Id: I3c5c1ff116dabe06c3d2e3cc59850ad3c66f8f83
2019-12-06 14:30:40 +01:00
Adam Wight 3d80501829 Narrow message localizer interface
We never access Language directly, so proxy its method instead of
returning the full object.

I believe I've found a bug, but not fixing here: the footnote body
numeric backlinks like "2.1" behave as if they were decimals rather
than two numbers stuck together with a dot.  So they are localized
to "2,1".

Bug: T239725
Change-Id: If386bf96d48cb95c0a287a02bedfe984941efe30
2019-12-06 12:17:09 +01:00
jenkins-bot 9622c4fb8f Merge "Comments to help understand the message localizer" 2019-12-05 14:49:58 +00:00
Adam Wight 01c76f46a6 Use message localizer in CiteKeyFormatter
Makes more tests easier.

Change-Id: I222ba61bfcf0be3e29cb04e39f44f0be7a9e0778
2019-12-05 14:57:32 +01:00
Adam Wight 1ce4079ce2 Use message localizer in FootnoteMarkFormatter
Completes test coverage.

Change-Id: Ib2ec24cf4a9de52769744d1888cb13d2bf08ae3b
2019-12-05 14:56:53 +01:00
Adam Wight 430086cb6b Use the message localizer in Cite
Allows us to convert another integration test into a unit test.

Change-Id: Id7a4036e64920acdeccb4dfcf6bef31d0e5657ab
2019-12-05 13:23:31 +01:00
jenkins-bot 6255ab85d9 Merge "Drop unused variable" 2019-12-05 10:10:20 +00:00
jenkins-bot faf0b38fd9 Merge "Add visual whitespace to concat code" 2019-12-05 10:08:47 +00:00
Adam Wight b575835c63 Comments to help understand the message localizer
Change-Id: Ic7c9a12a78f358d11d997abf9a3a8e996f451c8f
2019-12-05 09:06:37 +01:00
Adam Wight 646dc5f974 Drop unused variable
Change-Id: I393a89ca909a632729c88ff73543bcf71061a8bf
2019-12-05 09:03:17 +01:00
Adam Wight 3bcb8dc39f Add visual whitespace to concat code
Change-Id: I1e32c942d27db9f9c20fae0c684be256877aef2b
2019-12-04 18:12:47 +01:00
Adam Wight 5705228d17 Complete validateRef coverage
Change-Id: Id61fba34a8815a0c512ecf4bc57da3be4e15c8bb
2019-12-04 18:00:13 +01:00
Thiemo Kreuz a7c4e14f42 Remove obsolete ParserBeforeTidy hook handler
I was able to track this code down to I093d85d from 2012, which was done
right after the ParserAfterParse hook was introduced. I believe the
redundant code path was left to keep the Cite extension compatible with
old MediaWiki versions that did not had this hook yet.

I also noticed this code path is most probably entirely redundant with
the current version of MediaWiki. The *only* thing this code does is
blocking the ParserBeforeTidy hook from doing the same thing a second
time if the ParserAfterParse hook was called before. But it does *not*
block any other compination, e.g. if the two hooks are called the other
way around, or the same hook twice.

In core, it looks like it is impossible for the ParserBeforeTidy hook
being fired without the ParserAfterParse hook being fired before. If this
is true, this is in fact dead code.

Change-Id: Iacf8b600c7abdeaf89c22c2fc31e646f57245e47
2019-12-04 16:56:43 +01:00
Thiemo Kreuz 31bda4777b Don't indent refs with forbidden extends="…"
Change-Id: Ied2e3f56ce66d2a8ccf60df2bdbf99acad461595
2019-12-04 15:17:03 +00:00
Adam Wight 81261493c2 Show error when extending a subreference
Change-Id: Iaa47e302e5e49dfc190fde37567a3e7a2e743d67
2019-12-04 13:49:31 +01:00
Adam Wight c09d90aff3 Use message localizer in FootnoteBodyFormatter
Makes the class more easily testable.

Patch also changes an integration test into a unit test.

Change-Id: I545730404aceed7e3857d96f4fd3c1b0a900c0c2
2019-12-04 13:41:44 +01:00
Adam Wight bccb92335f Introduce ReferenceMessageLocalizer
Encapsulate the language interfaces, this will be used to replace
global wfMessage calls in future patches.

Change-Id: I7857f3e5154626e0b29977610b81103d91615f65
2019-12-04 13:40:05 +01:00
Adam Wight d04cc36fa4 Replace reference parameter with return value
This makes it obvious that our function isn't sensitive to the input value.

Also rearranges a string concatenation to make the element wrapping clearer.
I probably should have switched to the HTML class here, but I'm not sure what
the advantages would be.

Change-Id: Ife3424ce68588f73f168b10e63e6cd81c4a60084
2019-12-04 11:28:38 +01:00
Adam Wight 97e0cd2321 Minor cleanups
Change-Id: I895d16a17e7d7e30a2118e798fb453192ea282b3
2019-12-04 09:26:52 +00:00
Thiemo Kreuz 54333c9bd6 Stop formatting "1.2" as "1,2" in languages like German
The new extends="…" feature is using numbers like "1.2". These should be
localized in languages like Hebrew that uses other symbols for the digits.
But the "." should not change.

The existing feature when a <ref> is reused multiple times does have the
same "issue". But it seems this is intentional, because it is covered by
a test. Note this is not visible in German, because German uses custom
labels "a", "b", and so on.

This patch also improves the so called "smoke" tests and makes one cover
numbers up to "1,10" for a <ref> that is reused that often.

Bug: T239725
Change-Id: Iffcb56e1c7be09cefed9dabb1d6391eb6ad995ce
2019-12-04 09:43:04 +01:00
Adam Wight 1b82b93835 Fix function signature in phpdoc
Change-Id: I3329ca19d465c6ad7ed23385021a051fcc23ea8e
2019-12-03 13:26:50 +01:00
jenkins-bot 0f9c306748 Merge "Inline and streamline code in the formatter classes" 2019-12-03 10:42:28 +00:00
Thiemo Kreuz b145869980 Inline and streamline code in the formatter classes
* Don't use string comparisons to compare numbers.
* Avoid isset() for variables that are guaranteed to exist.
* Inline two small "gen…" functions that are only called once.
* Move the fallback code path out of getLinkLabel(). Before it was
  always called. Now it's only called when needed.

Change-Id: I42073f57f21d32c7936954da776ef3a393410020
2019-12-03 10:26:38 +01:00
Adam Wight 96db7944eb Cover rollback with tests
Fixed an unsafe array access during rollback.

Change-Id: Id9ee8976e3bae24501c18abf462e3e19894caff0
2019-12-03 09:53:23 +01:00
Adam Wight 008526b3aa Can use extends before its parent
If `extends` is encountered before the parent ref, we reserve the
sequence number and leave a placeholder to record the link between
ref name and number.  This is necessary to render a list like,
"[1] [2.1] [2]", or to use subreferencing when the parent ref is
declared in the references tag.

When a placeholder is encountered during references section rendering,
it means that the parent was never declared.

Change-Id: I611cd1d73f775908926a803fae90d039ce122ab6
2019-12-02 17:14:11 +01:00
Adam Wight a27c33a2e7 Cleanups
Address some code review comments from I75bd6644

Change-Id: I433c08318c137ecca4d4ef77f0863d5da42b567c
2019-12-02 16:36:49 +01:00
Adam Wight e9958d569b Formatter takes responsibility for rendering footnote mark
Pass the full ref structure from ReferenceStack to FootnoteMarkFormatter,
to give it control over the final rendering.  This is aligned with how
the FootnoteBodyFormatter directly scans over groupRefs.

Change-Id: I3294fd9366f01daa4250a5d481f4adbae84c72b1
2019-12-02 10:17:24 +01:00
Adam Wight 3f276388bf Split ref.number field
This was carrying the entire footnote marker, but subreferences need
to extract just the first (group ref sequence) part.  Storing number
and extendsIndex in two separate fields gives us more flexibility
during rendering, for example these might use two different symbol sets.

Change-Id: I75bd6644c336036f9e84ba91e1c35e05bc1ca7f3
2019-12-02 10:17:24 +01:00
Adam Wight 5dfe633b33 Include name in ref structure
This will become useful in I611cd1d7, when we calculate ref link text
in FootnoteMarkFormatter.

Change-Id: I729701614829ccbca4c243c181ded13f354d1103
2019-12-02 10:17:24 +01:00
Adam Wight 00f3be7c7f Reset extendsCount after each group
This was a bug which would affect book references, if the same group
and parent ref name combination occur twice in an article.

Change-Id: I608f58aac0cec31c8650835fc80195a87bc851d3
2019-12-02 10:17:18 +01:00
Thiemo Kreuz 504db2c46a Add strict PHP 7 type hints to most code
This patch does two things:
* Add strict PHP 7 type hints to most code.
* Narrow the interface of the checkRefsNoReferences() method to not
  require a ParserOptions object any more.

Change-Id: I91c6a2d9b76915d7677a3f735ee8e054c898fcc5
2019-12-02 08:51:42 +01:00
jenkins-bot 469a1e6364 Merge "Don't leave unclosed <li> behind" 2019-12-01 10:36:56 +00:00
jenkins-bot 76ef4af51f Merge "Clean up text and name conditionals" 2019-12-01 10:02:41 +00:00
Thiemo Kreuz 3cf1e99cc2 Don't leave unclosed <li> behind
This fixes a FIXME I left in the code. Previously, I just stripped the
closing </li> to make sure the nested <ol> is *inside* of the <li>.
This relies on (Remex) Tidy to clean the incomplete HTML up.

This patch remembers the stripped </li> and adds it back.

This also makes sure the nested <ol> is closed, even if it was the
last element in the data structure.

Notice how this does not influence any test. I find this a bit
confusing. It looks like (Remex) Tidy is executed, even if the tests
are not marked as "html/php+tidy".

Bug: T237241
Change-Id: Idb804df46dc24406d6bba40414675b6ff4812d48
2019-12-01 10:54:01 +01:00