Commit graph

230 commits

Author SHA1 Message Date
Adam Wight d04cc36fa4 Replace reference parameter with return value
This makes it obvious that our function isn't sensitive to the input value.

Also rearranges a string concatenation to make the element wrapping clearer.
I probably should have switched to the HTML class here, but I'm not sure what
the advantages would be.

Change-Id: Ife3424ce68588f73f168b10e63e6cd81c4a60084
2019-12-04 11:28:38 +01:00
Adam Wight 97e0cd2321 Minor cleanups
Change-Id: I895d16a17e7d7e30a2118e798fb453192ea282b3
2019-12-04 09:26:52 +00:00
Thiemo Kreuz 54333c9bd6 Stop formatting "1.2" as "1,2" in languages like German
The new extends="…" feature is using numbers like "1.2". These should be
localized in languages like Hebrew that uses other symbols for the digits.
But the "." should not change.

The existing feature when a <ref> is reused multiple times does have the
same "issue". But it seems this is intentional, because it is covered by
a test. Note this is not visible in German, because German uses custom
labels "a", "b", and so on.

This patch also improves the so called "smoke" tests and makes one cover
numbers up to "1,10" for a <ref> that is reused that often.

Bug: T239725
Change-Id: Iffcb56e1c7be09cefed9dabb1d6391eb6ad995ce
2019-12-04 09:43:04 +01:00
Adam Wight 1b82b93835 Fix function signature in phpdoc
Change-Id: I3329ca19d465c6ad7ed23385021a051fcc23ea8e
2019-12-03 13:26:50 +01:00
jenkins-bot 0f9c306748 Merge "Inline and streamline code in the formatter classes" 2019-12-03 10:42:28 +00:00
Thiemo Kreuz b145869980 Inline and streamline code in the formatter classes
* Don't use string comparisons to compare numbers.
* Avoid isset() for variables that are guaranteed to exist.
* Inline two small "gen…" functions that are only called once.
* Move the fallback code path out of getLinkLabel(). Before it was
  always called. Now it's only called when needed.

Change-Id: I42073f57f21d32c7936954da776ef3a393410020
2019-12-03 10:26:38 +01:00
Adam Wight 96db7944eb Cover rollback with tests
Fixed an unsafe array access during rollback.

Change-Id: Id9ee8976e3bae24501c18abf462e3e19894caff0
2019-12-03 09:53:23 +01:00
Adam Wight 008526b3aa Can use extends before its parent
If `extends` is encountered before the parent ref, we reserve the
sequence number and leave a placeholder to record the link between
ref name and number.  This is necessary to render a list like,
"[1] [2.1] [2]", or to use subreferencing when the parent ref is
declared in the references tag.

When a placeholder is encountered during references section rendering,
it means that the parent was never declared.

Change-Id: I611cd1d73f775908926a803fae90d039ce122ab6
2019-12-02 17:14:11 +01:00
Adam Wight a27c33a2e7 Cleanups
Address some code review comments from I75bd6644

Change-Id: I433c08318c137ecca4d4ef77f0863d5da42b567c
2019-12-02 16:36:49 +01:00
Adam Wight e9958d569b Formatter takes responsibility for rendering footnote mark
Pass the full ref structure from ReferenceStack to FootnoteMarkFormatter,
to give it control over the final rendering.  This is aligned with how
the FootnoteBodyFormatter directly scans over groupRefs.

Change-Id: I3294fd9366f01daa4250a5d481f4adbae84c72b1
2019-12-02 10:17:24 +01:00
Adam Wight 3f276388bf Split ref.number field
This was carrying the entire footnote marker, but subreferences need
to extract just the first (group ref sequence) part.  Storing number
and extendsIndex in two separate fields gives us more flexibility
during rendering, for example these might use two different symbol sets.

Change-Id: I75bd6644c336036f9e84ba91e1c35e05bc1ca7f3
2019-12-02 10:17:24 +01:00
Adam Wight 5dfe633b33 Include name in ref structure
This will become useful in I611cd1d7, when we calculate ref link text
in FootnoteMarkFormatter.

Change-Id: I729701614829ccbca4c243c181ded13f354d1103
2019-12-02 10:17:24 +01:00
Adam Wight 00f3be7c7f Reset extendsCount after each group
This was a bug which would affect book references, if the same group
and parent ref name combination occur twice in an article.

Change-Id: I608f58aac0cec31c8650835fc80195a87bc851d3
2019-12-02 10:17:18 +01:00
Thiemo Kreuz 504db2c46a Add strict PHP 7 type hints to most code
This patch does two things:
* Add strict PHP 7 type hints to most code.
* Narrow the interface of the checkRefsNoReferences() method to not
  require a ParserOptions object any more.

Change-Id: I91c6a2d9b76915d7677a3f735ee8e054c898fcc5
2019-12-02 08:51:42 +01:00
jenkins-bot 469a1e6364 Merge "Don't leave unclosed <li> behind" 2019-12-01 10:36:56 +00:00
jenkins-bot 76ef4af51f Merge "Clean up text and name conditionals" 2019-12-01 10:02:41 +00:00
Thiemo Kreuz 3cf1e99cc2 Don't leave unclosed <li> behind
This fixes a FIXME I left in the code. Previously, I just stripped the
closing </li> to make sure the nested <ol> is *inside* of the <li>.
This relies on (Remex) Tidy to clean the incomplete HTML up.

This patch remembers the stripped </li> and adds it back.

This also makes sure the nested <ol> is closed, even if it was the
last element in the data structure.

Notice how this does not influence any test. I find this a bit
confusing. It looks like (Remex) Tidy is executed, even if the tests
are not marked as "html/php+tidy".

Bug: T237241
Change-Id: Idb804df46dc24406d6bba40414675b6ff4812d48
2019-12-01 10:54:01 +01:00
Adam Wight f24f77d4c4 Fix comments
Change-Id: Ie99f172bf555af4e2c51928152043d12ea735d76
2019-11-29 23:01:07 +01:00
Adam Wight 6922f5201b Drop single-use variable
Change-Id: Idb9801cc1f414088841afc2f18a056be65b695d1
2019-11-29 23:01:06 +01:00
Adam Wight ffedf86a19 Clean up text and name conditionals
The "no key" error should have been unreachable, but I'm afraid that
null `$text` and empty string `$text` were reporting slightly different errors.

Unfortunately, we still have to care about `$text = '0'` because PHP, so
the expressions don't reduce to `if ( !$text ...`

Change-Id: Id1028611ec3bc462dca413f31f7f59637bd7cc7b
2019-11-29 19:26:12 +00:00
Thiemo Kreuz 22627f074d Make the normalizeKey() method private
There was a call in the API that was *not* using normalizeKey(). Now
that the API is gone, we can inline this.

This patch also contains a bunch of cleanups that might already been
resolved in the previous patches.

Change-Id: Id3767b5830268c8cfe9c10efabfa4a31e9dafeb8
2019-11-29 19:02:48 +01:00
Adam Wight 367de442c1 Move string contatenation out of sprintf
Simplifies what's actually happening here.

Change-Id: I7b561355506c1f4aa757cf551aa859a32fe23567
2019-11-29 16:22:35 +01:00
Adam Wight a40b1b10be Extract footnote body rendering
Change-Id: I9537849cbd700d5dc7ec1a53d852d69b0fe0dc35
2019-11-29 16:22:35 +01:00
Adam Wight c236524138 Extract footnote mark rendering
Change-Id: I79de89e46da36dc1f0ee2b2fdb9a139e6434fde2
2019-11-29 16:22:35 +01:00
jenkins-bot 4b58a25459 Merge "Extract key formatting" 2019-11-29 15:19:52 +00:00
jenkins-bot 50ccd05def Merge "Render nested references" 2019-11-29 15:19:50 +00:00
Thiemo Kreuz f8affe8eee Remove redundant variable names from @var docs
Change-Id: I5ba2f1041d3d6770c58f496482d2799bb24786be
2019-11-29 14:37:58 +01:00
Adam Wight a4c056f59b Extract key formatting
Change-Id: I155ec6f3e21075587dbcfdfdc346f28f958e3c15
2019-11-29 13:41:12 +01:00
Thiemo Kreuz 13598ba11e Render nested references
Forked from Icd933fc983.

Bugs and unimplemented features are documented as TODOs in the parser test
fixtures.

Bug: T237241
Change-Id: I9427e025ea0bcf2fa24fd539a775429cc64767cc
2019-11-29 13:40:34 +01:00
jenkins-bot 3beb5c3634 Merge "Remove ApiQueryReferences support" 2019-11-29 11:30:38 +00:00
Adam Wight a176e22097 Remove ApiQueryReferences support
This API was never used in Wikimedia production, and would have caused
performance problems.  Removing the dead code will simplify our refactoring.

Bug: T238195
Change-Id: I7088f257ec034c0d089e0abdaa5a739910598300
2019-11-28 11:08:46 +01:00
Adam Wight 44cbc60d40 Count extends refs
Has no effect, this is a "safe" split from I9427e025ea0.

Change-Id: I842673cd1226ec5c9248d8f069766a00a7c27f35
2019-11-28 02:10:28 +01:00
Adam Wight ab78df8d5c Wire extends into ReferenceStack
Takes no action, just shuttle the value between functions.

(Split from I9427e025ea0)

Change-Id: I271043e9161835f3278098787bf58b50ed93c892
2019-11-28 02:10:11 +01:00
jenkins-bot dbf4c56896 Merge "[Refactor] Pass validation error with StatusValue" 2019-11-27 21:23:30 +00:00
jenkins-bot e4a961a6c1 Merge "Rename $valid to $status for clarity" 2019-11-27 21:23:29 +00:00
Adam Wight 249982e353 Rewrite loop in ReferenceStack
Change-Id: I3bdf26fd14573abdcad989c7ebfea48e49ef42aa
2019-11-27 17:13:44 +00:00
Thiemo Kreuz f00b21943b Minor fixups to the ReferenceStack class
Change-Id: Ie7d72b13f987443c0e118fb9ac0f0af016f00392
2019-11-27 18:13:22 +01:00
Adam Wight 22a0350d84 [Refactor] Pass validation error with StatusValue
This has clearer semantics than checking for a `false` attribute.

Change-Id: I68f777eda40f8f157deafacaed02d4bd10cbf25c
2019-11-27 18:05:19 +01:00
Thiemo Kreuz 99ee9e443b Rename $valid to $status for clarity
This also splits some code a little bit to make the next patch smaller.

Change-Id: Ibc02fa3d683043de86d21a7aa3feef373502552a
2019-11-27 17:51:22 +01:00
Adam Wight b30340ba2b Clean up pushRef
This is motivated by I9427e025e, which demonstrates that the existing
logic was hard to integrate into.  There's a lot of redundant expressions
which make the function difficult to read, and code paths which have
less effect than they appear to.

Change-Id: Ida9612d1457f2593647b8fc02930d2e9ae824814
2019-11-27 16:40:51 +01:00
Thiemo Kreuz 0013943a4a Rewrite argument parsing and use for both <ref> & <references>
We realized the trim() are not needed. This does not leave much behind
in the existing refArg() method, except that it checks for unknown keys.

I tried a few strategies and ended using the pretty new possibility to
have keys in list(), as well as use [] instead of list(). Both is
supported since PHP 7.1.

Change-Id: I569bfa14e68b64402519bd39022c197553881dde
2019-11-27 14:01:52 +01:00
jenkins-bot 7ed54a3f3d Merge "Remove redundant attribute trimming" 2019-11-27 12:26:00 +00:00
Thiemo Kreuz 99d23ac841 Remove redundant attribute trimming
We noticed the group="…" attribute was the only one that was not
trimmed. Does this mean it was possible to have two groups "a" and
" a"? It turns out: no. This was never possible because the parser
already trims all attributes before calling this code.

I tried to come up with the worst possible test case, but it succeeds,
even with very old versions of this codebase.

I suggest to remove the extra trimming from this codebase and rely on
what the parser provides.

Note the content is special and *not* trimmed by default.

Change-Id: Idff015447d7156ba7b5c03a5c423f199a71349f2
2019-11-27 12:12:51 +00:00
Thiemo Kreuz 04f784bc02 Remove non-existing property from ReferenceStack
Change-Id: Id789897dd92c7692e36f54b828c097820ab46b43
2019-11-27 12:57:55 +01:00
Thiemo Kreuz a6a16f0703 Update and increase ReferenceStack test coverage
Functional changes:
* hasGroup() will return false when a group exists, but is empty. This
  is in line with what other methods like getGroups() already do.
  Shouldn't have any effect on the existing code, but feels more clean
  and consistent.
* getGroupRefs() won't fail any more when asked for an unknown group.

Tests:
* Add missing @covers for the constructor.
* Simplify test setup by always returning a spy. All tests need it
  anyway.
* Cover 3 more methods.

Change-Id: Ie93e9af6258b757d842b30b0b059344733aad434
2019-11-27 11:08:00 +01:00
Adam Wight 97746db93c More specific function signature
Change-Id: Id1d21b9abdb11bbf441650ea0a1cccc8b258d598
2019-11-26 18:27:16 +01:00
Adam Wight ec091fe906 Reorder keys
This doesn't make any functional difference, but helps minimize later
patch Ida9612d14

Change-Id: Ice89bad02e077437d0df6fa9f51f90b4cab4837c
2019-11-26 17:06:01 +01:00
Adam Wight 38d7c09495 Tweak comment
This makes it show up nicely in my IDE.

Change-Id: Ic203a9dcb83c96c3324996e183b4dfc239f65eca
2019-11-26 17:03:40 +01:00
Adam Wight 55099a7b0c Remove impossible condition
Numeric `$name` is caught during validation.

Change-Id: Id1c3e6717af38b0b1393c135732e084d261b53f6
2019-11-26 16:43:21 +01:00
Adam Wight 7cdcc2b075 Alphabetize returned array of attributes
That was annoying me.  Since we're passing a bare list, alphabetical
order helps make the code and tests readable.

Change-Id: I6384094e429e0e2a6fa810fdc28ae0643a0ccf7c
2019-11-26 15:28:02 +01:00
Adam Wight 6a9e51fc30 Tag every usage of "extends" even when invalid
We weren't recording usages which failed certain `refArg` validations.

Bug: T237531
Change-Id: Ibcb875c5d0ed6c2279e0e34ab415ac63d7ebe584
2019-11-26 15:28:02 +01:00
Adam Wight 2229f22899 [Refactor] Nest conditionals with common term
Previewing is an important exception, so I wanted to consolidate and
emphasize its edge cases.

Change-Id: Iae343ed8c225407e8184ff09e426d531c9f6ab00
2019-11-26 15:28:02 +01:00
Adam Wight 301b1fbcaa Move a follow edge case to validation
Change-Id: I06cf5291c258322e16449d61879bf7a18129b174
2019-11-26 15:28:02 +01:00
Adam Wight 44a2599ab4 [Refactor] Handle extends attribute in validation
Moves logic so that `refArg` handles parsing and `validateRef` more
of the validation.

Change-Id: I2c0a789d5f2c20b1968c4809b5780d9fe738fd9a
2019-11-26 15:28:02 +01:00
Adam Wight d1e0e6309b Add some comments
Change-Id: I012ad6cc52dc65f1b329febdc8e6441ac03c6463
2019-11-26 11:07:48 +01:00
Adam Wight 9fa22b15be Signature: these are all nullable
Change-Id: I79432de9007f9016604cb871b68577f1cf596be5
2019-11-26 11:07:39 +01:00
Adam Wight d65bb667b7 Drop unused param
The "dir" param is not validated, so don't pass through.

Change-Id: I78eaa3bf7067c6283a2d8e452b68f93ffab43875
2019-11-26 11:07:39 +01:00
Adam Wight 33014bb8f5 Comments in TODOs
Stuff we have to fix in future work.  Follows up on I2d9904b7631d0d6

Change-Id: I7bca73d3e9d3d2a224604efbe81d48948d2a3d76
2019-11-26 11:07:12 +01:00
Adam Wight 22b0bdf526 Test for numeric extends attribute
This is invalid, because it would allow access to internal, autoindexed,
anonymous refs.  These would break when refs are reordered.

Bug: T151305
Change-Id: Ib4bb8270d810b64e4c160f377ce52ce2fc70bab4
2019-11-26 11:07:11 +01:00
Adam Wight feaa724efa Test for numeric attributes earlier
This introduces a slight behavior change, but for the better:
* When pointing to the name "0", the non-numeric error will be displayed,
  which is correct whereas "no key" is not.

Change-Id: I33467b27cd447812fe67204831909c4d9869db08
2019-11-26 11:06:15 +01:00
Adam Wight 3fbeed0304 [Refactor] Consolidate validation
Validation logic can be split from arg parsing, default values and
other side-effects.

No behavior was changed.

Change-Id: I2d9904b7631d0d6be13e0aaed0106f186d388c4f
2019-11-26 09:25:06 +01:00
Adam Wight 9e2468882d Cache parser previewing state
We need to access this in several places, store as instance state for
convenience.

Change-Id: I4ea8f279a34cd8f819d9c07c75e3e8e160786f9b
2019-11-25 14:06:55 +01:00
Adam Wight 8453e3ecd7 Extract stack and state to a new class
Most of this state is used to manage interactions with other state,
and encapsulation allows us to hide data structures and access behind
self-explanatory function names.

The interface is still much wider than I'd like, but it can be improved in
future work.

There is one small behavior change in here: in the `follows` edge case
demonstrated by I3bdf26fd14, we prepend if the splice point cannot be
used because it has a non-numeric key.  I believe this was the original
intention of the logic, and is how the numeric case behaves.  I've verified
that when array_splice throws a warning about non-numeric key, it fails to
add anything to the original array, so the broken follows ref disappeared.

Bug: T237241
Change-Id: I091a0b71ee9aa78e841c2e328018e886a7217715
2019-11-25 14:06:32 +01:00
Adam Wight 10e4a4353d Finish renaming "key" to "name"
Change-Id: Iffd49268cfefefdce9c970f60b5d231e79cbc267
2019-11-25 13:25:40 +01:00
Adam Wight a0f019e1ac Clean up variable names in two more functions
Change-Id: I7e0eb97123a53463133226ae2067d2396e8ceda3
2019-11-25 12:39:57 +01:00
Adam Wight 3ec5a7c3e7 Clean up comments
Clarify where different keys come from and what they're used for.

Change-Id: I534de4952c5b0e053dcba95f31c837547b4a68a6
2019-11-25 12:37:07 +01:00
Adam Wight b9c51b81a1 Rename variable to $name
This is still directly fromt the "name" attribute.

Change-Id: I2c3d0faf591be7e5032c0b26cf5eb7542390bd64
2019-11-25 12:35:42 +01:00
Adam Wight 17c5d6e981 Use "name" to be consistent with attributes
Call the variable the same thing as the attribute it comes from.

Change-Id: I012e29018bdabadc7ec87ed13fc396a7a653933e
2019-11-25 12:31:32 +01:00
Adam Wight fb430f257e Fix variadic parameters in error reporter
Give these functions the same signature as `->msg()`

Change-Id: Ib90df52d6752512d7d9dddf51777c9c23c847e06
2019-11-25 11:02:31 +00:00
Thiemo Kreuz (WMDE) 5401fcd190 Revert a part of "Add missing test cases for follow="…""
This partly reverts commit 8e42a6ecdf.

The variabe $k as created by the foreach() loop is not necessarily
numeric, because the $this->mRef structure contains data both for
named and unnamed <ref>s. The array key is a (non-numeric) string for
named, and an integer for unnamed <ref>s.

array_splice() requires a position, not an array key.

Note that both implementations are wrong. The foreach() might return a
string $k, which makes array_splice() complain and do unwanted things.
The for() loop assumes there are count() array elements with integer
keys, which might not be true. Luckily this was not a problem, because
the isset() check would stop the (to long) loop eary enough.

A better rewrite as well as a test case for this will be added with
I3bdf26f.

Change-Id: I5568d3084197f1861f9dc8983d8b606a961e201f
2019-11-23 22:11:17 +01:00
Thiemo Kreuz c76a5e84f9 Fix misleading method names in CiteErrorReporter
I realized especially the method name html() was wrong. It does not
return HTML. What it returns is still wikitext and must still be parsed.
It only applies some early steps of the parsing process, e.g. expanding
extension <tags>.

Change-Id: I2c403a77eef843940f34f0933e4bfe58e6200ce5
2019-11-22 15:08:39 +01:00
Thiemo Kreuz 177c9cc1eb Fix inconsistencies and deep nesting for follow="…"
* This fixes the refArg() function. If there is nothing wrong with the
follow="…" attribute, it should not return null.

* However, *everything* is false if an unknown error (e.g. an unknown
attribute) occurs.

* A trivial check for `if ( $follow )` is fine because all keys are
guaranteed to not be the string "0".

Change-Id: Ia4e37781e01db1ee6615ffc30bb68e47023c6634
2019-11-22 15:01:09 +01:00
Thiemo Kreuz a823fa23d9 Merge two more code paths in Cite::referencesFormatEntry()
There was another, duplicated special case for previews. It was using
the same message as a <ref> with multiple uses. Now it's only one code
path.

The goal here is to reduce the number of code paths to make it much
easier to implement proper rendering for the extends="…" use cases.

Bug: T237241
Change-Id: I863ac3b5234d3a6f7f2371a2a85385c3aea276e5
2019-11-22 15:01:09 +01:00
jenkins-bot 7f4cff9523 Merge "Move bad dir="…" error reporting down to the renderer" 2019-11-22 13:46:44 +00:00
jenkins-bot 15985a7fa7 Merge "Fix internal presentation of the dir="…" attribute" 2019-11-22 13:26:49 +00:00
Thiemo Kreuz 8e42a6ecdf Add missing test cases for follow="…"
One of the test cases was duplicated, but a lot of the possible code
paths never had tests, including the happy code path!

I found this issue while trying to rework some of the more confusing
loops in this codebase. These changes are still part of this patch. All
loops still do the same as before, but are (I hope) more readable now.

Bug: T238187
Change-Id: I85baeadd9b149025a14c7522bcc4182339c66972
2019-11-22 11:32:28 +01:00
Thiemo Kreuz 55707a745e Rewrite Cite::inReferencesGuardedRef()
Change-Id: I74960a92b3530ef97565cd2e2f79e9696e97f975
2019-11-22 10:14:42 +01:00
Thiemo Kreuz ea6cea93ed Move bad dir="…" error reporting down to the renderer
… and make the error message for bad dir="…" shorter and more to the
point.

Now I understand why the error reporting was not done when $text was
empty: the error was actually appended to $text, which messes with
everything else that also works with the $text variable! This even
includes the API. This error message was exposed via the API. That was
certainly a bug.

With this patch, all error checking for the dir="…" attribute is now
done way down, when rendering the <references> section.

Note this also fixes a bug where the dir="…" was *not* rendered when
previewing a section.

Change-Id: I4ab0cb510973ed879c606bfaa394aacc91129854
2019-11-22 10:07:28 +01:00
Thiemo Kreuz 65c8967c32 Fix internal presentation of the dir="…" attribute
This fixes a whole bunch of inconsistencies:

* The dir attribute is now trimmed, as most others already are. This is
an actual user-facing change.

* The internal representation is now false in case the value was invalid,
not an empty string any more.

* Null means the attribute was not present. This is now always used,
even in the return values that are meant to represent an error state. No
existing behavior changes.

* The internal representation does not contain an HTML snippet any more,
but the raw value "ltr" or "rtl", or null. Note this might influence the
API, because the API actually exposes the internal representation.
However, we are pretty sure the API is not used anywhere. Even if,
exposing HTML code was most certainly an unwanted and unexpected effect
of the patch that introduced the dir attribute. This does make this a
bugfix, I would argue.

Change-Id: Ic385d9ab36fa0545c374d3d63063028ae4e449d4
2019-11-21 12:52:47 +01:00
Thiemo Kreuz ab3063fee5 Move all code to PSR-4 compatible namespaces
This patch does intentionally not touch any file name. Some of the
file names are a little weird now, e.g. \Cite\Cite. These can more
easily be renamed in later patches.

I used https://codesearch.wmflabs.org/search/?q=new%20Cite%5C( and it
looks like this code is not used anywhere else.

Change-Id: I5f93a224e9cacf45b7a0d68c216a78723364dd96
2019-11-20 17:00:13 +01:00