Commit graph

111 commits

Author SHA1 Message Date
Thiemo Kreuz 504db2c46a Add strict PHP 7 type hints to most code
This patch does two things:
* Add strict PHP 7 type hints to most code.
* Narrow the interface of the checkRefsNoReferences() method to not
  require a ParserOptions object any more.

Change-Id: I91c6a2d9b76915d7677a3f735ee8e054c898fcc5
2019-12-02 08:51:42 +01:00
Thiemo Kreuz 22627f074d Make the normalizeKey() method private
There was a call in the API that was *not* using normalizeKey(). Now
that the API is gone, we can inline this.

This patch also contains a bunch of cleanups that might already been
resolved in the previous patches.

Change-Id: Id3767b5830268c8cfe9c10efabfa4a31e9dafeb8
2019-11-29 19:02:48 +01:00
Adam Wight a40b1b10be Extract footnote body rendering
Change-Id: I9537849cbd700d5dc7ec1a53d852d69b0fe0dc35
2019-11-29 16:22:35 +01:00
Adam Wight a4c056f59b Extract key formatting
Change-Id: I155ec6f3e21075587dbcfdfdc346f28f958e3c15
2019-11-29 13:41:12 +01:00
Thiemo Kreuz 13598ba11e Render nested references
Forked from Icd933fc983.

Bugs and unimplemented features are documented as TODOs in the parser test
fixtures.

Bug: T237241
Change-Id: I9427e025ea0bcf2fa24fd539a775429cc64767cc
2019-11-29 13:40:34 +01:00
jenkins-bot 3beb5c3634 Merge "Remove ApiQueryReferences support" 2019-11-29 11:30:38 +00:00
Adam Wight a176e22097 Remove ApiQueryReferences support
This API was never used in Wikimedia production, and would have caused
performance problems.  Removing the dead code will simplify our refactoring.

Bug: T238195
Change-Id: I7088f257ec034c0d089e0abdaa5a739910598300
2019-11-28 11:08:46 +01:00
Adam Wight ab78df8d5c Wire extends into ReferenceStack
Takes no action, just shuttle the value between functions.

(Split from I9427e025ea0)

Change-Id: I271043e9161835f3278098787bf58b50ed93c892
2019-11-28 02:10:11 +01:00
jenkins-bot dbf4c56896 Merge "[Refactor] Pass validation error with StatusValue" 2019-11-27 21:23:30 +00:00
Adam Wight 22a0350d84 [Refactor] Pass validation error with StatusValue
This has clearer semantics than checking for a `false` attribute.

Change-Id: I68f777eda40f8f157deafacaed02d4bd10cbf25c
2019-11-27 18:05:19 +01:00
Thiemo Kreuz 38a38ae472 Add smoke tests for previously uncovered combinations
I noticed a possible issue related to the $this->refSequence counter
in the patch Ida9612d. Some of these counters might get messes up, but
there was never a test that checked what will happen to the *next*
reference then.

I checked the test cases in this patch with a very old version of the
codebase.

Change-Id: If6e56f727dce5d0e5e38e048e602437597248a42
2019-11-27 16:34:46 +01:00
Thiemo Kreuz 0013943a4a Rewrite argument parsing and use for both <ref> & <references>
We realized the trim() are not needed. This does not leave much behind
in the existing refArg() method, except that it checks for unknown keys.

I tried a few strategies and ended using the pretty new possibility to
have keys in list(), as well as use [] instead of list(). Both is
supported since PHP 7.1.

Change-Id: I569bfa14e68b64402519bd39022c197553881dde
2019-11-27 14:01:52 +01:00
jenkins-bot 7ed54a3f3d Merge "Remove redundant attribute trimming" 2019-11-27 12:26:00 +00:00
Thiemo Kreuz 99d23ac841 Remove redundant attribute trimming
We noticed the group="…" attribute was the only one that was not
trimmed. Does this mean it was possible to have two groups "a" and
" a"? It turns out: no. This was never possible because the parser
already trims all attributes before calling this code.

I tried to come up with the worst possible test case, but it succeeds,
even with very old versions of this codebase.

I suggest to remove the extra trimming from this codebase and rely on
what the parser provides.

Note the content is special and *not* trimmed by default.

Change-Id: Idff015447d7156ba7b5c03a5c423f199a71349f2
2019-11-27 12:12:51 +00:00
Thiemo Kreuz 04f784bc02 Remove non-existing property from ReferenceStack
Change-Id: Id789897dd92c7692e36f54b828c097820ab46b43
2019-11-27 12:57:55 +01:00
jenkins-bot 36952a55a1 Merge "Add test to cover Cite::listToText()" 2019-11-27 11:46:44 +00:00
Thiemo Kreuz 2ffcae0425 Add seperate unit test cases for Cite::testValidateRef()
Change-Id: I6008b834d18c2008304b51dd41f0387c28e53d94
2019-11-27 12:09:31 +01:00
Thiemo Kreuz f5b9360467 Add test to cover Cite::listToText()
As far as I can see this must (for now) be an integration test because
it is calling wfMessage().

Change-Id: Ic581c38128364990ccf81539996d1dda53bdcda5
2019-11-27 11:59:53 +01:00
jenkins-bot 40942620b2 Merge "Rename ambiguous tests to …UnitTest" 2019-11-27 10:51:19 +00:00
Thiemo Kreuz 9f1521a773 Rename ambiguous tests to …UnitTest
These exist two times, one time in the unit/ folder as a unit test, and
another time in the parent folder as an integration test. This confused
me already several times.

Change-Id: I147b8af8a7edba2582496468b4878faecc6d8110
2019-11-27 11:15:39 +01:00
Thiemo Kreuz a6a16f0703 Update and increase ReferenceStack test coverage
Functional changes:
* hasGroup() will return false when a group exists, but is empty. This
  is in line with what other methods like getGroups() already do.
  Shouldn't have any effect on the existing code, but feels more clean
  and consistent.
* getGroupRefs() won't fail any more when asked for an unknown group.

Tests:
* Add missing @covers for the constructor.
* Simplify test setup by always returning a spy. All tests need it
  anyway.
* Cover 3 more methods.

Change-Id: Ie93e9af6258b757d842b30b0b059344733aad434
2019-11-27 11:08:00 +01:00
Adam Wight ec091fe906 Reorder keys
This doesn't make any functional difference, but helps minimize later
patch Ida9612d14

Change-Id: Ice89bad02e077437d0df6fa9f51f90b4cab4837c
2019-11-26 17:06:01 +01:00
Adam Wight 890e86a7fb Fix tests: cannot have name and follow
This was impossible and is prevented by validation, so do not test.

Change-Id: I836b38c700f41f692e5c6a893be0076febfc9c4d
2019-11-26 16:57:56 +01:00
Adam Wight 55099a7b0c Remove impossible condition
Numeric `$name` is caught during validation.

Change-Id: Id1c3e6717af38b0b1393c135732e084d261b53f6
2019-11-26 16:43:21 +01:00
Adam Wight 7cdcc2b075 Alphabetize returned array of attributes
That was annoying me.  Since we're passing a bare list, alphabetical
order helps make the code and tests readable.

Change-Id: I6384094e429e0e2a6fa810fdc28ae0643a0ccf7c
2019-11-26 15:28:02 +01:00
Adam Wight 301b1fbcaa Move a follow edge case to validation
Change-Id: I06cf5291c258322e16449d61879bf7a18129b174
2019-11-26 15:28:02 +01:00
Adam Wight 8453e3ecd7 Extract stack and state to a new class
Most of this state is used to manage interactions with other state,
and encapsulation allows us to hide data structures and access behind
self-explanatory function names.

The interface is still much wider than I'd like, but it can be improved in
future work.

There is one small behavior change in here: in the `follows` edge case
demonstrated by I3bdf26fd14, we prepend if the splice point cannot be
used because it has a non-numeric key.  I believe this was the original
intention of the logic, and is how the numeric case behaves.  I've verified
that when array_splice throws a warning about non-numeric key, it fails to
add anything to the original array, so the broken follows ref disappeared.

Bug: T237241
Change-Id: I091a0b71ee9aa78e841c2e328018e886a7217715
2019-11-25 14:06:32 +01:00
Subramanya Sastry 2cfb76f8b6 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 7dfc2e931a6afeb62d2a0d791cda88fd8d39c070

Change-Id: I7edd1f293530653ae1bbfe47028e585f2b46927b
2019-11-22 18:44:22 +00:00
Thiemo Kreuz c76a5e84f9 Fix misleading method names in CiteErrorReporter
I realized especially the method name html() was wrong. It does not
return HTML. What it returns is still wikitext and must still be parsed.
It only applies some early steps of the parsing process, e.g. expanding
extension <tags>.

Change-Id: I2c403a77eef843940f34f0933e4bfe58e6200ce5
2019-11-22 15:08:39 +01:00
Thiemo Kreuz 177c9cc1eb Fix inconsistencies and deep nesting for follow="…"
* This fixes the refArg() function. If there is nothing wrong with the
follow="…" attribute, it should not return null.

* However, *everything* is false if an unknown error (e.g. an unknown
attribute) occurs.

* A trivial check for `if ( $follow )` is fine because all keys are
guaranteed to not be the string "0".

Change-Id: Ia4e37781e01db1ee6615ffc30bb68e47023c6634
2019-11-22 15:01:09 +01:00
jenkins-bot 7f4cff9523 Merge "Move bad dir="…" error reporting down to the renderer" 2019-11-22 13:46:44 +00:00
jenkins-bot 15985a7fa7 Merge "Fix internal presentation of the dir="…" attribute" 2019-11-22 13:26:49 +00:00
Thiemo Kreuz 8e42a6ecdf Add missing test cases for follow="…"
One of the test cases was duplicated, but a lot of the possible code
paths never had tests, including the happy code path!

I found this issue while trying to rework some of the more confusing
loops in this codebase. These changes are still part of this patch. All
loops still do the same as before, but are (I hope) more readable now.

Bug: T238187
Change-Id: I85baeadd9b149025a14c7522bcc4182339c66972
2019-11-22 11:32:28 +01:00
Thiemo Kreuz ea6cea93ed Move bad dir="…" error reporting down to the renderer
… and make the error message for bad dir="…" shorter and more to the
point.

Now I understand why the error reporting was not done when $text was
empty: the error was actually appended to $text, which messes with
everything else that also works with the $text variable! This even
includes the API. This error message was exposed via the API. That was
certainly a bug.

With this patch, all error checking for the dir="…" attribute is now
done way down, when rendering the <references> section.

Note this also fixes a bug where the dir="…" was *not* rendered when
previewing a section.

Change-Id: I4ab0cb510973ed879c606bfaa394aacc91129854
2019-11-22 10:07:28 +01:00
Thiemo Kreuz 65c8967c32 Fix internal presentation of the dir="…" attribute
This fixes a whole bunch of inconsistencies:

* The dir attribute is now trimmed, as most others already are. This is
an actual user-facing change.

* The internal representation is now false in case the value was invalid,
not an empty string any more.

* Null means the attribute was not present. This is now always used,
even in the return values that are meant to represent an error state. No
existing behavior changes.

* The internal representation does not contain an HTML snippet any more,
but the raw value "ltr" or "rtl", or null. Note this might influence the
API, because the API actually exposes the internal representation.
However, we are pretty sure the API is not used anywhere. Even if,
exposing HTML code was most certainly an unwanted and unexpected effect
of the patch that introduced the dir attribute. This does make this a
bugfix, I would argue.

Change-Id: Ic385d9ab36fa0545c374d3d63063028ae4e449d4
2019-11-21 12:52:47 +01:00
Thiemo Kreuz ab3063fee5 Move all code to PSR-4 compatible namespaces
This patch does intentionally not touch any file name. Some of the
file names are a little weird now, e.g. \Cite\Cite. These can more
easily be renamed in later patches.

I used https://codesearch.wmflabs.org/search/?q=new%20Cite%5C( and it
looks like this code is not used anywhere else.

Change-Id: I5f93a224e9cacf45b7a0d68c216a78723364dd96
2019-11-20 17:00:13 +01:00
Thiemo Kreuz b10dd4ec27 Block de-facto empty <ref> as if it's empty
The use case we care about is this:
<ref extends="some_book"> </ref>

It doesn't make sense that works, but the following doesn't:
<ref extends="some_book"></ref>

We decided that both need to behave the same.

For consistency this patch is applying the same change to all references,
no matter if they use the extends attribute or not. This is an actual
change and might make existing wikitext render differently. However, I
would like to argue that all wikitext that was using this was broken. The
effect of a <ref> </ref> with some whitespace is that the <references>
section at the end of the article will contain – well – an empty footnote.

Bug: T237241
Change-Id: Iaee35583eabcb416b0a06849b89ebbfb0fb7fef9
2019-11-20 15:07:54 +00:00
jenkins-bot 32e1f8e7c3 Merge "Don't pass a Title object around that's not needed" 2019-11-19 16:09:13 +00:00
jenkins-bot 7018e82352 Merge "Extract all error reporting to a CiteErrorReporter" 2019-11-19 15:53:29 +00:00
Thiemo Kreuz 9d2d61ff09 Don't pass a Title object around that's not needed
Change-Id: Iea9c366c4b45ba4cd9171c8b4fffc307c852b6e2
2019-11-19 16:48:36 +01:00
Thiemo Kreuz 342e231a22 Extract all error reporting to a CiteErrorReporter
Change-Id: Icf61c9a27fd03266c98caf443bb9f00a421e31f6
2019-11-19 14:53:31 +01:00
Thiemo Kreuz 7157c7f494 Add @license to all files
Note this codebase appears to be dual-licensed. Some files mention MIT,
but extension.json and some other files mention GPL.

Since WMDE typically uses GPL, I will continue to mark the files we
created as such.

Change-Id: I126da10f7fb13a6d4c99e96e72d024b2e5ecee06
2019-11-19 11:31:08 +01:00
Thiemo Kreuz d50c169612 Minor test updates for more complete test coverage
The main motivation here is to cover the fallback code that was moved
in I20c814d. At some point we might touch this code again.

Bug: T238194
Change-Id: I0ab8a34b09790f42b10376eb3730c3b3c4ef53d2
2019-11-14 14:42:22 +00:00
jenkins-bot 668ad80c58 Merge "Pass ParserOutput as parameter to Cite::checkRefsNoReferences" 2019-11-13 08:48:30 +00:00
Thiemo Kreuz 7920ec3150 Pass ParserOutput as parameter to Cite::checkRefsNoReferences
Change-Id: Ibc4455dfde9f60bb27eac0d71064796878994bc5
2019-11-12 16:33:52 +01:00
Thiemo Kreuz e68b96f75c Add more basic tests for API and RL modules
Change-Id: I5e54fae041ec8431c170be468c12f0622e355b9b
2019-11-12 16:32:15 +01:00
jenkins-bot 0782f24d31 Merge "Make most existing Cite tests pure unit tests" 2019-11-12 14:44:24 +00:00
Thiemo Kreuz f94b400474 Make most existing Cite tests pure unit tests
1. Most existing CiteTests can be unit tests. They run so much faster
this way.

2. I modified some test cases to cover all trim() in the code.

3. The strict type hint in CiteHooks is removed because the parameter
is not used. Having a hard type hint for what is effectively dead code
makes the code more brittle for changes done outside of this codebase.

Change-Id: I1bff1d6e02d9ef17d5e6b66aeec3ee42bba99cf4
2019-11-12 14:56:40 +01:00
Thiemo Kreuz d8fbbd0037 Remove dependency on PPFrame from Cite class
This fixes a series of issues:
* There is nothing about a "frame" in the Cite class any more.
* There is no addModules() call in the Cite class any more.

Change-Id: I20c814d46c26825c5c07eab0a5586de3a531eee7
2019-11-12 13:06:39 +01:00
jenkins-bot f36be06996 Merge "Add basic unit tests for all 3 hook classes" 2019-11-12 11:38:13 +00:00