Commit graph

110 commits

Author SHA1 Message Date
jenkins-bot 75ca493b68 Merge "Add test cases for (almost) all possible parsing errors" 2021-09-20 13:11:09 +00:00
Thiemo Kreuz d253fa4e28 Merge code paths in assertTemplateData() helper method
This streamlines the code of the helper method a bit, mostly by
avoiding duplication.

What actually happens is the exact same as before, with one
exception: When a test case doesn't have an expected "output",
the default (mostly empty) output does not run through the
roundtrip test. While doing this is not wrong, it doesn't tell
us anything about the specific test case.

Change-Id: I4a3d8a22c3dd6a9c5c3766195e5aef3cf37a6441
2021-09-06 08:48:01 +02:00
jenkins-bot 34503ed210 Merge "Use more strict assertSame() when comparing strings" 2021-09-04 05:19:49 +00:00
Thiemo Kreuz a7e1d60c64 Revert some unnecessary en→qqx changes
This reverts parts of I6c6342c.

Change-Id: I6e0e60604e393ffde21c3528ff3a26bfc01e66a2
2021-08-31 15:26:52 +02:00
jenkins-bot 324502fd88 Merge "Make all tests use dummy language qqx instead of en" 2021-08-30 17:14:04 +00:00
jenkins-bot 2b1cdff0af Merge "Fix several type hints in JavaScript code" 2021-08-30 17:05:56 +00:00
Thiemo Kreuz eb12e48b14 Add test cases for (almost) all possible parsing errors
"Almost" because I found at least one that appears to be
unreachable (the very first check for null). But changing this
code is out of scope of this patch.

This also updates some of the error messages to explain the
location of the error better. It appears like the incomplete
paths are copy-paste mistakes.

I also found one duplicate test case and removed it.

Change-Id: Ic0ee9d04f5cd1060ade385ef308e70d221dd2f18
2021-08-30 15:17:21 +00:00
Thiemo Kreuz 696e3ed98f Make all tests use dummy language qqx instead of en
I find this good practice. It makes the tests more robust (e.g.
changes to a text don't make the test fail) and is potentially
faster, as no localization needs to be loaded.

Change-Id: I6c6342c80a40ab7260c35e7f1e3052aa4a9b9358
2021-08-30 17:15:41 +02:00
Thiemo Kreuz 2232a44638 Dramatically improve performance of random string generator
This test was reported as being slow (approx. 0.1s, but still).
This new implementation is 10 times faster, while still
fulfilling the requirements. While the new algorithm is more
predictable (every chunk is guaranteed to contain every
character exactly onece), it's obviously still good enough.

Neither the exact length of the generated string nor the exact
length of the gzipped string matter. PHP's random number
generator might be different – possibly generating a string
that compresses different. Newer versions of the gzip library
possibly save an extra byte. Who knows. This test shouldn't
care, as long as the gzipped string is long enough.

Compatibility with PHP 7.1 can be dropped as it is not
supported any more since MediaWiki 1.34, as far as I can tell.

Change-Id: I8d63390c9f4baa6084f932fa34068f606696cafc
2021-08-29 14:21:52 +02:00
Thiemo Kreuz 2cb03827cc Fix several type hints in JavaScript code
Two main mistakes:
* The {...foo} syntax is for a variable number of parameters.
  But this is not the case here.
* Optional parameters should be marked as such via [foo].

Change-Id: I0c26ea44fab6094616443ce8fae4fd47c61fd7c4
2021-08-28 12:08:28 +02:00
Thiemo Kreuz 930edf2419 Use more strict assertSame() when comparing strings
assertEquals() does have weird effects, like not reporting a
method that is expected to return null but returns an empty
string, and such. While it's usually not a problem, I learned
to avoid it.

Change-Id: I4f27ed5b200278021e051f1ab4d272f48e0bf344
2021-08-28 11:08:24 +02:00
jenkins-bot b745f0961a Merge "Make parameter order when sorting by status independent of language" 2021-08-02 12:51:37 +00:00
jenkins-bot 73d5a6504d Merge "Change param duplicate numbering to start with 2" 2021-07-30 14:17:47 +00:00
jenkins-bot ffeaea6921 Merge "Fix handling of duplicate parameter names" 2021-07-30 14:11:09 +00:00
Thiemo Kreuz c2508a78ad Make parameter order when sorting by status independent of language
At the moment, when the user clicks the "Status" column to
sort by status, the statuses are ordered alphabetically,
which gives widely varying results depending on the language.
But there is an inherent order for these, even hard-coded in
the code: When a parameter is deprecated, nothing else matters.
Otherwise it's required → suggested → optional. Doesn't it
make much more sense to order the column this way? Especially
because there are never more than these 4 hard-coded values.

This is one of the (few remaining) issues mentioned on
https://de.wikipedia.org/wiki/Vorlage:TemplateData#Vorlagendokumentationsseite_verbessern_%E2%80%93_MediaWiki_ungen%C3%BCgend

This patch also makes it so that a CSS class name is always
added to all status fields, not only to the required ones.
This allows for per-wiki or per-user styling.

Change-Id: Id3f1ffafe09a3817972a4ee4bd4a3ded7be6f039
2021-07-30 13:42:00 +00:00
jenkins-bot 654e1ba037 Merge "Render docs with HTML5 <section>/<header> markup" 2021-07-30 13:40:14 +00:00
libraryupgrader a09e939fe7 build: Updating dependencies
composer:
* mediawiki/mediawiki-codesniffer: 36.0.0 → 37.0.0

npm:
* postcss: 7.0.35 → 7.0.36
  * https://npmjs.com/advisories/1693 (CVE-2021-23368)
* glob-parent: 5.1.0 → 5.1.2
  * https://npmjs.com/advisories/1751 (CVE-2020-28469)
* trim-newlines: 3.0.0 → 3.0.1
  * https://npmjs.com/advisories/1753 (CVE-2021-33623)

Change-Id: I23a441a089501a97f329b7b6d37bc658481e682f
2021-07-24 02:43:18 +00:00
Thiemo Kreuz 4b110c6c85 Render docs with HTML5 <section>/<header> markup
Change-Id: I91083561f7c4bcd3ee7184aee114a735de153fc4
2021-07-12 12:13:34 +02:00
Adam Wight 7b32bcefb4 Add suggested values parameter
Parameters may include a `suggestedvalues` property, which is rendered
in the UI for some parameter types.

TemplateData editor UI elements are implemented behind the
TemplateDataSuggestedValuesEditor feature flag.

Bug: T271897
Change-Id: I14012c79b3fa0d48c58fd8999584cc03ec03575e
2021-04-09 12:05:37 +02:00
jenkins-bot f26a5d37ec Merge "More strict validation of InterfaceTexts" 2021-02-13 17:23:44 +00:00
jenkins-bot 8b9c2f8b13 Merge "Remove inconsistent capitalization of parameter names" 2021-01-25 16:26:17 +00:00
Umherirrender b97d851664 Add missing @param to test documentation
Change-Id: Ie99e77e7e6f543eaf041eb8530c6f318538c8da5
2021-01-17 20:17:57 +01:00
Thiemo Kreuz 563a44ca72 Change param duplicate numbering to start with 2
… instead of 0. Conditionally add a dash in front as well to
avoid confusing results like '1' + sequence number = '12'.

Change-Id: I345704b00ba3812c4905f85e35cf21a6dfd05437
2021-01-08 14:50:04 +00:00
Thiemo Kreuz 954211d8ba Fix handling of duplicate parameter names
I looks like the Model.params data structure is build in a way
that it allows mismatching parameter "keys" and "names". E.g.

{
    "a": { "name": "a" },
    "a0": { "name": "a" },
}

There are comments in the code that suggest this is
intentional.

I found code that confused these two values and tries to use
the name as a key, for example. This fails, messes up the
paramOrder, and such.

This should not have much, if any effect for users because
users are blocked from doing this anyway, e.g. buttons get
disabled.

Change-Id: I2067024ad8d5b8e985a4f162cf6875f523777a6c
2021-01-08 15:48:55 +01:00
Thiemo Kreuz 294786223d Remove inconsistent capitalization of parameter names
Example:

On
https://en.wikipedia.beta.wmflabs.org/wiki/Template:Anschutz
the two parameters "state" and "capitalization_test" don't have
a label in the <templatedata> JSON structure. Instead the
internal parameter name is shown. But it's capitalized for an
unknown reason. I guess this is done to make the table look
"nice". But it causes confusion – see the ticket.

This capitalization is there since the very first commits from
2013, see I16d3f9e.

Compare with VisualEditor: Edit the template on
https://en.wikipedia.beta.wmflabs.org/wiki/Conflict-title-0.8542952978413387-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
or use the old wikitext editor and TemplateWizard to insert
the template. In both cases the parameter names are not
capitalized.

Another argument why this capitalization is misplaced: When
there is no <templatedata> JSON blob, the TemplateData editor
auto-detects the parameters and semi-automatically creates a
minimal JSON blob. This is the moment where labels should be
created and stored so the user can edit them. But this doesn't
happen (for good reasons).

The user can't do anything about the current capitalization.
The only way to change it is to add a label that does nothing
but repeat the parameter name, just to undo the capitalization.
That should not be the way this works, I would like to argue.

Bug: T174771
Change-Id: Ia8133d3f0d6b79fe89c63bb0392a334c0a185a65
2020-12-23 11:59:12 +00:00
Adam Wight 93b7bc0756 Drop unnecessary top-level closures
Change-Id: Ie8fbde864cd08f0657e4239f525ef9dd087aee41
2020-12-07 15:24:31 +01:00
Adam Wight bda6215309 Migrate to package files: ext.templateDataGenerator.data
Bug: T260343
Change-Id: Ifd171b40de704bff8c063da46b711a384238a0f5
2020-12-04 19:41:30 +01:00
Cindy Cicalese b10862a333 Make extension data JSON-serializable #1 - forward-compat.
Transitioning of ParserCache to JSON serialization will add
a requirement that all the extension data is JSON-serializable.

This is the first step in transition - making it forward-compatible.

Bug: T266252
Change-Id: If1c9d9bb5b0039df80a9d9b30c247206d8844c0a
2020-10-30 12:33:20 -04:00
Thiemo Kreuz c36cdcf862 Fix @covers tags in TemplateDataBlobTest
There is gzip-related code in this test, i.e. it is testing
the …Compressed… subclass as well.

Change-Id: I26c011be318427143a7d527f9f51f5c32c1e4fd3
2020-10-08 11:12:13 +02:00
jenkins-bot 86a6865512 Merge "Fix parameter extraction from wikitext source" 2020-10-06 23:07:23 +00:00
Thiemo Kreuz de434822b4 Fix all eslint warnings
Change-Id: I731ea4a54b001a8d30743ca32188edb1c039253e
2020-09-24 13:31:18 +01:00
Thiemo Kreuz f6de5d3493 Fix parameter extraction from wikitext source
This fixes a series of issues:
* The JS implementation had a trim() in one place that was
  missing in PHP.
* The actual parameter name in the paramNames/$params array was
  trimmed, but the "normalized" name (this is only for duplicate
  detection) was not trimmed.
* It was possible for an empty parameter to show up.

This resulted in very strange behavior, e.g. {{{ 1}}}{{{1 }}}
was detected as "1" and "10" (?), i.e. it would try to renumber
the duplicate in a strange way (string "1" plus a counter that
starts with 0).

Change-Id: I0a6371f3633b03b5b21809ecd06ea4c72d7d914d
2020-09-22 18:27:23 +02:00
Marielle Volz 9760fed11d Add map to Qunit tests
Add roundtrip test where a maps object is present.

Bug: T261406
Change-Id: I620e60599a28c677f1af6231f36bdb39658afb7a
2020-08-30 15:07:03 +01:00
Thiemo Kreuz c355a8fac4 More strict validation of InterfaceTexts
This fixes parts of a TODO in the code. Specifically:
* When an InterfaceText is an object, it can not be any object,
  but must be an stdClass.
* It can't be empty.
* Language codes must be non-empty strings.
* Values must be strings.

I'm intentionally not adding more validation for the language
codes, as this needs discussion (what values should be allowed?),
and can potentially break existing pages.

Same for empty strings. This can easily happen when users
manually create a <templatedata> tag and copy-paste pieces that
are meant to be filled in later. Empty strings are not really
invalid (again, this needs discussion), but might be something a
client want's to ignore.

Change-Id: I0facaa08cffe5a5a038423a58d55bc90a40b2d75
2020-08-19 15:29:55 +02:00
Siddharth VP 512911b92e Make getRawParams() ignore parameters in comments and nowiki tags
This is used by the includeMissingTitles option of API action=templatedata.
Parameter syntax existing within nowiki tags or comments will not be valid
for the template.

Bug: T237195
Change-Id: Ibbfa3e21488f2a37fc494862e929baf50607d4c9
2020-08-17 00:33:06 +00:00
adham-khatean 55a423a5ba Remove Sortable class from tables that have one row or less
Removing the sorting arrows from tables that have one row or less in
templates. By using variable $sorting that changes according to the
coung of elements in the array $data->params, to determine whether the
table should have "sortable" class or not.

Bug: T126150
Change-Id: I414c2375d4eb4da5d78f92f6b4e99b55e314ce4d
2020-06-22 09:13:17 +01:00
James D. Forrester c88aed986c build: Upgrade mediawiki-codesniffer to v28.0.0
Change-Id: I39190ff2056c24184f128eba71d619fc748b0ca5
2019-10-11 12:04:42 -07:00
saper ce13175017 Do not always store template data compressed
Only MySQL can handle blobs in page_props.pp_value

Bug: T203850
Change-Id: Ie20d412d89855e451d48f7656d97fb305b1c4c55
2019-09-30 14:24:39 +02:00
Ed Sanders 49d469b2d1 build: Update eslint-config-wikimedia to 0.10.1
Change-Id: Ia8baa12906d4f23fcdf07a0bb664b4a3435d18df
2019-02-10 22:14:11 +00:00
Timo Tijhof 4aaa9dfcf8 Remove default value for 'paramOrder'
When normalising a TemplateData blob for API consumers, we previously
automatically generated the 'paramOrder' with the order of the keys
as they were specified in the JSON blob (which, unlike in JS, is
known to be reliable in PHP).

While this was useful to some extent, it made it mandatory for
Parsoid and VisualEditor to always re-order properties during
edits to match the specified order.

In order to allow the order to remain flexible/unspecified, the
original Specification made paramOrder optional, but during the
implementation I gave it a default, which kind of defeated that
intention. This patch fixes that.

Bug: T138200
Change-Id: Ib40d23dac7e75274083f95a25c5aa1c22dfffb22
2019-01-08 19:55:59 +00:00
Timo Tijhof 498c66f451 tests: Add coverage for TemplateDataBlob::getHtml
Change-Id: I04caf657ac72f90d1dc0797f293d5a4f61cfdf08
2018-12-29 20:43:02 +00:00
jenkins-bot 4221945fc9 Merge "Ignore bang magic word in table syntax when guessing parameters" 2018-12-10 23:55:08 +00:00
Ed Sanders 47c3c69baa Update eslint-config-wikimedia to 0.9.0
Also bump stylelint (no fixes required)

Change-Id: Ia6bd3a8c0392eafb939f9ff05827e1320f56b706
2018-11-29 14:19:33 +00:00
Sam Wilson 75e54ae999 Ignore bang magic word in table syntax when guessing parameters
Three-braces-and-a-bang can be a table starting construct in templates.

This also fixes an overlooked bug in which the test wasn't checking
array element key names.

Bug: T157029
Change-Id: I69ed4fc9fe3bb126b7b39abea0f58ad56adf3885
2018-11-27 11:46:23 +08:00
Ed Sanders 9a8c807f51 eslint: Introduce wikimedia/qunit ruleset
Change-Id: I3972918c00e69d325c5cb2055d4996a8c07b1034
2018-11-15 23:21:32 +00:00
Bartosz Dziewoński 9695c4fbb2 TemplateDataBlobTest: Ensure identical results on PHP 7.1
The implementation of mt_srand()/mt_rand() was changed in PHP 7.1 to
a better one. Pass MT_RAND_PHP to get the old implementation.
http://php.net/manual/en/function.mt-rand.php#refsect1-function.mt-rand-changelog

Bug: T206287
Change-Id: Ia2f79f2364583f7ee8f145519ae4ae1b7210c450
2018-10-05 11:08:01 -07:00
Sam Wilson 79e5347d90 Prevent leading hash from starting param names when guessing
Parameter names can be constructed with parser functions, so
we should not include these as raw parameter names. There
might be more characters to add here.

Bug: T203605
Change-Id: If57a9ed7edf1e881cd121d9a1bcb2e7455c04ec9
2018-09-12 10:04:45 +08:00
jenkins-bot 783e99618a Merge "Use Parser::stripOuterParagraph" 2018-08-31 06:43:45 +00:00
Sam Wilson bc1422b788 Ignore subsequent similarly-named parameters when extracting
When extracting raw parameters from wikitext, ignore those that
only differ from already-found ones by spaces, underscores, or
hyphens.

Bug: T193265
Change-Id: I012e06bf4353eaaf0613124d7d6b88f3444d248a
2018-08-27 11:14:27 +08:00
Umherirrender 0f96d4c650 Use Parser::stripOuterParagraph
Since 1.24

Change-Id: Id919f85655008a7e5f4caa03bf28808e56254b98
2018-08-13 18:57:20 +02:00