wikimedia/mediawiki-extensions-VisualEditor
1
0
Fork 0
mirror of https://gerrit.wikimedia.org/r/mediawiki/extensions/VisualEditor synced 2024-11-25 06:46:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
mediawiki-extensions-Visual.../tests/parser/parserTests-whitelist.js

72 lines
6.9 KiB
JavaScript
Raw Normal View History

Add a parser test whitelist for manually-checked tests, and an option to print JSON-serialized parser output for failing tests, which can then be added to the whitelist if appropriate.
2011-12-01 10:58:12 +00:00
/* A map of test titles and their manually verified output. If the parser
* output matches the expected output listed here, the test can be marked as
* passing in parserTests.js. */
testWhiteList = {};
Make whitelist usage an option; tweak comment a bit
2011-12-01 11:47:22 +00:00
// The nesting of italic/bold tags is changed in this test, but the resulting
// formatting is identical
Convert quote handling (italic/bold) to a core extension operating on the token stream. This is the first token transformation exercising the TokenTransformer class as its dispatcher. Template expansions, wiki link formatting, tag sanitation and extensions should be able to use the same dispatcher by registering for specific token types. The parser performance is very slightly improved as the token stream is only traversed once.
2011-12-12 20:53:14 +00:00
testWhiteList["Italics and bold"] = "<ul><li> plain</li><li> plain<i>italic</i>plain</li><li> plain<i>italic</i>plain<i>italic</i>plain</li><li> plain<b>bold</b>plain</li><li> plain<b>bold</b>plain<b>bold</b>plain</li><li> plain<i>italic</i>plain<b>bold</b>plain</li><li> plain<b>bold</b>plain<i>italic</i>plain</li><li> plain<i>italic<b>bold-italic</b>italic</i>plain</li><li> plain<b>bold<i>bold-italic</i>bold</b>plain</li><li> plain<i><b>bold-italic</b>italic</i>plain</li><li> plain<i><b>bold-italic</b></i><b>bold</b>plain</li><li> plain<i>italic<b>bold-italic</b></i>plain</li><li> plain<b>bold<i>bold-italic</i></b>plain</li><li> plain l'<i>italic</i>plain</li><li> plain l'<b>bold</b> plain</li></ul>";
Add a parser test whitelist for manually-checked tests, and an option to print JSON-serialized parser output for failing tests, which can then be added to the whitelist if appropriate.
2011-12-01 10:58:12 +00:00
Prefix all internal data- attributes with data-mw- and adjust the whitelist and test output normalization accordingly. 235 tests passing.
2012-02-13 13:54:07 +00:00
testWhiteList["Bug 2702: Mismatched <i>, <b> and <a> tags are invalid"] = "<p><i><a href=\"http://example.com\">text</a></i><a href=\"http://example.com\" data-mw-sourcePos=\"30:61\"><b>text</b></a><i data-mw-sourcePos=\"62:106\">Something <a href=\"http://example.com\">in italic</a></i><i data-mw-sourcePos=\"107:164\">Something <a href=\"http://example.com\">mixed</a></i><a href=\"http://example.com\"><b>, even bold</b></a><i data-mw-sourcePos=\"165:204\"><b data-mw-sourcePos=\"165:204\">Now <a href=\"http://example.com\">both</a></b></i></p>";
Improve external links and definition lists, now 133 tests passing ;) Also add printwhitelist option to test runner, provides js code copy/pastable to whitelist.
2011-12-01 14:25:59 +00:00
Prefix all internal data- attributes with data-mw- and adjust the whitelist and test output normalization accordingly. 235 tests passing.
2012-02-13 13:54:07 +00:00
testWhiteList["Unclosed and unmatched quotes"] = "<p data-mw-sourcePos=\"0:66\"><i><b>Bold italic text </b>with bold deactivated<b> in between.</b></i></p><p><i><b>Bold italic text </b></i><b>with italic deactivated<i> in between.</i></b></p><p><b>Bold text..</b></p><p>..spanning two paragraphs (should not work).<b></b></p><p><b>Bold tag left open</b></p><p><i>Italic tag left open</i></p><p>Normal text.<!-- Unmatching number of opening, closing tags: -->\n</p><p><b>This year'</b>s election <i>should</i> beat <b>last year'</b>s.</p><p><i>Tom<b>s car is bigger than </b></i><b>Susan</b>s.</p>";
Add empty tables to the whitelist (legal in HTML5). Also add one more functionally identical italic/bold/link permmutation on the whitelist.
2011-12-06 22:05:43 +00:00
Fix quote handling and tweak the whitelist a bit. 'any' token registrations are now merged with specific registrations by rank. Not yet clear if that is a good idea overall, need to check use cases when implementing template expansion and other functionality. 183 parser test now passing.
2012-01-04 14:09:05 +00:00
// The expected result for this test is really broken html.
Change wikilink tokenization strategy to split on pipes. This makes it possible to support template / template argument expansion in image options, and causes little trouble for wikilinks. Non-image wikilinks with multiple text pipes are quite rare in the dumps, and concatenating description tokens with a plain '|' is quite easy. 261 parser tests passing.
2012-03-05 12:00:38 +00:00
testWhiteList["Link containing double-single-quotes '' in text embedded in italics (bug 4598 sanity check)"] = "<p data-mw-sourcePos=\"0:45\"><i>Some <a data-mw-type=\"internal\" href=\"/wiki/Link\">pretty </a></i><a data-mw-type=\"internal\" href=\"/wiki/Link\">italics<i> and stuff</i></a><i>!</i></p>";
Add a few more items to the whitelist
2011-12-07 11:44:38 +00:00
testWhiteList["External link containing double-single-quotes in text embedded in italics (bug 4598 sanity check)"] = "<p><i>Some <a href=\"http://example.com/\">pretty </a></i><a href=\"http://example.com/\">italics<i> and stuff</i></a><i>!</i></p>";
Convert quote handling (italic/bold) to a core extension operating on the token stream. This is the first token transformation exercising the TokenTransformer class as its dispatcher. Template expansions, wiki link formatting, tag sanitation and extensions should be able to use the same dispatcher by registering for specific token types. The parser performance is very slightly improved as the token stream is only traversed once.
2011-12-12 20:53:14 +00:00
// This is a rare edge case, and the new behavior is arguably more consistent
Fix quote handling and tweak the whitelist a bit. 'any' token registrations are now merged with specific registrations by rank. Not yet clear if that is a good idea overall, need to check use cases when implementing template expansion and other functionality. 183 parser test now passing.
2012-01-04 14:09:05 +00:00
testWhiteList["5 quotes, code coverage +1 line"] = "<p>'<i></i></p>";
Add a few more items to the whitelist
2011-12-07 11:44:38 +00:00
Clean up 'END' token handling a bit.
2012-01-17 20:01:21 +00:00
// The comment in the test already suggests this result as correct, but
// supplies the old result without preformatting.
Prefix all internal data- attributes with data-mw- and adjust the whitelist and test output normalization accordingly. 235 tests passing.
2012-02-13 13:54:07 +00:00
testWhiteList["Bug 6200: Preformatted in <blockquote>"] = "<blockquote data-mw-sourcePos=\"0:12\"><pre>\nBlah</pre></blockquote>";
Clean up 'END' token handling a bit.
2012-01-17 20:01:21 +00:00
Add a few more items to the whitelist
2011-12-07 11:44:38 +00:00
Add empty tables to the whitelist (legal in HTML5). Also add one more functionally identical italic/bold/link permmutation on the whitelist.
2011-12-06 22:05:43 +00:00
// empty table tags / with only a caption are legal in HTML5.
testWhiteList["A table with no data."] = "<table></table>";
testWhiteList["A table with nothing but a caption"] = "<table><caption> caption</caption></table>";
Prefix all internal data- attributes with data-mw- and adjust the whitelist and test output normalization accordingly. 235 tests passing.
2012-02-13 13:54:07 +00:00
testWhiteList["Fuzz testing: Parser22"] = "<p data-mw-sourcePos=\"0:23\"><a href=\"http://===r:::https://b\">http://===r:::https://b</a></p><table></table>";
Add empty tables to the whitelist (legal in HTML5). Also add one more functionally identical italic/bold/link permmutation on the whitelist.
2011-12-06 22:05:43 +00:00
Add a few more items to the whitelist
2011-12-07 11:44:38 +00:00
// MediaWiki changes the order of attributes in tables, ignore that
testWhiteList["Multiplication table"] = "<table border=\"1\" cellpadding=\"2\"><caption>Multiplication table</caption><tbody><tr><th> × </th><th> 1 </th><th> 2 </th><th> 3</th></tr><tr><th> 1</th><td> 1 </td><td> 2 </td><td> 3</td></tr><tr><th> 2</th><td> 2 </td><td> 4 </td><td> 6</td></tr><tr><th> 3</th><td> 3 </td><td> 6 </td><td> 9</td></tr><tr><th> 4</th><td> 4 </td><td> 8 </td><td> 12</td></tr><tr><th> 5</th><td> 5 </td><td> 10 </td><td> 15</td></tr></tbody></table>";
testWhiteList["Nested table"] = "<table border=\"1\"><tbody><tr><td> α</td><td><table bgcolor=\"#ABCDEF\" border=\"2\"><tbody><tr><td>nested</td></tr><tr><td>table</td></tr></tbody></table></td><td>the original table again</td></tr></tbody></table>";
Refactor table productions to support table fragments in templates (table start / row / table end). The old productions are not deleted yet to make it easy to compare the output on more complex articles. 181 tests passing after adding two table tests with whitespace-only differences to the whitelist.
2011-12-22 11:43:55 +00:00
// Very minor whitespace difference at end of cell (MediaWiki inserts a
// newline before the close tag even if there was no trailing space in the cell)
Prefix all internal data- attributes with data-mw- and adjust the whitelist and test output normalization accordingly. 235 tests passing.
2012-02-13 13:54:07 +00:00
testWhiteList["Table rowspan"] = "<table border=\"1\" data-mw-sourcePos=\"0:121\"><tbody><tr><td> Cell 1, row 1 </td><td rowspan=\"2\"> Cell 2, row 1 (and 2) </td><td> Cell 3, row 1 </td></tr><tr><td> Cell 1, row 2 </td><td> Cell 3, row 2 </td></tr></tbody></table>";
Refactor table productions to support table fragments in templates (table start / row / table end). The old productions are not deleted yet to make it easy to compare the output on more complex articles. 181 tests passing after adding two table tests with whitespace-only differences to the whitelist.
2011-12-22 11:43:55 +00:00
// Inter-element whitespace only
testWhiteList["Indented table markup mixed with indented pre content (proposed in bug 6200)"] = " \n\n<table><tbody><tr><td><pre>\nText that should be rendered preformatted\n</pre></td></tr></tbody></table>";
Add a few more items to the whitelist
2011-12-07 11:44:38 +00:00
/* Missing token transform functionality */
Reworked percent encoding handling for URIs to get closer to the 'url construction' part of the HTML5 spec: http://www.whatwg.org/specs/web-apps/current-work/multipage/urls.html#url-manipulation-and-creation Removed a few whitelisted test cases that are now passing directly. The encoding canonicalization could also be moved to the Sanitizer. Doing this early in token stream processing however has the advantage of providing further transformations uniform data to work with. We could even consider to move this even further into the tokenizer.
2012-03-06 13:49:37 +00:00
// Single quotes are legal in HTML5 URIs. See
// http://www.whatwg.org/specs/web-apps/current-work/multipage/urls.html#url-manipulation-and-creation
testWhiteList["Link containing double-single-quotes '' (bug 4598)"] = "<p><a href=\"/wiki/Lista_d''e_paise_d''o_munno\" data-mw-type=\"internal\">Lista d''e paise d''o munno</a></p>";
Add a few more items to the whitelist
2011-12-07 11:44:38 +00:00
// Sanitizer
testWhiteList["Invalid attributes in table cell (bug 1830)"] = "<table><tbody><tr><td Cell:=\"\">broken</td></tr></tbody></table>";
testWhiteList["Table security: embedded pipes (http://lists.wikimedia.org/mailman/htdig/wikitech-l/2006-April/022293.html)"] = "<table><tbody><tr><td> |<a href=\"ftp://|x||\">[1]</a>\" onmouseover=\"alert(document.cookie)\"&gt;test</td></tr></tbody></table>";
Reworked percent encoding handling for URIs to get closer to the 'url construction' part of the HTML5 spec: http://www.whatwg.org/specs/web-apps/current-work/multipage/urls.html#url-manipulation-and-creation Removed a few whitelisted test cases that are now passing directly. The encoding canonicalization could also be moved to the Sanitizer. Doing this early in token stream processing however has the advantage of providing further transformations uniform data to work with. We could even consider to move this even further into the tokenizer.
2012-03-06 13:49:37 +00:00
// Sanitizer, but UTF8 in link is ok in HTML5
Change wikilink tokenization strategy to split on pipes. This makes it possible to support template / template argument expansion in image options, and causes little trouble for wikilinks. Non-image wikilinks with multiple text pipes are quite rare in the dumps, and concatenating description tokens with a plain '|' is quite easy. 261 parser tests passing.
2012-03-05 12:00:38 +00:00
testWhiteList["External link containing double-single-quotes with no space separating the url from text in italics"] = "<p><a href=\"http://www.musee-picasso.fr/pages/page_id18528_u1l2.htm\" data-mw-type=\"external\" data-mw-rt=\"{&quot;sourcePos&quot;:[0,146]}\"><i>La muerte de Casagemas</i> (1901) en el sitio de </a><a href=\"/wiki/Museo_Picasso_(París)\" data-mw-type=\"internal\">Museo Picasso</a>.</p>";
Add a few more items to the whitelist
2011-12-07 11:44:38 +00:00
Change wikilink tokenization strategy to split on pipes. This makes it possible to support template / template argument expansion in image options, and causes little trouble for wikilinks. Non-image wikilinks with multiple text pipes are quite rare in the dumps, and concatenating description tokens with a plain '|' is quite easy. 261 parser tests passing.
2012-03-05 12:00:38 +00:00
testWhiteList["External links: wiki links within external link (Bug 3695)"] = "<p><a href=\"http://example.com\" data-mw-type=\"external\" data-mw-sourcePos=\"0:54\"></a><a data-mw-type=\"internal\" href=\"/wiki/Wikilink\">wikilink</a> embedded in ext link</p>";
Pluck a few low-hanging fruit in external link tokenization, and add a simple localurl parser function implementation. 230 parser tests now passing.
2012-02-07 10:28:23 +00:00
* Instead of sorting attributes, whitelist the one parserTest where it matters.
2012-02-20 22:26:24 +00:00
testWhiteList["<pre> with forbidden attribute values (bug 3202)"] = "<pre width=\"8\" style=\"\">Narrow screen goodies</pre>";
Add a few more items to the whitelist
2011-12-07 11:44:38 +00:00
Accept empty table cell attribute sections, and consider percent-encoded %2525 valid. 270 tests passing.
2012-03-06 14:32:45 +00:00
// This is valid, just confusing for humans. The reason for disallowing this
// might be history by now. XXX: Check this!
testWhiteList["Link containing % as a double hex sequence interpreted to hex sequence"] = "<p><a href=\"/wiki/7%2525_Solution\" data-mw-type=\"internal\">7%25 Solution</a></p>";
Fix invalid external link representation. 268 tests passing.
2012-03-05 18:06:29 +00:00
Add a parser test whitelist for manually-checked tests, and an option to print JSON-serialized parser output for failing tests, which can then be added to the whitelist if appropriate.
2011-12-01 10:58:12 +00:00
if (typeof module == "object") {
module.exports.testWhiteList = testWhiteList;
}
Reference in a new issue Copy permalink