Commit graph

36 commits

Author SHA1 Message Date
Siddharth VP c82db2de3e Add structure test to verify contents of pygments.generated.css
Verify that running Pygmentize::fetchGeneratedCss() (which gets invoked
from the updateCss.php maintenance script) gives CSS that matches the
generated file pygments.generated.css.

Change-Id: I454f287cb461b596178ee47c44634e2e127fe576
2024-09-28 21:59:47 +05:30
Fomafix 173e215bde Use overrideConfigValues instead of setMwGlobals
Also use MainConfigNames.

Change-Id: Ic51c376be4642eecbeb29d2ec2d2b2de388c2f61
2024-08-05 18:34:51 +00:00
Isabelle Hurbain-Palatin 3bc2ef11b5 Re-enable and fix tests for Parsoid CI
Bug: T214241
Depends-On: I022501851ea45cfd2554f9e914aa6f4fe2b07548
Change-Id: I34e32192047aaf268001b04bb936b3562686118b
2024-04-08 12:00:23 +00:00
Isabelle Hurbain-Palatin 2550f53d9c Disable a test for Parsoid CI
Bug: T214241
Needed-By: I022501851ea45cfd2554f9e914aa6f4fe2b07548
Change-Id: Ie3985837bc3fa679cf9b64cd6ab15b482402c18a
2024-04-03 03:01:10 +00:00
Subramanya Sastry 9c125346a8 Re-enable Parsoid testing
I think the syntax for parsoid-compatible may have changed at some
point and these tests probably never ran against Parsoid for a while now.

Change-Id: I9d7b23b180995eeadb4e3d183a0e970040ef6c78
2023-10-02 14:48:04 -05:00
Timo Tijhof 682fe922f9 Pygmentize: Treat Shellbox network loss like non-zero exit code
Prior to the shellbox migration, if during the parsing of a page,
pygmentize failed (i.e. non-zero exit from its local shell command,
pretty much the only way a php shell exec could fail), then
SyntaxHighlight would fallback to outputting a preformatted plain
`<pre>`.

The logic still exists in the code, and is still triggered for cases
where the command reached shellbox and its result was "successfully"
communicated to MediaWiki (HTTP 200), with the boxed result reporting
the non-zero exit code on the shellbox server.

However, the more likely scenario in the new setup is that the command
times out or never reaches the server in the first place, in which
case we don't get any shell exit code. Instead, we get a Shellbox
exception since the result is unknowable.

Instead of fatalling the entire pageview with a PHP exception and
HTTP 500 from MW, use the same graceful fallback.

Bug: T292663
Change-Id: Icaa8c34ff97ad8a99d044beab529ef943071269c
2023-06-03 14:25:50 +01:00
Bryan Davis 9f9c13bb7c Update Pygments to 2.15.0
Bump embedded Pygments version from 2.11.2 to 2.15.0. Changes
<https://github.com/pygments/pygments/compare/2.11.2...2.15.0>.

Pygments 2.15.0 includes support for MediaWiki's wikitext for the first time
ever (T29828).

New lexers: art, arturo, be, berry, carbon, comal, comal80, cplint,
css+ul4, dax, fc, fif, fift, func, gap-console, gap-repl, hcl, html+ul4,
jmespath, jp, js+ul4, jsonnet, k, linuxconfig, lobas, macaulay2, mcf,
mcfunction, mcschema, mediawiki, mips, oobas, phix, portugol,
postgres-explain, py+ul4, q, qlik, qlikscript, qliksense, qlikview,
snbt, sobas, sql+jinja, tal, tlb, ul4, unixconfig, uxntal, wsgl,
wikitext, wowtoc, wren, x++, xml+ul4, xpp

Bug: T334433
Change-Id: I80576c0782b8b6d4b5bfd0596f72d81c4f9346a8
2023-04-10 21:33:05 +00:00
C. Scott Ananian 2f4f3c962f parserTests.txt: Add nohtml option to accomodate test runner change
This makes the parserTests compatible with change
I61a67a0e6928463e3872be9a42ff6992c6754662 in core.

Change-Id: I5c22f456a3ae5ea25b59c4246d68965099c465cc
2023-02-09 23:36:00 -05:00
Sébastien Beyou b08c0a7cb9 Fix the case of empty <syntaxhighlight /> tags
Bug: T315740
Change-Id: I685806d4e8992a54f17d29a9187807bb30e31ef8
2022-08-21 13:46:17 +02:00
Subramanya Sastry 0eef7add67 Add Parsoid support for syntaxhighlight
* Added Parsoid config, and refactored code slightly to
  add native Parsoid handlers for parser tags exposed
  by this extension.
* Enabled parsoid mode testing on the test file.
* Added html/parsoid sections on a few tests.
* Marked rest of tests as wt2html and wt2wt only since
  html2wt and html2html will fail without a html/parsoid section
  and there is no real benefit to adding them to all tests.
* Added a couple tests to the known failures list:
  - One is because of T299103.
  - The other is because Parsoid always emits attributes in the
    form <tag .. foo="bar"..> instead of just <tag ... foo ..>
    Since Parsoid needs to accept this format that is present on
    wikis, I added a html/parsoid section for this test and
    added the failures to the known failures list.

Bug: T272939
Change-Id: Ie30aa6b082d4fc43c73296ff2ed6cb8c3873f48f
2022-08-08 20:07:46 -04:00
Subramanya Sastry 360ec88ec1 Get rid of +tidy, +untidy references from parser tests
Bug: T198214
Change-Id: I4b3346b167d13a1e4361ba78a44f6922e1828b55
2021-09-30 13:45:21 -05:00
Ed Sanders 2d3af74c39 Add mw-content-ltr/rtl classes to inline snippets
We already add the dir=ltr/rtl HTML attribute so this
should be a no-op and makes it consistent with block
snippets.

Change-Id: I53e9204cc3bd54ba167f6f91e718a9d35b5bdfd0
2021-01-15 17:33:15 +00:00
Ed Sanders e8add72d66 Move all HTML wrapping into #highlight
This means all callers to #highlight get code wrapped
in the correct HTML.

This was done outside of #highlight before as the transformation
depended on $parser, so optionally pass in a $parser object if
the contents are going to go through the parser.

Change-Id: Ic5d5c341687e965804cb33da07dda23913718ff5
2021-01-01 18:27:29 +00:00
DannyS712 f5b61126da Add tracking categories when deprecated syntax is used
When the `enclose` attribute is used `syntaxhighlight-enclose-category`
is added, and when a <source> tag is used, the
`syntaxhighlight-source-category` is used.

Parser tests verify the tracking category is added when appropriate.

Bug: T241636
Bug: T237267
Change-Id: I7a21c635de426ab024703c04acdc6fa2184daedb
2020-04-09 19:18:32 +00:00
C. Scott Ananian 48b20d9b67 Update parserTests to v2 (tidy by default)
Tidy does a slightly more-aggressive escaping of &gt;.

Bug: T174199
Bug: T249194
Change-Id: I7d80557ae3b43b8da03cef56430e11a0f5dc5793
2020-04-02 13:44:54 -04:00
Derk-Jan Hartman c1e371b8e6 Syntaxhighlight: Classnames for the highlighted language
Bug: T189106
Change-Id: If7f678bd875dcfe581cd08a04b998970ed28d6d2
2020-02-18 20:13:53 +01:00
Arlo Breault e6acaf8f5f Update tests to match parser changes
Bug: T208070
Bug: T218702
Change-Id: I8b6a8c8241a8766fdccdcac45553298fa23a5046
2019-03-19 13:38:21 -04:00
Daimona Eaytoy 66c56f5c86 Migrate to new Phan
Bug: T216944
Change-Id: I5278ea60a17283dca6b0aee843ff60bf5c4103e2
2019-03-02 13:43:16 +01:00
Kunal Mehta ab5ce23881 Remove tidy workaround for armoring tabs
RemexHtml does not mangle tabs, so we don't need to armor them! We'll
now output literal tab characters, so this does change the HTML output,
but does not have any visual change.

Depends-On: Idf246d05d116f63a73105b50a1929a7721fbe7b9
Change-Id: I2ba871acf9ff5fdc81c13e03eee3a8fe670e1f0f
2018-09-18 21:30:20 +00:00
Kunal Mehta 61a6b2c8c2 Add parser tests for tabs and tidy
These tests cover the fixes for T32930 and T59826.

Change-Id: I944b55c7668ac3dd11ba872ba8077ba188ed55e2
2018-09-07 18:19:13 -07:00
Brad Jorsch 41466e274b Update parser test for core change
I42e747f1 stopped changing a regular space before percent-escapes (like
"%25") to a non-breaking space in certain cases, causing tests to fail
where they were expecting that non-breaking space. Update the test to
match this new behavior.

Bug: T199647
Change-Id: If5d90e5701f240b1bc71a3c3458e952e0e2c7df4
2018-07-16 09:23:25 -04:00
Kunal Mehta 1755781a3c Don't manually suppress PhanDeprecatedClass
Already disabled by default in mediawiki-phan-config 0.3.0.

Change-Id: I2ad9dbefd6c167429100070b834fcbdbb216008f
2018-06-08 13:44:56 -07:00
Jayprakash12345 a79bf7638c Update extensions to take advantage of parser test autodiscovery
Bug: T170037
Change-Id: I99e68197bbe0438c7a5a55815e8000f4deb49234
2018-04-13 15:25:52 +00:00
Kunal Mehta 0615fe8444 build: Updating mediawiki/mediawiki-phan-config to 0.2.0
Change-Id: I1e5dc6d8fd131db8dd3296337ee73ae07b42f931
2018-04-05 17:23:15 -07:00
James D. Forrester df6eec698e Parser tests: Update for breaking whitespace change in MediaWiki core
Changed as part of T157418.

Change-Id: I2b71eed948fe211389fb3ac166f04cd39d118328
2018-03-20 11:43:33 -07:00
Kunal Mehta 14d34dfe56 Update parser tests for RemexHtml as tidy implementation
Bug: T188167
Depends-On: Ic7c34d57a300dbd36a37f03fbfe33391b2950b44
Change-Id: I807a99dc9c873f3e6eb67fd62574f7916530d62e
2018-02-23 22:31:41 -08:00
Kunal Mehta cda15b184c If there is no input, don't try and shell out to pygments
pygments will just wait for stdin if the stdin that was passed
to it was empty.

Bug: T188019
Change-Id: I46271f31df74822bc71b3ac283fa572f2d149d67
2018-02-22 13:29:58 -08:00
Kunal Mehta e75fda881e Migrate to mediawiki-phan-config
Change-Id: I038c599b428683cee669e69e9a5106f178b129a2
2018-02-15 14:29:11 -08:00
Rafid Aslam 46700b5fdb Add phan configuration for static analysis and fix phan warnings
Add phan configuration for static analysis, and fix phan warnings.
`PhanDeprecatedClass` and `PhanDeprecatedFunction` rules are supressed.

Bug: T179554
Change-Id: I7cbb410ed88ba58198d0557cafd9e6df968ed885
2017-12-25 11:16:49 +07:00
Kunal Mehta db681d158b Update pygments to 2.2
Notable changes:

* Added new token types and lexing for magic methods and variables in
  Python and PHP.
* Added a new token type for heredoc (and similar) string delimiters and
  lexing for them in C++, Perl, PHP, Postgresql and Ruby lexers.
* Improved the CSS lexer.
* Added Varnish config lexer

Full changelog is found at:
<bc45808f83/CHANGES>

Bug: T156573
Change-Id: I4dc1782f19881ba1294308e1cdea1b2e063f438a
2017-04-27 15:10:16 +02:00
Brian Wolff 8714354068 SECURITY: Replace strip markers before syntax highlighting
The raw strip markers can't be syntax highlighted properly anyhow.

Bug: T51256, T110143, T78456
Change-Id: I822fa2bbfde790df22c00bb87819c9027fb5dcc9
2016-05-20 10:21:06 -07:00
Brian Wolff f85be74e33 Fix parser tests for core commit ee4d5c6eed (removal $wgWellFormedXML)
Change-Id: Iac351ced0154c0c3c61f47a0e8ef0b9166fc0811
2016-05-16 06:02:39 -04:00
Timo Tijhof 206f0fd585 Fix "MWException: Unexpected output from Pygments" in highlight fallback
Follows-up 043969f84e.

This caused a fatal MWException when saving/reading pages that contain a
<source> that couldn't be highlighted (e.g. no lang attribute, unknown
lang, or too large).

Specifically when wgWellFormedXml=false, in which case $out isn't
just from Pygments, but actually from our own Html::element.

Change-Id: Ib299a274d28021b2c7bba52d763dd1e17c1f09ec
2015-07-07 22:12:53 +01:00
Bartosz Dziewoński 043969f84e Refactor final output formatting
* Use 'nowiki' strip marker to prevent list processing (also known as
  doBlockLevels()). This resolves various issues related to using
  <syntaxhighlight/> blocks in lists and lists suddenly appearing
  inside <syntaxhighlight/> blocks. Fixes T17333, T25674, T104067.

* To prevent <p/>-wrapping resulting from the above, add our own
  wrapper <div/> around the output.

* Since we already have our own wrapper, remove Pygments' one and
  extend it with custom attributes. This resolves some regressions
  from the GeSHi migration. Fixes most of T103964.

Bug: T17333
Bug: T25674
Bug: T103964
Bug: T104067
Change-Id: I3afd1224a18549c62cd4a95fd046affa6d1d3b3f
2015-07-05 22:03:24 +00:00
Timo Tijhof e033d2db9e Add parser tests for block and inline behaviour
Change-Id: I0af26e8b86f2f9060bd88db6d36449d983978f09
2015-06-25 05:46:42 +01:00
Kunal Mehta fb0e430f47 Add basic parser tests for functionality of non-recognized languages
Change-Id: I07cecb588818fd4abb0b1b8e8de893cf6f72014a
2015-06-22 22:26:22 -07:00