Prior to the shellbox migration, if during the parsing of a page,
pygmentize failed (i.e. non-zero exit from its local shell command,
pretty much the only way a php shell exec could fail), then
SyntaxHighlight would fallback to outputting a preformatted plain
`<pre>`.
The logic still exists in the code, and is still triggered for cases
where the command reached shellbox and its result was "successfully"
communicated to MediaWiki (HTTP 200), with the boxed result reporting
the non-zero exit code on the shellbox server.
However, the more likely scenario in the new setup is that the command
times out or never reaches the server in the first place, in which
case we don't get any shell exit code. Instead, we get a Shellbox
exception since the result is unknowable.
Instead of fatalling the entire pageview with a PHP exception and
HTTP 500 from MW, use the same graceful fallback.
Bug: T292663
Change-Id: Icaa8c34ff97ad8a99d044beab529ef943071269c
This makes the parserTests compatible with change
I61a67a0e6928463e3872be9a42ff6992c6754662 in core.
Change-Id: I5c22f456a3ae5ea25b59c4246d68965099c465cc
Use the '--json' flag to get Pygments to output its list of supported
lexers in a machine-readable format. Support for this flag was added (at
our request) to Pygments and included in the 2.11 release[1].
Tested by running updateLexerList.php and confirming empty diff.
[1]: https://github.com/pygments/pygments/issues/1437
Change-Id: I0f1d7fceca9034e6034bafa6a8dd312b99d379d1
When using a non-bundled Pygments (which is required on Windows, as the
bundled version is an ELF binary), we call into the Pygments executable
to generate the list of supported languages (lexers). This list seems to
occasionally include carraige returns, causing some languages to not be
processed correctly. Trim those CRs out so the language list is
accurate.
Change-Id: If8b1f145dd10e2c4707d6d32927e85d1d2459f15
Replacing the HIGHLIGHT_MAX_LINES and HIGHLIGHT_MAX_BYTES constants with $wgSyntaxHighlightMaxLines and $wgSyntaxHighlightMaxBytes respectively, so sysadmins can adjust the limits to their needs if performance is not of their concern.
Bug: T322293
Bug: T104109
Change-Id: I80768d3cb45ac01c004fc812832878c83ca4ecdb
Python on Windows requires the SystemRoot environment variable in order
to initialize its internal RNG, so make sure that is passed along to the
subprocess.
Bug: T300223
Change-Id: I170ce627a3f00c023f4b1f11613f4fe2cb17bd31
Steps to reproduce:
* Make a wikitext page with <syntaxhighlight line="sure">…</…>.
* Edit with VisualEditor and make an unrelated change to the content.
* Save.
* Before the attribute was always replaced with line="1".
* With this patch the old value stays untouched.
Change-Id: I433a206905c1a0a1bc06ef8b4037f6db9d935810
Skip the expensive check,
for example when no highlighting is wanted because there is no lexer
Also all validation of the tag is now processed and
invalid tags also not counted.
Bug: T316858
Change-Id: Ifad9a9a14fae92463c345fb12defb41f14c2e1f3
The shell out to get styled text is expensive.
Call Parser::incrementExpensiveFunctionCount to limit the highlighted
text snippet on a page and not reaching a timeout.
This would count each tag and not deduplicate the text snippet to count
only once or if pygmentize needs to call or is in the cache.
This also not affect Parsoid, not sure if the concept of expensive
parser function exists there
Bug: T316858
Change-Id: I8afe61e9be4a34e5f0725a9b65ef43c345e1be5f
