Changes to the use statements done automatically via script
Addition of missing use statement done manually
Change-Id: Iede38f051529229fce7660dce7d0dba64724ee8a
Update the CSS generation script to generate styles for both light and
dark modes. The Pygments style 'default' is used for light mode, as
before, and 'monokai' for dark mode.
Bug: T365926
Change-Id: I0cc1f9a10d4ff8e0c279a12cacbb1c300225328f
In Scribunto modules, link page names used in require(), mw.loadData()
and mw.loadJsonData() invocations.
Inspired from the CodeLinks gadget[0] but rewritten for brevity,
avoiding Wiktionary specific code, making the code more generic,
and thereby adding support for mw.loadJsonData().
[0]: https://en.wiktionary.org/wiki/MediaWiki:Gadget-CodeLinks.js
Bug: T368166
Change-Id: Idc554269ee52a05660fa41f065a2b3c73e2e1b9b
<syntaxhighlight> blocks with a boolean "copy" param will now have a
button next to them for copying the code to the clipboard. Not
applicable for inline code blocks.
Adapted from the mediawiki.org gadget written by Krinkle.
Bug: T40932
Change-Id: Ic8ef030514c3b6dd2cb9b137f032588869ab3762
This patch migrates the `syntaxhighlight_shell` metric to the new Prometheus
format while copying the equivalent graphite counterpart to Graphite for
backwards compatibility with the existing Grafana dashboards.
Bug: T359270
Change-Id: I95bc2e749324092aae5db9135847c28aa16c64ea
It's common practise to use [[wikilink]] syntax to mention page names
in comments of JS/CSS/Scribunto code where the links ordinarily don't
work. Using JavaScript to actually make the syntax clickable makes
navigation easier.
{{Templates links}} and external links are also supported.
Bug: T368166
Change-Id: I999937c1f6303ecc64adb6285e73a9ce10f67bd8
Most python error messages are reported to stderr rather than stdout. In
the event of a catastrophic failure executing the Pygments binary we are
likely to need to report stderr so that folks can debug the problem with
the executable.
Bug: T364249
Change-Id: Id5e5dbc515fdcdeb6eec61aacbbb9cbeddc79fab
Old HTML IDs had to start with Latin letters, but
in HTML5 IDs can use any characters.
Bug: T359214
Change-Id: I6b6733eb07267faca1990bb7445a967405f9327e
The PHPDoc/JSDoc updates are mostly about generic "array" types that
can be made more specific.
In PHP we can remove documentation when it is 100% identical to the
type declarations in the code.
A few mistakes are fixed as well, e.g. a missing "null".
This patch also made a major mistake visible. It looks like the
$geshi2pygments compatibility map was broken since 2018. The array was
changed from values to keys via I7a852dd and some usages updated, but
one was forgotten.
Change-Id: I480999d21f2f69cba84166bb877aa75882778966
This does the same as before. Only newlines are trimmed from the
left, but all whitespace from the right.
Change-Id: I6b7c860d8a2fc2a1f28428447ee8f18ab4bbe46c
The content of the SyntaxHighlight extension is not wikitext and
annotations should be stripped from it before rendering.
Bug: T341009
Depends-On: I4e9a7a8bec3cb9532ef8a729fd2c6c4acca5d8a0
Change-Id: Ibada54d517830b1112b59513b090dc4bbdc7c917
The use of "HookHandlers" attribute in extension.json makes it possible
to inject services into hook handler classes in a future patch.
Bug: T271029
Change-Id: I6df44cf4a160e618a6546fb9eec36070bf4b868e
Prior to the shellbox migration, if during the parsing of a page,
pygmentize failed (i.e. non-zero exit from its local shell command,
pretty much the only way a php shell exec could fail), then
SyntaxHighlight would fallback to outputting a preformatted plain
`<pre>`.
The logic still exists in the code, and is still triggered for cases
where the command reached shellbox and its result was "successfully"
communicated to MediaWiki (HTTP 200), with the boxed result reporting
the non-zero exit code on the shellbox server.
However, the more likely scenario in the new setup is that the command
times out or never reaches the server in the first place, in which
case we don't get any shell exit code. Instead, we get a Shellbox
exception since the result is unknowable.
Instead of fatalling the entire pageview with a PHP exception and
HTTP 500 from MW, use the same graceful fallback.
Bug: T292663
Change-Id: Icaa8c34ff97ad8a99d044beab529ef943071269c
Use the '--json' flag to get Pygments to output its list of supported
lexers in a machine-readable format. Support for this flag was added (at
our request) to Pygments and included in the 2.11 release[1].
Tested by running updateLexerList.php and confirming empty diff.
[1]: https://github.com/pygments/pygments/issues/1437
Change-Id: I0f1d7fceca9034e6034bafa6a8dd312b99d379d1
When using a non-bundled Pygments (which is required on Windows, as the
bundled version is an ELF binary), we call into the Pygments executable
to generate the list of supported languages (lexers). This list seems to
occasionally include carraige returns, causing some languages to not be
processed correctly. Trim those CRs out so the language list is
accurate.
Change-Id: If8b1f145dd10e2c4707d6d32927e85d1d2459f15
Replacing the HIGHLIGHT_MAX_LINES and HIGHLIGHT_MAX_BYTES constants with $wgSyntaxHighlightMaxLines and $wgSyntaxHighlightMaxBytes respectively, so sysadmins can adjust the limits to their needs if performance is not of their concern.
Bug: T322293
Bug: T104109
Change-Id: I80768d3cb45ac01c004fc812832878c83ca4ecdb
Python on Windows requires the SystemRoot environment variable in order
to initialize its internal RNG, so make sure that is passed along to the
subprocess.
Bug: T300223
Change-Id: I170ce627a3f00c023f4b1f11613f4fe2cb17bd31
Skip the expensive check,
for example when no highlighting is wanted because there is no lexer
Also all validation of the tag is now processed and
invalid tags also not counted.
Bug: T316858
Change-Id: Ifad9a9a14fae92463c345fb12defb41f14c2e1f3
The shell out to get styled text is expensive.
Call Parser::incrementExpensiveFunctionCount to limit the highlighted
text snippet on a page and not reaching a timeout.
This would count each tag and not deduplicate the text snippet to count
only once or if pygmentize needs to call or is in the cache.
This also not affect Parsoid, not sure if the concept of expensive
parser function exists there
Bug: T316858
Change-Id: I8afe61e9be4a34e5f0725a9b65ef43c345e1be5f
* Added Parsoid config, and refactored code slightly to
add native Parsoid handlers for parser tags exposed
by this extension.
* Enabled parsoid mode testing on the test file.
* Added html/parsoid sections on a few tests.
* Marked rest of tests as wt2html and wt2wt only since
html2wt and html2html will fail without a html/parsoid section
and there is no real benefit to adding them to all tests.
* Added a couple tests to the known failures list:
- One is because of T299103.
- The other is because Parsoid always emits attributes in the
form <tag .. foo="bar"..> instead of just <tag ... foo ..>
Since Parsoid needs to accept this format that is present on
wikis, I added a html/parsoid section for this test and
added the failures to the known failures list.
Bug: T272939
Change-Id: Ie30aa6b082d4fc43c73296ff2ed6cb8c3873f48f
Follow up to ae07430. The method needs to be public so that
WANObjectCache can call it from a callback, but we don't expect any
external callers.
Follows-Up: I424926d071e1cfd454a0c2d45a83693f41bdea55
Change-Id: Ia96d3132782435c693d2eaa77fd551fe9590b113
* Add rationale for each cache key's strategy being in Memc vs APCU.
* Extend pygmentize-lexers from 1 day to 1 week. It rarely changes
and already varies by version. Few things survive the day, but
there's not a reason to explicitly expire it sooner I think.
* Add a layer of Memc to the pygments-version APCU cache given that
it has a short expiry and thus relatively high miss rate.
The main rationale for this is noise in mwdebug logs since this
is currently the only thing we log by default in Logstash with prod
severity (exec INFO) during every pageview (after a php-fpm restart
which clears APCU). By adding Memc here we lose less of the cache
churn by reviving it via Memcached, and we keep the sense of there
being nothing in the logs "by default" at prod severity after restart,
e.g. don't get used to any fatigue.
Unlike the other cache keys and hooks, getVersion is the only
thing that gets called widely regardless of whether syntaxhighlight
is in use on the given page.
Change-Id: I424926d071e1cfd454a0c2d45a83693f41bdea55
Extensions using Phan need to be updated simultaneously with core due
to T308443.
Bug: T308718
Depends-On: Id08a220e1d6085e2b33f3f6c9d0e3935a4204659
Change-Id: Ie1356c582baf9a66b868f7349cc71c26f8f1ead3
