Commit graph

53 commits

Author SHA1 Message Date
Subramanya Sastry 0eef7add67 Add Parsoid support for syntaxhighlight
* Added Parsoid config, and refactored code slightly to
  add native Parsoid handlers for parser tags exposed
  by this extension.
* Enabled parsoid mode testing on the test file.
* Added html/parsoid sections on a few tests.
* Marked rest of tests as wt2html and wt2wt only since
  html2wt and html2html will fail without a html/parsoid section
  and there is no real benefit to adding them to all tests.
* Added a couple tests to the known failures list:
  - One is because of T299103.
  - The other is because Parsoid always emits attributes in the
    form <tag .. foo="bar"..> instead of just <tag ... foo ..>
    Since Parsoid needs to accept this format that is present on
    wikis, I added a html/parsoid section for this test and
    added the failures to the known failures list.

Bug: T272939
Change-Id: Ie30aa6b082d4fc43c73296ff2ed6cb8c3873f48f
2022-08-08 20:07:46 -04:00
Bryan Davis 3bee59df01 fix: Mark Pygmentize::fetchVersion as public, but @internal
Follow up to ae07430. The method needs to be public so that
WANObjectCache can call it from a callback, but we don't expect any
external callers.

Follows-Up: I424926d071e1cfd454a0c2d45a83693f41bdea55
Change-Id: Ia96d3132782435c693d2eaa77fd551fe9590b113
2022-07-15 19:13:52 -06:00
Timo Tijhof ae074306e8 Pygmentize: Cache pygments-version in memc (in addition to APCU)
* Add rationale for each cache key's strategy being in Memc vs APCU.

* Extend pygmentize-lexers from 1 day to 1 week. It rarely changes
  and already varies by version. Few things survive the day, but
  there's not a reason to explicitly expire it sooner I think.

* Add a layer of Memc to the pygments-version APCU cache given that
  it has a short expiry and thus relatively high miss rate.

  The main rationale for this is noise in mwdebug logs since this
  is currently the only thing we log by default in Logstash with prod
  severity (exec INFO) during every pageview (after a php-fpm restart
  which clears APCU). By adding Memc here we lose less of the cache
  churn by reviving it via Memcached, and we keep the sense of there
  being nothing in the logs "by default" at prod severity after restart,
  e.g. don't get used to any fatigue.

  Unlike the other cache keys and hooks, getVersion is the only
  thing that gets called widely regardless of whether syntaxhighlight
  is in use on the given page.

Change-Id: I424926d071e1cfd454a0c2d45a83693f41bdea55
2022-07-12 05:56:16 +00:00
Tim Starling 956aa8ecd7 Use new ResourceLoader namespace
Extensions using Phan need to be updated simultaneously with core due
to T308443.

Bug: T308718
Depends-On: Id08a220e1d6085e2b33f3f6c9d0e3935a4204659
Change-Id: Ie1356c582baf9a66b868f7349cc71c26f8f1ead3
2022-05-27 03:42:55 +00:00
Reedy 39b4f0c7c1 Namespace rest of the extension
Global alias of SyntaxHighlight left behind for migration

Change-Id: I35b2caa42ac91454abe359949e360d1601748121
2022-03-18 01:42:11 +00:00
C. Scott Ananian e0cece9bc6 Passing a string to ParserOutput::addModules()/addModuleStyles() is deprecated
Bug: T296123
Change-Id: If14866f76703aa62d33e197bb18a5eacde7a55c0
2022-01-11 17:01:22 -05:00
Derk-Jan Hartman 34c936a8d3 Include generated styles before Mediawiki overrides
The order of style inclusion matters, some of our overrides were no
longer in effect.

Follow-up to: I2e82e5aa2a71604b87ffb4936204201d06678341
Bug: T292736
Change-Id: If202c26d2c29994cb3680eb76a86bb7efacc3ff9
2021-10-10 22:30:13 +02:00
Kunal Mehta b8a5dd08ee Expose Pygments version on Special:Version
Change-Id: Ia4eccc4f16873b16e106c8196d7582ca5b27b365
2021-09-10 11:47:28 -07:00
Kunal Mehta af6654e5f9 Port to BoxedCommand
All of the interactions with `pygmentize` have been refactored into a
new class, conviently called Pygmentize. It is responsible for getting

* pygments version (cached in APCu for 1 hour)
* generated CSS (cached in WAN by version for 1 week)
* lexer list (cached in APCu by version for 1 day)

and actually highlighting stuff! Most code paths differentiate whether
we're using a bundled version of pygments or one that has been
explicitly configured. If using the bundled one, we take shortcuts since
we already know the lexer list, have the CSS generated, etc.

ResourceLoaderPygmentsModule is added to switch between loading
generated CSS from the bundled file or Shellboxing out to get it from
pygments.

Bug: T289227
Change-Id: I2e82e5aa2a71604b87ffb4936204201d06678341
2021-09-10 11:47:28 -07:00
Kunal Mehta c8bd606cab Remove dead Shell::isDisabled() check
With "ability-shell" set in extension.json's requirements as of
commit b5a904e2ec this extension will refuse to load if shelling
out is disabled.

Change-Id: Ie8f446fbb33e585ffcc7d0adda1894a5497f2dad
2021-09-02 21:42:13 +00:00
libraryupgrader d6176eb862 build: Updating dependencies
composer:
* mediawiki/mediawiki-codesniffer: 36.0.0 → 37.0.0

npm:
* postcss: 7.0.35 → 7.0.36
  * https://npmjs.com/advisories/1693 (CVE-2021-23368)
* glob-parent: 5.1.0 → 5.1.2
  * https://npmjs.com/advisories/1751 (CVE-2020-28469)
* trim-newlines: 3.0.0 → 3.0.1
  * https://npmjs.com/advisories/1753 (CVE-2021-33623)

Change-Id: I63d2cc4c790d9b7d89187b72fed8ed11e89712a2
2021-07-24 02:37:26 +00:00
Alexander Vorwerk 47a18809ef Avoid using ContentHandler::getContentText()
ContentHandler::getContentText() is deprecated and should be
replaced with Content::getText() for TextContent instances.

Change-Id: I8767a925148c31b3a64761f1173a2a85bd28dfe0
2021-05-20 01:01:43 +00:00
C. Scott Ananian 0e26a6b3bf Replace use of Parser::$mStripState, deprecated in 1.35
The replacement, Parser::getStripState(), was added to MediaWiki in
1.34.  This extension already requires MediaWiki >= 1.34.

Bug: T275160
Change-Id: I7806068e1cd6e4da66adfe7bb75095d4bfb5d6bc
2021-02-19 17:08:41 -05:00
Timo Tijhof fa20f69cf4 SyntaxHighlightVisualEditorModule: Use Context::encodeJson() instead
Also fixes the Phan warning about Xml::encodeJsCall/FormatJson
needing booleaen where int inDebugMode() is passed.

Change-Id: Id8de16ab683948eae096b43462118ea837f53038
2021-02-18 05:07:32 +00:00
Kunal Mehta 9bdf728fc1 Move default CSS/JS model mapping out of extension.json
These are for MediaWiki core models, so it seems reasonable to specify
them in the PHP source.

Change-Id: Iab6f9969d2bf72122b2661e139aa21a3475a92a8
2021-02-10 23:54:21 -08:00
Ed Sanders 2d3af74c39 Add mw-content-ltr/rtl classes to inline snippets
We already add the dir=ltr/rtl HTML attribute so this
should be a no-op and makes it consistent with block
snippets.

Change-Id: I53e9204cc3bd54ba167f6f91e718a9d35b5bdfd0
2021-01-15 17:33:15 +00:00
Ed Sanders 16af4f3c6b Register VE module unconditionally
Change-Id: Ifccc0223c2b57c0de6f6c14355850213b090f8fb
2021-01-03 00:25:21 +00:00
Ed Sanders 583e3b3db8 Add support for line anchors on code pages
Bug: T29531
Change-Id: Ic09086c19d37bdff8bb7e68bbb0f676ef87896fe
2021-01-03 00:19:13 +00:00
Ed Sanders e8add72d66 Move all HTML wrapping into #highlight
This means all callers to #highlight get code wrapped
in the correct HTML.

This was done outside of #highlight before as the transformation
depended on $parser, so optionally pass in a $parser object if
the contents are going to go through the parser.

Change-Id: Ic5d5c341687e965804cb33da07dda23913718ff5
2021-01-01 18:27:29 +00:00
Ed Sanders 10ec5067c5 Enable line numbers on code content pages
Bug: T32773
Change-Id: I2eb8dcfe4d7bf751f998e1b2dd26a23cce69bf34
2020-12-30 21:35:05 +00:00
Ed Sanders 98c644a639 Improve appearance of line numbers
* Render a solid gutter that can take 3-4 digit line numbers
* Position line numbers absolutely in the gutter
* Add padding to code so that it doesn't wrap into the gutter

Change-Id: I7abb87452ad61808dad32b41c1d2d86b8ababb28
2020-12-30 21:35:04 +00:00
Ed Sanders fe357519c7 Add correct classes to full page output
Previously the full page output was missing the
-lang-<languagename> and mw-content-<dir> classes.

Change-Id: I54f4ed0a86e78a3a7ff1d670ebbdfdb6f05f86cc
2020-12-30 20:31:38 +00:00
Ed Sanders 6e558b37db Add small gap between line numbers and code
Change-Id: Ie65128872f318a42677e52420338684ad8d814f8
2020-12-30 19:43:03 +00:00
Freephile 24d418bced The word Python was misspelled
Change-Id: Icb38cd1cc07d2a2be95cdf3254502af0417afdf8
2020-10-07 17:30:05 +00:00
Reedy 0f876f17b1 Fix PSR12.Properties.ConstantVisibility.NotFound
Bug: T253169
Change-Id: Ibbf708a030504b1db61050cd2ef05280e9d52f2d
2020-09-19 12:11:07 +01:00
jenkins-bot 3bc07c4dfa Merge "Pass associative array to Sanitizer::validateAttributes" 2020-06-12 02:44:29 +00:00
C. Scott Ananian c281e5f423 Pass associative array to Sanitizer::validateAttributes
Bug: T255049
Change-Id: I68f122d5a3fa06b0434863cff73851a39dd10514
2020-06-10 14:01:21 -04:00
Reedy 16243d424f Also remove GeSHi class
I think two years is more than long enough for this one too

Change-Id: I6dd34c3fd60d6ade0206bc3d62759d87359c0b7a
2020-04-19 00:52:50 +01:00
Reedy 2d2119801a Kill SyntaxHighlight_GeSHi alias
I think 3 years is more than long enough

Change-Id: I03563e45be48f03622cbeda09a20b2f6ffe2b998
2020-04-19 00:48:44 +01:00
DannyS712 f5b61126da Add tracking categories when deprecated syntax is used
When the `enclose` attribute is used `syntaxhighlight-enclose-category`
is added, and when a <source> tag is used, the
`syntaxhighlight-source-category` is used.

Parser tests verify the tracking category is added when appropriate.

Bug: T241636
Bug: T237267
Change-Id: I7a21c635de426ab024703c04acdc6fa2184daedb
2020-04-09 19:18:32 +00:00
Umherirrender b59f43ce8e Improve param docs
Change-Id: Ifcb9ed2fbbdc38e11dfcc44951314baed81f2410
2020-02-29 22:17:22 +01:00
Derk-Jan Hartman c1e371b8e6 Syntaxhighlight: Classnames for the highlighted language
Bug: T189106
Change-Id: If7f678bd875dcfe581cd08a04b998970ed28d6d2
2020-02-18 20:13:53 +01:00
libraryupgrader 4f7ed16f87 build: Updating mediawiki/mediawiki-codesniffer to 29.0.0
The following sniffs are failing and were disabled:
* MediaWiki.Commenting.FunctionComment.MissingReturn

Change-Id: I6576c262bf717aa9b3b0577caa27c05cff0cb44b
2020-01-14 12:44:57 +00:00
libraryupgrader 0916348c84 build: Updating mediawiki/mediawiki-phan-config to 0.9.0
Change-Id: Id77528732abaa59ab1e2610c2acc87cb2858b8cd
2020-01-10 10:49:20 -08:00
jenkins-bot 4ba73a94a5 Merge "Remove 2 unused functions deprecated 5 years ago" 2019-11-18 10:34:45 +00:00
Max Semenik b591e2cdab Remove 2 unused functions deprecated 5 years ago
Change-Id: I8dc549f3c681616a8331c0666c60972db9ff41bf
2019-11-14 21:47:18 -08:00
Max Semenik 310de5114f Stop passing objects by reference
Bug: T193950
Change-Id: I968b53ebd6ab2780c952511469cb73856f354e1a
2019-11-14 21:41:37 -08:00
James D. Forrester 1087f347e6 Drop use of wgParser, replaced in 1.32 and to be removed in 1.35
Bug: T160811
Change-Id: I7c40256638c4bd17d8baf568685a794147cbeabc
2019-10-28 20:13:31 +00:00
Derick Alangi 2f0a6a0c9a Avoid usage of deprecated ObjectCache::getMainWANInstance()
Replacement with services made available in 1.28 and this extension
requires (1.33). So, the replacement is good.

Change-Id: I12bb8496ff0aa518acc6f5b4c023cb075db8e64b
2019-07-03 14:59:01 +01:00
libraryupgrader 207172de43 build: Updating mediawiki/mediawiki-phan-config to 0.5.0
Change-Id: If6f3dd1c6c4cec6880c196d29278feba6e3169d9
2019-03-10 22:30:03 -07:00
Timo Tijhof 10d0a248bf Override supportsURLLoading() instead of getScriptURLsForDebug()
This is more semantically correct and avoid duplicating the
url logic in an extension.

This was the only FileModule subclasses I found that overrode
getScriptURLsForDebug for this purpose.

Change-Id: Id940067437be190760559ec452583347c6b94901
2018-11-28 00:57:37 +00:00
Thalia 118dcdb405 Separate node classes into individual files
Bug: T205231
Change-Id: Id175f839f0ac8714c44407c52145d4140cd8e343
2018-09-24 18:48:22 +01:00
Kunal Mehta ab5ce23881 Remove tidy workaround for armoring tabs
RemexHtml does not mangle tabs, so we don't need to armor them! We'll
now output literal tab characters, so this does change the HTML output,
but does not have any visual change.

Depends-On: Idf246d05d116f63a73105b50a1929a7721fbe7b9
Change-Id: I2ba871acf9ff5fdc81c13e03eee3a8fe670e1f0f
2018-09-18 21:30:20 +00:00
C. Scott Ananian 14e3167406 Use MWTidy::isEnabled() instead of deprecated $wgUseTidy
The $wgUseTidy global has been soft-deprecated since MW 1.26, which is
when MWTidy::isEnabled() was added.  This extension currently requires
MediaWiki >= 1.32 according to extension.json.

Change-Id: Ie30c9174e6e3b60bce5a692296a9de1e30192e2c
2018-09-07 17:12:45 -04:00
libraryupgrader 74dbf4bba9 build: Updating mediawiki/mediawiki-codesniffer to 22.0.0
Change-Id: I1c75e007be52cd22d0a72d228e56a3a9683fbb58
2018-09-03 19:50:49 +00:00
Kunal Mehta 27b6687848 Use wfMakeStaticArrayFile()
wfMakeStaticArrayFile() expects an associative array, so let's take this
opportunity to turn the lexer list into one with true as the value. This
allows us to use isset() instead of the slower in_array() when checking
to see if a lexer is known.

Bug: T200626
Change-Id: I7a852ddbcfa7c8ed19ac933205cabd176b20d0cb
2018-07-28 21:29:57 -07:00
Gergő Tisza cc44c3c7a8 Add notice about resource loading to highlight()
Co-Authored-By: Timo Tijhof <krinklemail@gmail.com>
Change-Id: Ide7ab4b84d1aacad2db8400de9e350a09c80e8c0
2018-05-01 09:17:58 +00:00
Ed Sanders af1a91ade9 Use OOUI 'markup' icon instead of VE's deprecated 'alienextension'
Change-Id: Iad1882c20c17792c1339cf9922b9c4c43bf6122b
2018-03-20 19:00:41 +00:00
Timo Tijhof 8c3a3ffda0 Update highlight() to use getWithSetCallback() idiom
* Use backend-specific makeGlobalKey() method instead of
  deprecated wfGlobalCacheKey().

* Call makeCacheKey() inline and reduce to only producing
  part of the key (makeCacheKeyHash).

* Add a $ttl (1 month), instead of the previously unspecified
  expiry (default: indefinite).

Change-Id: I913a768a3b61cc31ce14fa086a563ceecca7c603
2018-02-22 23:53:56 +00:00
Timo Tijhof ae9b8419dc Factor out plainCodeWrap to avoid recursive call to self
The recursive call is somewhat counter-intuitive, and seems like it could
easily break if forgotten. Instead, move the part that we want to re-use
out into its own method.

Change-Id: I64ced0da211155299744d3c86f7818e751ad027e
2018-02-22 23:40:19 +00:00