mirror of
https://gerrit.wikimedia.org/r/mediawiki/extensions/VisualEditor
synced 2024-11-16 02:51:50 +00:00
8368e17d6a
* All parser pipelines including tokenizer and DOM stuff are now constructed
from a 'recipe' data structure in a ParserPipelineFactory.
* All sub-pipelines of these can now be cached
* Event registrations to a pipeline are directly forwarded to the last
pipeline member to save relatively expensive event forwarding.
* Some APIs for on-demand expansion / format conversion of parameters from
parser functions are added:
param.to('tokens/expanded', cb)
param.to('text/wiki', cb) (this does not work yet)
All parameters are additionally wrapped into a Param object that provides
method for positional parameter naming (.named() or conversion to a dict
(.dict()).
* The async token transform manager is now separated from a frame object, with
the frame holding arguments, an on-demand expansion method and loop checks.
* Only keys of template parameters are now expanded. Parser functions or
template arguments trigger an expansion on-demand. This (unsurprisingly)
makes a big performance difference with typical switch-heavy template
systems.
* Return values from async transforms are no longer used in favor of plain
callbacks. This saves the complication of having to maintain two code paths.
A trick in transformTokens still avoids the construction of unneeded
TokenAccumulators.
* The results of template expansions are no longer buffered.
* 301 parser tests are passing
Known issues:
* Cosmetic cleanup remains to do
* Some parser functions do not support async expansions yet, and need to be
modified.
Change-Id: I1a7690baffbe8141cadf67270904a1b2e1df879a
|
||
|---|---|---|
| .. | ||
| dumpGrepPatterns | ||
| __patched-html5-parser.coffee | ||
| dumpGrepper.js | ||
| dumpReader.js | ||
| expansionTest.js | ||
| fetch-parserTests.txt.js | ||
| parserTests-whitelist.js | ||
| parserTests.js | ||
| parserTests.pegjs | ||
| README | ||
| roundtrip-test.js | ||
| roundtrip.js | ||
| tests.html | ||
| wikia-rte-roundtrip-tests.xml | ||
| worker.js | ||
Work in progress. :) Please see https://www.mediawiki.org/wiki/Future/Parser_development for some information on how to get started with these tests and the current parser architecture. Install dependencies and run tests: cd tests/parser make test == Running parserTests.js == For parserTests, you also need MediaWiki's parser test cases (parserTests.txt) either in a phase3 checkout parallel to extensions (tried by default). You can also specify a test case file as an argument. The "make test" target in this directory downloads parserTests.txt and specifies it as the test case automatically. Of course, you may also run tests manually: node ./parserTests.js Several options are available for parserTests: node ./parserTests.js --help Enjoy! == Running roundtrip tests == Round-trip tests are not yet ported to the current parser architecture. The focus so far has been on parserTests and getting the parser architecture in shape, but we'll get back to round-trip tests eventually.