wikimedia/mediawiki-extensions-VisualEditor
1
0
Fork 0
mirror of https://gerrit.wikimedia.org/r/mediawiki/extensions/VisualEditor synced 2024-11-25 14:56:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
mediawiki-extensions-Visual.../modules/parser
History
Gabriel Wicke 06ae53fdfe Drastically reduce memory usage for template-heavy pages 
Only call back a few callbacks per reactor iteration from the template fetch
request queue. This changes the expansion pattern from a (memory intensive)
breadth-first expansion to something quite close to depth-first expansion.
Additionally, retrieved pages are quickly added to the page cache so that a
lot of request queuing is avoided in favor of synchronous expansion from the
cache. On pages like Barack Obama that previously ran out of memory after
consuming node's 1.6G heap limit, expansion now runs in relatively constant
100-300M resident (so far, still running).

Change-Id: Ie34a1eeff00d868416de45ef8d289898258f560c
2012-04-13 14:31:03 +02:00
..
html5 Land big TokenTransformDispatcher and eventization refactoring. 2012-01-03 18:44:31 +00:00
ext.Cite.js Token representation clean-up. Now all tokens are differentiated using 2012-03-07 20:06:54 +00:00
ext.cite.taghook.ref.js Moving parser stuff back into the modules folder (oops) 2011-11-02 21:45:57 +00:00
ext.core.AttributeExpander.js Token stream transform improvements 2012-04-12 15:42:09 +02:00
ext.core.BehaviorSwitchHandler.js "magic words" are tokenized and used to set parser.environment flags 2012-04-04 11:25:29 -07:00
ext.core.LinkHandler.js Improve template tokenization in template arguments 2012-04-10 16:40:49 +02:00
ext.core.NoIncludeOnly.js Token stream transform improvements 2012-04-12 15:42:09 +02:00
ext.core.ParserFunctions.js Token stream transform improvements 2012-04-12 15:42:09 +02:00
ext.core.PostExpandParagraphHandler.js Don't eat newline / space in template parameters 2012-04-04 11:22:31 +02:00
ext.core.QuoteTransformer.js Replace console.log with console.warn in all debug statements 2012-02-14 20:56:14 +00:00
ext.core.Sanitizer.js Shorten data-mw-rt to data-mw and clean up whitelist 2012-04-02 18:12:49 +02:00
ext.core.TemplateHandler.js Drastically reduce memory usage for template-heavy pages 2012-04-13 14:31:03 +02:00
ext.Util.js Nominate more HTML5 sectioning and heading elements for block-level treatment 2012-04-11 12:53:49 +02:00
ext.util.TokenCollector.js Token stream transform improvements 2012-04-12 15:42:09 +02:00
mediawiki.DOMConverter.js Replace console.log with console.warn in all debug statements 2012-02-14 20:56:14 +00:00
mediawiki.DOMPostProcessor.js Replace console.log with console.warn in all debug statements 2012-02-14 20:56:14 +00:00
mediawiki.HTML5TreeBuilder.node.js Shorten data-mw-rt to data-mw and clean up whitelist 2012-04-02 18:12:49 +02:00
mediawiki.LinearModelConverter.js Add HTML DOM -> linear model converter 2012-03-29 12:47:14 -07:00
mediawiki.parser.defines.js Move link types to data-mw-rt, and support some more template tokenization 2012-03-13 12:32:31 +00:00
mediawiki.parser.environment.js Token stream transform improvements 2012-04-12 15:42:09 +02:00
mediawiki.parser.js Token stream transform improvements 2012-04-12 15:42:09 +02:00
mediawiki.Title.js Add basic thumb rendering support 2012-04-09 23:04:26 +02:00
mediawiki.tokenizer.peg.js Convert external link syntax stops to stack 2012-04-13 11:08:42 +02:00
mediawiki.TokenTransformManager.js Token stream transform improvements 2012-04-12 15:42:09 +02:00
package.json As much as I have loved writing Makefiles... I've replaced its functionality with package.json, mostly so we can avoid non-node dependencies. This is one of the recommended practices. We should consider moving tests/parser into modules/parser/tests, other node projects keep all module code in one directory. 2012-04-04 11:02:58 -07:00
parse.js Fixes to template expansion / token transform managers, 296 tests passing. 2012-04-11 16:34:27 +02:00
pegTokenizer.pegjs.txt Convert external link syntax stops to stack 2012-04-13 11:08:42 +02:00
README.txt As much as I have loved writing Makefiles... I've replaced its functionality with package.json, mostly so we can avoid non-node dependencies. This is one of the recommended practices. We should consider moving tests/parser into modules/parser/tests, other node projects keep all module code in one directory. 2012-04-04 11:02:58 -07:00

README.txt 

A combined Mediawiki and html parser in JavaScript running on node.js. Please
see (https://www.mediawiki.org/wiki/Future/Parser_development) for an overview
of the current implementation, and instructions on running the tests.

You might need to set the NODE_PATH environment variable,
  export NODE_PATH="node_modules"

Download the dependencies:
  npm install

Run tests:
  npm test