Commit graph

68 commits

Author SHA1 Message Date
Subramanya Sastry f733af93c4 Code cleanup.
* Moved tokensToString and KVtoHash from parser.environment
  to Util -- got rid of excessive warnings from tokensToString.
* Replaced used of Util.lookupKV(blah).v with Util.lookup(blah)

Change-Id: Ic48d5d0c79045b9789dc61a2b14df3fc2acf7aa5
2012-08-24 15:29:53 -05:00
Subramanya Sastry 2d15258803 Fixed template meta-wrapping to handle nesting and attributes.
* Nested parser functions are no longer meta-wrapped similar to
  nested templates.
* HTML attributes which derive all or part of their key/value
  from a template are not template-wrapped, but are marked to
  be tracked via different rdfa meta attributes.  The actual
  tracking and addition of new meta attrs. is yet to be
  implemented.
* Parser pipeline construction updated to use an options object
  rather than individual option args -- used to support the two
  features above.
* With this fix, we can now distinguish between the following
  scenarios:

  Templates:
  - {{echo|[[Foo|bar]]}} -- regular wrapping of the entire link
  - [[{{echo|Foo}}|bar]] -- no wrapping, but href marked for tracking
  - [[Foo|{{echo|bar}}]] -- link text is wrapped.

  Parser functions:
  - {{uc:foo}} -- regular wrapping.
  - {{uc:{{lc:{{uc:foo}}}}}} -- only the top-level parser function
    is wrapped, the nested lc and uc parser functions are not.

Change-Id: I512b8de9838ad42d9255ebcd73074b3898294cd8
2012-08-19 15:21:36 -05:00
Gabriel Wicke 4900d85606 Reorganize Parsoid repo after split from VisualEditor
* /js/ for JavaScript / nodjs implementation
* /js/lib for js modules
* /js/tests for js test runners and unit tests

Change-Id: I7da1ea2a62b9312be0f1b492447340427f6a14fb
2012-07-30 13:10:22 -07:00
Subramanya Sastry 4ae497db1e Renamed ext.Util.js to mediawiki.Util.js
Change-Id: I909847686c8239a0b00cbaa9a0b1583826ee1487
2012-07-24 13:07:53 -05:00
Subramanya Sastry 194c7afb34 Added utility methods to ext.Util.js
* Copied over utility methods from mediawiki.parser.environment.js
  to ext.Util.js.
* Moved over utility method from mediawiki.parser.defines.js to
  ext.Util.js.
* Converted Util to be a singleton object rather than an allocatable
  class.  There is no reason to allocate a new utility class everywhere
  since this utility object has no useful state.
* Fixed up use of utility methods to use Util rather than env.

Change-Id: Ib81f96b894f6528f2ccbe36e1fd4c3d50cd1f6b7
2012-07-20 18:12:37 -05:00
Subramanya Sastry c0ab29779c In trace mode, wrap transform to output trace info
- Added extra debug_name parameter to addTransform which is
  used in addTransform to output useful trace info.

Change-Id: I160ba0c45f681149375e32ab19f97baa439b09a8
2012-07-20 18:12:37 -05:00
Gabriel Wicke 0a4f1d435b Very basic interwiki support
Pages titles with a wikipedia interwiki prefix now load the page from
corresponding Wikipedia. Links in a page then stay within the given language.

Note that Parsoid currently makes no effort to recognize localized namespaces,
so it won't render media files, categories etc correctly.

Change-Id: I7bc4102e81a402772ea23231170734d580ea15b9
2012-06-05 11:19:58 +02:00
Gabriel Wicke 89c5f4aae4 Forward-port Cite extension
* Adapted Cite extension to use current interfaces and token formats
* Improved TokenCollector

Change-Id: I20419b19edd9bbad2c2abf17a2ff1411b99c0c04
2012-05-03 13:22:01 +02:00
Gabriel Wicke 42af5277ab Token representation clean-up. Now all tokens are differentiated using
constructors instead of type attributes.
2012-03-07 20:06:54 +00:00
Gabriel Wicke 06b257089b Replace console.log with console.warn in all debug statements 2012-02-14 20:56:14 +00:00
Gabriel Wicke 142e40c563 Change token format to plain strings for text tokens, and specific objects for
other tokens. This is only the first half of the conversion. The next step is
to drop the type attribute on most tokens and match on the constructor in the
token transform machinery.
2012-02-01 16:30:43 +00:00
Gabriel Wicke 840941546b Two batteries worth of token transform manager refactoring.
* TokenTransformDispatcher is now renamed to TokenTransformManager, and is
  also turned into a base class
* SyncTokenTransformManager and AsyncTokenTransformManager subclass
  TokenTransformManager and implement synchronous (phase 1,3) and asynchronous
  (phase 2) transformation stages.
* Communication between stages uses the same chunk / end events as all the
  other token stages.
* The AsyncTokenTransformManager now supports the creation of nested
  AsyncTokenTransformManagers for template expansion.
  The AsyncTokenTransformManager object takes on the responsibilities of a
  preprocessor frame. Transforms are newly created (or potentially resurrected
  from a cache), so that transforms do not have to worry about concurrency.
* The environment is pushed through to all transform managers and the
  individual transforms.
2012-01-09 17:49:16 +00:00
Gabriel Wicke 0866116fe4 Comment out a stray console.log 2011-12-14 23:44:58 +00:00
Gabriel Wicke fc41c8887c A collection of small bug fixes to the grammar, Cite, the Token format
converter and the HTML DOM -> WikiDom converter. The tokenizer now digests all
parserTests.
2011-12-14 23:38:46 +00:00
Gabriel Wicke 730b75ca21 Add rough HTML DOM to WikiDom conversion. You can see serialized WikiDom of
parser tests using 'node parserTests.js --wikidom'.
2011-12-14 15:15:41 +00:00
Gabriel Wicke 7a6f71db2f Minor tweak to comment. 2011-12-13 18:55:44 +00:00
Gabriel Wicke 8e8362b8f0 Clean up and comment the Cite extension a bit. 2011-12-13 18:45:09 +00:00
Gabriel Wicke 0fed6f9c79 Convert the Cite extension to a token stream transformer.
This required a few further additions to the TokenTransformDispatcher. In
particular, there is now an 'any' token match whose callbacks are executed
before more specific callbacks. This is used by the Cite extension to eat all
tokens between ref and /ref tags. This need is very common, so should be
broken out to an intermediate layer in the future.

In general, the requirements for the TokenTransformDispatcher API are now
clearer, and the API should likely be cleaned up / simplified.
2011-12-13 14:48:47 +00:00