Commit graph

37 commits

Author SHA1 Message Date
Arlo Breault f75943282f Avoid serialize/parse of mw:dom-fragment-token content
* The contents of "mw:dom-fragment-token"s was being serialized
   after processing to the DOM and stored on the token to be
   shuttled through tree building.  Only to be reparsed in the
   unpacking phase.

 * Here we store a pointer to the contents in a fragment map.

 * Doing less work results in a performance improvement, though
   only slightly because the content still needs to be adopted
   by the main document.

Change-Id: Ia0aec7de469101a2a93342ea89daac0f0e73cf1a
2017-10-25 22:41:18 -04:00
Arlo Breault 84c80a7ed2 T133334: Ref marker in caption in data-mw
* Replace ref markers instead of waiting for cleanup to remove them
   since that doesn't happen on embedded html.

Change-Id: Ied746f025a0ac7f14d922aff6640fef3aa4b55b0
2017-10-26 00:12:27 +00:00
Arlo Breault 27799ea9b5 There's only one refInRefProcessor
* Avoid the unnecessary binding.

Change-Id: I80e558a86eb7197ce39a07062499ec58f170a3af
2017-10-26 00:06:20 +00:00
Arlo Breault 23db74cd9c Remove unnecessary serialize/parse of references content
* Also, change the input parameter of buildDOMFragmentTokens
   to uniformly accepts a <body>, rather than a doc or string,
   so that we can pass it nodes from our dummy document.

Change-Id: I4bb44573fe7203277d51e804a4a6423100a34f03
2017-10-25 23:52:19 +00:00
C. Scott Ananian fc4967ac5f Remove unnecessary calls to Node#childNodes.length
These calls force us to allocate a backing array for childNodes,
deoptimizing a linked list representation.  Use the standard
Node#hasChildNodes() method instead, which can be efficiently
implemented without allocating a backing array.

Change-Id: I1706bfa15263564bc981d689947835a3d0d4a68f
2017-10-24 23:04:21 +00:00
Arlo Breault 3aa2e1a106 Bump version to 0.8.0 for release
Change-Id: I2279cbd8d56d3cd097a086b4f52741234a014346
2017-10-23 17:37:34 -04:00
Arlo Breault 0dba785946 T176425: Transfer about group to reference wrapper
* Follow up to 74acc71e

Change-Id: Ia403bf88dd48702dbbbf53a247ea2714243a0acc
2017-09-25 12:59:46 -04:00
Arlo Breault ab0c40f496 Fix crasher from empty references list
* See on /en.wikipedia.org/v3/page/html/Manuela_Maleeva/800363347

 * Folow up to 74acc71e

Change-Id: Iae89d4b0073dafce5fe4c28dc94fd6d5779d71c6
2017-09-13 22:35:37 +00:00
Arlo Breault 74acc71e98 T159894: Add support for Cite's responsive parameter
* Ports commit 04c3ad01 in core's Cite extension.

 * Until $wgCiteResponsiveReferences is exported, a number of wikis
   which have enabled this won't be getting the right default.
   https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings.php#L14945-L14970

   However, it looks like enwiki's {{reflist}} explicitly sets the
   parameter either way, so not a bad start.

 * The "ext.cite.styles" css resource is added from core's Cite
   extension 05cb5cc1, since that's where the responsive css lives.

 * The blacklist changes for existing tests are because we now only
   serialize the children of the div wrappers.  Those tests probably
   deserve Parsoid specific sections..

Depends-On: I2404999ab11b5cf7b740ae43696c4676ab1b6d22
Change-Id: I8f9277b3ecb253e0bee7ee55eef7af4935821527
2017-09-13 21:24:52 +00:00
Arlo Breault ff09c1667d Add 'references' class to reflist node
* Matches the core Cite extension commit cec36fed

 * This change means we can get rid of some of wiki specific styling in,
   https://github.com/wikimedia/integration-visualdiff/blob/master/lib/parsoid.custom_styles.yaml

Change-Id: I763b8086d057ea0cac8ba8fd4c9f4f4279766fed
2017-09-11 15:13:55 -04:00
Arlo Breault 6e5b366ff6 [eslint] Enforce lines-around-directive
* Done with --fix

Change-Id: Ie9b34c69615493870427be5f318a14f31a361d6a
2017-04-27 16:05:00 -04:00
Arlo Breault d13cfa4f71 [eslint] Use eslint-config-node-services
* Lots of overrides here that we should either fix or be explicit about
   wanting to diverge and maybe talk about upstreaming the change.

Change-Id: I927dd325e49ef72ccbe5e8b926f7984b82dd0f2e
2017-04-27 16:04:53 -04:00
Arlo Breault 863e53ff8e T163744: Switch to eslint
* .eslintrc was generated from .jshintrc and .jscsrc using polyjuice
   and some manual tweaking.

 * Forced to follow the convention in eslint #4174

 * However, in a follow up, we'll use wikimedia/eslint-config-node-services

Change-Id: I2500a6520a5c9f41d5333e937c151228aec88be0
2017-04-27 14:00:51 -04:00
Arlo Breault dea21633ba T64270: Support video and audio content
* Spec'd at https://www.mediawiki.org/wiki/Specs/HTML/1.3.0#Audio.2FVideo_.28Proposal.29

Depends-On: Icb26ffb802150c1d1f8c6e0aab05b25e503077e2
Change-Id: I53c6fa8d862c152bb1c8dd3ae96f4ee464a60d32
2017-04-21 20:44:44 +00:00
Subramanya Sastry 4b242825cf Update HISTORY.md + bump version numbers to 0.7.1 for deb release
Change-Id: I32cf92c566dc7d501ec0b28552ef6a4fbb879d82
2017-04-05 11:13:30 -05:00
Subramanya Sastry d4140cb46f Update HISTORY.md + bump version numbers to 0.7.0 for deb release
Change-Id: I46189fae2dc0b01b473c471a8a62d2f6204f3692
2017-04-03 17:25:07 -05:00
Subramanya Sastry 2368fba08f Escape cite ids with Sanitizer.escapeId
* Without this, refs like <ref name=":0"> won't generate the same
  links that the PHP parser generates.

* Updated an existing test to add the :0 key that require escapeId
  to be encoded properly.

Change-Id: I69e5f16ccf64bd1c9cf05bdea7a379e679d36b1a
2016-12-20 15:07:07 -06:00
Subramanya Sastry 18c965dec0 T102134: Fix cite hrefs to render properly
* With Parsoid's base href pointing to the wiki, plain #-fragment
  links won't resolve properly. Add the page title to the href
  for the links to start resolving properly again.

* Updated parser tests accordingly.

Change-Id: I280c41a0382bd2acd82cc586212695aa3b920171
2016-12-20 15:07:07 -06:00
jenkins-bot 0b1277434f Merge "T104662: Allow nested ref tags only in templates" 2016-12-12 15:15:10 +00:00
Arlo Breault ea599bb074 Let extensions supply the pp tracing name
* They won't all have tags, like the linter.

Change-Id: Idf309a82bf229219d98d693302c305726fa25fcd
2016-12-09 19:03:35 -08:00
Arlo Breault 439bc3af2d T104662: Allow nested ref tags only in templates
* To support the #tag parser function.

 * An aside, #tag is nutty,

     {{#tag:nowiki|test<ref>haha</ref>}}

Change-Id: I0ab93f6ae959b30a887e967952c904ef0400b189
2016-12-08 10:33:00 -08:00
Subramanya Sastry a16445812f Bump version to 0.6.1 (0.6.0 deb pkg was botched)
Change-Id: I601f31c79d8c2361ca33a6d0e6d6196c1b6c5f0f
2016-11-14 16:44:35 -06:00
Subramanya Sastry c8a0ec200a Bump version to 0.6.0 for new deb release + update HISTORY.md
* Updated HISTORY file based on Parsoid deployment logs and
  added only the most pertinent entries.

Change-Id: If07bc75fcb09507cae30fc82d7d89b8d87a4c69b
2016-11-07 16:30:16 -06:00
Arlo Breault 8c60ef0f79 Let native extensions add stylesheets
Change-Id: Ib418544236ddf2d9a075934d72712734d90b4537
2016-09-12 17:09:05 -07:00
Arlo Breault e9383e0877 Use tagWidths for content in Util.getArgInfo
* Use nestedRefsHTML.length to determine when we need a dataMw.body,
   the regexps there were unnecessary.

 * Pulled out of https://gerrit.wikimedia.org/r/#/c/264026/

Change-Id: I4602594e468322c8b6b8653eee33047ef9af9ebc
2016-08-31 13:33:03 -07:00
Arlo Breault 419db6b794 Bump version after release
Change-Id: Iaed8279cb06c0c7b55940b1e4aed3f8b21262e79
2016-06-14 11:18:15 -07:00
Arlo Breault 53c9863382 Keep the data-* attributes at the edges of the DOM
* FIXME: stop skipping jsapi tests

 * DU.serializeNode is renamed to DU.toXML, to avoid confusion with
   wikitext serialization (and a name clash when grepping).

 * DU.serializeToXML is renamed to DU.ppToXML, to distinguish it both in
   name and use from DU.toXML.  This is a special xml serializer that's
   meant to be used in Parsoid's DOM post-processing phase.  It's aware
   of a node's .dataobject and automatically transfers that to json
   stringified attributes before serializing a node.

 * DU.ppToDOM is added as helper for loading attributes when parsing
   html, as in DU.parseHTML.  It's the converse to DU.ppToXML

 * Diff markers are no longer stored as json stringified attributes on
   nodes while serializing.  Only when dom dumping, and even then only
   on clones.

 * Once T100856 is fixed, we can look into discarding data-parsoid from
   html that's stored in data-mw (ie. captions, references, etc.)

 * Filed T133334 to note that some of the html we're stuffing in data-mw
   needs further processing, since we're storing ref markers.

Bug: T91700
Change-Id: Ia1a60951e6292b5eb073eedca0b79938094809d2
2016-04-28 18:49:22 -04:00
C. Scott Ananian 8a5316e0c6 Add autoload mechanism for user extension code.
This parallels the autoload mechanism in mediawiki core.  In fact, small
wikis can use the same extensions directory for both core and Parsoid.

We also export a very basic "extension API" for external extensions
and convert our current "native extensions" to use this to show how it
is to be done.

Bug: T133320
Change-Id: I8e05d5bfdff873f28a58dead68aaca0e4823cf32
2016-04-26 22:25:59 -04:00
Arlo Breault 87296241d5 Rename DU.serializeChilden to DU.serializeToXML
* And use it where appropriate.  This has the effect of reducing the
   dom size because of the use of smartQuote (see all the parserTests /
   blacklist changes).

 * The serializeChildren name clashes with the one in the
   serializeState, causing confusion and making grepping harder.

 * Bonus: removed some dsr info in parserTests and fixed a html2wt typo
   that was prevented a test from running.

Change-Id: I287f1efd4afe06b158844e0de9f16494d4a89f93
2016-04-12 14:01:04 -07:00
Arlo Breault 126ed00d40 Suppress errors raised when getting debugging info
* An example of an href from eswiki/Kate_Gosselin?oldid=90347467
   is <a href="#cite_note-13.3F_It_Can't_Be!-3">
   which domino should do a better job validating, instead of throwing
   "Object [object Object] has no method '''".

Change-Id: Ie4bd702d7eee23ad021f2ad31b47cfbada947cd9
2016-04-08 19:36:01 -07:00
Arlo Breault df00a42c5e Fix log type
* Broken in ae45ffe9

Change-Id: Ife2a42e8a805e68fb4bfe83076e39b64be6dbf71
2016-03-30 18:22:07 -07:00
Subramanya Sastry 53117cef54 T113331: Move auto-generated refs flag from data-parsoid to data-mw
VisualEditor requires this info and presumably other clients might
and this is, in reality, semantic information about wikitext.

The html2html failure is because:
* even when the references html has been autogenerated,
  in non-rt-testing html2wt mode, we always generate the
  <references /> tag.
* we moved the flag from data-parsoid to data-mw but ignore changes
  to data-parsoid while comparing test output.
* So, in html -> wt mode, <references /> is always generated
  => data-mw.autoGenerated flag doesn't show up in the wt -> html
     phase of the (html -> wt -> html) test
  => the html2html test fails.

Change-Id: I8e79f2a436a1ca276b9351228a3d8f02d7ebd0c6
2016-03-24 23:17:34 -05:00
Arlo Breault 3b44e7e544 Remove extsrc bloat from references data-mw
* This hasn't been used since 2013! See 46ddbf3f

Change-Id: Iea382829f38788541b3bc4f8595a0d6bf600c167
2016-02-05 18:26:33 -08:00
Arlo Breault ae45ffe95e Move extension content serialization to cite
* See the follow up for why references extsrc is irrelevant.

Change-Id: I1db0f3c3f3e85706125418bb431aa40c17a97407
2016-02-05 18:26:22 -08:00
Arlo Breault 41756cb70b Flatten ext/
Change-Id: I083dc0d7c5ab37c8f0f5c051e82e4705c891812d
2016-01-13 15:34:40 -08:00
Arlo Breault 1c10f71b47 Move processRefs.js to ext/cite/
Change-Id: I10ab294900f090dcdb85b205d9eb0472fd3e9157
2015-10-28 15:31:53 -07:00
Subramanya Sastry 05bf16d3eb T115665: Reorg parsoid repo
* First pass moving around files into different directories.

* Renamed files to remove unnecessary prefixes or align the name
  closer to what the file contains.

* Added temporary soft links to bin/parse.js and bin/roundtrip-test.js
  in the tests/ directory since jenkins jobs seems to have hardcoded
  refs to those paths.

* Deleted:
  - a couple of stale scripts in tests/ that are no longer relevant.
  - a couple of state scripts in api/ that didn't look relevant.
  - swagger spec since it was incomplete, stale, and unmaintained.

Change-Id: I97c30467b190b417eec9e750238704330ae91137
2015-10-27 12:50:47 -05:00