Commit graph

936 commits

Author SHA1 Message Date
Trevor Parscal 0245f6a6a8 Fixes for 44fe109f14 which mistakingly added .children to a few uses of batchedSplice
Change-Id: I2c876a66b64c6f1ae3cbd1577b7e1102618997ba
2012-04-27 15:08:15 -07:00
Trevor Parscal 44fe109f14 Added test data and fixed test suite links
Change-Id: Idb5de70b58c525a67f16b21f7adc53214af9b486
2012-04-27 14:59:52 -07:00
Trevor Parscal 42eaba05e7 First go at ve.dm.Document and ve.dm.DocumentFragment
Change-Id: I6099ec0dbdac8d00cdde138b6b9a1b3a5781daea
2012-04-26 19:44:44 -07:00
Catrope 69df3eefbc Implement ve.NodeFactory and add tests
Change-Id: I34fdf24c0099072fe5f7178400abbc323be975d4
2012-04-23 11:46:30 -07:00
Catrope a239b73746 Don't instantiate classes in ve.extendClass(), it's unnecessary
Change-Id: Iebb9c871750a19a806ebdd911e1ef476a0acaddc
2012-04-23 11:46:30 -07:00
Catrope e84b6b656b Rename ve.dm.NodeFactory to ve.NodeFactory
Change-Id: Ic2a0389ca4855ac1afc3fb9a151145fea2b4795e
2012-04-23 11:46:30 -07:00
Catrope d1d70ddf3e Add a note that I need to implement insert and remove as replace
Change-Id: Id076921707bf6295a6bbac3264fa0074178e9321
2012-04-23 11:46:30 -07:00
Trevor Parscal 3d6391419d Added more nodes and removed canHave[Grandc|C]hildren methods
Replacing them with static members on each node type

Change-Id: I455debf880bef4e280eea072364f5f57308ec2b1
2012-04-20 16:34:47 -07:00
Trevor Parscal 5eca12cca2 Added skeleton class for ve.dm.NodeFactory
Roan will finish the rest

Change-Id: I8da5c493b13f00ca9abdc94ef20e0d2923893d28
2012-04-20 14:40:31 -07:00
Catrope d4d2ae88ad Update lengthDifference in ve.dm.Transaction.pushReplace()
This makes the tests pass

Change-Id: Iede7f47183378d80419f616ff62732a07d136d87
2012-04-20 11:29:30 -07:00
Catrope 0a43cf1f32 Remove documentation for nonexistent parameter
Change-Id: I3237eca92d5962a415c596da84cd3f51d3d8cebb
2012-04-20 11:29:30 -07:00
Catrope 08f792a198 Remove the parameter from the ve.dm.Transaction constructor
It's not being used at all, and it's broken because
this.lengthDifference is set to zero regardless of what length
difference the operations passed into the constructor might cause

Change-Id: I3b7a312a1920347e7bf34df88a05bf6f2ff11f7d
2012-04-20 11:29:30 -07:00
Catrope d93f2f2bab Copy ve.dm.Transaction.js from modules/ve
Change-Id: I629376bfa935ff120d93d8cfc020884524a0ae2f
2012-04-20 11:29:30 -07:00
Trevor Parscal b004b22241 Added tests for all exceptions
We are now checking for the exception messages as well.

Change-Id: I3a306ce9fe82afe6fd1e46a2e4da4d0a70952688
2012-04-19 18:05:48 -07:00
Trevor Parscal 9a3784301b Improved test coverage for ve.dm.BranchNode.splice and ve.dm.TwigNode.splice
* Changed splice to check all elements about to be inserted are allowed before inserting any of them so that catching an exception leaves you in a sane state
* Fixed the order of execution of parent class constructors in ve.dm.LeafNode and ve.dm.TwigNode so that canHaveChildren and canHaveGrandchildren produce correct values and added tests to ensure these methods are correctly inherited in subclasses
* Added tests that check for exceptions when adding nodes that can have children to nodes that can not have grandchildren
* Added test that check for events being emitted before and after splicing, including that beforeSplice should be emitted even in cases where a splice fails and throws an exception because the nodes are incompatible (but afterSplice is not called in this case) since beforeSplice might modify the nodes in some way before the compatibility tests are run

Change-Id: Id12aea995a42c26ff63a74ae3d31f2bf455759e3
2012-04-19 17:45:58 -07:00
Trevor Parscal f081c0932a A few fix ups for fd49e8d
* Moved getParent and getRoot from ve.dm.Node back to ve.Node
* Fixed use of getElementLength that should have been changed to getOuterLength, but was changed to getLength (oops)

Change-Id: Ibe5b855aef533dcd493f762a8a02c6a11ce6e7de
2012-04-19 16:47:40 -07:00
Trevor Parscal fd49e8df32 Added more tests for ve.*Node and ve.dm.*Node classes
In this commit several methods (child node add/remove and parent/root modification) were also moved to ve.dm.BranchNode ve.dm.Node respectively. ve.Node and ve.BranchNode are immutable. ve.dm.Node and ve.dm.BranchNode are mutable. Other subclasses of ve.Node and ve.BranchNode should implement functionality to mimic changes made to a data model.

Change-Id: Ia9ff78764f8f50f99fc8f9f9593657c0a0bf287e
2012-04-19 16:03:59 -07:00
Trevor Parscal c9ce7dbffe Added some basic coverage for ve.*Node classes
* prototype.canHaveChildren
* prototype.canHaveGrandchildren
* prototype.getType
* prototype.getParent
* prototype.getRoot
* prototype.setRoot
* prototype.attach
* prototype.detach

Change-Id: I920f7c9504e467f4818df537608760165c28d432
2012-04-19 14:43:58 -07:00
Trevor Parscal b16ed2b12d Setup tests, which are still empty
Had to fix a few namespace typos too

Change-Id: I3ebdc418f374bd3e151c516e1c0cfe85398772f0
2012-04-19 14:17:59 -07:00
Trevor Parscal 7ea475ebcf Initial commit of data model rewrite
Ground-up rewrite of the data model. Putting this in the ve2 directory for now so we still have the old code floating around.

Main changes so far in this rewrite:
* Renamed hasChildren() to canHaveChildren()
* Added canHaveGrandchildren()
* Added a new node type TwigNode that can have children but not grandchildren (so all of its children must be LeafNodes)
* Implemented push/pop/shift/unshift as wrappers around splice()
* Renamed getElementType() to getType(). Nodes now take a string as a type, and the element stuff is gone and won't be back
* Removed clearRoot(), replaced it with setRoot( this ) where needed

Change-Id: I23f3bb1b4a2473575e5446e87fdf17af107bacf6
2012-04-19 13:54:34 -07:00
Catrope bd900a4bd4 Add ve2 directory for rewriting code in
Change-Id: Id71ba248c73baa05e5fbeb8ad91f9225c1d1f412
2012-04-16 19:38:13 -07:00
Gabriel Wicke afa5b95bc1 Don't work around html5 library tokenizer attribute reordering
The HTML5 parser we are using to normalize expected HTML output in parserTests
reverses the order of attributes (see
https://github.com/aredridel/html5/pull/53 for the fix). Remove whitelist
entries concerned with this and use the proper order in external image
attributes.

Change-Id: If1868cae05396a150757c85a20473ab756cbcd97
2012-04-16 17:09:06 +02:00
Gabriel Wicke c688b039de Collected tweaks
* less verbose logging in noinclude processing and template expansion
* Give priority to the processing of templates transcluded from transclusions
  to get closer to depth-first processing. This serves to minimize memory
  usage from queued-up tokens.
* Increase the maximum outstanding requests per template retrieval. 10000
  amazingly proved too low a limit on some big pages.
* Only process a single template request callback at a time for now
* Add a debug print in the treebuilder wrapper
* Don't treat multiple comments on a single line as a single comment to match
  the PHP parser's behavior

Change-Id: I9a86b6d7bec3b9e1f17415daf1bf74170240721a
2012-04-16 15:47:03 +02:00
Gabriel Wicke 1bf8a9e5e1 Small tweak in comment about onlyinclude forcing buffered expansion
Change-Id: Ib324e24c51c97e07e6737bf23f16db07043b69ab
2012-04-16 15:42:29 +02:00
Gabriel Wicke efd4c026ea Disallow < and > in external link urls
Change-Id: Id865c3d46b33b182bb5b244e77e815c0afd7fa49
2012-04-16 15:36:56 +02:00
Gabriel Wicke 25523f4cf0 Implement urlencode parser function
Change-Id: I4fca3134c9c3eb9a7d6f3360be6de054fb47477c
2012-04-16 14:54:03 +02:00
Gabriel Wicke 421ef44621 Match the empty string as whitespace too
Change-Id: I1a8ed882021804f62855b9db4368270feebbfc16
2012-04-16 14:48:39 +02:00
Trevor Parscal 387b3160ca Merge "Add and update an offset map in DocumentNode" 2012-04-13 23:48:57 +00:00
Catrope 7465b670e1 Add and update an offset map in DocumentNode
This has some TODOs still but I want to land it now anyway, and fix the
TODOs later.

* Add this.offsetMap which maps each linear model offset to a model tree node
* Refactor createNodesFromData()
** Rename it to buildSubtreeFromData()
** Have it build an offset map as well as a node subtree
** Have it set the root on the fake root node so that when the subtree
   is attached to the main tree later, we don't get a rippling root
   update all the way down
** Normalize the way the loop processes content, that way adding offsets
   for content is easier
* Add rebuildNodes() which uses buildSubtreeFromData() to rebuild stuff
* Use rebuildNodes() in DocumentSynchronizer
* Use pushRebuild() in TransactionProcessor
* Optimize setRoot() for the case where the root is already set correctly

Change-Id: I8b827d0823c969e671615ddd06e5f1bd70e9d54c
2012-04-13 16:46:02 -07:00
Rob Moen efb8384c1e Merge "Cleanup terms used in annotation method" 2012-04-13 21:37:07 +00:00
Gabriel Wicke 08453199df Increase number of callbacks per reactor iteration to 4
In experiments this dropped the memory consumption further, and reduces the
queuing overhead in the node reactor.

Change-Id: I9409b6ca863b43b7557663bbec9572365059c078
2012-04-13 14:50:36 +02:00
Gabriel Wicke 06ae53fdfe Drastically reduce memory usage for template-heavy pages
Only call back a few callbacks per reactor iteration from the template fetch
request queue. This changes the expansion pattern from a (memory intensive)
breadth-first expansion to something quite close to depth-first expansion.
Additionally, retrieved pages are quickly added to the page cache so that a
lot of request queuing is avoided in favor of synchronous expansion from the
cache. On pages like Barack Obama that previously ran out of memory after
consuming node's 1.6G heap limit, expansion now runs in relatively constant
100-300M resident (so far, still running).

Change-Id: Ie34a1eeff00d868416de45ef8d289898258f560c
2012-04-13 14:31:03 +02:00
Gabriel Wicke df050e4481 Convert external link syntax stops to stack
Eat unbalanced external link parts within template parameters. This does not
produce the same output as the PHP parser
(try echo '{{YouTube}}' | node parse.js), but preserves a level of sanity.
Need to check how common this is for external links. If it is rare enough,
moving the ']' after the parser function manually would fix the rendering for
the YouTube case.

Change-Id: I597d808efff36baa22191e7946a0061cc31120e8
2012-04-13 11:08:42 +02:00
Rob Moen d37a325f44 Cleanup terms used in annotation method
Change-Id: I509a1f1b680cc8a2973188f0ed7c7d67b8a15f4f
2012-04-12 16:37:23 -07:00
Rob Moen 1fa462ce92 Fix up FormatDropdownTool
Effectively stopping & starting polling prior to conversion
Getting Selection from model
Reselecting after conversion (TODO: modify selection to entire block ?)

Change-Id: I9ba331b5393bf568cc8d137646b43244ae2640a8
2012-04-12 16:21:10 -07:00
Rob Moen 961f83d326 Elminiated isSelection method
Determine actual selection from model by checking length

Change-Id: I1c58a4de31b26d49b55c7a2c55c9c6462741bdfb
2012-04-12 14:05:14 -07:00
Robmoen 3e6e736cc0 Merge "Removed unused fake cursor div" 2012-04-12 18:53:17 +00:00
Rob Moen 28e86c8688 New Method isSelection() in Surface View
Used to properly hide/show Context menu and UI elements.

Change-Id: I8b3750c270651498b7e6c54a1921f5012cf33ea6
2012-04-12 10:22:42 -07:00
Gabriel Wicke 5bb2d96869 Token stream transform improvements
* add past paths for empty arguments etc
* cache attribute token transform pipelines
* fix bugs in TokenCollector and NoIncludeOnly handler, and improve its
  efficiency by only registering for 'end' tokens on demand
* Remove empty reset methods from a few handlers
* Add a simple 'ap' debug print function that makes it easy to only print some
  debug prints by temporarily changing 'dp' to 'ap'
* Improvements and bug fixes in AttributeExpander

Change-Id: Ie69729c8f62d48bba922712e44ebce484c621c50
2012-04-12 15:42:09 +02:00
Gabriel Wicke 3124deca2c Track inclusion status on CachedTokenPipeline
Non-include attribute pipelines are not cached for now. Adding separate
caching for non-include attribute pipelines is very likely worth it, but
deferred for now.

Change-Id: I13f949d9f0a04536f9ccfcb73a2be69c5c08be01
2012-04-12 10:21:50 +02:00
Erik Moeller 9f5a45f138 Add simple warning to indicate that VE is still in early dev.
Change-Id: Iea2abf9660556eca1d981fbbaab30d75a813e92d
2012-04-11 18:32:45 -07:00
Trevor Parscal f6a8619b95 Removed unused fake cursor div
This was an artifact from experimentation with multiple cursors long long ago in a land far far away

Change-Id: I14491c4adbd40bb8df4b1c31725cb1621351bef2
2012-04-11 14:56:15 -07:00
Rob Moen b5befb1d57 On annotate, get current selection from model
Fixes link annotation, and any annotation that pops up an inspector as selection is lost.

Change-Id: Idd8812b05f4fddfa4e613dea28a1a8421792656d
2012-04-11 14:15:52 -07:00
Gabriel Wicke 23331abf21 Merge "Set inclusion flag for attribute transform managers too" 2012-04-11 20:47:01 +00:00
Rob Moen 224cc1f9a0 Migrate tools from SurfaceObserver Class
ContextView positioning, Italic, and Bold annotations working

Change-Id: Ifc68bd61c8f27b6a14d314dc2999753af6e0ee04
2012-04-11 13:19:58 -07:00
Gabriel Wicke efa41370d3 Set inclusion flag for attribute transform managers too
Change-Id: Ice15d8fde6de4a3e850a028db9917e976218fc43
2012-04-11 21:55:52 +02:00
Gabriel Wicke bff43938f6 Support noinclude/includeonly/onlyinclude in attributes
Fun test case:
{|
|-<includeonly>
foo
</includeonly>
|Hello
|}

Change-Id: I353bb287d3967ade549fbcb4ae64511a1f1f7e36
2012-04-11 17:37:25 +02:00
Gabriel Wicke 9ae572cca0 Fixes to template expansion / token transform managers, 296 tests passing.
* Convert isNoInclude logic to positive isInclude throughout and set it
  properly on attribute pipelines. Also don't cache non-include pipelines.
* Add a --pagename parameter to parse.js, which sets the page name in the
  environment. This is then returned by {{PAGENAME}}. Not the final solution,
  but useful for taxobox testing as taxons are selected based on PAGENAME.
* Add rudimentary pagenamebase parser function

Change-Id: If9c0be4c255200d0f2a30f02e5619437b4fd8f12
2012-04-11 16:34:27 +02:00
Gabriel Wicke bbae66cd69 Nominate more HTML5 sectioning and heading elements for block-level treatment
Block-level (in HTML4 lingo) elements are not wrapped into paragraphs.

Change-Id: I4a01c9721be30b526172952915d528dea79e2f30
2012-04-11 12:53:49 +02:00
Gabriel Wicke 5a33099875 Improve template tokenization in template arguments
Taxobox tables now render pretty much correctly.

Change-Id: I5a0564138ff0c688d8a5a69b7867646fd3763946
2012-04-10 16:40:49 +02:00