Commit graph

392 commits

Author SHA1 Message Date
Catrope c9605066cd Merge remote-tracking branch 'origin/master' into dmrewrite 2012-06-12 11:57:32 -07:00
Rob Moen 2ea1030ab4 Rewrite document annotation getters to return collection objects
vs arrays

Change-Id: Ib48a39fff501cdb769f5b9b9fc7717805b74236e
2012-06-12 11:10:29 -07:00
Catrope ab0c37def9 Rip out the offset map
The offset map was broken from the start because it wasn't updated when
adjusting the length of a text node, and if we fix that bug it's
doubtful whether the costs of updating the offset map outweigh the
benefits, especially considering that adjusting the length of a text
node is something we do for almost every keypress. If it turns out
having an offset map does make sense, we can always reintroduce it
later.

Change-Id: I59e8bc154f7d07aa1bab2f473c13ff466d0e463f
2012-06-11 18:10:08 -07:00
Catrope 77e923e323 Update tests for Parsoid workarounds
Change-Id: Ifc7e7dc53eafa62af5652a2dba6cdc1fad199b4c
2012-06-11 18:10:07 -07:00
Trevor Parscal c8c4ed8f99 Added ve.dm.Transaction.newFromContentBranchConversion
Change-Id: I0e272b24dddae028e44d4c693297b1875000ef57
2012-06-11 15:58:44 -07:00
Trevor Parscal 20d1c290d4 Fixed multi-auto-wrapping issue and added tests to track regressions
Roan mostly wrote this

Change-Id: Id292c84a4da97d6608aa7c17ab6a4cf5a2e3f409
2012-06-08 15:16:55 -07:00
Trevor Parscal 773be35b16 Fixed element summaries to not rely on HTML serialization, which can render attributes in any order and tends to differ between browsers
Change-Id: I9cb7c3d26efaf983f667c9490cc4be0ea2fe5ab5
2012-06-08 15:15:22 -07:00
Trevor Parscal 5969f55550 Delete HTMLConverter and update demo&sandbox
Also update the demo and sandbox for ve.{dm,ce}.TableSectionNode

Change-Id: I5a73a8250d283e8666b93561ce9e9374739936a5
2012-06-08 12:40:43 -07:00
Trevor Parscal faa58d1e4a Merge branch 'refs/heads/master' into dmrewrite 2012-06-08 12:29:57 -07:00
Trevor Parscal 61ba00aaef Add a tableSection to the example data and update offsets and stuff in almost all tests
This fixes the converter tests.

Change-Id: I72f6d6c103c3a8bbee201c070a2f7c1bfe401235
2012-06-08 12:27:39 -07:00
Trevor Parscal 2e0d4ef094 Make tableCell represent both <td> and <th>
Adding a 'style' attribute which is set to either 'data' or 'header'

This breaks even more tests because of missing style attributes

Change-Id: I0a75d8c1578b4414eeae8c484f6c4d6f8a59472a
2012-06-08 12:27:22 -07:00
Trevor Parscal c614d3391b Add ve.dm.TableSectionNode and ve.ce.TableSectionNode
Yes, this breaks a bunch of tests, because tableRows are now required to be inside tableSections

Change-Id: Idb52d988df69173647c85f7602344650456e078e
2012-06-08 12:26:45 -07:00
Trevor Parscal 911ee3229a Test getDomFromData using getDataFromDom's tests
Move the test cases for getDataFromDom out into ve.dm.example and use them to test getDomFromData as well

Change-Id: Ib33854a7a1e8cb6a8ebf2c5b88cd5c17e25644b2
2012-06-07 22:10:53 -07:00
Trevor Parscal f2b3df8027 Cleaned up some useless/unused functions
Change-Id: I6aa5d0ed3d21993ed653ee64b4928ef98874db2d
2012-06-07 15:02:40 -07:00
Trevor Parscal 453bec4555 Fixed alienInline and alienBlock conversion (was missing closings)
Change-Id: Ie8e51eb8d9a452c656aebf341071da5627dd80b5
2012-06-07 14:30:06 -07:00
Trevor Parscal fa9fe8b08a Added support for alienInline nodes
Change-Id: I15b9cbcdb66b966dca28edcb26bf5d54a4f17be6
2012-06-07 14:16:19 -07:00
Catrope 3c1de3dc4f Add ve.dm.Document.getBalancedData()
Change-Id: I2420ef5369b9eae053962898b64dd7324e714bbd
2012-06-07 13:17:57 -07:00
Trevor Parscal c8ec9fc6f5 Added a test for removing all the content of a paragraph in the middle of a document
Change-Id: Ibbbc401a6484616dca52133b605d643ba3f810eb
2012-06-07 10:57:03 -07:00
Gabriel Wicke b49102281f Remove a few entries we now care about from the whitelist
They are mostly about whitespace, but there is also a debatable quote test
that outputs an empty bold element at the end of the line. We should perhaps
strip this empty bold in the QuoteTransformer, as the preservation of an empty
bold tag in round-tripping does not seem to be too useful.

Change-Id: I1d8f3ebabcd9f6249e5170de420ba52e8aea22ca
2012-06-07 10:04:20 +02:00
Trevor Parscal b62beed2e2 Added implementation for ve.dm.Converter.getDataFromDom
Change-Id: I8561d4a00ba630b86fb7bcd3e347fe2c6c2f6aa3
2012-06-06 17:47:27 -07:00
Catrope f34b41fe3f Add a test case for a complex removal
Change-Id: I9e57bf2f1381b1da196308f1cd27c9be58bf8509
2012-06-06 15:36:08 -07:00
Catrope 5e47bdc57f Maintain a transaction-wide offset adjustment
This fixes a bug where the second replace operation in a transaction
would cause the rebuild of the wrong range, or the adjustment of the
wrong text node.

Change-Id: I9b1c68d84999d538fe10bb193f4dfdd694121d2a
2012-06-06 15:34:55 -07:00
Catrope 02b19f337f Add zero-length text nodes to empty content nodes
This is needed to make the results of certain transactions' tree sync
round-trip cleanly through the ve.dm.Document constructor

Change-Id: I2ab0758ec6bd7afba5b6645c7330f9fa2d45205d
2012-06-06 15:33:42 -07:00
Catrope 1429a3da59 Rewrite newFromRemoval() using 'covered' mode
Change-Id: If0736aff6d194ec09c2a109f046dd6362ddc64fb
2012-06-06 11:47:35 -07:00
Catrope 1566306937 Add 'covered' mode in selectNodes
Change-Id: I4d24c011ce7a94bc75508dea410fb0ed9ebce6a1
2012-06-06 11:47:35 -07:00
Catrope 61229a5313 Make selectNodes tests actually compare nodeOuterRange
Change-Id: I5c251eb6972e8064ab7e9b2cb9fe2e93060c830f
2012-06-06 11:47:35 -07:00
Catrope 1498ac74eb Fix broken input data in selectNodes tests
Change-Id: I8b81d23d8c501d8f0db5d2922ed3408a6c9033b0
2012-06-06 11:47:35 -07:00
Rob Moen 2439c0149a Created secondary method for getMatchingAnnotations
new static method looks for annotation in annotation object.
ve2 Cleanup on annotate method and surface model.
Partially revive UI tools by exchaning old method usage
for ve2 methods.

Change-Id: Id0ac58330292d76801bbcf1d71a919b493f8ab9e
2012-06-06 11:27:23 -07:00
Trevor Parscal 5d20f45120 Added getHtmlElementFromDataElement and getDataElementFromHtmlElement
Change-Id: Ie5f4fd86612b5a6c34b5843d1e9a521edc626a63
2012-06-06 10:17:51 -07:00
Gabriel Wicke 0f9d939b00 Use word diff if --color is enabled
Change-Id: Ib8d3de75ac306974abfdaca22bfc7b69bc62891d
2012-06-05 16:10:13 +02:00
Catrope 528728558b Fix bug in selectNodes's logic for traversing back up the tree
Change-Id: I0fc5a2ad2c9a8d162e8ddbf3cc6d31684d364928
2012-06-05 06:50:09 -07:00
Trevor Parscal 9111e34a0b Added nodeOuterRange to selectNodes
Change-Id: I9ef0c383fbb2515c752d2d3c52e8632aac73d811
2012-06-04 16:21:29 -07:00
Trevor Parscal a0ea700090 Added tests for transactions that deal with removing alien nodes
Change-Id: Id36413c62386dbe5ebc8c8b3a1d3c5e301a8175a
2012-06-04 16:21:29 -07:00
Catrope f7445b37b8 Retain attributes when reopening closed nodes
* Push entire elements onto openingStack rather than type strings
* When closing an element, build a clone of the opening and push it onto
  closedElements, then insert that clone when reopening the element

Change-Id: I8b0fb44394aed6c471dc6dacaab03e44c2333733
2012-06-04 16:07:16 -07:00
Catrope e67b2fd793 Add some more tests for newFromInsertion
Change-Id: I3c403928a8176d8685e48a63759fefec4657ca96
2012-06-04 16:01:14 -07:00
Rob Moen f958297102 Create tests for surface model annotate method.
Fix up surface model tests

Change-Id: I02b57edc5c3faeb39e0b3c1f473f03fbd49d85b4
2012-06-04 15:02:56 -07:00
Trevor Parscal 386faa5569 Fixed ve.dm.Transaction.newFromRemoval so it supports trimming off extra openings and closings
Change-Id: Icbcf4129a51aa92d47aa89738facd0d288337cbd
2012-06-01 14:34:59 -07:00
Trevor Parscal e7f03c6d70 More cleanup, removed ObjectAnnotation (they are inline elements now)
Change-Id: I2c53c145eed1a801392c954001de7641f928216c
2012-05-31 17:18:28 -07:00
Trevor Parscal 042ffb4e3b Added foundation for new converter
* Added converters to all relevant node implementations
* Added new annotation objects with their own factory

Change-Id: I9870d6d5eac45083929d74d2e58917d0939ca917
2012-05-31 16:50:16 -07:00
Trevor Parscal f6864b0c04 Changed use of "factory" to "nodeFactory" to make way for other kinds of factories
Change-Id: I96db0f28bb220f1c3b23990824e9523278cb8f9b
2012-05-31 15:20:58 -07:00
Trevor Parscal 32bddaf088 Added ve.dm.Transaction.newFromRemoval
Also:
* Refactored tests
* Added tests for ve.dm.Transaction.newFromInsertion
* Added tests for ve.dm.Transaction.newFromRemoval
* Fixed problems with ve.dm.Transaction.newFromInsertion
* Added ve.dm.Node.canBeMergedWith which is partially a port of ve.Node.getCommonAncestorPaths merged with canMerge from within ve.dm.DocumentNode.prepareRemoval from the old ve codebase

Change-Id: Ibbc3887d08286d8ab33fd6296487802d65b319fa
2012-05-31 14:39:34 -07:00
Catrope 2f18605a1a Fix buggy replace behavior when inserting content that contains nodes
Copy-pasting things like "text<IMAGE>moretext" failed spectacularly,
this commit fixes that.

* Check for content rather than structure in the inserted/removed data
* In the content case
** Run selectNodes() over the removal range, rather than just the cursor
*** i.e. no longer assume that content replacements only affect one node
** If there is structure involved, rebuild all affected nodes

Change-Id: I80e40b5b7c514a3fb105d57e4a17770d0fefaaea
2012-05-31 07:27:36 -07:00
Catrope f6ca37926d Add ve.dm.Document.isContentData()
Some of the replacement code was assuming that "does not contain
elements" and "is content" were the same. They're not any more, because
we have content nodes (like image) now, so I need a separate function
to distinguish between these cases.

Change-Id: I206ccdf082b7baddf99d382eb3cdd77ea34fb479
2012-05-31 06:19:34 -07:00
Catrope e92db86046 Fix bug in ve.dm.Document constructor for input ending with text
If the last element of the input data array was text, the resulting text
node would have length=0 rather than the expected length value.

Change-Id: I3d089a80b8a447a12ba411b2e11c1b84f14f2959
2012-05-31 06:17:23 -07:00
Gabriel Wicke e53bc93a8e Check out old results before running tests
Change-Id: Ia56aec22194a14c94620237041c30c269ac2e56a
2012-05-30 17:37:10 +02:00
Gabriel Wicke 36084c5d93 Preserve original newlines in HTML and serialization
254 round-trip tests (up from 184) are now passing.

Also:
* tweaked runtests.sh slightly (use less -R instead of -r).
* made sure the EOFTk is preserved in phase 3 transforms

Change-Id: I1de22186bdb78e52019370e43f096877005b8f5a
2012-05-29 23:29:03 +02:00
Subramanya Sastry 8174c9dafc First attempt implementing rewriting rules on the DOM
- This is implemented as a post-processing pass.
- Might require additional checks to verify rewriteability.
- Implemented as a pair-wise tag DOM minimization strategy,
  i.e. it takes tag pairs (B, I) for ex, and attempts to
  normalize the tree just for those tag pairs.  Normalizing
  across multiple tags is implemented as pairwise rewriting
  across all pairs:  Ex:(b,i), (b,u),(i,u) for (b,i,u)
- Copied over attributes as part of rewriting, but some of the
  attributes lose their meaning on rewriting since tags are
  reordered (ex: sourcePosn, sourceTagPosn). How do we handle this?

Output examples and possible issues to fix:
   <i><b><u>biu</u></b></i><b><u>bu</u></b><u>u</u>
gets rewritten to:
   <u><b><i>biu</i>bu</b>u</u>

But, the equivalent wikitext form:
   '''''<u>biu</u>''''''''<u>bu</u>'''<u>u</u>
does not get rewritten because of parsing differences.
This wikitext gets parsed into:
   <i><b><u>biu</u>'''</b></i><u>bu<b>u</b></u>
The extra ''' token in the middle thwarts DOM rewriting.

However, a slightly different version:
   "'''''<u>biu</u>''<u>bu</u>'''<u>u</u>"
gets properly normalized to:
   <u>'''''biu''bu'''u</u>

An alternative, but fun strategy to play with is to use the following
two normalization primitives: S(wap) and M(erge).
- S rewrites T1(T2(x)) into T2(T1(x))
  (ex: <b><i>foo</i></b> ==> <i><b>foo</b></i>)
- M rewrites (T(x),T(y)) into (T(x,y)).
  (ex: <b>foo</b><b>bar</b> ==> <b>foobar</b>)

The current rewriting strategy could possibly be re-implemented as S-M
rewriting.  The problem to solve there would be to find an efficient
rewriting strategy that is guaranteed to lead to a normal form.  I may
not play with it now, but just documenting it for later (to play with
in my spare time).

This commit is just as a record of fun/experimental code where I get to
learn details of JS, wikitext, parsing, and DOM manipulation.  Next
version of this code will attempt to introduce minimal DOM restructuring
across multiple tags at once which can be more efficient.

gwicke: Removed now passing test from whitelist, and updated another whitelist
entry which is now improved.

Change-Id: Ie97bcb164eb62c34ba61aa76ba2f4c232aa713d8
2012-05-29 08:17:57 +02:00
Trevor Parscal 14862a1212 Fixed bug in isStructuralOffset which didn't consider neighboring a non-content leaf to be a structural offset
This was caused by the fact that a non-structural leaf can not have children, which makes it appear incompatible as a sibling to an arbitrary structural element (like a paragraph) but since it can not contain content we can check that instead.

Change-Id: Ie3c58b4b43f2aa6921f8f82aa82511e231207854
2012-05-25 13:19:56 -07:00
Gabriel Wicke c52c24b0cb Slightly improve formatting of web service; test commit message tweak
Change-Id: Ibac3ce3dd9aa2c4faf11eed351fea941ebf1e4b3
2012-05-25 16:36:14 +02:00
Gabriel Wicke c692bc2307 Use '/bin/sh' instead of '/bin/bash'
Bash omits the time output for some reason. We would like to keep a record of
performance too.

Change-Id: I7c435b1cf2e2f237f78a45b2819a195a240e3aa4
2012-05-25 16:19:46 +02:00