Commit graph

154 commits

Author SHA1 Message Date
Ed Sanders 44b1fdebe4 Use FlatLinearData for storing converter results
Previously we returned ElementLinearData from the converter, then
stripped out the MetaLinearData. This meant that before processing
the ElementLinearData from the converter actually contained metadata
which is confusing.

The new document constructor stores the converter results in a
FlatLinearData object and simultaneously populates element and meta
data stores.

Also in this commit I have moved various methods from ElementLinearData
to FlatLinearData, from which ElementLinearData inherits.

Change-Id: I64561bde2c31d8f703c13ac7b0a0c5f7ade9f3d4
2013-10-06 20:27:32 +01:00
Ed Sanders caa05f3315 Fix exception thrown by converter when data ends in text
Conversions usually end in a close tag so this hasn't been a problem
before, but the mini-conversion we do in onCopy may not be balanced.

Change-Id: Ia2db29f116ca84ee886b4c5cafd4ac45cd926b79
2013-09-20 18:26:14 +00:00
Ed Sanders e7aed52d93 Fix check for preformatted when stripping whitespace
Previous was checking by string match to 'preformatted' which didn't
match 'mwPreformatted'. Now using node factory to see if
hasSignificantWhitespace is set.

Bug: 54083
Change-Id: I1be2d4568df7848e08074d200fb374acd508f6fa
2013-09-16 16:41:45 +01:00
Ed Sanders f153f932a2 Remove inserted leading whitespace
This is bit of a hack, as leading whitespace could be
significant if styled with white-space:pre.

Long term VE shouldn't be editing the user's HTML, and
should just highlight potential formatting issues.

We avoid the stripping in preformatted elements as we
expect they will have that styling.

Bug: 51462
Change-Id: I654d98e17dd604cb2a192831ff3f3597f95b9962
2013-08-02 19:01:22 +00:00
Timo Tijhof 44623c9b2a ve.copy: Remove obsolete copyArray and copyObject
These have been pointing to the same method for a while now,
we can safely remove these obsolete aliases and just use it
as generic copy.

* Each file touched by my editor had its new line at EOF fixed
  where absent
* Don't copy an otherwise unused empty object
  (ve.dm.Converter)
* Use common ve#copy syntax instead to create a link
  (ve.dm.Document, ve.dm.example)
* Remove redundant conditionals for isArray/copyArray/copyObject
  (ve.dm.example)

Change-Id: If560e658dc1fb59bf01f702c97e3e82a50a8a255
2013-07-30 01:44:22 +02:00
jenkins-bot 6355d9937f Merge "Fix the newline bunny-hop bug" 2013-07-27 18:07:07 +00:00
Ed Sanders a9d391272e Speed up openAndCloseAnnotations by using store indexes
By using annotation indexes only we can avoid a lot of
ve.getHash calls. This reduces the number of getHash calls
on load of [[:en:Argentina]] from ~60,000 to ~2,000.

Bug: 52013
Change-Id: I0bc9aa8feea5f7e4e90a5fcd829de57cab803c15
2013-07-26 18:11:01 -07:00
Roan Kattouw 4041d3f342 Fix the newline bunny-hop bug
We would dirty-diff "</span>\n<!-- comment -->\n<span>" to
"</span>\n\n<!-- comment --><span>", i.e. the second newline made
a bunny-hop to the left over the comment.

The actual bug turned out to involve a double bunny-hop, with
"</span> <!-- comment -->\n<span>" turning into
"</span>\n <!--comment --><span>", i.e. the newline bunny-hops
both the comment and the space.

This happened because outputWrappedMetaItems() didn't take
wrappedWhitespace into account when restoring meta items and
associated whitespace. I hacked a check for wrappedWhitespace into it,
but we should really just rewrite this pile of hacks into a unified
system for queuing and processing both whitespace and metadata.

Change-Id: I4375f4c07983ffec6877d0371aeaa9bf6e65fd6e
2013-07-26 16:24:27 -07:00
Ed Sanders 017b947464 Quick optimisation to avoid containsComparableForSerialization
As that method makes expensive ve.compare calls, we can quickly
avoid most cases by testing AnnotationSet#contains first.

On page load this reduces the number of ve.compare calls on
[[:en:Argentina]] from ~6,000,000 to about ~2,000.

Also reduces ve.compare calls per backspace keystroke from
~300 to 2.

We can optimise this further, but this is a good simple start.

Bug: 52013
Change-Id: Ie3b4517fd13383c48acb64b3c4e82051c34e7484
2013-07-27 00:03:11 +01:00
Ed Sanders cdfed7039f Handle meta-only transclusions as meta items
To achieve this we need to evaluate the DOM contents of
transclusion nodes to see if it consists solely of meta items
and whitespace.

To check for meta items we do a model registry match, but with an
additional parameter to exclude mwTransclusion types as a possible
result (as the first item may be a meta tag, but with a mw:Transclusion
typeof attribute).

Bug: 51322
Change-Id: I89a220350fb7e10e15f3682d21438539196a5846
2013-07-24 14:27:24 +00:00
Roan Kattouw 46c5192477 Allow annotations to render nothing
This effectively unwraps the annotation. Annotations can do this by
returning an empty array from their toDomElements() function.

Right now this is only supported for annotations, but once the converter
is rewritten to be entirely bottom-up, this is trivial to support for
other model types, and could even be used to implement unwrapping of
wrapper paragraph.

Change-Id: Ia572fd0610afccccfe795c257c0de9d003330f13
2013-07-16 15:09:08 -07:00
Roan Kattouw 2942fa8fa0 Pass child DOM elements to annotations' toDomElements()
This is special-cased for annotations but should be supported for
all model types once the converter is rewritten to be bottom-up
entirely.

Add a toDomElements() stub to ve.dm.Annotation overriding the stub
in ve.dm.Model so there's a place for the new parameter to be documented.

Change-Id: Id81da87b8b83d556a3618cc6187b22443a1e37e6
2013-07-16 15:09:08 -07:00
Roan Kattouw 15a7822df1 Process annotations bottom-up rather than top-down in data->DOM
This means that instead of creating a DOM element for an annotation,
then appending stuff to that DOM element, we queue up the things to
append and only create the DOM element after we've built everything
that's going to be in it.

Most significantly, this moves the toDomElements() call to the close
function, which paves the way for passing in the annotation's contents.

Change-Id: I98a7d3ebb0f3eb8627c22348b48596906db2646e
2013-07-16 15:09:05 -07:00
Ed Sanders aaf4f5b134 Extend empty document check to check for meta-only data
An empty document is one which contains no 'real' data, so
we should check for meta-only documents when deciding whether
to add in a wrapper paragraph.

Bug: 50289
Change-Id: Ib3ebf0717aa0c6c51fd1d0b14e95de50b2842647
2013-07-15 12:12:00 +01:00
Ed Sanders d2778b1df4 Fix annotated element alien metadata inside wrapped paragraph
Add check that we can close the wrapper to meta items, also
annotate them correctly.

Bug: 51235
Change-Id: Idd4a2304f588ea74b9a9814ce86c1fd0c026e441
2013-07-12 20:13:15 +01:00
Ed Sanders 37cf529d73 Allow annotations to be additive
Set a static property on big, small, sup, sub to allow them
to be added multiple times to an annotationSet.

Fix the converter to count out annotations when opening/closing.

Bug: 49755
Change-Id: Ifbede9345a66434022dbd681eada447ab81ab025
2013-07-02 21:43:47 +01:00
jenkins-bot c75e379086 Merge "Fix bug where whitespace in an empty node was doubled" 2013-07-01 17:15:58 +00:00
jenkins-bot 74ec334a30 Merge "Fix reordering of metadata bug" 2013-07-01 17:09:30 +00:00
Roan Kattouw f93b051acf Fix bug where whitespace in an empty node was doubled
<table>\n\n</table> round-tripped to <table>\n\n\n\n</table> because
we would store '\n\n' in both the innerPre and innerPost fields.
Fixed by not setting innerPost if the element is empty.

Change-Id: I0393bfaf9793fdebc8fff72c8760113fa69bb2bd
2013-06-30 00:29:38 -07:00
Roan Kattouw 46c3d48ba7 Also annotate empty annotations
The converter wasn't setting .annotations on meta items created to
represent empty annotations, which meant that HTML like
<i>Foo<b></b></i> would end up as <i>Foo</i><b></b> in the linmod.

Change-Id: I13d7d9820beeee1e8c3673e08051361d6c6ac4cd
2013-06-29 23:51:34 -07:00
Roan Kattouw 414709c5cb Fix reordering of metadata bug
If you had <meta /><b>Annotated text</b> in a wrapper paragraph,
the converter would swap them and output the linear model equivalent
of <p wrapper><b>Annotated text</b></p><meta />.

This happened because the meta item was queued, and annotations didn't
trigger metadata queue flushes. The fix is to trigger a metadata queue
flush whenever we're about to write something that isn't itself queued.

Change-Id: I168abc0392fbec5503271d1653ee5c38518f857d
2013-06-29 22:35:21 -07:00
Roan Kattouw 92ffa0739d Fix reserialization of mw-data.body.html
This bug caused all references containing complex content (e.g. links
or templates) to be dirty-DOMed and reformatted by Parsoid.

ve.dm.MWReferenceNode.js:
* Parse the original body.html and check if it's semantically equal to
  the new value. If so, don't set it.

ve.dm.Converter.js:
* .normalize() the converter output to remove empty text nodes and
  merge adjacent text nodes

ve.dm.example.js:
* Update reused reference test to have body.html absent, not empty
* Add a link to one of the reference tests so this bug is triggered
** The link's attributes are ordered specifically so that toDomElements
   will reorder them, at least in Chrome (may behave differently in
   other browsers)
** This test fails without this fix in place

Change-Id: Idc091a14422fbb117a3d06fc6bb9497768086fc3
2013-06-25 19:24:18 -07:00
jenkins-bot 2b7e47b8ab Merge "Don't drop annotated comments in wrappers" 2013-06-25 22:54:42 +00:00
Roan Kattouw f9e698db22 Don't drop annotated comments in wrappers
HTML like <td><span>Foo<!--bar--></span></td> would lose the comment
when converting to linmod, and so we'd drop the comment when converting
back to HTML.

This happened because we were queueing up meta items to possibly move
them and their whitespace out of the wrapper paragraph even if we were
inside of an annotation and there was no way we could move things out
at all. The fix is simply to detect these case and not queue up meta
items in that case.

[12:11]	marktraceur	Let the comments hit the floor, let the comments hit the floor
[12:11]	marktraceur	(to the tune of the Drowning Pool "Let the Bodies Hit the Floor")

Bug: 50071
Change-Id: I5a844e2e0655bc1db152b3805718324ca0bb04df
2013-06-25 12:38:17 -07:00
Ed Sanders 29d61dbea4 Only use HTML attributes when comparing annotations for serialisation
The UI still needs a simplified version of annotation comparison
so it can clear/set annotations, whereas for serailisation we need
to compare selected HTML attributes (and once Parsoid is fixed,
compare all attributes).

Change-Id: I1a5521a3f51ea373a21cfbc9a007a2381c12f4fe
2013-06-25 17:57:08 +01:00
Ed Sanders 2352613b66 Code style fix: (X) -> ( X )
Change-Id: I7f3ac95621f70d9a89e70e9a51905673240cb51f
2013-06-24 18:51:59 +01:00
Ed Sanders 4bcc07b120 Allow metadata to be annotated
To prevent dirty diffs when inline metadata appears inside annotations.

Bug: 50060
Change-Id: I40da3d5e8e987571195142304b1a853e177e36ff
2013-06-23 18:48:32 +01:00
Ed Sanders 8bfbef1c5e Handle nested empty annotations
At the moment we create an alien meta item if an annotation
wraps contents of zero length, however we also need to handle
nested empty annotations, i.e. where the contents is other
empty annotations (i.e. alien meta items).

As alien meta items simply preserve DOM elements we don't
mind what type of alien meta items they are, so this also fixes
the case of an annotation wrapping another alien meta
e.g. a comment or <meta> tag.

Bug: 48605

Change-Id: I98d88b341efcd6384ee1fda5d6d7e1e1acb107aa
2013-06-21 17:20:06 +01:00
Roan Kattouw 95733390dc Use indexes in .children, not .childNodes, for attribute preservation
The difference is that .children excludes non-element nodes (text nodes
and comment nodes). These can't have attributes anyway, so there's
nothing lost by skipping them, and this way we avoid bugs where a
text node split causes the indexes to be off.

(Text node splits are probably due to an interaction between whitespace
preservation and paragraph unwrapping, and aren't necessarily bad. We
just shouldn't rely on indexes into .childNodes)

Change-Id: I905a50e1c299ebafcbd4eaa0f938b06a1b5849ff
2013-06-17 15:41:22 -07:00
Roan Kattouw 393925a868 Build the internalList linmod even if it's empty
Otherwise fun exceptions occur when you have a <references /> tag
with no <ref>s, for instance.

Also disregard the internalList in the data->DOM conversion, to prevent
a nasty interaction where the whitespace information on the last element
is considered invalid because it doesn't match the internalList's.

Plus test updates from hell because this touches ve.dm.example.data

Change-Id: I62881d9fc27fa081123856d1b35a6021af469271
2013-06-17 13:23:20 +01:00
Roan Kattouw 0c16066eb3 Kill attribute order preservation
Per the bug report, it's useless because Firefox does not preserve order
in the .attributes array (but sorts it alphabetically instead), and so we
actually get the same exact behavior regardless of whether we use ordering
cleverness or just a straight-up plain object: order is preserved in Chrome,
and alphabetized in Firefox.

Bug: 48980
Change-Id: I1463d06db4900083dd4c565292bbabef09194b43
2013-06-13 23:17:08 -07:00
Ed Sanders da51566d56 Store empty annotations as alienMeta nodes
Not a perfect solution, but better than deleting them completely.

Bug: 48605
Change-Id: I0f3a194f629b054f193a61af105733a555775d79
2013-06-12 19:46:41 +01:00
Ed Sanders 59bfc081df Store alien meta items' dom nodes only
We don't need to decompose and rebuild alien meta items
as they are uneditable, so just store the dom node and return that
in toDomElements.

Update test cases and several tests now need to use
assert.deepEqualWithDomElements.

Change-Id: I4d2eed267dac7855cb929a17d7aef566eaf0e287
2013-06-12 16:37:37 +01:00
jenkins-bot 1bab467c61 Merge "Rename character to (grapheme) cluster when dealing with unicode." 2013-06-06 17:49:12 +00:00
Inez Korczyński 5079bbae9a ve.dm.Converter: Fix too-much-ascending bug in Converter.
Change-Id: I748003d316614717c8eaf58dcf5735246951532a
2013-06-05 14:04:42 -07:00
Ed Sanders 605fe147dc Rename character to (grapheme) cluster when dealing with unicode.
Change-Id: Ia72e19289ba7f15a921d22698ab5d8c354ef120d
2013-06-05 17:30:28 +01:00
Catrope 1365c4a5d5 Fix internal node weirdness
* In getDataElementOrSlice(), we were slicing one too far
* When encountering a closing for an internal node, don't traverse
  up. Doing this caused weird bugs, like inserting text where it didn't
  belong in some cases an exceptions in others, but these issues were
  parly masked by the off-by-one error in the data slice.

Change-Id: Ieda9afa95b7c1953d09e391774350a9b4148c2fe
2013-05-28 15:30:15 -07:00
jenkins-bot 5add8cc4c0 Merge "Cleanup internalList method names and data stores" 2013-05-28 14:39:14 +00:00
Ed Sanders aa6eca10db Cleanup internalList method names and data stores
Rename some methods and erase the HTML data after conversion.

Change-Id: Ic7317db2c7693591fda4bea459631981a69003f3
2013-05-28 15:11:17 +01:00
Trevor Parscal 8511f8ab3e Treat acronyms like words in camel case names
No matter what the W3C says, getDomFromHtmlString is more legible than
getDOMFromHTMLString.

Change-Id: Ic843b6671871024cce8acd82b6be435599ed168b
2013-05-28 12:51:41 +00:00
Ed Sanders 90e4f4d665 Store multi byte characters as one element
Getting & setting the cursor is done with byte offsets
instead of data model offset (characters) so we need to
be able to convert between the two as well as splitting
characters.

TODO: The regex only works on surrogate pairs, not
yet combining accents.

fixupInsertion will combine a combining mark with the
character to its left it it can.

Bug: 48630

Change-Id: I8d936fb15d82f73cd45fac142c540a7950850d55
2013-05-25 00:50:11 +02:00
jenkins-bot f25572a992 Merge "Fix for check if data is balanced" 2013-05-22 22:05:43 +00:00
Inez Korczyński 02d522500c Fix for check if data is balanced
Previous check wouldn't make sense, cause the last offset in the data
could be that one that makes data balanced (and j is increased always
after iteration).


Change-Id: Ie9498d0ac9e3417d09b8b3043bf3281e7dfbf9db
2013-05-22 14:19:42 -07:00
Inez Korczyński bf6df1ff37 Get rid of try..catch by using ve.dm.nodeFactory.lookup.
Change-Id: I017ed60f7d7ce8af34c94fe237cc4101192518f1
2013-05-22 12:51:51 -07:00
Catrope 9ff6737f4c Refactor HTML attribute preservation
Rather than using namespaced linmod attributes, store the preserved
HTML attributes in the .htmlAttributes property of the linear model
element, in a nested structure to allow for easier treatment of child
nodes. Also added attribute order preservation by storing attributes
as an object plus an array of keys.

ve.ce.Node.js:
* Remove html/* attribute synchronization. Doesn't make sense any more
  because these things aren't in the attributes object any more. I don't
  think it ever made sense because these attributes were never supposed
  to be changed anyway.

ve.ce.View.js:
* Replace renderAttributes() with a simple wrapper around
  renderHtmlAttributeList()

ve.dm.Converter.js:
* Add buildHtmlAttributeList() and renderHtmlAttributes() for building
  and rendering HTML attribute lists

ve.dm.Model.js:
* Add getter for .htmlAttributes

ve.dm.Node.js:
* Drop .htmlAttributes on clone, and remove logic dropping html/*

ve.ui.MWCategoryWidget.js:
* Remove html/0/about hack, was already unnecessary and now doesn't
  work any more

tests/:
* UPDATE ALL THE TESTS

Change-Id: I620573afd70d36ade6b80413075b6e1f4a435abe
2013-05-17 20:57:33 -07:00
Inez Korczyński 531c049679 Support for nodes that handles their own children in ve.dm.Converter.prototype.getDomSubtreeFromData
Change-Id: I784dec71d3f0b23001075234e324a8e06a6cfc6b
2013-05-16 16:01:46 -07:00
Inez Korczyński abe81d42d7 Added method getDataFromDomRecursionClean to ve.dm.Converter - used by nodes that handle their own children.
Change-Id: I604cf201515152824ec159b8c86f2010afcd3369
2013-05-15 10:02:08 -07:00
Ed Sanders 328dd8e21f Whitespace between unwrapped inline nodes assigned to paragraph
In stopWrapping we assign any left over whitespace to the paragraph
in position 3, however we weren't clearing this whitespace buffer
if an inline content node followed it.

Change-Id: I8b3ee3915044abd6bafda386430bf7f992ca4aa8
2013-05-10 16:40:11 +01:00
Catrope 31c2165770 Also render attributes of the form html/i-j/attrName in CE
Factored the parsing of html/* attributes out into a static function.
Factored attribute (re)rendering out into ce.View, attribute updates
are much simpler now.

Change-Id: I4caa6d5e1e2c21c28ddff61c3c864e47f66cc6b2
2013-05-08 18:10:46 +00:00
Catrope 05828cc3f1 Preserve HTML attributes recursively
For nodes that handle their own children (as well as leaf nodes and
meta items), store the first child's attributes in html/0-0/*, the
second child's attributes in html/0-1/*, the second element's third
child's fourth child's attributes in html/1-2-3/* , etc.

This obsoletes the ad-hoc code that basically did the same thing in
MWInlineImageNode.

Change-Id: If5abd2d5d9c361b359617ff4b0f3d6ba4c9b0142
2013-05-08 11:10:07 -07:00