The converter was misbehaving when handling <p>s inside <span>s. This
can't be expressed in the linmod, but it would try to anyway. <span><p>
would result in too many paragraph closing elements, leading to an
exception in ve.dm.Document complaining about unbalanced input.
<span>\n<p> would result in an exception in the converter itself while
trying to perform whitespace preservation on the newline.
This change makes the converter detect these scenarios and alienate the
offending node. So <span><p>Foo</p></span> converts to a wrapper
paragraph containing an alienInline whose HTML is "<p>Foo</p>" and which
is annotated with a TextStyleSpanAnnotation.
ve.dm.Converter.getDomFromData():
* Change the criteria for alienBlock vs alienInline
** Only infer from the node type if we're in wrapping mode AND we're at
the same level where the wrapping started (wrappingIsOurs). If the
latter isn't the case, we can't split the wrapper in the block case
because we're at the wrong level.
** Use alienInline not only if the branch is a content branch, but also
if there are active annotations. This catches e.g. <li><b><p>
(and generally <span><p> on the top level).
* Before converting a child element, check that the child isn't "bad".
Bad children are non-content children in content branches, and
non-content children encountered within a wrapper that we can't split.
Only good children are converted, and bad children are alienated (cue
Santa/Sinterklaas jokes).
* Add childIsContent and rename branchIsContent to branchHasContent
Change-Id: If420ae80ab0777424a9a5517335ef9d0170e87ae
HTML DOM has annoying behavior for <pre>s where .innerHTML eats the
first newline in a <pre>. Work around this by explicitly adding a
newline in the data->DOM converter if the <pre> already contained a
newline.
There is a separate bug in Parsoid that causes the newline to be lost
anyway, filed as bug 42666
Change-Id: Ia26cd4a4c61afbe439b0562deb7f24ee8b8147d7
Was broken both on the way in and on the way out.
* Move alien restoration (data->DOM) out of the main getDomFromData()
function and into getDomElementFromDataElement(). This means the
comment about District 9 is gone (sniff), but moving this here ensures
all code paths hit it (previously, it was assumed annotated nodes
could never be aliens).
* In the DOM->data converter, add annotation application to
getDataElementFromDomElement() (for content nodes) and createAlien()
(for aliens). Previously, these nodes would not get annotations.
** ve.AnnotationSet doesn't have a constructor that takes an array, we
should fix that.
Change-Id: I65f8e9a322111ca3af275bf9997b0b1e7ee93769
This is cleaner than passing around the attributes separately, and it
allows us to access the annotations in dm.LeafNode as well.
Change-Id: Ie5b90988114835831cbe5cdccf63c7cd45719e31
When editing a new page, or loading an empty page into the editor, the
converter generates a paragraph so the document isn't completely empty.
This paragraph is then unwrapped on the way out, potentially destroying
change markers and generally producing strange HTML output.
Mark this paragraph with generated=empty rather than generated=wrapper,
and only unwrap it on the way out if it's still empty. This means we
cleanly round-trip empty documents (and empty list items and the like),
but if the user enters text, we create a paragraph like we're supposed
to.
Change-Id: Id0241221a67b769445676b833b5741320d99ea5f
When alienating in wrapping mode, we need to look at the type of tag to
decide whether to create a wrapped alienInline, or to interrupt the
paragraph for an alienBlock.
This was being done just fine for the general alienation case
(unrecognized tag), but not for the special cases (mw:unrecognized,
about groups).
* Centralize the logic for ending a wrapper in stopWrapping()
* Move the wrapping-contingent block/inline detection logic into
createAlien()
* Simplify the terrible if statement to decide whether a future decision
requires us to stop wrapping. Instead, detect the cases in each code
path separately and call stopWrapping() as appropriate
* Add tests
Change-Id: I4054584ae05e7d5daa71edead3e6a6588cf5d3bb
<span typeof="mw:Entity"> tags are now correctly represented in the
model, and rendered in CE. There are still issues with cursor movement
etc. in CE.
Because the prioritization mechanism for annotations vs nodes is broken
in the current "node API", I had to hack two special cases for mw:Entity
into the converter. I also had to change the converter to ignore the
children of inline nodes (this was a legitimate bug, but had never come
up before).
Change-Id: Ib9f70437c58b4ca06aa09f7272bf51d9c41b18f2
* Make converter generate meta nodes with 'style': 'comment'
* Handle style==='comment' in MetaBlockNode toDOM converter
* Add some comments to the meta test case
** Update other tests accordingly
* Change getDomElementSummary() to actually assert presence of comment
nodes (specifically, all non-text child nodes)
Change-Id: Ieef9418f4c47df3541477d9420aa2ab8df6e3df1
Editing the text of a list item results in a change marker on the
paragraph within that list item. However, that paragraph usually isn't
present in the HTML, so the converter unwraps it when converting back to
HTML, and the change markers are lost. Instead, transfer the change
markers to the <li>.
Change-Id: Id675075d19c08d69bc8e990174841dc393b749fc
About groups are HTML structures like the following:
<div about="#mwt1">....</div>
<span about="#mwt1">...</span>
<div about="#mwt1">...</div>
When about groups are alienated, they are now merged into one alien
node, rather than producing a separate alien node for each sibling.
This is very basic about group handling, because it only works for
groups of directly adjacent siblings (text nodes are permitted in
between, but nothing else) assumes all about groups are aliens (which
is currently true).
* Before processing an element in the DOM->data converter, perform about
grouping on its children. This temporarily wraps about groups in
<div data-ve-aboutgroup="value of about attribute">
* Extended createAlien() to handle single nodes as well as wrappers
holding multiple nodes.
* In the data->DOM converter, temporarily wrap multi-node aliens in
<div data-ve-multi-child-alien-wrapper="true"> . This makes the rest
of the algorithm easier.
Change-Id: I2df5f62bc222b570fc11a89fe43d353f8363ead8
This causes the converter not to strip inner whitespace in them, and
causes CE to suppress the whitespace mangling logic that is normally
applied (↵ for newlines, ➞ for tabs, alternating s for spaces).
Change-Id: I738a750c91a4ca4836c485e282865bb7525bf30a
* Add map of change markers per offset to Transaction
* Map is populated by TransactionProcessor
* Markers are reversed on rollback
* Removals aren't marked, Parsoid can detect these using DSR
discontinuities
Change-Id: I2290886ab411c6ad6162044ed85c091313613e51
* ve.dm.Converter still generates metaInline/metaBlock elements as
before, it's not affected by this change
* ve.dm.Document constructor splits its input into "real" data and
metadata
** Metadata is stored in this.metadata (the meta-linmod) as a sparse
array of arrays, with an element for each offset in this.data
** this.data itself does not contain the metadata
** This means the node tree also doesn't contain the metadata
** Which means CE doesn't know about it at all
* All splice operations on the linear model are sent through
ve.dm.Document.spliceData(), which performs the splice and syncs the
meta-linmod
** Metadata in the removed range is reaped and added to the metadata for
the offset immediately following the removal
* ve.dm.Document.getFullData() splices the linmod and meta-linmod back
into each other; this "full data" is then fed back to ve.dm.Converter
Change-Id: Ief6dfd5b59cc13a8457993ed85c725413029c4fb
The HTML "1<br/>2" was being converted to a linmod that looked like
"<p>1</p><br></br><p>2</p>". This commit fixes the wrapping logic such
that the result is "<p>1<br></br>2</p>" instead. In general, inline
nodes (content nodes) should not interrupt the wrapping, but block nodes
should.
This creates a problem for alien nodes: normally, we determine whether an
alien node is a block alien or an inline alien based on context, but if
we're in wrapping mode we're unsure of the context. We can't tell the
difference between "1<tt>Foo</tt>2" (should be wrapped as one, because
tt is inline) and "1<figure></figure>2" (1 and 2 should be wrapped
separately, because figure is block) using context alone, so in these
cases (and ONLY in these cases) we look up whether the HTML tag in
question is an inline tag or a block tag and use that to decide.
Change-Id: I75e7f3da387dd401d9b93e09a21751951eccbb83
This node type represents <meta> or <link> (transparently, based on the
style attribute). I had to make two node types for this and hack the
toData conversion code directly into ve.dm.Converter, because we don't
have native support for node types that can be both inline and block.
(We should add this in the node API rewrite.)
The CE implementation renders a placeholder (with the same styles as an
alien node) right now. I'm not sure how nice that is, but it's better
than rendering raw <meta>/<link> tags.
This whole thing is a total pile of hacks to make VE deal with
<meta>/<link> tags until we have a proper node types API.
Change-Id: Id6783fcfc35a896db088ff424ff9faaabcaff716
This was broken in three different ways:
* On the way in, we were applying whitespace to an array of elements
rather than the actual element, so the whitespace wasn't stored.
* Whitespace processing on the way out was skipped for aliens because
they had their own code path. Refactored this so alien openings and
regular openings share much more code, including whitespace output.
* Somewhat unrelatedly, innerPost output was broken for paragraphs
containing inline elements, because the inline elements' processing
polluted lastOuterPost. Discovered this because my test with inline
aliens also happened to be the first test of whitespace preservation
in paragraphs with inline content elements. Fixed by explicitly
skipping content nodes when outputting whitespace.
Fixed these issues and added a test case.
Change-Id: I8edb61a008e60ace886b1a841b3417682ec39c32
Introduced the ve.AnnotationSet class to manage sets of annotations. This
is a generalization of ve.OrderedHashSet, a class that manages a set
using an array and an object keyed by hash.
Converted everything that stores, tracks or passes around annotations to
use ve.AnnotationSet. In particular, this means the linear model now
contains AnnotationSets instead of hash-keyed objects.
This allows us to maintain the order of annotations in the linear model,
and will help fix bugs with annotation ordering and splitting.
Change-Id: I50975b0a95f4cc33017a0b59fdede9ed1eff0124
[jshint]
ce/ve.ce.Surface.js: line 670, col 9, Too many var statements.
ce/ve.ce.Surface.js: line 695, col 6, Missing semicolon.
ce/ve.ce.Surface.js: line 726, col 22, Expected '===' and instead saw '=='.
ce/ve.ce.Surface.js: line 726, col 41, Expected '===' and instead saw '=='.
ce/ve.ce.Surface.js: line 733, col 13, Too many var statements.
ce/ve.ce.Surface.js: line 734, col 24, Expected '===' and instead saw '=='.
ce/ve.ce.Surface.js: line 1013, col 13, Too many var statements.
ce/ve.ce.Surface.js: line 1019, col 17, Too many var statements.
ce/ve.ce.Surface.js: line 1023, col 18, Too many ar statements.
ce/ve.ce.Surface.js: line 1027, col 13, Too many var statements.
dm/annotations/ve.dm.LinkAnnotation.js: line 70, col 52, Insecure '.'.
dm/ve.dm.Converter.js: line 383, col 29, Empty block.
dm/ve.dm.Converter.js: line 423, col 33, Empty block.
Commands:
* jshint .
* ack '(if|else|function|switch|for|while)\('
* Sublime Text 2:
Find(*): (if|else|function|switch|for|while)\(
Replace: $1 (
* ack ' ' -Q # double spaces, except in certain comments
Change-Id: I8e34bf2924bc8688fdf8acef08bbc4f6707e93be
This commit fully utilizes all four positions in the internal.whitespace
array. Outer whitespace is now preserved as well, and is duplicated
either in the adjacent sibling (one node's outerPost is the next
sibling's outerPre) or in the parent (a branch node's innerPre is its
first child's outerPre, and its innerPost is its last child's
outerPost). Before restoring saved whitespace, we check that these two
agree with each other, and if they disagree we assume the user has been
moving stuff around and don't restore any whitespace in that spot. The
whitespace at the very beginning and the very end of the document (i.e.
the first node's outerPre and the last node's outerPost) isn't
duplicated anywhere, nor is inner whitespace in content nodes.
The basic outline of the implementation is:
* When we encounter whitespace, strip it and store it in the previous
node's outerPost. Also store it in nextWhitespace so we can put it in
the next node's outerPre once we encounter that node.
* When we encounter whitespace in wrapped bare text, we don't know in
advance if it's gonna be succeeded by more non-whitespace (in which
case it needs to be output verbatim), or not (in which case it's
leading whitespace and needs to be stripped and stored). The fact that
annotations are nodes in HTML makes this trickier. So we write the
whitespace to the temporary linmod and store it in wrappedWhitespace,
then if it turns out to be trailing whitespace we take it back out of
the data array and record it the usual way.
* Because text nodes can contain any combination of leading whitespace
actual text and trailing whitespace, and because we may or may not
already have opened a wrapping paragraph, there are a lot of different
combinations to handle. We handle all of them but the resulting code
is pretty dense and verbose.
More low-level list of changes:
In getDataFromDom():
* Added helper function addWhitespace() for storing whitespace for an
element
* Added helper function processNextWhitespace() for processing any
whitespace passed on from the previous node via the nextWhitespace var
* Rename paragraph to wrappingParagraph. Make wrapping default to
alreadyWrapped so we can simplify wrapping||alreadyWrapped and
!wrapping&&!alreadyWrapped. Add wrappingIsOurs to track whether the
wrapping originated in this recursion level (needed for deciding when
to close the wrapper).
* Add prevElement to track the previous element so we can propagate
whitespace to it, and nextWhitespace so we can propagate whitespace to
the next element.
* Remove previous newline stripping hacks
* Integrate the logic for wrapping bare content with the outer
whitespace preservation code
* Remove wrapperElement, no longer needed because we have a dedicated
variable for the wrapping paragraph now and what was previously inner
whitespace preservation for wrapper paragraphs is now covered by the
outer whitespace preservation code.
In getDomFromData():
* Reinsert whitespace where appropriate
** outerPre is inserted when opening the element
** This covers outerPost as well except for the last child's outerPost,
which is handled as the parent's innerPost when closing the parent.
** innerPre and innerPost are inserted when closing the element. Care is
taken not to insert these if they're duplicates of something else.
* Propagate each node's outerPost to the next node (either the next
sibling or the parent) using parentDomElement.lastOuterPost. We can't
get this using .lastChild because we will have destroyed that child's
.veInternal by then, and we can't tell whether a node will be its
parent's last child when we process it (all other processing,
including first child handling is done when processing the node itself,
but this cannot be).
* Special handling is needed for the last node's outerPost, which ends
up in the container's .lastOuterPost property.
Tests:
* Allow .html to be null in data<->DOM converter tests. This indicates
that the test is a one-way data->DOM test, not a DOM->data->DOM
round-trip test. The data will be converted to HTML and checked
against .normalizedHtml
* Update existing tests as needed
* Add tests for outer whitespace preservation and storage
* Add test for squashing of whitespace in case of disagreement (this
requires .html=null)
Change-Id: I4db4fe372a421182e80a2535657af7784ff15f95
data-mw-gc is ancient and unused. We do need to detect and alienate
generated nodes, but that is now based on RDFa types. Removing the
data-mw-gc stuff for now because it doesn't work anyway, will replace it
with proper detection later.
Replaced instances of data-mw-gc in the test suite with unregistered
node types.
Change-Id: If3f5898d382a436fa57929013264c53af5e840ba
domToData wraps bare content in paragraph elements, which were then
converted to <p> tags by domToData. With this fix, HTML with "missing"
<p> tags actually round-trips through the editor correctly now, rather
than having <p> tags added wherever VE believes they should exist.
* Mark generated paragraph elements with .internal.generated = 'wrapper'
** This signifies the wrapper was generated but its contents were not,
so the right thing to do when converting back to HTML is to remove
the wrapper and keep the contents. We might want to use other values
of generated in the future.
* Unwrap nodes with generated=wrapper when converting to HTML
Tests:
* Add 'generated': 'wrapper' as appropriate. Only affects 1 test
* Remove 'normalizedHtml' for this test because it is no longer needed
** Need to keep 'normalizedHtml' for now because we normalize hrefs
* Eventually the main example should test bare content, but that
requires touching a lot of stuff. The main example could use some
beefing up anyway.
Change-Id: I277ad5fe3f64e07c1bbf49007d6bbaecc90b7466
This allows us to put other internal data in there in the future. Also
passing it through the Node constructor properly now.
* ve.dm.Node
** Rename fringeWhitespace property to internal
** Add internal parameter to constructor
** Remove setFringeWhitespace()
* Increase the number of parameters passed through by ve.Factory to 3
* Pass through .internal from linmod to nodeFactory in ve.dm.Document
* ve.dm.Converter
** Rename .fringeWhitespace to .internal.whitespace and make it an array
** Store a temporary reference to .internal in domElement.veInternal
* Add internal to all node constructors except TextNode
Tests:
* Update for fringeWhitespace->internal rename
* Add third parameter to ve.Factory tests
* Add .internal to getNodeTreeSummary
Change-Id: If20c0bb78fee3efa55f72e51e7fc261283358de7
To do this, we are using the replacer callback of JSON.stringify, which is supported by all of our target browsers including IE8. We are also leaning on Object.keys and Array.reduce, the latter of which required adding a new fallback implementation for some browsers which do not support it yet.
Change-Id: Ifa285ca3da4d94d962464f09414591532bbea79c
Because the Parsoid prefix format changed from /mw:Foo to /mw/Foo , the
href format for internal links has changed from "/Foo" to "Foo". So the
href is now simply the title, except that it may be preceded by one or
more "../" if the title of the page we're on contains a '/'.
So instead of stripping the leading slash from internal link hrefs and
putting it back on the way out, only strip any leading "../"s and dump
the titles directly into the hrefs on the way out.
Also update the link test case for this, and add a test case for the ../
stripping.
Change-Id: I3e0bdde20f22cda34eb45fc351de5e780419b6a2
Annotation types with more than one slash such as 'link/ExtLink/URL'
weren't being processed correctly because .split( '/', 2 ) throws away
everything after the second slash. Instead, don't pass a limit to
.split(); the code for reconstructing a slash-separated string from
multiple components was already in place.
Also add test cases for URL links and numbered links.
(Do you like the lines-of-code to lines-of-test ratio in this commit,
Trevor? ;) )
Change-Id: I7add87396447a01b1c23a4f9bfd63d2e8fd861ce
This makes things like
== Foo ==
* Bar
render without the leading and trailing spaces, while still
round-tripping those spaces.
* Added a .fringeWhitespace property to the linear model and ve.dm.Node
** Object containing innerPre, innerPost, outerPre, outerPost
** Only inner* are used right now, outer* are planned for future use
** Like .attributes , it's suppressed if it's an empty object
* In getDataFromDom():
** Store the stripped whitespace in .fringeWhitespace
** Move emptiness check up: empty elements with .fringeWhitespace have
to be preserved
** Move paragraph wrapping up: .fringeWhitespace has to be applied to
the generated paragraph, not its parent
** Add wrapperElement to keep track of the element .fringeWhitespace has
to be added to; this is either dataElement or the generated paragraph
or nothing, but we can't modify dataElement because it's used later
* In getDomFromData():
** When processing an opening, store the fringeWhitespace data in the
generated DOM node
** When processing a closing, add the stored whitespace back in
* In the ve.dm.Document constructor, pass through .fringeWhitespace from
the linear model data to the generated nodes
Tests:
* Change one existing test case to account for this change
* Add three new test cases for this behavior
* Add normalizedHtml field so I can test behavior with bare content
Change-Id: I0411544652dd72b923c831c495d69ee4322a2c14
Convert underscores in the href attribute to spaces in the linear model,
and back to underscores when going back to HTML. This ensures the link
targets displayed to and edited by the user look nice
Change-Id: I4855fce28ad8b724284c53881abc7b99b59b9079
This means we don't have to rely on data-rt.sHref. It also means that
we'll now be showing the canonical link target in the link inspector
rather than the link target as entered by the user, but that's fine.
Also change test to have href differ from sHref to show that we use
href.
Change-Id: Idabdbf2579663ef1efb47d6a73f39743c9f64f3b
This is ugly but makes things work again. I intend to clean this up once
we have a better attribute API
* Recognize mw:WikiLink, mw:SimpleWikiLink, mw:ExtLink,
mw:NumberedExtLink and mw:UrlLink
* Support is incomplete because we can't get to the annotation text with
the current API
* Preserve all unhandled attributes rather than special-casing data-mw
* Update remaining code using data-mw (sHref and stx extraction) to
account for the data-mw -> data-rt rename
* Update tests accordingly
Change-Id: Ia13d3008a6d4cdc8828f9acda5aa797566bc597f
