To help the selective serialiser we can return the original
HTML for generated content if it is unmodified.
As the output of toDomElements now depends on changes
to the dataElement we now have a 'modify' function in
the some test cases.
We also now have 'storeItems' to assert that the index-value
store is correctly populated and for loading values back
into the store for toDomElements tests.
Also make 'mw' an attribute and remove 'about' property.
Bug: 47394
Change-Id: I2bbb5d2d6a90c4eb87fa129671112c92a9b931e7
Also add detection for whether the browser is actually broken (most are,
but some, like Opera, aren't), treat <textarea> and <listing> in addition
to <pre>, and fix a bug where the function would crash if the <pre> was
empty (because .firstChild was undefined/null).
Change-Id: I541b57e9fd5c9c42d19d0a59f6e29fb43d35c9b6
By removing the transaction listeners from surface fragments we
no longer have to make sure they are always manually destroyed.
In order to retain the functionality of having fragments update
with transactions elsewhere we keep a pointer to a place in the
new complete history stack in the surface. The complete history
stack records all transactions, even undone ones.
Whenever getRange is called we replay all transactions in the
complete history (in the correct order) since the fragment was
last updated.
Also in this commit:
* Updated Format/IndentationAction to test undo(). This increases
coverage of surface fragment behaviour.
* .range is always accessed by .getRange now, although as an
optimisation we can use the noCopy mode when we a sure the
returned range will not be modified.
* Added undo test to .update (previously .onTransact)
Bug: 47343
Change-Id: I9e9818da1baa8319a3002f6d74fd1aad6732a8f5
This was broken, especially in wrappers.
Changed the wrapping algorithm so that meta items are placed outside
wrappers if possible. On the left-hand side, this is already the case:
we don't open wrappers for meta items. On the right-hand side, this is
accomplished by buffering the meta items and only inserting them when
we encounter either real text (not whitespace) or the end of the wrapper.
If we're interrupted by real text, we insert the meta items with the
unmodified whitespace. If we're interrupted by the end of the wrapper,
we insert the meta items outside of the wrapper with whitespace stripped.
Internally, this is done by stripping the whitespace into the whitespace[0]
of the meta item to its right. Then when we output the meta items, we
either decide to 'restore' the whitespace, or to 'fixup' by also setting
whitespace[3] on the element before the whitespace.
Change-Id: Ibeea2a9906c4aae9fe6d284613edd6ec853ca5e7
Before, it took an array of objects and translated those to indexes
using the store. Literally every caller outside of the test suite got
an array of indexes from the linear model, translated those to objects,
then passed them into the AnnotationSet constructor which translated
them right back to indexes.
The previous behavior was kind of ridiculous on its face, but the
reason we found it is because Inez was investigating the performance
degradation when bolding a line and found that half of it was due
to the hundreds of ve.getHash() calls caused by this behavior.
Change-Id: I38df8ae9f6392849dacf477ea2f804283c964417
Context-sensitive nodes are HTML elements like <caption> that can only
appear in certain contexts (<table> in this case). This means that
serializing them by throwing them in a <div> and calling .innerHTML
fails, because the browser knows a <caption> can't be in a <div> and
unwraps it. jQuery's .html() function is clever though and knows to wrap
<caption> in a <table> (and has similar rules for other elements).
So use jQuery's .html() rather than manual .innerHTML stuff.
Change-Id: Id7d3eff968b3a2ba345680772f7cc32e3dcdb529
In HTML like <td>Foo <b>Bar</b></td>, the space would be stripped and
registered as trailing whitespace in the <td>, so it wouldn't be visible
in the editor and would be inserted after the </b> on the way out.
Thanks to Subbu for reporting this. This case was kind of ridiculous and
we're lucky the JRuby article contained it. To trigger the bug, you had
to have:
* a table cell
* containing unwrapped content
* consisting of
** some text
** whitespace
** open annotation (bold in my example, link in Subbu's case)
** text
** close annotation
** and nothing else
Change-Id: I2b83f02764b311a32a50956d4c8930a9394e91a4
* Create MWTemplateBlockNode & MWTemplateInlineNode (in ce and dm)
* Move Alien's 'isInline' code to ve.dm.Node.static.isHybridInline
* Move definition of ce.AlienBlock/InlineNode inside ce.Aline.js file
to match dm.AlienBlock/InlineNode and MWTemplate
* Duplicate AlienBlock/Inline styles for templates
* Create test case for inline templates
* Count test cases in ve.dm.Converter.test.js automatically
Change-Id: Id9bc7f049ea974dd5e7f8b7a66080939e0948bbd
This node stores the rendered in the index-value store, hashed on
a custom hash of the dm (type + mw) which makes it unique it its
parameters.
Bug: 46571
Change-Id: I0ab4c9f7bca207121d5b42e83c821771b6139da8
ve.ce.ImageNode.js
* Moved in generic stuff from MWImageNode
* Added drag end handler (empty, will be used soon)
ve.ce.MWImageNode.js
* Changed to inherit ImageNode
* Moved generic stuff out
ve.dm.ImageNode.js
* Added attribute extraction/preservation for src, width and height
ve.dm.MWImageNode.js
* Changed to inherit ImageNode
* Re-using ImageNode's attribute handling to extract/preserve attributes on both the image and wrapper level
Change-Id: Ied4e1ece24e6804220eac35330790f7084df55de
This changes the annotation API to be the same as the node API, sans
a few boolean flags that don't apply. The APIs were different, but
there was really no good reason why, so this makes things simpler for
API users. It also means we'll be able to factor a bunch of things out
because they're now duplicated between nodes, meta items and annotations.
Linear model annotations are now objects with 'type' and 'attributes'
properties (rather than 'name' and 'data'), for consistency with elements.
They now also contain html/0/* attributes for HTML attribute preservation,
which obsoletes the htmlTagName and htmlAttributes properties.
dm.Annotation subclasses take a reference to such an object and implement
conversion using .static.toDataElement and .static.toDomElements just
like nodes do. The custom .getHash() functions are no longer necessary
because of the way HTML attribute preservation was reimplemented.
CE rendering has been moved out of dm.Annotation (it never made sense to
have CE rendering functions in DM classes, this was bothering me) and into
separate ce.Annotation subclasses. These are very similar to CE nodes in
that they have a this.$ generated based on something in the DM; the main
difference is that nodes listen to events and update themselves, whereas
annotations are static and are simply destroyed and rebuilt when they
change. This change also adds whitelisted HTML attribute rendering for
annotations, as well as class="ve-ce-FooAnnotation" attributes.
Now that annotation classes produce real DOM nodes rather than weird
objects describing HTML tags, we can't generate HTML as a string in
ce.ContentBranchNode anymore. getRenderedContents() has been rewritten
to be much more similar to the way the converter renders annotations;
in fact, significant parts of it were copied from the converter, so that
should be factored out in the future. This change actually fixes an
annotation rendering discrepancy between ce.ContentBranchNode and
dm.Converter; see the diff of ve.ce.ContentBranchNode.test.js.
ve.ce.MWEntityNode.js:
* Remove stray property
ve.dm.MWExternalLinkAnnotation.js:
* Store 'rel' attribute
ve.dm.TextStyleAnnotation.js:
* Put all the conversion logic in the abstract base class
ve.dm.Converter.js:
* Also feed annotations through getDomElementsFromDataElement() and
createDataElement()
ve.dm.Node.js:
* Fix undocumented property
ve.ce.ContentBranchNode.test.js:
* Add descriptive messages for each test case
* Compare DOM trees, not HTML strings
* Compare without all the class="ve-ce-WhateverAnnotation" clutter
ve.ui.LinkInspector.js:
* Replace direct .getHash() calls (evil!) with ve.getHash()
Bug: 46464
Bug: 44808
Change-Id: I31991488579b8cce6d98ed8b29b486ba5ec38cdc
* In matchTypeRegExps(), skip string types
** Didn't break because we currently have mixes of strings and regexes
* Combine types from rel, typeof and property rather than picking one
** Added test case in ve.dm.example that resembles actual Parsoid output
* If the element has extension-specific types, not only restrict type
matching to extension-specific types, but also require that *all* types
present on the element be matched
Change-Id: Iacf3851a0ca9081d2c813b42435484a47cec6230
Created an IndexValueStore class which can store any object and return
an integer index to its hash map.
Linear data is now stored in ve.dm.LinearData instances. Two subclasses
for element and meta data contain methods specific to those data types
(ElementLinearData and MetaLinearData).
The static methods in ve.dm.Document that inspected data at a given
offset are now instance methods of ve.dm.ElementLinearData.
AnnotationSets (which are no longer OrderedHashSets) have been moved
to /dm and also have to be instantiated with a pointer the store.
Bug: 46320
Change-Id: I249a5d48726093d1cb3e36351893f4bff85f52e2
Using element.height was returning 0 if the attribute was empty
when in fact what we mean to store is null (i.e. auto height).
This takes care of the writing of attributes in CE as jQuery
ignores an attribute-set command if the value is null.
Also in this commit I've implemented a basic toDomElements
that outputs the original HTML (code copied from AlienNode).
This stops the code from throwing an exception but will
eventually need to be rewritten to rebuild the HTML from
the attributes stored in the DM.
Bug: 56336
Change-Id: I297a1d0a07e9ebf9d0110fb1cdf266f8415f25b7
Rather than meta-things being special kinds of nodes, they are now a
separate class of things (MetaItems) along with Nodes and Annotations.
* Created a generic ve.dm.MetaItem that meta items inherit from.
There will be actual instances of this class as well in the upcoming
meta group code.
* Renamed MetaNode to AlienMetaItem, MWMetaNode to MWMetaItem,
'metaBlock'/'metaInline' to 'alienMeta'
* Created a MetaItemFactory, handle meta items in the ModelRegistry
* Kill ve.dm.Node.static.isMeta, now obsolete
ve.dm.Converter:
* Pass in the MetaItemFactory
* Look up data element types in the ModelRegistry rather than the
NodeFactory, because they can be either nodes or meta items
* Document createDataElement() and make explicit that modelClass can be
either a node or a meta item
* Handle meta items in getDataFromDom()
* In getDomFromData(), check the MetaItemFactory as well as the NodeFactory
Change-Id: I893709c6f3aa00f85c1b905b70f9f4e597bdeada
isolateAndUnwrap now unwraps to a level determined by a target type
i.e. the type you are going to convert to.
Also in this commit wrap/unwrap/rewrap have been refactored to use
getLengthDifference. unwrap now takes an inner/outer unwrap depth.
Change-Id: I3c6249de43232a9ef64f498a0aaf66b1c44973f2
This changes the node API to work with multiple elements, so we can
support about groups. Instead of passing in and returning single DOM
elements, we use arrays of DOM elements.
ve.dm.Converter:
* Pass modelRegistry into the constructor
* Remove onNodeRegister handler and its data
* Remove getDataElementFromDomElement() and
getDataAnnotationFromDomElement(). Most logic moved into
getDataFromDom(), some into createDataElement()
* Remove createAlien(), replaced with
createDataElement( ve.dm.AlienNode, ... )
* Replace doAboutGrouping() (which wrapped about groups) with
getAboutGroup() (which returns an array of the nodes in the group)
* Put in a hack so <meta>/<link> elements with an mw: property aren't
alienated
* Remove about group wrapping behavior in favor of just outputting
multiple nodes
ve.dm.AlienNode.js:
* For multi-element aliens, only choose inline if all elements are
inline, not just the first one
ve.dm.example.js:
* Add html/0 stuff for meta nodes
* Fix test case to reflect new alien behavior
Change-Id: I40dcc27430f778bc00a44b91b7d824bfb2718be6
The Parsoid output will also be expected to be a full HTML document. For
backwards compatibility, we allow for the Parsoid output to be a
document fragment as well. We don't send a full document back yet, also
for b/c -- we'll change this later once Parsoid has been updated in
production.
ve.dm.Converter.js:
* Make getDataFromDom() accept a document rather than a node
** Split off the recursion (which does use nodes) into its own function
** For now we just convert the <body>. In the future, we'll want to do
things with the <head> as well
* Pass the document around so we can use it when creating elements
* Make getDomFromData() return a document rather than a <div>
ve.init.mw.Target.js:
* Store a document (this.doc) rather than a DOM node (this.dom)
* Pass around documents rather than DOM nodes
* Detect whether the Parsoid output is an HTML document or a fragment
using a hacky regex
* When submitting to Parsoid, submit the innerHTML of the <body>
ve.init.mw.ViewPageTarget.js:
* s/dom/doc/
* Store body.innerHTML in this.originalHtml
ve.Surface.js:
* s/dom/doc/
demos/ve/index.php:
* Don't wrap HTML in <div>
* Pass HTML document rather than DOM node to ve.Surface
ve.dm.Converter.test.js:
* Construct a document from the test HTML, rather than a <div>
ve.dm.example.js:
* Wrap the HTML in the converter test cases in <body> tags to prevent
misinterpretation (HTML fragments starting with comments, <meta>,
<link> and whitespace are problematic)
Change-Id: I82fdad0a099febc5e658486cbf8becfcdbc85a2d
This means that <p data-foo="bar"> will now be converted to a paragraph
with attributes {"html/0/data-foo":"bar"} rather than {"html/foo":"bar"}
This paves the way for multi-element node (about group) handling in the
node API: nodes representing multiple DOM elements will have html/i/attr
to represent an attribute of the i'th DOM element.
Change-Id: Iea52bdccd721942ca708c8f9f47e934524809845
When encountering an inline node (i.e. content node that's not a text
node) within a branch node that's not a content branch node, the
converter should start a wrapper. But it doesn't do this, it only opens
wrappers for text nodes and annotations.
Fixed this in the converter, added a test for it, and fixed an existing
test that asserted the broken behavior.
Change-Id: I6e143e21e68b68f0d85b8772e24a2d3a5d465410
* Introduce context object as specified for
ve.dm.Node.static.toDataElement()
* Remove wrapping variable in favor of context.wrapping
* Remove wrappingIsOurs in favor of context.canCloseWrapper
* Introduce originallyExpectingContent and use it to repopulate
context.expectingContent after closing a wrapper
* Replace most uses of branchHasContent with context.expectingContent
** Except for two cases where we need originallyExpectingContent
These changes fix a case where a metaBlock was generated in an inline
position. Updated the tests to reflect this.
Change-Id: I6baf6053f8a3a0b7d91487f812b9235a7b2b3db1
* Made method descriptions imperative: "Do this" rather than "Does this"
* Changed use of "this object" to "the object" in method documentation
* Added missing documentation
* Fixed incorrect documentation
* Fixed incorrect debug method names (as in those VeDmClassName tags we add to functions so they make sense when dumped into in the console)
* Normalized use of package names throughout
* Normalized class descriptions
* Removed incorrect @abstract tags
* Added missing @method tags
* Lots of other minor cleanup
Change-Id: I4ea66a2dd107613e2ea3a5f56ff54d675d72957e
See CODING.md for how to run it.
Mistakes fixed:
* Warning: Unknown type function
-> Function
* Warning: Unknown type DOMElement
-> HTMLElement
* Warning: Unknown type DOM Node
-> HTMLElement
* Warning: Unknown type Integer
-> Mixed
* Warning: Unknown type Command
-> ve.Command
* Warning: Unknown type any
-> number
* Warning: Unknown type ve.Transaction
-> ve.dm.Transaction
* Warning: Unknown type ve.dm.AnnotationSet
-> ve.AnnotationSet
* Warning: Unknown type false
-> boolean
* Warning: Unknown type ve.dm.AlienNode
ve.dm doesn't have a generic AlienNode like ve.ce
-> Unknown type ve.dm.AlienInlineNode|ve.dm.AlienBlockNode
* Warning: Unknown type ve.ve.Surface
-> ve.ce.Surface
* ve.example.lookupNode:
-> Last @param should be @return
* ve.dm.Transaction.prototype.pushReplace:
-> @param {Array] should be @param {Array}
* Warning: ve.BranchNode.js:27: {@link ve.Node#hasChildren} links to non-existing member
-> (removed)
* Warning: ve.LeafNode.js:21: {@link ve.Node#hasChildren} links to non-existing member
-> (removed)
Differences fixed:
* Variadic arguments are like @param {Type...} [name]
instead of @param {Type} [name...]
* Convert all file headers from /** to /*! because JSDuck tries
to parse all /** blocks and fails to parse with all sorts of
errors for "Global property", "Unnamed property", and
"Duplicate property".
Find: \/\*\*([^@]+)(@copyright)
Replace: /*!$1$2
* Indented blocks are considered code examples.
A few methods had documentation with numbered lists that were
indented, which have now been updated to not be intended.
* The free-form text descriptions are parsed with Markdown,
which requires lists to be separated from paragraphs by an
empty line.
And we should use `backticks` instead of {braces} for inline
code in text paragraphs.
* Doc blocks for classes and their constructor have to be
in the correct order (@constructor, @param, @return must be
before @class, @abstract, @extends etc.)
* `@extends Class` must not have Class {wrapped}
* @throws must start with a {Type}
* @example means something else. It is used for an inline demo
iframe, not code block. For that simply indent with spaces.
* @member means something else.
Non-function properties are marked with @property, not @member.
* To create a link to a class or member, in most cases the name
is enough to create a link. E.g. Foo, Foo.bar, Foo.bar#quux,
where a hash stands for "instance member", so Foo.bar#quux,
links to Foo.bar.prototype.quux (the is not supported, as
"prototype" is considered an implementation detail, it only
indexes class name and method name).
If the magic linker doesn't work for some case, the
verbose syntax is {@link #target label}.
* @property can't have sub-properties (nested @param and @return
values are supported, only @static @property can't be nested).
We only have one case of this, which can be worked around by
moving those in a new virtual class. The code is unaltered
(only moved down so that it isn't with the scope of the main
@class block). ve.dm.TransactionProcessor.processors.
New:
* @mixins: Classes mixed into the current class.
* @event: Events that can be emitted by a class. These are also
inherited by subclasses. (+ @param, @return and @preventable).
So ve.Node#event-attach is inherited to ve.dm.BreakNode,
just like @method is.
* @singleton: Plain objects such as ve, ve.dm, ve.ce were missing
documentation causing a tree error. Documented those as a
JSDuck singleton, which they but just weren't documented yet.
NB: Members of @singleton don't need @static (if present,
triggers a compiler warning).
* @chainable: Shorthand for "@return this". We were using
"@return {classname}" which is ambiguous (returns the same
instance or another instance?), @chainable is specifically
for "@return this". Creates proper labels in the generated
HTML pages.
Removed:
* @mixin: (not to be confused with @mixins). Not supported by
JSDuck. Every class is standalone anyway. Where needed marked
them @class + @abstract instead.
Change-Id: I6a7c9e8ee8f995731bc205d666167874eb2ebe23
This happens when the <span> is the start of unwrapped content. The
converter logic to look at the tag name in wrapping mode doesn't kick in
because we're not yet in wrapping mode at that point.
The core issue was that previously, we relied on the document
structure/state to choose between alienBlock and alienInline, and only
used the tag name where the document structure was ambiguous (wrapping).
Changed this to be the other way around: we now rely primarily on the
tag name, and if that doesn't match what we expect based on the document
structure, we work around that if possible. Specifically:
* inline tag in our wrapper --> inline alien
* block tag in our wrapper --> close wrapper, block alien
* inline tag in wrapper that's not ours --> inline alien
* block tag in wrapper that's not ours --> *inline* alien
* inline tag in structural location --> open wrapper, inline alien
* block tag in structural location --> block alien
* inline tag in content location --> inline alien
* block tag in content location --> *inline* alien
only in the fourth and the last case do we need to use the "wrong" alien type to
preserve document validity, and it will always be inline where block was
expected, which should reduce UI issues.
The condensed version of the above, which is used in the code, is:
* If in a non-wrapper content location, use inline
* If in a wrapper that's not ours, use inline
* Otherwise, decide based on tag name
* Open or close wrapper if needed
ve.dm.Converter:
* Replace isInline logic in createAlien() with the above
* Factor out code to start wrapping (was duplicated) into startWrapping()
* Call startWrapping() if createAlien() returns an alienInline and we're
in a structural location
Tests:
* Add test cases with aliens at the start and end of unwrapped content
** The first one failed prior to these changes and now passes, the
second one was already passing
* Fix about group test case, was exhibiting the bug that this commit fixes
Change-Id: I657aa0ff5bc2b57cd48ef8a99c8ca930936c03b8
I noticed this bug on [[List of Presidents of the United States]]. When
there's HTML that looks like "<td>Foo\n<meta/></td>", the converter will
collect the newline in wrappedWhitespace, then attempt to splice it out
and store it in internal data. But instead, it ends up splicing out the
/metaBlock element, which causes strange unbalanced input, which causes
an empty table in the node tree.
Change-Id: I79ed2fa9a834cc42759c7d21250d8842f563d38f
The converter was misbehaving when handling <p>s inside <span>s. This
can't be expressed in the linmod, but it would try to anyway. <span><p>
would result in too many paragraph closing elements, leading to an
exception in ve.dm.Document complaining about unbalanced input.
<span>\n<p> would result in an exception in the converter itself while
trying to perform whitespace preservation on the newline.
This change makes the converter detect these scenarios and alienate the
offending node. So <span><p>Foo</p></span> converts to a wrapper
paragraph containing an alienInline whose HTML is "<p>Foo</p>" and which
is annotated with a TextStyleSpanAnnotation.
ve.dm.Converter.getDomFromData():
* Change the criteria for alienBlock vs alienInline
** Only infer from the node type if we're in wrapping mode AND we're at
the same level where the wrapping started (wrappingIsOurs). If the
latter isn't the case, we can't split the wrapper in the block case
because we're at the wrong level.
** Use alienInline not only if the branch is a content branch, but also
if there are active annotations. This catches e.g. <li><b><p>
(and generally <span><p> on the top level).
* Before converting a child element, check that the child isn't "bad".
Bad children are non-content children in content branches, and
non-content children encountered within a wrapper that we can't split.
Only good children are converted, and bad children are alienated (cue
Santa/Sinterklaas jokes).
* Add childIsContent and rename branchIsContent to branchHasContent
Change-Id: If420ae80ab0777424a9a5517335ef9d0170e87ae
HTML DOM has annoying behavior for <pre>s where .innerHTML eats the
first newline in a <pre>. Work around this by explicitly adding a
newline in the data->DOM converter if the <pre> already contained a
newline.
There is a separate bug in Parsoid that causes the newline to be lost
anyway, filed as bug 42666
Change-Id: Ia26cd4a4c61afbe439b0562deb7f24ee8b8147d7
Was broken both on the way in and on the way out.
* Move alien restoration (data->DOM) out of the main getDomFromData()
function and into getDomElementFromDataElement(). This means the
comment about District 9 is gone (sniff), but moving this here ensures
all code paths hit it (previously, it was assumed annotated nodes
could never be aliens).
* In the DOM->data converter, add annotation application to
getDataElementFromDomElement() (for content nodes) and createAlien()
(for aliens). Previously, these nodes would not get annotations.
** ve.AnnotationSet doesn't have a constructor that takes an array, we
should fix that.
Change-Id: I65f8e9a322111ca3af275bf9997b0b1e7ee93769
This is cleaner than passing around the attributes separately, and it
allows us to access the annotations in dm.LeafNode as well.
Change-Id: Ie5b90988114835831cbe5cdccf63c7cd45719e31
When editing a new page, or loading an empty page into the editor, the
converter generates a paragraph so the document isn't completely empty.
This paragraph is then unwrapped on the way out, potentially destroying
change markers and generally producing strange HTML output.
Mark this paragraph with generated=empty rather than generated=wrapper,
and only unwrap it on the way out if it's still empty. This means we
cleanly round-trip empty documents (and empty list items and the like),
but if the user enters text, we create a paragraph like we're supposed
to.
Change-Id: Id0241221a67b769445676b833b5741320d99ea5f
