Commit graph

155 commits

Author SHA1 Message Date
Catrope 05828cc3f1 Preserve HTML attributes recursively
For nodes that handle their own children (as well as leaf nodes and
meta items), store the first child's attributes in html/0-0/*, the
second child's attributes in html/0-1/*, the second element's third
child's fourth child's attributes in html/1-2-3/* , etc.

This obsoletes the ad-hoc code that basically did the same thing in
MWInlineImageNode.

Change-Id: If5abd2d5d9c361b359617ff4b0f3d6ba4c9b0142
2013-05-08 11:10:07 -07:00
Catrope 317a404ece Make .static.storeHtmlAttributes more versatile
It now allows you to specify which attributes to preserve in various
ways rather than just setting true or false.

Removed unused factory methods that exposed the old value.

Change-Id: I914164adcf1f0e48fa3fa85277e68c72dbad393e
2013-05-07 14:45:26 -07:00
jenkins-bot 76b277485e Merge "Fix placement of whitespace when element ends in meta" 2013-05-06 20:04:04 +00:00
jenkins-bot bf0a227fd6 Merge "Use a smarter comparison of annotations when creating open/close tags" 2013-05-06 18:52:30 +00:00
Ed Sanders 830de420e1 Code style fixes
* "function(" -> "function ("
* "{String}" -> "{string}"
* collapse unintentional double spaces

Change-Id: I3ce3f02d1e31d4797b44e04d28457dec363be296
2013-05-06 12:36:52 +01:00
Ed Sanders 8c87882633 Use a smarter comparison of annotations when creating open/close tags
Currently we just compare by store index, but a bold annotation
with data-parsoid attributes set should merge with a new clean bold
annotation. Similar rules apply to link annotations.

Bug: 48110
Change-Id: I93586919002c78732228e08b134e67e1a94f8ad7
2013-05-05 20:41:53 +01:00
Roan Kattouw ef24ac5879 Force about grouping for multi-element about groups
When converting an element that starts an about group with at
least one other element in it, we now only consider models that
support about grouping. This prevents the first node from being
converted to something else and leaving the others hanging.

In practical terms, this means that elements like
<link rel="mw:WikiLink/Category" typeof="mw:Object/Template">
get alienated and pull in the rest of their about group, rather than
being converted to a category or alienMeta or whatever and
leaving the other elements to be converted normally.

Added a test case that asserts this. Really the result should be an
MWtemplate rather than an alien, but that's a separate issue.
Also removed superfluous mustMatchAll checks; we've already
filtered the array by the time we get there.

Change-Id: I522ba4c56d5bc52c7e9aab1e2535385540c1315d
2013-05-05 00:07:29 -07:00
Ed Sanders 8cfb4ee62f Fix placement of whitespace when element ends in meta
This was broken for both normal elements, where the meta item is
inside the element, and wrapper paragraphs, where the meta item
gets moved outside the wrapper.

Bug: 47712
Change-Id: I42daaf142e548e5b221ff0a52df0ad24ec6a4fd0
2013-05-03 22:45:34 +01:00
Ed Sanders e29ed7f2b6 Remove all code related to change markers
Specifically by looking for "data-ve-changed",
"ChangeMarker*" and internal.changed.

Various tests, test counters and unused variables also
affected.

Bug: 45061
Change-Id: Ibd1ee68e0d650979d40574eff9cebded1a28499f
2013-04-30 23:15:47 +01:00
Ed Sanders 3d64c3043c Further AnnotationSet optimisation: create containsIndex
In most places we call .contains we already know the index, so we
can avoid store lookups by using .containsIndex.

Change-Id: I45a9a421473f9bec479ab8ccceceb162b7004c3a
2013-04-25 22:55:43 +01:00
Ed Sanders 6ad61d4ddb Add data model support for MediaWiki references
So far just read-only.

Bug: 39599
Change-Id: I6daff5c5969e5fdc871f8f346cf790b4302ae080
2013-04-23 10:17:42 +01:00
jenkins-bot 196123e7a5 Merge "MWTemplateNode should serialise original HTML if unchanged" 2013-04-22 21:15:41 +00:00
Ed Sanders 2bd6f8576a MWTemplateNode should serialise original HTML if unchanged
To help the selective serialiser we can return the original
HTML for generated content if it is unmodified.

As the output of toDomElements now depends on changes
to the dataElement we now have a 'modify' function in
the some test cases.

We also now have 'storeItems' to assert that the index-value
store is correctly populated and for loading values back
into the store for toDomElements tests.

Also make 'mw' an attribute and remove 'about' property.

Bug: 47394
Change-Id: I2bbb5d2d6a90c4eb87fa129671112c92a9b931e7
2013-04-22 20:44:21 +00:00
Catrope 3848c3f220 Factor the <pre> newline hack out of the converter into ve.properInnerHTML()
Also add detection for whether the browser is actually broken (most are,
but some, like Opera, aren't), treat <textarea> and <listing> in addition
to <pre>, and fix a bug where the function would crash if the <pre> was
empty (because .firstChild was undefined/null).

Change-Id: I541b57e9fd5c9c42d19d0a59f6e29fb43d35c9b6
2013-04-22 20:09:52 +01:00
Catrope 83a592f312 Fix whitespace preservation around meta items
This was broken, especially in wrappers.

Changed the wrapping algorithm so that meta items are placed outside
wrappers if possible. On the left-hand side, this is already the case:
we don't open wrappers for meta items. On the right-hand side, this is
accomplished by buffering the meta items and only inserting them when
we encounter either real text (not whitespace) or the end of the wrapper.
If we're interrupted by real text, we insert the meta items with the
unmodified whitespace. If we're interrupted by the end of the wrapper,
we insert the meta items outside of the wrapper with whitespace stripped.

Internally, this is done by stripping the whitespace into the whitespace[0]
of the meta item to its right. Then when we output the meta items, we
either decide to 'restore' the whitespace, or to 'fixup' by also setting
whitespace[3] on the element before the whitespace.

Change-Id: Ibeea2a9906c4aae9fe6d284613edd6ec853ca5e7
2013-04-18 16:06:58 -07:00
Catrope eac44c39f4 Make the AnnotationSet constructor take an array of indexes
Before, it took an array of objects and translated those to indexes
using the store. Literally every caller outside of the test suite got
an array of indexes from the linear model, translated those to objects,
then passed them into the AnnotationSet constructor which translated
them right back to indexes.

The previous behavior was kind of ridiculous on its face, but the
reason we found it is because Inez was investigating the performance
degradation when bolding a line and found that half of it was due
to the hundreds of ve.getHash() calls caused by this behavior.

Change-Id: I38df8ae9f6392849dacf477ea2f804283c964417
2013-04-18 10:56:03 -07:00
Catrope 2f1ee49213 Fix a weird whitespace stripping bug
In HTML like <td>Foo <b>Bar</b></td>, the space would be stripped and
registered as trailing whitespace in the <td>, so it wouldn't be visible
in the editor and would be inserted after the </b> on the way out.

Thanks to Subbu for reporting this. This case was kind of ridiculous and
we're lucky the JRuby article contained it. To trigger the bug, you had
to have:
* a table cell
* containing unwrapped content
* consisting of
** some text
** whitespace
** open annotation (bold in my example, link in Subbu's case)
** text
** close annotation
** and nothing else

Change-Id: I2b83f02764b311a32a50956d4c8930a9394e91a4
2013-04-17 17:08:47 -07:00
Catrope 04516bb02e Whitespace preservation was broken after the first run
The first run of getDomFromData() would preserve whitespace just fine,
but it blanked out the .veInternal.whitespace[1] element in certain
cases, contaminating the linear model and making the whitespace data
inconsistent. Subsequent runs of getDomFromData() would then refuse to
serialize that whitespace because the information about it was
inconsistent.

In getDomFromData(), we sometimes unset .veInternal.whitespace[1] (i.e.
set it to undefined) to prevent double processing. Because we're
potentially going to modify .veInternal, don't assign it by reference,
but copy the object.

Added tests asserting that the linear model is unchanged after calling
getDomFromData(), because that function should never modify
linear model data. This test failed in 4 cases (all whitespace-related)
before I added the copyObject() call.

Bug: 43543
Change-Id: Ic4c93510518163894201a693ab50331413715967
2013-04-17 11:28:05 +00:00
Ed Sanders 6dacc54954 Hybridise MWTemplateNode
* Create MWTemplateBlockNode & MWTemplateInlineNode (in ce and dm)
* Move Alien's 'isInline' code to ve.dm.Node.static.isHybridInline
* Move definition of ce.AlienBlock/InlineNode inside ce.Aline.js file
  to match dm.AlienBlock/InlineNode and MWTemplate
* Duplicate AlienBlock/Inline styles for templates
* Create test case for inline templates
* Count test cases in ve.dm.Converter.test.js automatically

Change-Id: Id9bc7f049ea974dd5e7f8b7a66080939e0948bbd
2013-04-14 02:34:18 +00:00
Catrope 54a232a92b Allow nodes to handle their own children
For data->DOM, this is easy: .toDataElements() can optionally return an
array instead of an object, and that will be treated as the data to
insert. If this happens, the converter won't descend. The node handler
can recursively invoke the converter if it needs to (although I suspect
the current implementation is broken when converting block content in an
inline context).

For DOM->data, this is a bit more complex. The node sets
.static.handlesOwnChildren = true; , which triggers the converter to
pass a data slice rather than a single data element, and not to
descend. The node handler can invoke the converter to recursively
convert DOM subtrees to data.

ve.dm.Converter (data->DOM):
* Renamed createDataElement() to createDataElements()
** .toDataElement() may return element or array, handle this
* Renamed childDataElement to childDataElements, is now an array
* Actually alienate if .toDataElement() returns null
** Shockingly, this claimed to be supported before but wasn't
* Rather than pushing to data, concat to it
** Add closing if needed
* Don't descend if .toDataElement() returned an array of length >1, or
  if the node has .handlesOwnChildren = true

ve.dm.Converter (DOM->data):
* Split getDomSubtreeFromData() and getDomFromData()
* When converting a node that handles its own children, pass in a data
  slice and skip over that data

Change-Id: I196cb4c0895cbf0b428a189adb61b56565573ab3
2013-04-11 22:41:18 +00:00
Catrope 1b5a376c28 Allow hybrids across Model subclasses
A node could already implement a toDataElements() function that
returns a data element of another node type, but it couldn't return
an annotation or a meta item. This is fixed now, and any dm.Model
subclass can now morph into any other dm.Model subclass.

I didn't originally plan to do this today at all, but doing this now
makes my upcoming converter changes easier. Surprise feature!

Change-Id: Ief6ac302094df084221a5a97c32a522b929c2960
2013-04-11 11:12:44 -07:00
Catrope 76b080dce1 Pass the converter object to the node handler in toDataElement()
This will allow node handlers to recursively invoke getDataFromDomRecursion()

Change-Id: I12cd4b31614a549bfbe8fbdc7d0607ece32aa98a
2013-04-11 11:12:44 -07:00
Catrope daaf255f13 Make getDataFromDomRecursion() use a context stack to pass context info
This will allow toDataElement() functions to just call this function
with a DOM element, rather than having to have all the recursion context
data to pass in.

Also expose this information using getters.

Change-Id: I89574c42385267e08704f018c0892d63014376a6
2013-04-11 11:12:39 -07:00
Ed Sanders 62c06d0253 Create GeneratedContentNode which can store rendered HTML in IV store
AlienNode is now a subclass of GCNode, but doesn't use the IV store yet.

Bug: 46571
Change-Id: If0717afdf557a2aa681d1bae3a6e98299631091a
2013-04-10 19:34:19 +01:00
Catrope 27875c8220 Reduce code duplication for annotation rendering
ve.dm.Converter and ve.ce.ContentBranchNode were duplicating a fair bit
of logic for annotation rendering. Moved the annotation opening and
closing logic into ve.dm.Converter.openAndCloseAnnotations, and
implemented both annotation rendering code paths in terms of that
function with callbacks for caller-specific behavior.

Change-Id: I7cba7d2fda7002287b07949a1b8120ba80bfe854
2013-04-09 23:38:03 +00:00
Catrope 0b55bb8cdc Move common Node/Annotation/MetaItem code into ve.dm.Model
ve.dm.Model is now the common base class for these three. ve.dm.Node
inherited from ve.Node before, so it now uses it as a mixin instead.
This required changing ve.Node's usage of ve.EventEmitter from
inhertiance to a mixin as well, because inherited methods apparently
don't get mixed in correctly.

* Change annotation terminology from linmodAnnotation to element for
  consistency with Node, MetaItem and Model
* Reimplement getClonedElement() in Node for .internal treatment

Change-Id: Ifd3922af23557c0b0f8984d36b31c8a1e2ec497e
2013-04-09 12:05:05 -07:00
Catrope 2eb0d2a6b2 Great Annotation Refactor of 2013
This changes the annotation API to be the same as the node API, sans
a few boolean flags that don't apply. The APIs were different, but
there was really no good reason why, so this makes things simpler for
API users. It also means we'll be able to factor a bunch of things out
because they're now duplicated between nodes, meta items and annotations.

Linear model annotations are now objects with 'type' and 'attributes'
properties (rather than 'name' and 'data'), for consistency with elements.
They now also contain html/0/* attributes for HTML attribute preservation,
which obsoletes the htmlTagName and htmlAttributes properties.
dm.Annotation subclasses take a reference to such an object and implement
conversion using .static.toDataElement and .static.toDomElements just
like nodes do. The custom .getHash() functions are no longer necessary
because of the way HTML attribute preservation was reimplemented.

CE rendering has been moved out of dm.Annotation (it never made sense to
have CE rendering functions in DM classes, this was bothering me) and into
separate ce.Annotation subclasses. These are very similar to CE nodes in
that they have a this.$ generated based on something in the DM; the main
difference is that nodes listen to events and update themselves, whereas
annotations are static and are simply destroyed and rebuilt when they
change. This change also adds whitelisted HTML attribute rendering for
annotations, as well as class="ve-ce-FooAnnotation" attributes.

Now that annotation classes produce real DOM nodes rather than weird
objects describing HTML tags, we can't generate HTML as a string in
ce.ContentBranchNode anymore. getRenderedContents() has been rewritten
to be much more similar to the way the converter renders annotations;
in fact, significant parts of it were copied from the converter, so that
should be factored out in the future. This change actually fixes an
annotation rendering discrepancy between ce.ContentBranchNode and
dm.Converter; see the diff of ve.ce.ContentBranchNode.test.js.

ve.ce.MWEntityNode.js:
* Remove stray property

ve.dm.MWExternalLinkAnnotation.js:
* Store 'rel' attribute

ve.dm.TextStyleAnnotation.js:
* Put all the conversion logic in the abstract base class

ve.dm.Converter.js:
* Also feed annotations through getDomElementsFromDataElement() and
  createDataElement()

ve.dm.Node.js:
* Fix undocumented property

ve.ce.ContentBranchNode.test.js:
* Add descriptive messages for each test case
* Compare DOM trees, not HTML strings
* Compare without all the class="ve-ce-WhateverAnnotation" clutter

ve.ui.LinkInspector.js:
* Replace direct .getHash() calls (evil!) with ve.getHash()

Bug: 46464
Bug: 44808
Change-Id: I31991488579b8cce6d98ed8b29b486ba5ec38cdc
2013-04-08 18:10:16 -07:00
Ed Sanders fdf30b1ac8 Store data in LinearData class with an index-value store for objects
Created an IndexValueStore class which can store any object and return
an integer index to its hash map.

Linear data is now stored in ve.dm.LinearData instances. Two subclasses
for element and meta data contain methods specific to those data types
(ElementLinearData and MetaLinearData).

The static methods in ve.dm.Document that inspected data at a given
offset are now instance methods of ve.dm.ElementLinearData.

AnnotationSets (which are no longer OrderedHashSets) have been moved
to /dm and also have to be instantiated with a pointer the store.

Bug: 46320
Change-Id: I249a5d48726093d1cb3e36351893f4bff85f52e2
2013-03-30 10:06:34 +00:00
Catrope a835c03bc1 Change MetaNodes to MetaItems
Rather than meta-things being special kinds of nodes, they are now a
separate class of things (MetaItems) along with Nodes and Annotations.

* Created a generic ve.dm.MetaItem that meta items inherit from.
  There will be actual instances of this class as well in the upcoming
  meta group code.
* Renamed MetaNode to AlienMetaItem, MWMetaNode to MWMetaItem,
  'metaBlock'/'metaInline' to 'alienMeta'
* Created a MetaItemFactory, handle meta items in the ModelRegistry
* Kill ve.dm.Node.static.isMeta, now obsolete

ve.dm.Converter:
* Pass in the MetaItemFactory
* Look up data element types in the ModelRegistry rather than the
  NodeFactory, because they can be either nodes or meta items
* Document createDataElement() and make explicit that modelClass can be
  either a node or a meta item
* Handle meta items in getDataFromDom()
* In getDomFromData(), check the MetaItemFactory as well as the NodeFactory

Change-Id: I893709c6f3aa00f85c1b905b70f9f4e597bdeada
2013-03-14 23:35:50 -07:00
Ed Sanders 9a7b8aacf8 Only unwrap { generated: wrapper } based on context.
Wrapper paragraphs should only be unwrapped if they are the first
element in their parent - or if there is a block level element separating
them from the previous unwrapped paragraph.

Empty paragraphs should only be unwrapped if they are empty and the
last element in their parent.

Also in this commit is a simple test for IndentationAction.decrease().

Bug: 45590
Change-Id: I1f47d12db6d57d984fd4607f667a3b62c53f3dd6
2013-03-13 00:42:16 +00:00
Catrope 1463d03f44 Change one last .storeHTMLAttributes to .storeHtmlAttributes
Change-Id: I161b2e8bf22c3784ca660ab0961a528b23601022
2013-02-22 16:13:47 -08:00
Catrope 04c72f6871 Add MWMetaNode to clean up <meta>/<link> hack in the converter
Change-Id: I4c69bff4981eef78415b43d31c3fd2ee271450ef
2013-02-22 15:21:40 -08:00
Catrope 2e36f1542b (bug 45062) Implement the new node API in the converter
This changes the node API to work with multiple elements, so we can
support about groups. Instead of passing in and returning single DOM
elements, we use arrays of DOM elements.

ve.dm.Converter:
* Pass modelRegistry into the constructor
* Remove onNodeRegister handler and its data
* Remove getDataElementFromDomElement() and
  getDataAnnotationFromDomElement(). Most logic moved into
  getDataFromDom(), some into createDataElement()
* Remove createAlien(), replaced with
  createDataElement( ve.dm.AlienNode, ... )
* Replace doAboutGrouping() (which wrapped about groups) with
  getAboutGroup() (which returns an array of the nodes in the group)
* Put in a hack so <meta>/<link> elements with an mw: property aren't
  alienated
* Remove about group wrapping behavior in favor of just outputting
  multiple nodes

ve.dm.AlienNode.js:
* For multi-element aliens, only choose inline if all elements are
  inline, not just the first one

ve.dm.example.js:
* Add html/0 stuff for meta nodes
* Fix test case to reflect new alien behavior

Change-Id: I40dcc27430f778bc00a44b91b7d824bfb2718be6
2013-02-22 15:21:40 -08:00
Catrope 5e16141750 Change context.wrapping to context.inWrapper
It's what the docs say it should be, so let's call it that :)

Change-Id: Ia10861ce77872243beb3d3a1886877824103f6c9
2013-02-22 15:21:40 -08:00
Catrope bbe3783d58 Add .static.storeHtmlAttributes
Defaults to true, but set to false, so we don't do redundant work for
aliens.

Change-Id: If35db3a67afd78124b4b2b46bb78ad60cbac46f5
2013-02-22 15:21:31 -08:00
James D. Forrester 82114467f1 Bump copyright notice year range to -2013 over -2012
199 files touched. Whee!

Change-Id: Id82ce4a32f833406db4a1cc585674f2bdb39ba0d
2013-02-19 15:37:34 -08:00
Catrope 3035c311ec Make the converter work with full HTML documents rather than fragments
The Parsoid output will also be expected to be a full HTML document. For
backwards compatibility, we allow for the Parsoid output to be a
document fragment as well. We don't send a full document back yet, also
for b/c -- we'll change this later once Parsoid has been updated in
production.

ve.dm.Converter.js:
* Make getDataFromDom() accept a document rather than a node
** Split off the recursion (which does use nodes) into its own function
** For now we just convert the <body>. In the future, we'll want to do
   things with the <head> as well
* Pass the document around so we can use it when creating elements
* Make getDomFromData() return a document rather than a <div>

ve.init.mw.Target.js:
* Store a document (this.doc) rather than a DOM node (this.dom)
* Pass around documents rather than DOM nodes
* Detect whether the Parsoid output is an HTML document or a fragment
  using a hacky regex
* When submitting to Parsoid, submit the innerHTML of the <body>

ve.init.mw.ViewPageTarget.js:
* s/dom/doc/
* Store body.innerHTML in this.originalHtml

ve.Surface.js:
* s/dom/doc/

demos/ve/index.php:
* Don't wrap HTML in <div>
* Pass HTML document rather than DOM node to ve.Surface

ve.dm.Converter.test.js:
* Construct a document from the test HTML, rather than a <div>

ve.dm.example.js:
* Wrap the HTML in the converter test cases in <body> tags to prevent
  misinterpretation (HTML fragments starting with comments, <meta>,
  <link> and whitespace are problematic)

Change-Id: I82fdad0a099febc5e658486cbf8becfcdbc85a2d
2013-02-19 10:38:39 -08:00
Catrope 591f2e7396 Change the HTML attribute prefix from html/ to html/0/
This means that <p data-foo="bar"> will now be converted to a paragraph
with attributes {"html/0/data-foo":"bar"} rather than {"html/foo":"bar"}

This paves the way for multi-element node (about group) handling in the
node API: nodes representing multiple DOM elements will have html/i/attr
to represent an attribute of the i'th DOM element.

Change-Id: Iea52bdccd721942ca708c8f9f47e934524809845
2013-02-06 12:00:43 -08:00
Catrope 57ad316988 Fix bug where inline nodes didn't trigger wrapping
When encountering an inline node (i.e. content node that's not a text
node) within a branch node that's not a content branch node, the
converter should start a wrapper. But it doesn't do this, it only opens
wrappers for text nodes and annotations.

Fixed this in the converter, added a test for it, and fixed an existing
test that asserted the broken behavior.

Change-Id: I6e143e21e68b68f0d85b8772e24a2d3a5d465410
2013-02-01 16:06:17 -08:00
Catrope a97f777685 Introduce context object in getDataFromDom()
* Introduce context object as specified for
  ve.dm.Node.static.toDataElement()
* Remove wrapping variable in favor of context.wrapping
* Remove wrappingIsOurs in favor of context.canCloseWrapper
* Introduce originallyExpectingContent and use it to repopulate
  context.expectingContent after closing a wrapper
* Replace most uses of branchHasContent with context.expectingContent
** Except for two cases where we need originallyExpectingContent

These changes fix a case where a metaBlock was generated in an inline
position. Updated the tests to reflect this.

Change-Id: I6baf6053f8a3a0b7d91487f812b9235a7b2b3db1
2013-01-31 15:00:00 -08:00
Catrope 1927330352 Use AnnotationSet rather than array in getDataFromDom()
Change-Id: Ie5f1f94bd62739ccf74c027dd0ba418fe2d91fa6
2013-01-31 15:00:00 -08:00
Catrope 99df776543 Allow matchTagNames = null in ve.dm.Converter
This won't usefully register the node with the converter right now, but
we need to allow this because the ModelFactory tests will need to have
stub nodes with tag-only matches.

Change-Id: I023cc8ff647363ab55c73dff39b17ca47e9e6681
2013-01-22 17:45:43 -08:00
Catrope aa372b6c16 Actually use this.nodeFactory and this.annotationFactory in ve.dm.Converter
Change-Id: I138a437d2e64577ad905ff70ecedf1eb7e6c8360
2013-01-22 15:55:11 -08:00
Catrope de6193734d Add annotation-like static properties to nodes
Add static properties for matching, data<->DOM conversion, and name. Use
matchTagNames, toDataElement and toDOMElement. name isn't used yet.

Change-Id: I5e7df3303bbd65e6968e931b568c23d76003a9a4
2013-01-18 14:51:40 -08:00
Trevor Parscal 8d33a3de0d Major Documentation Cleanup
* Made method descriptions imperative: "Do this" rather than "Does this"
* Changed use of "this object" to "the object" in method documentation
* Added missing documentation
* Fixed incorrect documentation
* Fixed incorrect debug method names (as in those VeDmClassName tags we add to functions so they make sense when dumped into in the console)
* Normalized use of package names throughout
* Normalized class descriptions
* Removed incorrect @abstract tags
* Added missing @method tags
* Lots of other minor cleanup

Change-Id: I4ea66a2dd107613e2ea3a5f56ff54d675d72957e
2013-01-16 15:37:59 -08:00
jenkins-bot d349f18d4b Merge "(bug 43056) Inline tags like <span> are block-alienated sometimes" 2013-01-08 20:41:28 +00:00
Timo Tijhof b11bbed7a6 JSDuck: Generated code documentation!
See CODING.md for how to run it.

Mistakes fixed:
* Warning: Unknown type function
  -> Function
* Warning: Unknown type DOMElement
  -> HTMLElement
* Warning: Unknown type DOM Node
  -> HTMLElement
* Warning: Unknown type Integer
  -> Mixed
* Warning: Unknown type Command
  -> ve.Command
* Warning: Unknown type any
  -> number
* Warning: Unknown type ve.Transaction
  -> ve.dm.Transaction
* Warning: Unknown type ve.dm.AnnotationSet
  -> ve.AnnotationSet
* Warning: Unknown type false
  -> boolean
* Warning: Unknown type ve.dm.AlienNode
  ve.dm doesn't have a generic AlienNode like ve.ce
  -> Unknown type ve.dm.AlienInlineNode|ve.dm.AlienBlockNode
* Warning: Unknown type ve.ve.Surface
  -> ve.ce.Surface
* ve.example.lookupNode:
  -> Last @param should be @return
* ve.dm.Transaction.prototype.pushReplace:
  -> @param {Array] should be @param {Array}
* Warning: ve.BranchNode.js:27: {@link ve.Node#hasChildren} links to non-existing member
  -> (removed)
* Warning: ve.LeafNode.js:21: {@link ve.Node#hasChildren} links to non-existing member
  -> (removed)

Differences fixed:
* Variadic arguments are like @param {Type...} [name]
  instead of @param {Type} [name...]
* Convert all file headers from /** to /*! because JSDuck tries
  to parse all /** blocks and fails to parse with all sorts of
   errors for "Global property", "Unnamed property", and
  "Duplicate property".
  Find: \/\*\*([^@]+)(@copyright)
  Replace: /*!$1$2
* Indented blocks are considered code examples.
  A few methods had documentation with numbered lists that were
  indented, which have now been updated to not be intended.
* The free-form text descriptions are parsed with Markdown,
  which requires lists to be separated from paragraphs by an
  empty line.
  And we should use `backticks` instead of {braces} for inline
  code in text paragraphs.
* Doc blocks for classes and their constructor have to be
  in the correct order (@constructor, @param, @return must be
  before @class, @abstract, @extends etc.)
* `@extends Class` must not have Class {wrapped}
* @throws must start with a {Type}
* @example means something else. It is used for an  inline demo
  iframe, not code block. For that simply indent with spaces.
* @member means something else.
  Non-function properties are marked with @property, not @member.
* To create a link to a class or member, in most cases the name
  is enough to create a link. E.g. Foo, Foo.bar, Foo.bar#quux,
  where a hash stands for "instance member", so Foo.bar#quux,
  links to Foo.bar.prototype.quux (the is not supported, as
  "prototype" is considered an implementation detail, it only
  indexes class name and method name).
  If the magic linker doesn't work for some case, the
  verbose syntax is {@link #target label}.
* @property can't have sub-properties (nested @param and @return
  values are supported, only @static @property can't be nested).
  We only have one case of this, which can be worked around by
  moving those in a new virtual class. The code is unaltered
  (only moved down so that it isn't with the scope of the main
  @class block). ve.dm.TransactionProcessor.processors.

New:
* @mixins: Classes mixed into the current class.
* @event: Events that can be emitted by a class. These are also
  inherited by subclasses. (+ @param, @return and @preventable).
  So ve.Node#event-attach is inherited to ve.dm.BreakNode,
  just like @method is.
* @singleton: Plain objects such as ve, ve.dm, ve.ce were missing
  documentation causing a tree error. Documented those as a
  JSDuck singleton, which they but just weren't documented yet.
  NB: Members of @singleton don't need @static (if present,
  triggers a compiler warning).
* @chainable: Shorthand for "@return this". We were using
  "@return {classname}" which is ambiguous (returns the same
  instance or another instance?), @chainable is specifically
  for "@return this". Creates proper labels in the generated
  HTML pages.

Removed:
* @mixin: (not to be confused with @mixins). Not supported by
  JSDuck. Every class is standalone anyway. Where needed marked
  them @class + @abstract instead.

Change-Id: I6a7c9e8ee8f995731bc205d666167874eb2ebe23
2013-01-05 01:16:32 +01:00
Catrope 7bcf35e0e8 (bug 43056) Inline tags like <span> are block-alienated sometimes
This happens when the <span> is the start of unwrapped content. The
converter logic to look at the tag name in wrapping mode doesn't kick in
because we're not yet in wrapping mode at that point.

The core issue was that previously, we relied on the document
structure/state to choose between alienBlock and alienInline, and only
used the tag name where the document structure was ambiguous (wrapping).
Changed this to be the other way around: we now rely primarily on the
tag name, and if that doesn't match what we expect based on the document
structure, we work around that if possible. Specifically:
* inline tag in our wrapper --> inline alien
* block tag in our wrapper --> close wrapper, block alien
* inline tag in wrapper that's not ours --> inline alien
* block tag in wrapper that's not ours --> *inline* alien
* inline tag in structural location --> open wrapper, inline alien
* block tag in structural location --> block alien
* inline tag in content location --> inline alien
* block tag in content location --> *inline* alien
only in the fourth and the last case do we need to use the "wrong" alien type to
preserve document validity, and it will always be inline where block was
expected, which should reduce UI issues.

The condensed version of the above, which is used in the code, is:
* If in a non-wrapper content location, use inline
* If in a wrapper that's not ours, use inline
* Otherwise, decide based on tag name
* Open or close wrapper if needed

ve.dm.Converter:
* Replace isInline logic in createAlien() with the above
* Factor out code to start wrapping (was duplicated) into startWrapping()
* Call startWrapping() if createAlien() returns an alienInline and we're
  in a structural location

Tests:
* Add test cases with aliens at the start and end of unwrapped content
** The first one failed prior to these changes and now passes, the
   second one was already passing
* Fix about group test case, was exhibiting the bug that this commit fixes

Change-Id: I657aa0ff5bc2b57cd48ef8a99c8ca930936c03b8
2012-12-22 12:27:11 +01:00
Timo Tijhof 4fa57b469a Phase out $.toJSON, use JSON.stringify directly.
Although $.toJSON optimises heavily for modern browsers (it
becomes a direct reference to JSON.stringify), we still load the
extra plugin.

JSON is specified as part of ECMAScript 5, but most browsers
supported this one before they supported the rest of ES5.

http://caniuse.com/#search=JSON

Cut off for native JSON is IE7, Firefox 3.0 (3.6 supports it) and
Safari 3. Not any of our concern as VE will most likely never
support those (certainly not at this point in time, and less
likely as time goes on).

Change-Id: I4e8f26ac94763fa38d29e41264de0247f53a21e5
2012-12-13 01:33:46 +01:00
Catrope 045b597253 Fix the "list of US Presidents" bug
I noticed this bug on [[List of Presidents of the United States]]. When
there's HTML that looks like "<td>Foo\n<meta/></td>", the converter will
collect the newline in wrappedWhitespace, then attempt to splice it out
and store it in internal data. But instead, it ends up splicing out the
/metaBlock element, which causes strange unbalanced input, which causes
an empty table in the node tree.

Change-Id: I79ed2fa9a834cc42759c7d21250d8842f563d38f
2012-12-11 11:23:31 -08:00
Catrope 085a6f0985 (bug 42487) Don't crash the converter for "<span>\n<p>Foo</p></span>"
The converter was misbehaving when handling <p>s inside <span>s. This
can't be expressed in the linmod, but it would try to anyway. <span><p>
would result in too many paragraph closing elements, leading to an
exception in ve.dm.Document complaining about unbalanced input.
<span>\n<p> would result in an exception in the converter itself while
trying to perform whitespace preservation on the newline.

This change makes the converter detect these scenarios and alienate the
offending node. So <span><p>Foo</p></span> converts to a wrapper
paragraph containing an alienInline whose HTML is "<p>Foo</p>" and which
is annotated with a TextStyleSpanAnnotation.

ve.dm.Converter.getDomFromData():
* Change the criteria for alienBlock vs alienInline
** Only infer from the node type if we're in wrapping mode AND we're at
   the same level where the wrapping started (wrappingIsOurs). If the
   latter isn't the case, we can't split the wrapper in the block case
   because we're at the wrong level.
** Use alienInline not only if the branch is a content branch, but also
   if there are active annotations. This catches e.g. <li><b><p>
   (and generally <span><p> on the top level).
* Before converting a child element, check that the child isn't "bad".
  Bad children are non-content children in content branches, and
  non-content children encountered within a wrapper that we can't split.
  Only good children are converted, and bad children are alienated (cue
  Santa/Sinterklaas jokes).
* Add childIsContent and rename branchIsContent to branchHasContent

Change-Id: If420ae80ab0777424a9a5517335ef9d0170e87ae
2012-12-05 17:20:07 -08:00
Catrope e95cc34978 (bug 42469) Leading newlines in <pre>s get eaten
HTML DOM has annoying behavior for <pre>s where .innerHTML eats the
first newline in a <pre>. Work around this by explicitly adding a
newline in the data->DOM converter if the <pre> already contained a
newline.

There is a separate bug in Parsoid that causes the newline to be lost
anyway, filed as bug 42666

Change-Id: Ia26cd4a4c61afbe439b0562deb7f24ee8b8147d7
2012-12-03 17:14:33 -08:00
Catrope e123a39b4e Handle annotated inline nodes in the converter
Was broken both on the way in and on the way out.

* Move alien restoration (data->DOM) out of the main getDomFromData()
  function and into getDomElementFromDataElement(). This means the
  comment about District 9 is gone (sniff), but moving this here ensures
  all code paths hit it (previously, it was assumed annotated nodes
  could never be aliens).
* In the DOM->data converter, add annotation application to
  getDataElementFromDomElement() (for content nodes) and createAlien()
  (for aliens). Previously, these nodes would not get annotations.
** ve.AnnotationSet doesn't have a constructor that takes an array, we
   should fix that.

Change-Id: I65f8e9a322111ca3af275bf9997b0b1e7ee93769
2012-11-27 14:41:40 -08:00
Inez Korczyński a9082e6dde Only apply HTML attributes to DOM nodes that are "safe"
* Added whitelist argument to setDomAttributes which allows filtering of attributes being set
* Added prefix argument to ve.dm.Node.getAttributes to allow extracting a subset of attributes by name prefix
* Added a whitelist to ve.ce.Node which was extracted from MediaWiki's Sanitizer class
* Replaced attribute copying code with a call to setDomAttributes using the whitelist argument, passing in attributes from a call to ve.dm.Node.getAttributes using the prefix argument

Also…

* Removed comment in constructor of ve.ce.Node, documentation for properties is usually in the getters/setters, and already was in this case
* Renamed ve.setDOMAttributes to ve.setDomAttributes
* Renamed ve.getDOMAttributes to ve.getDomAttributes
* Renamed ve.getDOMText to ve.getDomText
* Renamed ve.getDOMHash to ve.getDomHash
* Updated all callers of renamed methods

Change-Id: Id556172d5d18ea431044b9d402400e1f0e67a293
2012-11-27 14:34:29 -08:00
Trevor Parscal b6139ba65e Merge "(bug 42124) Store comments in the meta-linmod" 2012-11-21 22:12:41 +00:00
Catrope bf7b243627 (bug 42121) Change markers lost for first paragraph on new page
When editing a new page, or loading an empty page into the editor, the
converter generates a paragraph so the document isn't completely empty.
This paragraph is then unwrapped on the way out, potentially destroying
change markers and generally producing strange HTML output.

Mark this paragraph with generated=empty rather than generated=wrapper,
and only unwrap it on the way out if it's still empty. This means we
cleanly round-trip empty documents (and empty list items and the like),
but if the user enters text, we create a paragraph like we're supposed
to.

Change-Id: Id0241221a67b769445676b833b5741320d99ea5f
2012-11-21 13:54:52 -08:00
Catrope 662880605c (bug 42119) Handle alienation in wrapping mode properly
When alienating in wrapping mode, we need to look at the type of tag to
decide whether to create a wrapped alienInline, or to interrupt the
paragraph for an alienBlock.

This was being done just fine for the general alienation case
(unrecognized tag), but not for the special cases (mw:unrecognized,
about groups).

* Centralize the logic for ending a wrapper in stopWrapping()
* Move the wrapping-contingent block/inline detection logic into
  createAlien()
* Simplify the terrible if statement to decide whether a future decision
  requires us to stop wrapping. Instead, detect the cases in each code
  path separately and call stopWrapping() as appropriate
* Add tests

Change-Id: I4054584ae05e7d5daa71edead3e6a6588cf5d3bb
2012-11-21 13:42:13 -08:00
Catrope 1234a702c9 (bug 42218) Add MWEntityNode
<span typeof="mw:Entity"> tags are now correctly represented in the
model, and rendered in CE. There are still issues with cursor movement
etc. in CE.

Because the prioritization mechanism for annotations vs nodes is broken
in the current "node API", I had to hack two special cases for mw:Entity
into the converter. I also had to change the converter to ignore the
children of inline nodes (this was a legitimate bug, but had never come
up before).

Change-Id: Ib9f70437c58b4ca06aa09f7272bf51d9c41b18f2
2012-11-20 16:19:55 -08:00
Catrope 3a047e0208 (bug 42124) Store comments in the meta-linmod
* Make converter generate meta nodes with 'style': 'comment'
* Handle style==='comment' in MetaBlockNode toDOM converter
* Add some comments to the meta test case
** Update other tests accordingly
* Change getDomElementSummary() to actually assert presence of comment
  nodes (specifically, all non-text child nodes)

Change-Id: Ieef9418f4c47df3541477d9420aa2ab8df6e3df1
2012-11-19 20:01:09 -08:00
Trevor Parscal 2c8411eb62 (bug 41947) Propagate change markers when unwrapping generated nodes
Editing the text of a list item results in a change marker on the
paragraph within that list item. However, that paragraph usually isn't
present in the HTML, so the converter unwraps it when converting back to
HTML, and the change markers are lost. Instead, transfer the change
markers to the <li>.

Change-Id: Id675075d19c08d69bc8e990174841dc393b749fc
2012-11-16 15:39:35 -08:00
Catrope 3acc6cb8f4 Disable change marking by default
It's causing problems with Parsoid in production

Change-Id: Id47493baafe1ec7f7c0e2bbdb2ea60a82913dfaf
2012-11-14 11:58:32 -08:00
Catrope d4ea93b872 Add basic support for about groups
About groups are HTML structures like the following:
<div about="#mwt1">....</div>
<span about="#mwt1">...</span>
<div about="#mwt1">...</div>
When about groups are alienated, they are now merged into one alien
node, rather than producing a separate alien node for each sibling.

This is very basic about group handling, because it only works for
groups of directly adjacent siblings (text nodes are permitted in
between, but nothing else) assumes all about groups are aliens (which
is currently true).

* Before processing an element in the DOM->data converter, perform about
  grouping on its children. This temporarily wraps about groups in
  <div data-ve-aboutgroup="value of about attribute">
* Extended createAlien() to handle single nodes as well as wrappers
  holding multiple nodes.
* In the data->DOM converter, temporarily wrap multi-node aliens in
  <div data-ve-multi-child-alien-wrapper="true"> . This makes the rest
  of the algorithm easier.

Change-Id: I2df5f62bc222b570fc11a89fe43d353f8363ead8
2012-11-07 18:13:50 -08:00
Catrope 1f01100eb9 Flag pre nodes as having significant whitespace
This causes the converter not to strip inner whitespace in them, and
causes CE to suppress the whitespace mangling logic that is normally
applied (↵ for newlines, ➞ for tabs, alternating &nbsp;s for spaces).

Change-Id: I738a750c91a4ca4836c485e282865bb7525bf30a
2012-11-07 12:10:58 -08:00
Catrope 04a999f991 Add change marking for Parsoid's benefit
* Add map of change markers per offset to Transaction
* Map is populated by TransactionProcessor
* Markers are reversed on rollback
* Removals aren't marked, Parsoid can detect these using DSR
  discontinuities

Change-Id: I2290886ab411c6ad6162044ed85c091313613e51
2012-11-06 10:11:11 -08:00
Catrope 84e598953a Wrap inline elements properly
The HTML "1<br/>2" was being converted to a linmod that looked like
"<p>1</p><br></br><p>2</p>". This commit fixes the wrapping logic such
that the result is "<p>1<br></br>2</p>" instead. In general, inline
nodes (content nodes) should not interrupt the wrapping, but block nodes
should.

This creates a problem for alien nodes: normally, we determine whether an
alien node is a block alien or an inline alien based on context, but if
we're in wrapping mode we're unsure of the context. We can't tell the
difference between "1<tt>Foo</tt>2" (should be wrapped as one, because
tt is inline) and "1<figure></figure>2" (1 and 2 should be wrapped
separately, because figure is block) using context alone, so in these
cases (and ONLY in these cases) we look up whether the HTML tag in
question is an inline tag or a block tag and use that to decide.

Change-Id: I75e7f3da387dd401d9b93e09a21751951eccbb83
2012-10-17 13:50:29 -07:00
Catrope 735ee449e3 New annotation API: ve.dm.Converter integration
The annotation-related code in the converter is greatly simplified
because the API itself takes care of almost everything already.

Change-Id: Ib48f52bad6b650a05dc4e7ef82db4158c19b3cf5
2012-10-12 15:07:28 -07:00
Trevor Parscal daa7e76807 Merge "Add a node type for meta nodes" 2012-09-18 18:15:46 +00:00
Timo Tijhof ab7d6bf082 Documentation & clean up
* Commands for Sublime:

  Find*: "(\* @[a-z]+) ([^{].*) \{(.*)\}"
  Replace: "$1 {$3} $2"

  Save all && Close all

  Find: " function("
  Replace: " function ("

  Save all && Close all

  Find: "Intialization"
  Replace: "Initialization"

  Save all && Close all

* Consistent use of types (documented in CODING.rm):
  - Merged {Integer} into {Number}.
  - Merged {DOM Node} into {DOMElement}.

* Remove work-around /*jshint newcap: false */ from ve.js
  Calling Object() as a function to to use the internal
  toObject no longer throws a newcap warning in JSHint.
  It only does that normal functions now .

  (e.g. var a = Cap(); or var a = new uncap();)

* Add missing annotations (@static, @method, ..).

* Remove unused variables

* Remove null-assignments to variables that should just be
  undefined. There's a few variables explicitly set to null
  whereas they are set a few lines under and not used otherwise
  (e.g. 'tx' in ve.ce.Surface.prototype.onPaste)

Change-Id: I0721a08f8ecd93c25595aedaa1aadb0e08b83799
2012-09-17 16:02:52 +02:00
Catrope 7b96fbe3d2 Add a node type for meta nodes
This node type represents <meta> or <link> (transparently, based on the
style attribute). I had to make two node types for this and hack the
toData conversion code directly into ve.dm.Converter, because we don't
have native support for node types that can be both inline and block.
(We should add this in the node API rewrite.)

The CE implementation renders a placeholder (with the same styles as an
alien node) right now. I'm not sure how nice that is, but it's better
than rendering raw <meta>/<link> tags.

This whole thing is a total pile of hacks to make VE deal with
<meta>/<link> tags until we have a proper node types API.

Change-Id: Id6783fcfc35a896db088ff424ff9faaabcaff716
2012-09-10 15:35:30 -07:00
Catrope c6cb537f1a Fix bugs in whitespace preservation for aliens
This was broken in three different ways:
* On the way in, we were applying whitespace to an array of elements
  rather than the actual element, so the whitespace wasn't stored.
* Whitespace processing on the way out was skipped for aliens because
  they had their own code path. Refactored this so alien openings and
  regular openings share much more code, including whitespace output.
* Somewhat unrelatedly, innerPost output was broken for paragraphs
  containing inline elements, because the inline elements' processing
  polluted lastOuterPost. Discovered this because my test with inline
  aliens also happened to be the first test of whitespace preservation
  in paragraphs with inline content elements. Fixed by explicitly
  skipping content nodes when outputting whitespace.

Fixed these issues and added a test case.

Change-Id: I8edb61a008e60ace886b1a841b3417682ec39c32
2012-09-07 15:17:28 -07:00
Catrope 74ed8e8766 Rename ve_foo_bar back to VeFooBar per discussion
Change-Id: Ibf6d4f08c4761727b2e3952a76e474c8221b38f9
2012-09-06 16:15:55 -07:00
Timo Tijhof b1d9c83b5d Object management: Object create/inherit/clone utilities
* For the most common case:
  - replace ve.extendClass with ve.inheritClass (chose slightly
    different names to detect usage of the old/new one, and I
    like 'inherit' better).
  - move it up to below the constructor, see doc block for why.

* Cases where more than 2 arguments were passed to
  ve.extendClass are handled differently depending on the case.

  In case of a longer inheritance tree, the other arguments
  could be omitted (like in "ve.ce.FooBar, ve.FooBar,
  ve.Bar". ve.ce.FooBar only needs to inherit from ve.FooBar,
  because ve.ce.FooBar inherits from ve.Bar).

  In the case of where it previously had two mixins with
  ve.extendClass(), either one becomes inheritClass and one
  a mixin, both to mixinClass().

  No visible changes should come from this commit as the
  instances still all have the same visible properties in the
  end. No more or less than before.

* Misc.:
 - Be consistent in calling parent constructors in the
   same order as the inheritance.
 - Add missing @extends and @param documentation.
 - Replace invalid {Integer} type hint with {Number}.
 - Consistent doc comments order:
   @class, @abstract, @constructor, @extends, @params.
 - Fix indentation errors
   A fairly common mistake was a superfluous space before the
   identifier on the assignment line directly below the
   documentation comment.
   $ ack "^ [^*]" --js modules/ve
 - Typo "Inhertiance" -> "Inheritance".
 - Replacing the other confusing comment "Inheritance" (inside
   the constructor) with "Parent constructor".
 - Add missing @abstract for ve.ui.Tool.
 - Corrected ve.FormatDropdownTool to ve.ui.FormatDropdownTool.js
 - Add function names to all @constructor functions. Now that we
   have inheritance it is important and useful to have these
   functions not be anonymous.

   Example of debug shot: http://cl.ly/image/1j3c160w3D45

   Makes the difference between

   < documentNode;
   > ve_dm_DocumentNode
     ...
     : ve_dm_BranchNode
       ...
       : ve_dm_Node
         ...
         : ve_dm_Node
           ...
           : Object
             ...

   without names (current situation):

   < documentNode;
   > Object
     ...
     : Object
       ...
       : Object
         ...
         : Object
           ...
           : Object
             ...

   though before this commit, it really looks like this
   (flattened since ve.extendClass really did a mixin):

   < documentNode;
   > Object
     ...
     ...
     ...

   Pattern in Sublime (case-sensitive) to find nameless
   constructor functions:
   "^ve\..*\.([A-Z])([^\.]+) = function \("

Change-Id: Iab763954fb8cf375900d7a9a92dec1c755d5407e
2012-09-06 15:29:31 -07:00
Trevor Parscal 64209467df Merge "Alienate everything with an unrecognized mw: type." 2012-09-06 21:57:00 +00:00
Catrope 5e7c14c868 Manage annotations in ve.AnnotationSet object
Introduced the ve.AnnotationSet class to manage sets of annotations. This
is a generalization of ve.OrderedHashSet, a class that manages a set
using an array and an object keyed by hash.

Converted everything that stores, tracks or passes around annotations to
use ve.AnnotationSet. In particular, this means the linear model now
contains AnnotationSets instead of hash-keyed objects.

This allows us to maintain the order of annotations in the linear model,
and will help fix bugs with annotation ordering and splitting.

Change-Id: I50975b0a95f4cc33017a0b59fdede9ed1eff0124
2012-09-06 14:39:38 -07:00
Catrope e14d30f3cc Alienate everything with an unrecognized mw: type.
Currently this is done in a hacky way because we don't have a real
registry of RDFa types for node types, so we just hardcode the list of
recognized types (only links currently).

Change-Id: I5afcc55701fc6fa0ee2a360dcf5ca62b065292f5
2012-09-06 13:33:20 -07:00
Timo Tijhof 71020f7175 Remainder JSHint fixes on modules/ve/*
[jshint]
ce/ve.ce.Surface.js: line 670, col 9, Too many var statements.
ce/ve.ce.Surface.js: line 695, col 6, Missing semicolon.
ce/ve.ce.Surface.js: line 726, col 22, Expected '===' and instead saw '=='.
ce/ve.ce.Surface.js: line 726, col 41, Expected '===' and instead saw '=='.
ce/ve.ce.Surface.js: line 733, col 13, Too many var statements.
ce/ve.ce.Surface.js: line 734, col 24, Expected '===' and instead saw '=='.
ce/ve.ce.Surface.js: line 1013, col 13, Too many var statements.
ce/ve.ce.Surface.js: line 1019, col 17, Too many var statements.
ce/ve.ce.Surface.js: line 1023, col 18, Too many ar statements.
ce/ve.ce.Surface.js: line 1027, col 13, Too many var statements.
dm/annotations/ve.dm.LinkAnnotation.js: line 70, col 52, Insecure '.'.
dm/ve.dm.Converter.js: line 383, col 29, Empty block.
dm/ve.dm.Converter.js: line 423, col 33, Empty block.

Commands:
 * jshint .
 * ack '(if|else|function|switch|for|while)\('
 * Sublime Text 2:
   Find(*): (if|else|function|switch|for|while)\(
   Replace: $1 (
 * ack '  ' -Q # double spaces, except in certain comments

Change-Id: I8e34bf2924bc8688fdf8acef08bbc4f6707e93be
2012-09-05 13:45:09 -07:00
Catrope 452c759914 Preserve whitespace between elements
This commit fully utilizes all four positions in the internal.whitespace
array. Outer whitespace is now preserved as well, and is duplicated
either in the adjacent sibling (one node's outerPost is the next
sibling's outerPre) or in the parent (a branch node's innerPre is its
first child's outerPre, and its innerPost is its last child's
outerPost). Before restoring saved whitespace, we check that these two
agree with each other, and if they disagree we assume the user has been
moving stuff around and don't restore any whitespace in that spot. The
whitespace at the very beginning and the very end of the document (i.e.
the first node's outerPre and the last node's outerPost) isn't
duplicated anywhere, nor is inner whitespace in content nodes.

The basic outline of the implementation is:
* When we encounter whitespace, strip it and store it in the previous
  node's outerPost. Also store it in nextWhitespace so we can put it in
  the next node's outerPre once we encounter that node.
* When we encounter whitespace in wrapped bare text, we don't know in
  advance if it's gonna be succeeded by more non-whitespace (in which
  case it needs to be output verbatim), or not (in which case it's
  leading whitespace and needs to be stripped and stored). The fact that
  annotations are nodes in HTML makes this trickier. So we write the
  whitespace to the temporary linmod and store it in wrappedWhitespace,
  then if it turns out to be trailing whitespace we take it back out of
  the data array and record it the usual way.
* Because text nodes can contain any combination of leading whitespace
  actual text and trailing whitespace, and because we may or may not
  already have opened a wrapping paragraph, there are a lot of different
  combinations to handle. We handle all of them but the resulting code
  is pretty dense and verbose.

More low-level list of changes:

In getDataFromDom():
* Added helper function addWhitespace() for storing whitespace for an
  element
* Added helper function processNextWhitespace() for processing any
  whitespace passed on from the previous node via the nextWhitespace var
* Rename paragraph to wrappingParagraph. Make wrapping default to
  alreadyWrapped so we can simplify wrapping||alreadyWrapped and
  !wrapping&&!alreadyWrapped. Add wrappingIsOurs to track whether the
  wrapping originated in this recursion level (needed for deciding when
  to close the wrapper).
* Add prevElement to track the previous element so we can propagate
  whitespace to it, and nextWhitespace so we can propagate whitespace to
  the next element.
* Remove previous newline stripping hacks
* Integrate the logic for wrapping bare content with the outer
  whitespace preservation code
* Remove wrapperElement, no longer needed because we have a dedicated
  variable for the wrapping paragraph now and what was previously inner
  whitespace preservation for wrapper paragraphs is now covered by the
  outer whitespace preservation code.

In getDomFromData():
* Reinsert whitespace where appropriate
** outerPre is inserted when opening the element
** This covers outerPost as well except for the last child's outerPost,
   which is handled as the parent's innerPost when closing the parent.
** innerPre and innerPost are inserted when closing the element. Care is
   taken not to insert these if they're duplicates of something else.
* Propagate each node's outerPost to the next node (either the next
  sibling or the parent) using parentDomElement.lastOuterPost. We can't
  get this using .lastChild because we will have destroyed that child's
  .veInternal by then, and we can't tell whether a node will be its
  parent's last child when we process it (all other processing,
  including first child handling is done when processing the node itself,
  but this cannot be).
* Special handling is needed for the last node's outerPost, which ends
  up in the container's .lastOuterPost property.

Tests:
* Allow .html to be null in data<->DOM converter tests. This indicates
  that the test is a one-way data->DOM test, not a DOM->data->DOM
  round-trip test. The data will be converted to HTML and checked
  against .normalizedHtml
* Update existing tests as needed
* Add tests for outer whitespace preservation and storage
* Add test for squashing of whitespace in case of disagreement (this
  requires .html=null)

Change-Id: I4db4fe372a421182e80a2535657af7784ff15f95
2012-08-23 19:08:00 -07:00
Trevor Parscal 15791bef8b Merge "Remove references to data-mw-gc" 2012-08-21 21:49:32 +00:00
Catrope dfd78cb121 Remove references to data-mw-gc
data-mw-gc is ancient and unused. We do need to detect and alienate
generated nodes, but that is now based on RDFa types. Removing the
data-mw-gc stuff for now because it doesn't work anyway, will replace it
with proper detection later.

Replaced instances of data-mw-gc in the test suite with unregistered
node types.

Change-Id: If3f5898d382a436fa57929013264c53af5e840ba
2012-08-20 17:44:55 -07:00
Catrope 1a85b48c87 (bug 39512) Fix regression in I277ad5fe3f64e07c1bbf49007d6bbaecc90b7466
.childNodes is live, so iterating over it ends up skipping every other
child

Change-Id: I54ac480eca8e7ed36d299d68670a97ff176f3ebd
2012-08-20 15:18:41 -07:00
Catrope 7319038ed6 Strip generated <p> tags in dataToDom
domToData wraps bare content in paragraph elements, which were then
converted to <p> tags by domToData. With this fix, HTML with "missing"
<p> tags actually round-trips through the editor correctly now, rather
than having <p> tags added wherever VE believes they should exist.

* Mark generated paragraph elements with .internal.generated = 'wrapper'
** This signifies the wrapper was generated but its contents were not,
   so the right thing to do when converting back to HTML is to remove
   the wrapper and keep the contents. We might want to use other values
   of generated in the future.
* Unwrap nodes with generated=wrapper when converting to HTML

Tests:
* Add 'generated': 'wrapper' as appropriate. Only affects 1 test
* Remove 'normalizedHtml' for this test because it is no longer needed
** Need to keep 'normalizedHtml' for now because we normalize hrefs
* Eventually the main example should test bare content, but that
  requires touching a lot of stuff. The main example could use some
  beefing up anyway.

Change-Id: I277ad5fe3f64e07c1bbf49007d6bbaecc90b7466
2012-08-16 16:09:28 -07:00
Catrope ce60b54d33 Rename fringeWhitespace to internal
This allows us to put other internal data in there in the future. Also
passing it through the Node constructor properly now.

* ve.dm.Node
** Rename fringeWhitespace property to internal
** Add internal parameter to constructor
** Remove setFringeWhitespace()
* Increase the number of parameters passed through by ve.Factory to 3
* Pass through .internal from linmod to nodeFactory in ve.dm.Document
* ve.dm.Converter
** Rename .fringeWhitespace to .internal.whitespace and make it an array
** Store a temporary reference to .internal in domElement.veInternal
* Add internal to all node constructors except TextNode

Tests:
* Update for fringeWhitespace->internal rename
* Add third parameter to ve.Factory tests
* Add .internal to getNodeTreeSummary

Change-Id: If20c0bb78fee3efa55f72e51e7fc261283358de7
2012-08-16 15:14:01 -07:00
Catrope 70fa9c8aeb Recognize annotation types with multiple slashes correctly
Annotation types with more than one slash such as 'link/ExtLink/URL'
weren't being processed correctly because .split( '/', 2 ) throws away
everything after the second slash. Instead, don't pass a limit to
.split(); the code for reconstructing a slash-separated string from
multiple components was already in place.

Also add test cases for URL links and numbered links.

(Do you like the lines-of-code to lines-of-test ratio in this commit,
Trevor? ;) )

Change-Id: I7add87396447a01b1c23a4f9bfd63d2e8fd861ce
2012-08-13 18:22:31 -07:00
Trevor Parscal f99088b79f Merge "Strip and preserve inner leading&trailing whitespace in the linear model" 2012-08-13 16:56:50 +00:00
Trevor Parscal ea0467fb0e Merge "Refactor ve.js utilities and improve documentation" 2012-08-13 03:24:43 +00:00
Timo Tijhof f06952f2f3 Refactor ve.js utilities and improve documentation
Refactor:
* ve.indexOf
  Renamed from ve.inArray.
  This was named after the jQuery method which in turn has a longer
  story about why it is so unfortunately named. It doesn't return
  a boolean, but an index. Hence the native method being called
  indexOf as well.

* ve.bind
  Renamed from ve.proxy.
  I considered making it use Function.prototype.bind if available.
  As it performs better than $.proxy (which doesn't use to the native
  bind if available). However since bind needs to be bound itself in
  order to use it detached, it turns out with the "call()" and
  "bind()"  it is slower than the $.proxy shim:
  http://jsperf.com/function-bind-shim-perf
  It would've been like this:
  ve.bind = Function.prototype.bind ?
      Function.prototype.call.bind( Function.prototype.bind ) :
      $.proxy;
  But instead sticking to ve.bind = $.proxy;

* ve.extendObject
  Documented the parts of jQuery.extend that we use. This makes it
  easier to replace in the future.

Documentation:
* Added function documentation blocks.
* Added annotations to  functions that we will be able to remove
  in the future in favour of the native methods.
  With "@until + when/how".
  In this case "ES5". Meaning, whenever we drop support for browsers
  that don't support ES5. Although in the developer community ES5 is
  still fairly fresh, browsers have been aware for it long enough
  that thee moment we're able to drop it may be sooner than we think.
  The only blocker so far is IE8. The rest of the browsers have had
  it long enough that the traffic we need to support of non-IE
  supports it.

Misc.:
* Removed 'node: true' from .jshintrc since Parsoid is no longer in
  this repo and thus no more nodejs files.
 - This unraveled two lint errors: Usage of 'module' and 'console'.
   (both were considered 'safe globals' due to nodejs, but not in
   browser code).

* Replaced usage (before renaming):
 - $.inArray -> ve.inArray
 - Function.prototype.bind -> ve.proxy
 - Array.isArray -> ve.isArray
 - [].indexOf -> ve.inArray
 - $.fn.bind/live/delegate/unbind/die/delegate -> $.fn.on/off

Change-Id: Idcf1fa6a685b6ed3d7c99ffe17bd57a7bc586a2c
2012-08-12 20:32:45 +02:00
Catrope 63e6702c52 Strip and preserve inner leading&trailing whitespace in the linear model
This makes things like
== Foo ==
* Bar
render without the leading and trailing spaces, while still
round-tripping those spaces.

* Added a .fringeWhitespace property to the linear model and ve.dm.Node
** Object containing innerPre, innerPost, outerPre, outerPost
** Only inner* are used right now, outer* are planned for future use
** Like .attributes , it's suppressed if it's an empty object
* In getDataFromDom():
** Store the stripped whitespace in .fringeWhitespace
** Move emptiness check up: empty elements with .fringeWhitespace have
   to be preserved
** Move paragraph wrapping up: .fringeWhitespace has to be applied to
   the generated paragraph, not its parent
** Add wrapperElement to keep track of the element .fringeWhitespace has
   to be added to; this is either dataElement or the generated paragraph
   or nothing, but we can't modify dataElement because it's used later
* In getDomFromData():
** When processing an opening, store the fringeWhitespace data in the
   generated DOM node
** When processing a closing, add the stored whitespace back in
* In the ve.dm.Document constructor, pass through .fringeWhitespace from
  the linear model data to the generated nodes

Tests:
* Change one existing test case to account for this change
* Add three new test cases for this behavior
* Add normalizedHtml field so I can test behavior with bare content

Change-Id: I0411544652dd72b923c831c495d69ee4322a2c14
2012-08-10 17:23:53 -07:00
Catrope 8887a0731b Fix up c0ea02c0bf
Was stripping whitespace before and after annotations as well

Change-Id: Ie49319c15e68de6855fef2beadec0a11beea3ec6
2012-08-10 13:44:35 -07:00
Catrope baf965fc50 Merge "Make use of new jshint options" 2012-08-10 01:43:25 +00:00
Timo Tijhof 23c5b0d02c Make use of new jshint options
* Restricting "camelcase":
  No changes, we were passing all of these already

* Explicitly unrestricting "forin" and "plusplus"
  These are off by default in node-jshint, but some distro of jshint
  and editors that use their own wrapper around jshint instead of
  node-jshint (Eclipse?) may have different defaults. Therefor
  setting them to false explicitly. This also serves as a reminder
  for the future so we'll always know we don't pass that, in case
  we would want to change that.

* Fix order ("quotemark" before "regexp")

* Restricting "unused"
  We're not passing all of this, which is why I've set it to false
  for now. But I did put it in .jshintrc as placeholder.
  I've fixed most of them, there's some left where there is no clean
  solution.

* While at it fix a few issues:
 - Unused variables ($target, $window)
 - Bad practices (using jQuery context for find instead of creation)
 - Redundant /*global */ comments
 - Parameters that are not used and don't have documentation either
 - Lines longer than 100 chars @ 4 spaces/tab

* Note:
 - ve.ce.Surface.prototype.onChange takes two arguments but never
   uses the former. And even the second one can be null/undefined.
   Aside from that, the .change() function emits
   another event for the transaction already. Looks like this
   should be refactored a bit, two more separated events probably
   or one that is actually used better.
 - Also cleaned up a lot of comments, some of which were missing,
   others were incorrect
 - Reworked the contentChange event so we are no longer using the
   word new as an object key; expanded a complex object into multiple
   arguments being passed through the event to make it easier to work
   with and document

Change-Id: I8490815a508c6c379d5f9a743bb4aefd14576aa6
2012-08-10 02:50:30 +02:00
Catrope c0ea02c0bf Strip leading and trailing whitespace in text
Right now this means things like headings and list items are rendered
nicer (without the whitespace), but also get their whitespace normalized
when saving back. I'll submit code tomorrow that preserves this
whitespace.

Submitting this now because it's needed to make <br>s look reasonable

Change-Id: I4b5e5ad8ee1bbe2f1eaf0fb860dd59f6e401dc3d
2012-08-09 16:26:35 -07:00
Trevor Parscal d8ee3c2c29 After much research on error objects, native = good, custom = bad
Stack traces, line numbers, etc. All the approaches I've seen are bad hacks. This is the best way to go.

Change-Id: Ib12e9d2ecfe610bcc89d046005e35cc13efa3d99
2012-08-08 10:48:53 -07:00
Trevor Parscal b4de3ead08 Throw ve.Error instead of string literals
Throwing strings is bad because it doesn't include a lot of important
information that an error object does, such as a stack trace or where
the error was actually thrown from.

ve.Error inherits directly from Error. In the future we may create
more specific subclasses and/or do custom stuff.

Some interesting reading on the subject:
* http://www.devthought.com/2011/12/22/a-string-is-not-an-error/

Change-Id: Ib7c568a1dcb98abac44c6c146e84dde5315b2826
2012-08-08 06:19:00 +02:00
Trevor Parscal 255ce870e2 Puttin' em white-spacers where they aught'a be
function() -> function ()
){ -> ) {

Change-Id: I20a85fcf79d7aec64f7f2559e84c0279550d4eea
2012-08-06 18:52:19 -07:00
Trevor Parscal f0549e0075 Merge "Kill all but one of the Parsoid compat hacks, we don't need them any more" 2012-08-06 21:02:03 +00:00
Catrope 444961af2f Kill all but one of the Parsoid compat hacks, we don't need them any more
Also remove traces of Parsoid workarounds in tests. Tests are now
passing, yay :)

Change-Id: I8a59fc92c567c3595849e3e9377ce6eb6acde280
2012-08-06 13:56:40 -07:00
Catrope 68c5430872 Fix weird bug that was suddenly happening on the live site
When closing annotation nodes, we weren't popping them off
annotationStack. Not sure where this came from, but the code was
definitely bad and this fixes it.

Change-Id: I6d805e9aca3778666212135f76ff34c6baacbbc8
2012-08-06 13:51:51 -07:00
Trevor Parscal 13ccb68ae1 Cleanup - all jshint conditions are now met
Also:
* Removed a lot of dead code in Surface that was used in the now dead and gone sandbox.
* Changed from throwing an exception when calling getBalancedData on a range that produces no results from selectNodes to just returning []

Change-Id: Icf27094724eae5b90eec21308f9e26afe877e3ee
2012-08-03 18:56:04 -07:00
Catrope 67e11ebbc3 Make VE work again with the link RDFa changes in Parsoid
This is ugly but makes things work again. I intend to clean this up once
we have a better attribute API

* Recognize mw:WikiLink, mw:SimpleWikiLink, mw:ExtLink,
 mw:NumberedExtLink and mw:UrlLink
* Support is incomplete because we can't get to the annotation text with
  the current API
* Preserve all unhandled attributes rather than special-casing data-mw
* Update remaining code using data-mw (sHref and stx extraction) to
  account for the data-mw -> data-rt rename
* Update tests accordingly

Change-Id: Ia13d3008a6d4cdc8828f9acda5aa797566bc597f
2012-07-26 16:23:03 -07:00
Trevor Parscal a0f537712e Converted some instances of "var\t" to "var "
Change-Id: I02154e0381d5ae65b482bbcfc21ac93d0bf30d86
2012-07-19 17:24:54 -07:00