Transactions that replaced metadata twice at the same offset
(with retainMeta-replaceMeta-retainMeta-replaceMeta) were broken
because the processor for replaceMetadata didn't advance the
metadata cursor.
Change-Id: I7ad24e7ffb4c39b40ec9c347db301f8e28f3692d
In ve.dm.Document.getMetadataReplace(), we used to only merge metadata
if the amount removed is larger than the amount inserted. But this
could end up putting metadata in odd positions, for example if you
have Foo[[Category:Bar]]Baz and you delete 'ooBa' and replace it with
{image}xxx{/image}, then the category ends up inside the image.
We should always merge metadata when a segment is deleted, so that it
appears outside any new structure added.
There's a weird corner case here when a segment is removed but no
insertion is made: the removed metadata then needs to get glommed onto
the next element. We extend the insert/replace metadata array
when this happens.
Bug: 53444
Bug: 53445
Change-Id: I51d55fb370b473273f9cf152fdd0f356377d4109
op !== operation in all iterations of this loop except the first,
and using information from one operation to apply another one
doesn't work very well.
The fact that undefined cast to a number is NaN rather than 0
is very unhelpful :(
Also fixed some commas that should be semicolons.
Bug: 52238
Change-Id: I4138c023c955f2866881084506e24bb8b6db5a4d
It was being used correctly in the textual replace case, but it was
omitted in the structural replace case. This caused rare, insidious
metadata placement bugs that I haven't been able to reproduce outside
of newFromDocumentInsertion testing, but this might explain some of
the odd category bugs that have been reported.
Change-Id: I1424e482303853f285e4516a93c9609076648eff
The way it operated was evil. It did a depth-first search from the root,
finding the node using reference equality. For documents with deep
structures, this could take a long time. Inez did some profiling and
found it was called tens of millions of times on a complex document.
Kill getOffsetFromNode() and move its functionality to getOffset().
The logic has been completely rewritten: getOffset() now traverses
up from the node rather than down from the root, and pretty much does
the reverse of what getNodeFromOffset() does. This should be much more
efficient even without offset caching in the node objects (which we may
still implement later).
Change-Id: I125f9fa423c40db6472e2c4a7c94214218ba3bc7
Specifically by looking for "data-ve-changed",
"ChangeMarker*" and internal.changed.
Various tests, test counters and unused variables also
affected.
Bug: 45061
Change-Id: Ibd1ee68e0d650979d40574eff9cebded1a28499f
Before, it took an array of objects and translated those to indexes
using the store. Literally every caller outside of the test suite got
an array of indexes from the linear model, translated those to objects,
then passed them into the AnnotationSet constructor which translated
them right back to indexes.
The previous behavior was kind of ridiculous on its face, but the
reason we found it is because Inez was investigating the performance
degradation when bolding a line and found that half of it was due
to the hundreds of ve.getHash() calls caused by this behavior.
Change-Id: I38df8ae9f6392849dacf477ea2f804283c964417
ve.dm.Transaction
* Replace operations are now built directly from the
linear model and automatically determine what metadata
replace information they need to include:
** retainMetadata
** replaceMetadata
** insertMetadata
ve.dm.Document
* Metadata array created empty and padded out after data parsing
as we are no longer using Document.spliceData to build it (a new
test checks for correct metadata length)
* spliceData replaced with getMetadataReplace, which instead returns
transactional steps of spliceData (retain, replace, insert)
ve.dm.MetaLinearData
* Add function for merging metadata items together. Only used
once in the code (Document.getMetadataReplace) but useful
for generating test data.
ve.dm.MetaList
* Replace operations with metadata need to calculate new offset
and indices directly, but can't be applied immediately lest they
put a metaItem out of place and affect findItem.
ve.dm.MetaItem
* Add methods to support queued moves as required by MetaList
Test files
* Updated to match new pushReplace API
* Remove any instances of Document.spliceData
* Extra check on sparse metadata array length
* Rewrite spliceData tests as getMetadataReplace tests
* Count expected cases in Transaction(Processor) tests
Bug: 46954
Change-Id: I4edad1c2dd37c723bff2792bab7d694ef17a86dc
Use ve.BranchNode.getNodeFromOffset instead of
ve.dm.Document.getNodeFromOffset so correct nodes will be retrieved to
emit update events to.
Change-Id: Iaf559f0424584a3dde065e548e403c4a53207312
Created an IndexValueStore class which can store any object and return
an integer index to its hash map.
Linear data is now stored in ve.dm.LinearData instances. Two subclasses
for element and meta data contain methods specific to those data types
(ElementLinearData and MetaLinearData).
The static methods in ve.dm.Document that inspected data at a given
offset are now instance methods of ve.dm.ElementLinearData.
AnnotationSets (which are no longer OrderedHashSets) have been moved
to /dm and also have to be instantiated with a pointer the store.
Bug: 46320
Change-Id: I249a5d48726093d1cb3e36351893f4bff85f52e2
Previously these were static functions in TransactionProcessor
which instantiated a TP called .process() on it. These are now
methods of ve.dm.Document.
Also moved the emission of the 'transact' event on the document from
TransactionProcessor to Document itself, and moved the tests asserting
double application is protected against from TP to Document (because
the corresponding code moved as well).
Change-Id: I7c9f22a14accaf0ba1f70d5aa4f0573bb7e677d0
We'll need this to let a MetaList observe a Document and adjust to changes.
We should probably also have SurfaceFragment listen to this event instead.
Change-Id: I9e811e242969eb44afe0b4fa8153d0fb1b0071cd
Have created builders for insertion, removal, and single element replacement.
In adding Document.getMetadata which is nearly identical to Document.getData,
the two functions have been refactored to use a common static method
getDataSlice, with this.data/this.metadata as an argument.
Document.spliceMetadata has been added. It is essentially spliceData with
the data/metadata synchronisation issue.
Metadata cursor position is now tracked in the TransactionProcessor. Cursor
advancement has been moved to a function so the metadata cursor can be reset
every time the data cursor is moved.
There were unhit bugs in the TransactionProcessor run test section, where
the data being loaded into the test documents wasn't always being deep-copied,
and the assert was looking at getData instead of getFulldata (which wouldn't
be able to test metadata changes).
Change-Id: Ieb20ab3e7827bc7ff04148f147da6728eb1eb666
* Made method descriptions imperative: "Do this" rather than "Does this"
* Changed use of "this object" to "the object" in method documentation
* Added missing documentation
* Fixed incorrect documentation
* Fixed incorrect debug method names (as in those VeDmClassName tags we add to functions so they make sense when dumped into in the console)
* Normalized use of package names throughout
* Normalized class descriptions
* Removed incorrect @abstract tags
* Added missing @method tags
* Lots of other minor cleanup
Change-Id: I4ea66a2dd107613e2ea3a5f56ff54d675d72957e
See CODING.md for how to run it.
Mistakes fixed:
* Warning: Unknown type function
-> Function
* Warning: Unknown type DOMElement
-> HTMLElement
* Warning: Unknown type DOM Node
-> HTMLElement
* Warning: Unknown type Integer
-> Mixed
* Warning: Unknown type Command
-> ve.Command
* Warning: Unknown type any
-> number
* Warning: Unknown type ve.Transaction
-> ve.dm.Transaction
* Warning: Unknown type ve.dm.AnnotationSet
-> ve.AnnotationSet
* Warning: Unknown type false
-> boolean
* Warning: Unknown type ve.dm.AlienNode
ve.dm doesn't have a generic AlienNode like ve.ce
-> Unknown type ve.dm.AlienInlineNode|ve.dm.AlienBlockNode
* Warning: Unknown type ve.ve.Surface
-> ve.ce.Surface
* ve.example.lookupNode:
-> Last @param should be @return
* ve.dm.Transaction.prototype.pushReplace:
-> @param {Array] should be @param {Array}
* Warning: ve.BranchNode.js:27: {@link ve.Node#hasChildren} links to non-existing member
-> (removed)
* Warning: ve.LeafNode.js:21: {@link ve.Node#hasChildren} links to non-existing member
-> (removed)
Differences fixed:
* Variadic arguments are like @param {Type...} [name]
instead of @param {Type} [name...]
* Convert all file headers from /** to /*! because JSDuck tries
to parse all /** blocks and fails to parse with all sorts of
errors for "Global property", "Unnamed property", and
"Duplicate property".
Find: \/\*\*([^@]+)(@copyright)
Replace: /*!$1$2
* Indented blocks are considered code examples.
A few methods had documentation with numbered lists that were
indented, which have now been updated to not be intended.
* The free-form text descriptions are parsed with Markdown,
which requires lists to be separated from paragraphs by an
empty line.
And we should use `backticks` instead of {braces} for inline
code in text paragraphs.
* Doc blocks for classes and their constructor have to be
in the correct order (@constructor, @param, @return must be
before @class, @abstract, @extends etc.)
* `@extends Class` must not have Class {wrapped}
* @throws must start with a {Type}
* @example means something else. It is used for an inline demo
iframe, not code block. For that simply indent with spaces.
* @member means something else.
Non-function properties are marked with @property, not @member.
* To create a link to a class or member, in most cases the name
is enough to create a link. E.g. Foo, Foo.bar, Foo.bar#quux,
where a hash stands for "instance member", so Foo.bar#quux,
links to Foo.bar.prototype.quux (the is not supported, as
"prototype" is considered an implementation detail, it only
indexes class name and method name).
If the magic linker doesn't work for some case, the
verbose syntax is {@link #target label}.
* @property can't have sub-properties (nested @param and @return
values are supported, only @static @property can't be nested).
We only have one case of this, which can be worked around by
moving those in a new virtual class. The code is unaltered
(only moved down so that it isn't with the scope of the main
@class block). ve.dm.TransactionProcessor.processors.
New:
* @mixins: Classes mixed into the current class.
* @event: Events that can be emitted by a class. These are also
inherited by subclasses. (+ @param, @return and @preventable).
So ve.Node#event-attach is inherited to ve.dm.BreakNode,
just like @method is.
* @singleton: Plain objects such as ve, ve.dm, ve.ce were missing
documentation causing a tree error. Documented those as a
JSDuck singleton, which they but just weren't documented yet.
NB: Members of @singleton don't need @static (if present,
triggers a compiler warning).
* @chainable: Shorthand for "@return this". We were using
"@return {classname}" which is ambiguous (returns the same
instance or another instance?), @chainable is specifically
for "@return this". Creates proper labels in the generated
HTML pages.
Removed:
* @mixin: (not to be confused with @mixins). Not supported by
JSDuck. Every class is standalone anyway. Where needed marked
them @class + @abstract instead.
Change-Id: I6a7c9e8ee8f995731bc205d666167874eb2ebe23
The transaction builder would step around inline content elements when
building annotation transactions. This is now fixed.
I also tweaked the processor to tolerate attempts to annotate inline
closings. This allows the builder to generate simpler transactions,
because it doesn't have to step around the closing.
Change-Id: I1e0d7f95b38bad1b35b3e125a53350d2d126a7de
When working with a document containing only a slug in an empty paragraph, tree synchronization would break because it was trying to rebuild a non-existent text node.
This change makes the rebuild always occur on the outer range, rather than the inner range, which prevents absent text nodes from being asked to be rebuilt.
Thank you to Roan for debugging this for like 20 min.
Change-Id: I8c3dad921ace395f0694f77cec44305a680657fe
TransactionProcessor was using parentOuterRange without checking whether
it was present, so it was exploding for indexInNode results.
Now checking for parentOuterRange presence, and falling back to
nodeOuterRange when missing.
This fix causes inconsistencies with zero-length text nodes. We should
fix these eventually, but for now I've just made the unit tests
tolerant of zero-length-text-node deviations.
Change-Id: Id9eadd57a0d5fcbaf009c0781da0a03928aebb31
* Add map of change markers per offset to Transaction
* Map is populated by TransactionProcessor
* Markers are reversed on rollback
* Removals aren't marked, Parsoid can detect these using DSR
discontinuities
Change-Id: I2290886ab411c6ad6162044ed85c091313613e51
* ve.dm.Converter still generates metaInline/metaBlock elements as
before, it's not affected by this change
* ve.dm.Document constructor splits its input into "real" data and
metadata
** Metadata is stored in this.metadata (the meta-linmod) as a sparse
array of arrays, with an element for each offset in this.data
** this.data itself does not contain the metadata
** This means the node tree also doesn't contain the metadata
** Which means CE doesn't know about it at all
* All splice operations on the linear model are sent through
ve.dm.Document.spliceData(), which performs the splice and syncs the
meta-linmod
** Metadata in the removed range is reaped and added to the metadata for
the offset immediately following the removal
* ve.dm.Document.getFullData() splices the linmod and meta-linmod back
into each other; this "full data" is then fed back to ve.dm.Converter
Change-Id: Ief6dfd5b59cc13a8457993ed85c725413029c4fb
* Lift node assignment out of the if/else
* Flip the condition so we detect text-only replacements rather than
non-text-only replacements
* Additionally assert that there is exactly one selected node, and that
it is a text node
Change-Id: Iaaddf532f06709e860ac44457470e6d8bfcb6dd9
* Store the applied state in the Transaction
* Store the Transaction in the TransactionProcessor (previously, only
its operations were stored)
* Have commit() and rollback() throw exceptions when passed transactions
with the wrong applied state
* Add tests for this behavior
Change-Id: I27b7a96fdf4d3555d78f64c05a03702ea560c802
We were populating empty content nodes with zero-length text nodes to
make round-trip tests in the test suite work (otherwise blanking a
paragraph leaves behind a zero-length text node whereas creating an
empty paragraph does not), but the empty nodes are causing problems in
CE apparently.
* Do not create empty text nodes when constructing a node tree
* Be more careful with text-only replacements:
** Don't resize a text node to zero, remove it instead
** There may not be a text node to resize at all, build it in that case
** Switch nodeRange to nodeOuterRange, this was probably broken before
Tests:
* Change test case for zero-length text node to assert that there is
*no* zero-length text node :)
* Remove a test case concerning an empty text node from the
ve.ce.TextNode suite
Change-Id: Ie677457f2f0a7823a517ba3077b844ef52a20fcc
* Commands for Sublime:
Find*: "(\* @[a-z]+) ([^{].*) \{(.*)\}"
Replace: "$1 {$3} $2"
Save all && Close all
Find: " function("
Replace: " function ("
Save all && Close all
Find: "Intialization"
Replace: "Initialization"
Save all && Close all
* Consistent use of types (documented in CODING.rm):
- Merged {Integer} into {Number}.
- Merged {DOM Node} into {DOMElement}.
* Remove work-around /*jshint newcap: false */ from ve.js
Calling Object() as a function to to use the internal
toObject no longer throws a newcap warning in JSHint.
It only does that normal functions now .
(e.g. var a = Cap(); or var a = new uncap();)
* Add missing annotations (@static, @method, ..).
* Remove unused variables
* Remove null-assignments to variables that should just be
undefined. There's a few variables explicitly set to null
whereas they are set a few lines under and not used otherwise
(e.g. 'tx' in ve.ce.Surface.prototype.onPaste)
Change-Id: I0721a08f8ecd93c25595aedaa1aadb0e08b83799
* For the most common case:
- replace ve.extendClass with ve.inheritClass (chose slightly
different names to detect usage of the old/new one, and I
like 'inherit' better).
- move it up to below the constructor, see doc block for why.
* Cases where more than 2 arguments were passed to
ve.extendClass are handled differently depending on the case.
In case of a longer inheritance tree, the other arguments
could be omitted (like in "ve.ce.FooBar, ve.FooBar,
ve.Bar". ve.ce.FooBar only needs to inherit from ve.FooBar,
because ve.ce.FooBar inherits from ve.Bar).
In the case of where it previously had two mixins with
ve.extendClass(), either one becomes inheritClass and one
a mixin, both to mixinClass().
No visible changes should come from this commit as the
instances still all have the same visible properties in the
end. No more or less than before.
* Misc.:
- Be consistent in calling parent constructors in the
same order as the inheritance.
- Add missing @extends and @param documentation.
- Replace invalid {Integer} type hint with {Number}.
- Consistent doc comments order:
@class, @abstract, @constructor, @extends, @params.
- Fix indentation errors
A fairly common mistake was a superfluous space before the
identifier on the assignment line directly below the
documentation comment.
$ ack "^ [^*]" --js modules/ve
- Typo "Inhertiance" -> "Inheritance".
- Replacing the other confusing comment "Inheritance" (inside
the constructor) with "Parent constructor".
- Add missing @abstract for ve.ui.Tool.
- Corrected ve.FormatDropdownTool to ve.ui.FormatDropdownTool.js
- Add function names to all @constructor functions. Now that we
have inheritance it is important and useful to have these
functions not be anonymous.
Example of debug shot: http://cl.ly/image/1j3c160w3D45
Makes the difference between
< documentNode;
> ve_dm_DocumentNode
...
: ve_dm_BranchNode
...
: ve_dm_Node
...
: ve_dm_Node
...
: Object
...
without names (current situation):
< documentNode;
> Object
...
: Object
...
: Object
...
: Object
...
: Object
...
though before this commit, it really looks like this
(flattened since ve.extendClass really did a mixin):
< documentNode;
> Object
...
...
...
Pattern in Sublime (case-sensitive) to find nameless
constructor functions:
"^ve\..*\.([A-Z])([^\.]+) = function \("
Change-Id: Iab763954fb8cf375900d7a9a92dec1c755d5407e
Introduced the ve.AnnotationSet class to manage sets of annotations. This
is a generalization of ve.OrderedHashSet, a class that manages a set
using an array and an object keyed by hash.
Converted everything that stores, tracks or passes around annotations to
use ve.AnnotationSet. In particular, this means the linear model now
contains AnnotationSets instead of hash-keyed objects.
This allows us to maintain the order of annotations in the linear model,
and will help fix bugs with annotation ordering and splitting.
Change-Id: I50975b0a95f4cc33017a0b59fdede9ed1eff0124
Stack traces, line numbers, etc. All the approaches I've seen are bad hacks. This is the best way to go.
Change-Id: Ib12e9d2ecfe610bcc89d046005e35cc13efa3d99
Throwing strings is bad because it doesn't include a lot of important
information that an error object does, such as a stack trace or where
the error was actually thrown from.
ve.Error inherits directly from Error. In the future we may create
more specific subclasses and/or do custom stuff.
Some interesting reading on the subject:
* http://www.devthought.com/2011/12/22/a-string-is-not-an-error/
Change-Id: Ib7c568a1dcb98abac44c6c146e84dde5315b2826
Also:
* Removed a lot of dead code in Surface that was used in the now dead and gone sandbox.
* Changed from throwing an exception when calling getBalancedData on a range that produces no results from selectNodes to just returning []
Change-Id: Icf27094724eae5b90eec21308f9e26afe877e3ee
This license change is aimed at maximizing the reusability of this code
in other projects. VisualEditor is more than just an awesome editor for
MediaWiki, it's the new editor for the entire internet.
Added license and author files, plus mentions of the license to all
VisualEditor PHP, JavaScript and CSS files. Parser files have not been
modified but are effectively re-licensed since there's no overriding
license information. 3rd party libraries are not changed, but are all
already MIT licensed.
Change-Id: I895b256325db7c8689756edab34523de4418b0f2
* "onevar" warning sometimes solved by just merging var statements
other times solved by making it a function declaration instead
of a function expression.
* Also fixed several '_this' variable names in ve.es.Surface to
more descriptive names, and enabled warnings for dangling _
in identifiers.
Change-Id: I7d411881e3e06cf9a7fe56d689c29375881a81de
* Adjust both start and end for preceding operations
* Adjust end for the current operation as well
Change-Id: I2f96d609bddf3788aa5700ad1f0b46208f3517d7
