This function builds a transaction that takes a document slice and
inserts it back into the document it came from, applying any changes
that were made.
This makes editing document slices simple:
slicedDoc = doc.getDocumentSlice( captionNode );
// Edit slicedDoc using a surface
tx = ve.dm.Transaction.newFromDocumentInsertion( doc, captionNode, slicedDoc );
surface.change( tx );
Specifically, newFromDocumentInsertion replaces the node's contents
with the document's contents (meaning any changes made to the node in
the meantime are lost). It also merges the stores internal lists
of the two documents and remaps indexes accordingly. This means editing
of references inside of references is supported.
This functionality is not specific to slices, and can also be used to
safely insert data from a paste buffer, with internal list data being
transplanted correctly.
ve.dm.MetaLinearData:
* Make merge( [ undefined, undefined, ... ] ) return undefined rather
than [].
ve.dm.Document:
* In getDocumentSlice, store a pointer to the original dm.Document in
the new one, and also store the length of the internal list. This
allows us to figure out which internal list items the two documents
have in common when we insert the modified slice back into the main
document.
* In getMetadataReplace, optionally take the inserted metadata as a
parameter, to allow for operations that insert both data and metadata.
Per Ed's review, rewrite this function to return null rather than {}
if no metadata needs to be replaced.
ve.dm.InternalList:
* Add method to merge two internal lists
ve.dm.Transaction:
* Remove newFromNodeReplacement and replace it with newFromDocumentInsertion.
* In pushReplace, optionally take the inserted metadata as a parameter.
Change-Id: I786ee7bad796aa54bc242993b4de3ad18ad0773e
Also keep items in the order they appear in the document
and grouped by group and key.
Additions and removals are triggered by the new root/unroot events.
Change-Id: Ia3e90ccfdab88f352b89992b90554e5f03ff9952
Objectives:
* Fix compareObjects so it doesn't break when given arrays
* Remove compareArrays
* Rename compareObjects to compare and update callers of both methods
Changes:
ve.js
* Loosen check for whether to recurse so both arrays and objects qualify
* Remove compareArrays
* Rename compareObjects to compare
ve.dm.MWTemplateNode.js, ve.dm.AnnotationSet.js, ve.dm.Document.js,
ve.dm.Transaction.js
* Update uses of compare(Arrays|Objects) to use compare
Change-Id: I7d4f7ceb28c0389f0157b7598e291f21393b5b85
A document slice is a document built from a data slice of an existing
document. It's completely separate from the original document and has
its own store and internalList. The new document's data also contains
the entirety of the original document's internal list. It's possible
to create a document slice of data located inside the internal list,
in which case the resulting document will contain that data twice (one
mutable copy at the top level, and one immutable copy in the internal
list).
ve.dm.Document.js:
* Optionally take an internalList in the constructor. This allows us to
create a document with a clone of an existing internalList rather than
an empty one.
* Add edgeMetadata flag to getFullData()
ve.dm.IndexValueStore.js, ve.dm.InternalList.js:
* Make these classes cloneable
Change-Id: I93e06f764ace16aee9df941b07f8c2bff1a28e2b
Getting & setting the cursor is done with byte offsets
instead of data model offset (characters) so we need to
be able to convert between the two as well as splitting
characters.
TODO: The regex only works on surrogate pairs, not
yet combining accents.
fixupInsertion will combine a combining mark with the
character to its left it it can.
Bug: 48630
Change-Id: I8d936fb15d82f73cd45fac142c540a7950850d55
The code to record the first/last child stacks was written before
I decided on using a recursive call going only one step when such
a stack was encountered.
As we only ever test the length of the stack most of the code around
calculating the stack can be thrown away, and all we need is a simple
test to see if we are the first/last child of the immediate parent.
Change-Id: Iffb03c649e166c9f89061a4d944fa07633f16aba
The way it operated was evil. It did a depth-first search from the root,
finding the node using reference equality. For documents with deep
structures, this could take a long time. Inez did some profiling and
found it was called tens of millions of times on a complex document.
Kill getOffsetFromNode() and move its functionality to getOffset().
The logic has been completely rewritten: getOffset() now traverses
up from the node rather than down from the root, and pretty much does
the reverse of what getNodeFromOffset() does. This should be much more
efficient even without offset caching in the node objects (which we may
still implement later).
Change-Id: I125f9fa423c40db6472e2c4a7c94214218ba3bc7
fixUpInsertion now returns an object with both data and offset
which allows offset changes.
Within fixUpInsertion we lazy-generate first/lastChildStack which
is a list of parent nodes for which the current node is the first/last
child. Whenever we try to close off a node we check these stacks and
if they are populated we instead use a recursive call to start
fixUpInsertion again but with the offset shifted by 1.
Bug: 46799
Change-Id: Ic51dd03725c11f1f7e279929534ee3afea14d662
The fixUpStack is actually redundant code and closingStack
and openingStack handle all our cases. It was causing the
insertion to try to correct balance itself in the middle
of inserting two paragraphs, causing the creation on an
empty paragraph between them.
Added a test case for the fix and other cases to make
sure removing fixUpStack hasn't caused problems.
Bug: 46800
Change-Id: I35e54165709ac56e8116359a7c3b487eecf08ff7
Before, it took an array of objects and translated those to indexes
using the store. Literally every caller outside of the test suite got
an array of indexes from the linear model, translated those to objects,
then passed them into the AnnotationSet constructor which translated
them right back to indexes.
The previous behavior was kind of ridiculous on its face, but the
reason we found it is because Inez was investigating the performance
degradation when bolding a line and found that half of it was due
to the hundreds of ve.getHash() calls caused by this behavior.
Change-Id: I38df8ae9f6392849dacf477ea2f804283c964417
ve.dm.Transaction
* Replace operations are now built directly from the
linear model and automatically determine what metadata
replace information they need to include:
** retainMetadata
** replaceMetadata
** insertMetadata
ve.dm.Document
* Metadata array created empty and padded out after data parsing
as we are no longer using Document.spliceData to build it (a new
test checks for correct metadata length)
* spliceData replaced with getMetadataReplace, which instead returns
transactional steps of spliceData (retain, replace, insert)
ve.dm.MetaLinearData
* Add function for merging metadata items together. Only used
once in the code (Document.getMetadataReplace) but useful
for generating test data.
ve.dm.MetaList
* Replace operations with metadata need to calculate new offset
and indices directly, but can't be applied immediately lest they
put a metaItem out of place and affect findItem.
ve.dm.MetaItem
* Add methods to support queued moves as required by MetaList
Test files
* Updated to match new pushReplace API
* Remove any instances of Document.spliceData
* Extra check on sparse metadata array length
* Rewrite spliceData tests as getMetadataReplace tests
* Count expected cases in Transaction(Processor) tests
Bug: 46954
Change-Id: I4edad1c2dd37c723bff2792bab7d694ef17a86dc
Created an IndexValueStore class which can store any object and return
an integer index to its hash map.
Linear data is now stored in ve.dm.LinearData instances. Two subclasses
for element and meta data contain methods specific to those data types
(ElementLinearData and MetaLinearData).
The static methods in ve.dm.Document that inspected data at a given
offset are now instance methods of ve.dm.ElementLinearData.
AnnotationSets (which are no longer OrderedHashSets) have been moved
to /dm and also have to be instantiated with a pointer the store.
Bug: 46320
Change-Id: I249a5d48726093d1cb3e36351893f4bff85f52e2
Initially just with a Wordbreak module to implement Unicode standard
on 'Default Word Boundaries'. Due to it's standaloneability this has
been written as a separate library. Non-BMP characters are currently
not supported.
Bug: 44085
Change-Id: Ieafa070076f4c36855684f6bc179667e28af2c25
Previously these were static functions in TransactionProcessor
which instantiated a TP called .process() on it. These are now
methods of ve.dm.Document.
Also moved the emission of the 'transact' event on the document from
TransactionProcessor to Document itself, and moved the tests asserting
double application is protected against from TP to Document (because
the corresponding code moved as well).
Change-Id: I7c9f22a14accaf0ba1f70d5aa4f0573bb7e677d0
This is just a prototype - if it will be as useful as I expect
then it will get refactored (and probably get a better name +
location).
Change-Id: Ice1a2bd7d498d9d8438c35239216f01bd3db1826
We'll need this to let a MetaList observe a Document and adjust to changes.
We should probably also have SurfaceFragment listen to this event instead.
Change-Id: I9e811e242969eb44afe0b4fa8153d0fb1b0071cd
Before, replacement operations that both inserted and removed data at
the same time would be treated as removals followed by insertions,
so we'd reap the metadata from the affected range and move it to the
start of the range. For a pure replacement, this doesn't make any sense.
Instead, preserve the first min(insertLength, removeLength) elements in
the metadata array, then perform a pure insertion splice or a pure
removal splice for the length adjustment. Any metadata reaped in a
removal splice is restored at the offset where we started removing,
after the preserved portion.
These changes make the behavior of metadata reaping saner in general
(the previous behavior had the potential to move metadata around if it
was near a paragraph opening or closing and you converted the paragraph
to a heading), and makes the behavior match up with translateOffset(),
which is desirable for MetaList synchronization.
Change-Id: If9a1c6a7cf43ead7e3e1e8f6e081b139ca65fa53
Rather than meta-things being special kinds of nodes, they are now a
separate class of things (MetaItems) along with Nodes and Annotations.
* Created a generic ve.dm.MetaItem that meta items inherit from.
There will be actual instances of this class as well in the upcoming
meta group code.
* Renamed MetaNode to AlienMetaItem, MWMetaNode to MWMetaItem,
'metaBlock'/'metaInline' to 'alienMeta'
* Created a MetaItemFactory, handle meta items in the ModelRegistry
* Kill ve.dm.Node.static.isMeta, now obsolete
ve.dm.Converter:
* Pass in the MetaItemFactory
* Look up data element types in the ModelRegistry rather than the
NodeFactory, because they can be either nodes or meta items
* Document createDataElement() and make explicit that modelClass can be
either a node or a meta item
* Handle meta items in getDataFromDom()
* In getDomFromData(), check the MetaItemFactory as well as the NodeFactory
Change-Id: I893709c6f3aa00f85c1b905b70f9f4e597bdeada
Replacing [a-zA-Z] with a long unicode expression which encompasses
all the characters in the unicode 'letters' category. Similarly replacing
[0-9] with an expression for 'numbers'.
Bug: 44085
Change-Id: Idd403339caa24769ce08133dda06ab6d4b9d694e
Also in this commit is a minor fix to the regular expression so it
behaves as documented (the hyphen needed escaping).
Bug: 44085
Change-Id: Idc315e2dce79be8f028b5681c60f74e175b9d869
Have created builders for insertion, removal, and single element replacement.
In adding Document.getMetadata which is nearly identical to Document.getData,
the two functions have been refactored to use a common static method
getDataSlice, with this.data/this.metadata as an argument.
Document.spliceMetadata has been added. It is essentially spliceData with
the data/metadata synchronisation issue.
Metadata cursor position is now tracked in the TransactionProcessor. Cursor
advancement has been moved to a function so the metadata cursor can be reset
every time the data cursor is moved.
There were unhit bugs in the TransactionProcessor run test section, where
the data being loaded into the test documents wasn't always being deep-copied,
and the assert was looking at getData instead of getFulldata (which wouldn't
be able to test metadata changes).
Change-Id: Ieb20ab3e7827bc7ff04148f147da6728eb1eb666
This gets rid of the meta-specific hack in ve.dm.Document
Later on, I want to use this to get rid of meta-specific hacks in the
converter too, by pushing some of that logic into the ModelFactory.
Change-Id: I1dbee1654fa32d9c7cd521ec325b2553cde219d1
Continues where Ibb682332a6084e357104183641a104e3ae1e253f left off, adding tests and removing inconsistencies between the behavior of the document constructor, which was adding empty text nodes to empty paragraphs, and correcting other tests which expected empty text nodes to be there as well.
Change-Id: I414d061cdd494b8023f14e944eda2910a4dab0d4
The normalize method doesn't need to be explicitly called anymore because there's not any code that changes the properties of a range directly anymore.
A good way to prove it's not needed anymore is to move the normalization logic to the constructor and then add "console.log(this.from <= this.to );" to the normalize method - you will find that it's never actually doing anything at all because the range was normalized by the constructor.
ve.Range
* Moved normalization logic to constructor
* Removed calls to normalize method
* Removed normalize method
* Simplified documentation for flip method
* Whitespace fixes
ve.Document, ve.dm.Transaction, ve.dm.Surface, ve.dm.Document, ve.ce.Surface
* Removed calls to range.normalize
* Switched to using range.isCollapsed instead of comparing properties directly
Change-Id: I80bfd06f88579c34dce2083c2b70d63ab92f1275
* Made method descriptions imperative: "Do this" rather than "Does this"
* Changed use of "this object" to "the object" in method documentation
* Added missing documentation
* Fixed incorrect documentation
* Fixed incorrect debug method names (as in those VeDmClassName tags we add to functions so they make sense when dumped into in the console)
* Normalized use of package names throughout
* Normalized class descriptions
* Removed incorrect @abstract tags
* Added missing @method tags
* Lots of other minor cleanup
Change-Id: I4ea66a2dd107613e2ea3a5f56ff54d675d72957e
Follow up for I6a7c9e8ee8f995731bc205d666167874eb2ebe23
The first pass that Timo took missed the following cases
* "{Array|String}": string is just one of the values
* "{String[]}": string is followed by [] to indicate an array of strings
Change-Id: I65e595e8d37fb624802d84af9536a2d3c5d73c7d
See CODING.md for how to run it.
Mistakes fixed:
* Warning: Unknown type function
-> Function
* Warning: Unknown type DOMElement
-> HTMLElement
* Warning: Unknown type DOM Node
-> HTMLElement
* Warning: Unknown type Integer
-> Mixed
* Warning: Unknown type Command
-> ve.Command
* Warning: Unknown type any
-> number
* Warning: Unknown type ve.Transaction
-> ve.dm.Transaction
* Warning: Unknown type ve.dm.AnnotationSet
-> ve.AnnotationSet
* Warning: Unknown type false
-> boolean
* Warning: Unknown type ve.dm.AlienNode
ve.dm doesn't have a generic AlienNode like ve.ce
-> Unknown type ve.dm.AlienInlineNode|ve.dm.AlienBlockNode
* Warning: Unknown type ve.ve.Surface
-> ve.ce.Surface
* ve.example.lookupNode:
-> Last @param should be @return
* ve.dm.Transaction.prototype.pushReplace:
-> @param {Array] should be @param {Array}
* Warning: ve.BranchNode.js:27: {@link ve.Node#hasChildren} links to non-existing member
-> (removed)
* Warning: ve.LeafNode.js:21: {@link ve.Node#hasChildren} links to non-existing member
-> (removed)
Differences fixed:
* Variadic arguments are like @param {Type...} [name]
instead of @param {Type} [name...]
* Convert all file headers from /** to /*! because JSDuck tries
to parse all /** blocks and fails to parse with all sorts of
errors for "Global property", "Unnamed property", and
"Duplicate property".
Find: \/\*\*([^@]+)(@copyright)
Replace: /*!$1$2
* Indented blocks are considered code examples.
A few methods had documentation with numbered lists that were
indented, which have now been updated to not be intended.
* The free-form text descriptions are parsed with Markdown,
which requires lists to be separated from paragraphs by an
empty line.
And we should use `backticks` instead of {braces} for inline
code in text paragraphs.
* Doc blocks for classes and their constructor have to be
in the correct order (@constructor, @param, @return must be
before @class, @abstract, @extends etc.)
* `@extends Class` must not have Class {wrapped}
* @throws must start with a {Type}
* @example means something else. It is used for an inline demo
iframe, not code block. For that simply indent with spaces.
* @member means something else.
Non-function properties are marked with @property, not @member.
* To create a link to a class or member, in most cases the name
is enough to create a link. E.g. Foo, Foo.bar, Foo.bar#quux,
where a hash stands for "instance member", so Foo.bar#quux,
links to Foo.bar.prototype.quux (the is not supported, as
"prototype" is considered an implementation detail, it only
indexes class name and method name).
If the magic linker doesn't work for some case, the
verbose syntax is {@link #target label}.
* @property can't have sub-properties (nested @param and @return
values are supported, only @static @property can't be nested).
We only have one case of this, which can be worked around by
moving those in a new virtual class. The code is unaltered
(only moved down so that it isn't with the scope of the main
@class block). ve.dm.TransactionProcessor.processors.
New:
* @mixins: Classes mixed into the current class.
* @event: Events that can be emitted by a class. These are also
inherited by subclasses. (+ @param, @return and @preventable).
So ve.Node#event-attach is inherited to ve.dm.BreakNode,
just like @method is.
* @singleton: Plain objects such as ve, ve.dm, ve.ce were missing
documentation causing a tree error. Documented those as a
JSDuck singleton, which they but just weren't documented yet.
NB: Members of @singleton don't need @static (if present,
triggers a compiler warning).
* @chainable: Shorthand for "@return this". We were using
"@return {classname}" which is ambiguous (returns the same
instance or another instance?), @chainable is specifically
for "@return this". Creates proper labels in the generated
HTML pages.
Removed:
* @mixin: (not to be confused with @mixins). Not supported by
JSDuck. Every class is standalone anyway. Where needed marked
them @class + @abstract instead.
Change-Id: I6a7c9e8ee8f995731bc205d666167874eb2ebe23
ve.Range
* Rewrote truncate so that it works as expected, truncation should always reduce the length using the start/end values, not the from/to values
ve.ui.Inspector
* Added a comment about where the name argument to onBeforeInspectorOpen comes from, since it's a little bit confusing on first read (and I wrote it!)
* Calling onInitialize, onOpen and onClose methods directly, since we need to control the before or after-ness of listeners getting in there and doing their stuff - plus it's more direct
* Removed onRemove stub, which is never actually called
* Added before/after versions of initialize, open and close events
* Got rid of recursion guard since we don't need it anymore thanks to changes made in ve.dm.Surface (see below)
ve.ui.Context
* Updated event names to deal with new before/after naming of initialize, open and close events
* Removed fade-in logic since fading in doesn't even work anymore - since now we now annotate first, then open the inspector, the menu will actually exist and be open when we open the inspector even though you don't see it because it's quickly obscured
ve.ui.LinkInspector
* Made fragments non-auto selecting, in the case of onInitialize we actually call select(), which is silly since we were using an auto-selecting fragment - it's clear that this was a mistake
ve.dm.Surface
* Moved locking (polling stop and start) to the far outside edges of the change method
* I need a lot of eyes and testing on this change, it seems OK to me, but I'm suspicious that it may have side effects
* What was happening is that selection changes were being applied and then the poll was picking them up, and then the selection was coming in again as a change, but it wasn't a change at all, it was just feedback - this change event was then closing the inspector the instant it was opened - the odd part was that this only occurred when you selected backwards, which seems to be caused by the range being normalized, so it looked like a new selection even though it wasn't
ve.dm.Document
* trimOuterSpace from Range didn't consider annotated spaces to be spaces, by using the [0] trick (first character of a plain text character string or first element in an annotated character's array both are the character's value - but elements don't have a property named '0' so it skips those safely as well) we can always get the right value for comparison
Change-Id: I0873d906c058203b83b8d4bbe5a4b274f05a26fd
ve.dm.Document
* 2 of the 3 paths of getSlice still returned arrays instead of ve.dm.DocumentSlice objects
ve.ce.Surface
* Translate the range using the insertion transaction and truncate it, so the cursor ends up just after the pasted content
Change-Id: If7bae5e254ec84a847c1d3527f74d9c09c2d82b4
ve.ce.Surface
* Switched to using getSlice instead of getData in copy and paste handlers
* Added try/catch which attempts to build a transaction with the unbalanced data first, but falls back on the balanced data otherwise
ve.dm.*Node
* Added default style attributes (now used by ve.dm.NodeFactory)
ve.dm.Document
* Fixed bugs in fixupInsertions where parentType was being set with an object rather than a string
* Made use of getDataElement
* Added adoption capability so that inserting a</h1><p>b into <p>c[cursor]d</p> results in <p>ca</p><p>bd</p> rather than throwing an exception
* Renamed getBalancedData to getSlice, now retuning a ve.dm.DocumentSlice object
ve.dm.DocumentSlice
* Introduced new container for balanced data and a range of the original context - useful for copy/paste
ve.dm.NodeFactory
* Added getDataElement method, which uses default attributes to create a boilerplate version of a data element
ve.dm.Document.test
* Updated getBalancedData test to be a getSlice test
demos/ve/index, VisualEditor, test/index
* Added references to ve.dm.DocumentSlice
Change-Id: Id9269a29e51ca213508de8f155d3feec5e5b0774
ve.ce.Surface
* Added ve.ce.Surface.adjustCursor, which replaces repetitive and buggy code that was handling left and right arrow key presses
* New method only affects the selection target, so it won't collapse the selection on you - this was what caused bug 42401
* Made hasSlugAtOffset() actually return a boolean
ve.dm.Document
* Fixed turn-around issue in ve.dm.Document.getRelativeOffset - if the offset is already valid and we can't move in the direction we want, we should just leave it be, not turn around
* Since this method was being used by ve.ce.Surface to correct the cursor position on arrow key presses, it was causing the strange cursor jumping when you pressed an arrow key while at the edge of a document
ve.dm.SurfaceFragment
* Fixed typo where getAnnotationRangeFromSelection was preserving selection direction, but checking the wrong properties
ve.dm.Document.test
* Added tests that verify turn-around issue is fixed
Change-Id: Iba55cfc3d531e7d1333b78c94912ff22179aace8
This is cleaner than passing around the attributes separately, and it
allows us to access the annotations in dm.LeafNode as well.
Change-Id: Ie5b90988114835831cbe5cdccf63c7cd45719e31
