Let's experiment with this via our local Gruntfile. If it works
fine we can install it in Jenkins (similar to node-csslint).
Verify through $ npm install && npm test;
Fixed all outstanding violations.
Also:
* Added syntaxhighight to ignore.
* Added imetests (which contain unformatted JSON) to ignore.
* In ve.dm.ModelRegistry#matchTypeRegExps, removed redundant
!! cast from the [+!!withFunc] statement which was hitting
a bug in node-jscs. All callers to this local private function
pass a literal boolean true/false so no need to cast it.
* Removed "/* key .. , value */" from ve.setProp, though this
wasn't caught by node-jscs, found it when searching for " , ".
* Made npm.devDependencies fixed instead of using tilde-ranges.
This too often leads to strange bugs or sudden changes. Fixed
them at the version they were currently ranging to.
Bug: 54218
Change-Id: Ib2630806f3946874c8b01e58cf171df83a28da29
ve.ui.MWReferenceDialog.prototype.teardown:
* Pass ranges rather than nodes to transaction builders
* Don't do a removal in insertion mode, we know the removal range
will be empty
ve.dm.Transaction.newFromDocumentInsertion:
* Correctly splice the edited internalItem into listData in the case
that newDoc isn't a document slice of doc
* Rename range to spliceItemRange for clarity
* Introduce spliceListNodeRange and set it to either newDoc's
listNodeRange or doc's depending on whether newDoc is a slice of doc
Bug: 57683
Change-Id: Iae7de7701ae86bed89b707038407243d82249e1a
Allow pasting of rich (HTML) content.
ve.ce.Surface
* Use a sliced document clone for converting to DM HTML (copy)
* Add full context to pasteTarget before copying
* Add ve-pasteProtect class to spans to prevent them being dropped
* Implement external paste by converting HTML to data and inserting
with newFromDocumentInsertion
* Remove clipboard key placeholder after read so they aren't picked
up by rich paste. Hash no longer includes the placeholder.
* Detect the corruption of important spans and fallback to clipboard
data HTML if available.
ve.dm.LinearData
* Add clone method for copy
ve.dm.ElementLinearData
* Add compareUnannotated for use by context diffing.
* Add sanitize method for cleaning data according to a set of rules.
ve.dm.Transaction
* Add range parameter for inserting a range of a document only,
e.g. stripping the paste context.
ve.dm.Document
* Implement sliced document clone creation so that DM HTML
is generated correctly in onCopy
ve.dm.DocumentSlice
* Replaces LinearDataSlice. Now has two ranges for balanced data
and data with a full context.
ve.init.Target.js
* Define default, loose, paste rules (just remove aliens).
ve.init.mw.ViewPageTarget
* Define strict MW paste rules:
+ no links, spans, underlines
+ no images, divs, aliens
+ strip extra HTML attribues
ve.init.sa.Target, ve.init.mw.ViewPageTarget, ve.ui.Surface
* Pass through and store paste rules.
Bug: 41193
Bug: 48170
Bug: 50128
Bug: 53828
Change-Id: I38d63e31ee3e3ee11707e3fffed5174e1d633b42
In order to do this we have to separate out the removal
operation from NDFR, so it becomes newFromDocumentInsertion
(again, although actually, for the first time). As NFDI is
an insertion we can just run fixUpInsertion on the data
part of it.
In order for the removal operation to be a proper removal
we have to allow metadata removal (the default is to merge it).
Change-Id: I16d575b61b9796e7e889f2c27cfe02b4a40b7639
This fixes some of the problems with pasting references.
It's a bit overzealous in that references get renumbered even when
replacing, which is unnecessary but doesn't actually have any
noticeable effect.
Unfortunately, the internal list state depends so much on the converter
having run that we now need to add yet another hack, to set the counter
to the appropriate value.
Change-Id: I3c6514ce600af4f4c037f419554d34b5a5c86a63
* Our metadata insertions now need to be the same length as the data
insertion, not one more, so:
** Remove the +1 in the listMetadata splice
** Shorten the metadata variable by dropping the merging of the
metadata right before and right after the internal list; it was
also including the metadata right after the internal list twice
*** We still need to deal with this in some way though, left a TODO
** Fix the metadata insertion test for these changes
* Fix null reference keys in the test data; we made all references
keyed a while ago, but this test data was never updated for that
** The remapping of reference data doesn't remap auto/N keys yet,
left a FIXME for that
Change-Id: I8ef4e6ee7c1808574d81d0b83294848afd400cd7
The way we implemented undoing transactions was horrible. We'd process
the original transaction, but with a reversed=true flag. That meant we
had to keep track of the 'reversed' flag everywhere, and use ternaries
like insert = reversed ? op.remove : op.insert; all over the place to
access transaction operations. Redo then worked by reapplying the
transaction. We would verify that this was OK by tracking whether the
transaction was in an applied state or an undone state.
This commit makes it so every transaction can only be applied once. To
undo, you obtain a mirror image of the transaction with tx.reverse(),
then apply that. To redo, you clone the original transaction with
tx.clone() and apply that. All the code that had to use ternaries to
check whether the transaction was being applied in reverse or not is
gone now, because you can only apply a given transaction forwards,
never in reverse.
Bonus:
* Make ve.dm.Document's .completeHistory a simple array of
transactions, rather than transaction/boolean pairs
* In the protection of double application test, clone the example
document properly; it modified ve.dm.example.data, which was "fine"
because it ran .commit() and .rollback() the same number of times
Change-Id: I3050c5430be4a12510f22e20853560b92acebb67
Replaces newFromNodeReplacement(). newFromNodeReplacement was very
simplistic and didn't support metadata or internal list items, so
if you had comments or references inside of the data you were editing
(reference contents or an image caption), they'd get mangled.
With this, you can do:
newDoc = doc.getDocumentSlice( node );
// Edit newDoc
tx = ve.dm.Transaction.newFromDocumentReplace( doc, node, newDoc );
surface.change( newDoc );
and that takes care of metadata, internal list items, and things like
references that reference internal list items.
ve.dm.Document.js:
* In getDocumentSlice(), store a reference to the original document
and the number of items in its InternalList at the time of slicing
in the created slice. This is used for reconciliation when the
modified slice is injected back into the parent document with
newFromDocumentReplace().
ve.dm.InternalList.js:
* Add a method for merging in another InternalList. This provides a
mapping from old to new InternalList indexes so the linear model data
being injected by newFromDocumentReplace() can have its InternalList
indexes remapped.
ve.dm.Transaction.js:
* Replace newFromNodeReplacement() with newFromDocumentReplace()
ve.ui.MWMediaEditDialog.js, ve.ui.MWReferenceDialog.js:
* Use getDocumentSlice/newFromDocumentReplace for editing captions/refs
* Change insertion code path to insert an empty internalItem/caption, then
newFromDocumentReplace into that
* Add empty internalList to new mini-documents
ve/test/dm/ve.dm.Transaction.test.js:
* Replace newFromNodeReplacement tests with newFromDocumentReplace tests
ve-mw/test/dm/ve.dm.Transaction.test.js (new):
* Add tests for newFromDocumentReplace with mwReference nodes
ve.dm.mwExample.js:
* Add data for newFromDocumentReplace with mwReference tests
VisualEditor.hooks.php:
* Add new test file
Bug: 52102
Change-Id: I4aa980780114b391924f04df588e81c990c32983
Documentation says third argument is 'data' when in fact it is
'insertion' which is then re-used as the result of fixupInsertion :/
Change-Id: I17c959f858eddddc90a6fa839da0d32da69a784f
Correct the return value descriptions for the `newFromMetadataInsertion` and
`newFromMetadataRemoval` methods (which appear to have been cut-and-pasted
from another method).
Change-Id: I019b0ef9c75ff47fc0f2df8c95da9d090710a5a8
This was already possible in the operation structure by using
removeMetadata and insertMetadata, but no transaction builders
generated transactions like that.
With this change, pushReplace() can be instructed to overwrite the
metadata with arbitrary data rather than collapsing it by passing the
optional insertMeta parameter.
Change-Id: I0272fe38e053161c738591cf0a7b447a0827d4dc
Add some test cases for documents with trailing metadata, and fix an
off-by-one error in the metadata-mutating transactions (since the
document metadata array is one larger than the document data array).
Change-Id: I8f049466e03ed55010dfcf0a35702536edfa7b0a
This version pushes a `replaceMetadata` operation after a `replace` to
fixup trailing metadata if there is no inserted region and the removed
region includes metadata. This avoids a corner case where the
size of the metadata arrays inserted/removed in `Transaction.pushReplace()`
do not match the size of the data arrays inserted/removed.
We remove the now-unused `Document.getMetadataReplace()` method.
We also adjust `MetaList.onTransact()` so that it continues to work
properly when the number of metadata entries in `replace.insert` is
not the same as the number of metadata entries in `replace.remove`.
Change-Id: I1d600405b855ca1cb569853bb885b0752df47173
The `Transaction.pushReplace` method has a corner case if the removed
region has metadata and the inserted region is empty. This works fine
unless there are two adjacent `pushReplace` operations, which can occur
in `Transaction.newFromUnwrap`. Fix this by having `pushReplace` look
at a preceding replace and correctly merge the two operations if
possible (in particular in the tricky case where the previous case has
a zero-length insertion). Pleasantly, this can be done without a lot of
special-casing code in `pushReplace` or `newFromUnwrap`.
Add test cases verifying the `newFromUnwrap` works correctly (both
in commit and in rollback) when there is metadata present.
Change-Id: I6cfec0d2b1823dad724422f018a3c73dc0c7f186
Avoid making a copy of the entire data array by using `doc.data` and
`doc.data.getLength()` instead of `doc.getData()` and
`doc.getData().length`.
Get rid of some unnecessary conditionals in `Transaction.newFromWrap`: the
`tx.pushReplace()` and `tx.pushRetain()` methods will already gracefully
no-op if the replaced/retained region is empty; we don't need to add
extra control flow to handle that case.
Clarify offset math in `Transaction.newFromWrap`.
Fix a minor bug in `Transaction.newFromInsertion`: the final retain didn't
account for the length of the removed region, if `doc.fixupInsertion`
creates one. (This usually doesn't happen.)
Remove an inaccurate TODO in `Transaction.pushReplaceMetadata()` -- it is
no longer a straight copy/paste of `Transaction.pushReplace`, so a refactor
isn't really called for.
Change-Id: I7d86a2449978365d69d4a5ed43116c2e9945475d
In ve.dm.Document.getMetadataReplace(), we used to only merge metadata
if the amount removed is larger than the amount inserted. But this
could end up putting metadata in odd positions, for example if you
have Foo[[Category:Bar]]Baz and you delete 'ooBa' and replace it with
{image}xxx{/image}, then the category ends up inside the image.
We should always merge metadata when a segment is deleted, so that it
appears outside any new structure added.
There's a weird corner case here when a segment is removed but no
insertion is made: the removed metadata then needs to get glommed onto
the next element. We extend the insert/replace metadata array
when this happens.
Bug: 53444
Bug: 53445
Change-Id: I51d55fb370b473273f9cf152fdd0f356377d4109
These have been pointing to the same method for a while now,
we can safely remove these obsolete aliases and just use it
as generic copy.
* Each file touched by my editor had its new line at EOF fixed
where absent
* Don't copy an otherwise unused empty object
(ve.dm.Converter)
* Use common ve#copy syntax instead to create a link
(ve.dm.Document, ve.dm.example)
* Remove redundant conditionals for isArray/copyArray/copyObject
(ve.dm.example)
Change-Id: If560e658dc1fb59bf01f702c97e3e82a50a8a255
Whenever there is more than 2 spaces (except the extra space
on a continued line of an @ tag, or the extra space on a
continued line of a list item) it causes a <pre> context.
Removed both spurious spaces that caused a <pre> and ones that
didn't but looked like it could.
When making an ordered or unordered list, the first item needs
to be on a new line and in block context (e.g. an empty line
before it). Otherwise it is rendered inline as 1. Foo 2. Bar
(such as in #rebuildNodes where both the ordered and unordered
lists were broken).
Change-Id: Id0f154854afbdc8e5a8387da92e6b2cdf0875f69
Broke reference insertion because it removed
newFromNodeReplacement(), which is still in use.
This reverts commit 1765e39b40
Change-Id: I043997715474ad4850329ff903eb7a8c61c8b453
This function builds a transaction that takes a document slice and
inserts it back into the document it came from, applying any changes
that were made.
This makes editing document slices simple:
slicedDoc = doc.getDocumentSlice( captionNode );
// Edit slicedDoc using a surface
tx = ve.dm.Transaction.newFromDocumentInsertion( doc, captionNode, slicedDoc );
surface.change( tx );
Specifically, newFromDocumentInsertion replaces the node's contents
with the document's contents (meaning any changes made to the node in
the meantime are lost). It also merges the stores internal lists
of the two documents and remaps indexes accordingly. This means editing
of references inside of references is supported.
This functionality is not specific to slices, and can also be used to
safely insert data from a paste buffer, with internal list data being
transplanted correctly.
ve.dm.MetaLinearData:
* Make merge( [ undefined, undefined, ... ] ) return undefined rather
than [].
ve.dm.Document:
* In getDocumentSlice, store a pointer to the original dm.Document in
the new one, and also store the length of the internal list. This
allows us to figure out which internal list items the two documents
have in common when we insert the modified slice back into the main
document.
* In getMetadataReplace, optionally take the inserted metadata as a
parameter, to allow for operations that insert both data and metadata.
Per Ed's review, rewrite this function to return null rather than {}
if no metadata needs to be replaced.
ve.dm.InternalList:
* Add method to merge two internal lists
ve.dm.Transaction:
* Remove newFromNodeReplacement and replace it with newFromDocumentInsertion.
* In pushReplace, optionally take the inserted metadata as a parameter.
Change-Id: I786ee7bad796aa54bc242993b4de3ad18ad0773e
Objective:
* Allow editing reference groups and names in the reference dialog
Bonus:
* Modify attribute transaction builder to support multiple attribute
changes in a single transaction
Changes:
ve.ui.MWReferenceDialog.js
* Load ref name and group from model
* Save ref name and group, if changed, to model
ve.ui.ListAction.js, ve.ui.Transaction.test.js, ve.ce.ResizableNode.js
* Update use of newFromAttributeChange to newFromAttributeChanges
ve.dm.SurfaceFragment.test.js
* Add test for new changeAttributes method
ve.dm.InternalList.js
* Missing new line at end of file
ve.dm.Transaction.js
* Change newFromAttributeChange to accept an list of attribute changes and
produce a single transaction that applies one or more attribute changes
at once
ve.dm.SurfaceFragment.js
* Fix bug in getCoveredNodes where the wrong mode name was being used
* Add changeAttributes method, which applies attributes to all covered
nodes and allows filtering of which types of nodes the attributes are
applied to
ve.dm.MWReferenceNode.js
* Actually write key and group back to DOM
* Separate onRoot functionality into addToInternalList so it can be called
separately (similarly onUnroot/removeFromInternalList)
ve.ce.MWReferenceListNode.js
* Clone internal item CE node before appending to avoid rendering bug.
*.php
* Add links to messages and sort them
Change-Id: Ic4121e4fcfc09265d5863af6f078cdeb77926c8e
Objectives:
* Fix compareObjects so it doesn't break when given arrays
* Remove compareArrays
* Rename compareObjects to compare and update callers of both methods
Changes:
ve.js
* Loosen check for whether to recurse so both arrays and objects qualify
* Remove compareArrays
* Rename compareObjects to compare
ve.dm.MWTemplateNode.js, ve.dm.AnnotationSet.js, ve.dm.Document.js,
ve.dm.Transaction.js
* Update uses of compare(Arrays|Objects) to use compare
Change-Id: I7d4f7ceb28c0389f0157b7598e291f21393b5b85
Getting & setting the cursor is done with byte offsets
instead of data model offset (characters) so we need to
be able to convert between the two as well as splitting
characters.
TODO: The regex only works on surrogate pairs, not
yet combining accents.
fixupInsertion will combine a combining mark with the
character to its left it it can.
Bug: 48630
Change-Id: I8d936fb15d82f73cd45fac142c540a7950850d55
Objective:
Generalize the shield and phantom magic, so we can use it for pretty much
any node we like. Usually this will be used with generated content nodes,
but also with aliens (of course) and possible other stuff in the future.
Bonus:
Also fixes a bug in DM that would crash VE when you selected to the end
and hit backspace.
Changes:
*.php
* Added links to files
aliens.html
* Added attributes to aliens to make them aliens again
ve.ce.AlienNode.js
* Moved shield and phantom functionality to ve.ce.ProtectedNode
ve.ce.AlienNode.js, ve.ce.MWReferenceListNode.js,
ve.ce.MWReferenceNode.js, ve.ce.MWTemplateNode.js
* Mixed in ve.ce.ProtectedNode
ve.ce.Node.css
* Reorganized styles and updated class names
* Added simple light blue hover with outline (using inset box shadow) for
protected nodes, same style as before for aliens
ve.ce.Surface.css
* Moved phantom styles to ve.ce.Node.css
ve.ce.BranchNode.js
* Moved call to setLive(false) to happen before detach() so that the
surface object is still available and events can be disconnected
ve.ce.BranchNode.js, ve.ce.Document.js, ve.ce.js, ve.ce.Surface.js, ve.ce.SurfaceObserver.js
* Adjusted CSS class names
ve.ce.Node.js
* Moved shield template to ve.ce.ProtectedNode
ve.ce.ProtectedNode.js
* New class, mix into another class to protect it from editing
ve.ce.RelocatableNode.js
* Renamed temporary surface property to relocatingSurface to avoid
confusion when debugging
ve.ce.Surface.js
* Moved phantom template to ve.ce.ProtectedNode
ve.dm.Transaction.js
* Fixed bug where most of the internal list was being deleted when the
end of the document was selected and the user pressed backspace
Change-Id: I2468b16e1ba6785ad298e38190e33493135719c3
Specifically by looking for "data-ve-changed",
"ChangeMarker*" and internal.changed.
Various tests, test counters and unused variables also
affected.
Bug: 45061
Change-Id: Ibd1ee68e0d650979d40574eff9cebded1a28499f
Remove all manual changes to SF ranges as these are not
undoable. Instead change translate range to default to
outer expand and build functionality around that behaviour
never changing.
As translate range is always outer I don't think we need to
check for start and end crossing over?
Added more undo tests to assert these selections are maintained
properly, and added the test case to 'update' for when and undo
point is overwritten.
Insert content now results in a selection over the inserted
content. Most usages were expecting this anyway and were
followed up with an adjustRange(-length,0) which is no longer
necessary.
Noticed that the link inspector case was never being triggered
as word boundary was always expanding to at least one char (mainly
for Hanzi selection). This doesn't make much sense as single
spaces get auto selected so removed this functionality.
Split collapseRange out into collapseRangeToStart and
collapseRangeToEnd as this may be required to get the old
behaviour (range moves to end after insert).
Change-Id: I3dc0b4d00d37bad1ca3076a69b41c5f0b3fa0570
In this case, selectNodes() returns the paragraph, which is not a content
node, and so newFromContentBranchConversion() decides to do nothing at all.
This change makes newFromContentBranchConversion() more intelligent about
finding the content branch to operate on, fixing cases such as these
where selectNodes() returns the content branch itself rather than one of
its children.
Bug: 41203
Change-Id: I710fbf184ef5ef84d9c2f5bca2b115e0660f5b8f
fixUpInsertion now returns an object with both data and offset
which allows offset changes.
Within fixUpInsertion we lazy-generate first/lastChildStack which
is a list of parent nodes for which the current node is the first/last
child. Whenever we try to close off a node we check these stacks and
if they are populated we instead use a recursive call to start
fixUpInsertion again but with the offset shifted by 1.
Bug: 46799
Change-Id: Ic51dd03725c11f1f7e279929534ee3afea14d662
In most places we call .contains we already know the index, so we
can avoid store lookups by using .containsIndex.
Change-Id: I45a9a421473f9bec479ab8ccceceb162b7004c3a
ve.dm.Transaction
* Replace operations are now built directly from the
linear model and automatically determine what metadata
replace information they need to include:
** retainMetadata
** replaceMetadata
** insertMetadata
ve.dm.Document
* Metadata array created empty and padded out after data parsing
as we are no longer using Document.spliceData to build it (a new
test checks for correct metadata length)
* spliceData replaced with getMetadataReplace, which instead returns
transactional steps of spliceData (retain, replace, insert)
ve.dm.MetaLinearData
* Add function for merging metadata items together. Only used
once in the code (Document.getMetadataReplace) but useful
for generating test data.
ve.dm.MetaList
* Replace operations with metadata need to calculate new offset
and indices directly, but can't be applied immediately lest they
put a metaItem out of place and affect findItem.
ve.dm.MetaItem
* Add methods to support queued moves as required by MetaList
Test files
* Updated to match new pushReplace API
* Remove any instances of Document.spliceData
* Extra check on sparse metadata array length
* Rewrite spliceData tests as getMetadataReplace tests
* Count expected cases in Transaction(Processor) tests
Bug: 46954
Change-Id: I4edad1c2dd37c723bff2792bab7d694ef17a86dc
Created an IndexValueStore class which can store any object and return
an integer index to its hash map.
Linear data is now stored in ve.dm.LinearData instances. Two subclasses
for element and meta data contain methods specific to those data types
(ElementLinearData and MetaLinearData).
The static methods in ve.dm.Document that inspected data at a given
offset are now instance methods of ve.dm.ElementLinearData.
AnnotationSets (which are no longer OrderedHashSets) have been moved
to /dm and also have to be instantiated with a pointer the store.
Bug: 46320
Change-Id: I249a5d48726093d1cb3e36351893f4bff85f52e2
rewrapAllNodes is effectively the same as unwrap then wrap
except it operates as one transaction as so avoids a potentially
invalid intermediate state.
Added some more comments for unwrapAllNodes and renamed a varaible
for consistency.
Fixed a typo in Transaction.newFromWrap comment.
Bug: 45242
Change-Id: Ie752a788d087055d97c7c6f75f59c6a2680d26c7
Previously, we would translate to the right of an insertion, but for
wrapping transactions that means we end up with something like
<ul><li>|<p>...</p></li></ul>|, which doesn't make any sense. This
change changes this to <ul><li>|<p>...</p>|</li></ul>.
* Add parameter to translateOffset() that toggles the behavior for
the offset before an insertion
* In translateRange(), translate start and end differently
** In some cases this can map them across each other, fix that
Change-Id: Ia2197b08d9f6763be3f2db5a59546ddc3f74a281
We already combine consecutive operations of other types, but didn't do
that for remove. We have to do this, though, because translateOffset()
gets confused otherwise.
Change-Id: I4285a3efd9f2297398f57e2f24adb26adafdf465
In our test case, offset 12 was mapped to 16, but that should be 11.
The problem here was that the offset right before the removal was
mapped to right after the removal, but that's only valid if we're
dealing with a removal, not when we're dealing with a replacement
where we're both removing and inserting data.
Change-Id: Ibf3c1463c0de009578cd50736f19bae82669ced8
