Previously we returned ElementLinearData from the converter, then
stripped out the MetaLinearData. This meant that before processing
the ElementLinearData from the converter actually contained metadata
which is confusing.
The new document constructor stores the converter results in a
FlatLinearData object and simultaneously populates element and meta
data stores.
Also in this commit I have moved various methods from ElementLinearData
to FlatLinearData, from which ElementLinearData inherits.
Change-Id: I64561bde2c31d8f703c13ac7b0a0c5f7ade9f3d4
InternalList.clone() assumed that all properties are automatically rebuilt
when a new document is built, but that's not true for .nextUniqueNumber
(or for .itemHtmlQueue for that matter). This meant that, in practice,
.nextUniqueNumber was being reset to 0 after auto/N numbers for existing
references had been assigned, but before assigning numbers to newly
created references. This caused all sorts of naming collision fun.
Bug: 54712
Change-Id: I1d087a5f3c23979d7d488e3ab32eb064ebc23e94
Document slice only ever contained linear data, with extra functionality
to preserve the range. It pre-dated LinearData, but now we should
refactor it to reflect its purpose.
Change-Id: Ifc908f7526c83a43a51372c8d2494d7260e7facd
We already getSlice which returns a ve.dm.DocumentSlice, so using
the word slice in this method is very confusing. What we are actually
doing is creating a ve.dm.Document from a range. Also remove argument
overloading as it's not particularly helpful and would make the new
name a lie.
Change-Id: I93da3419510410b170396e6765fbe2a87f9795be
The way we implemented undoing transactions was horrible. We'd process
the original transaction, but with a reversed=true flag. That meant we
had to keep track of the 'reversed' flag everywhere, and use ternaries
like insert = reversed ? op.remove : op.insert; all over the place to
access transaction operations. Redo then worked by reapplying the
transaction. We would verify that this was OK by tracking whether the
transaction was in an applied state or an undone state.
This commit makes it so every transaction can only be applied once. To
undo, you obtain a mirror image of the transaction with tx.reverse(),
then apply that. To redo, you clone the original transaction with
tx.clone() and apply that. All the code that had to use ternaries to
check whether the transaction was being applied in reverse or not is
gone now, because you can only apply a given transaction forwards,
never in reverse.
Bonus:
* Make ve.dm.Document's .completeHistory a simple array of
transactions, rather than transaction/boolean pairs
* In the protection of double application test, clone the example
document properly; it modified ve.dm.example.data, which was "fine"
because it ran .commit() and .rollback() the same number of times
Change-Id: I3050c5430be4a12510f22e20853560b92acebb67
Replaces newFromNodeReplacement(). newFromNodeReplacement was very
simplistic and didn't support metadata or internal list items, so
if you had comments or references inside of the data you were editing
(reference contents or an image caption), they'd get mangled.
With this, you can do:
newDoc = doc.getDocumentSlice( node );
// Edit newDoc
tx = ve.dm.Transaction.newFromDocumentReplace( doc, node, newDoc );
surface.change( newDoc );
and that takes care of metadata, internal list items, and things like
references that reference internal list items.
ve.dm.Document.js:
* In getDocumentSlice(), store a reference to the original document
and the number of items in its InternalList at the time of slicing
in the created slice. This is used for reconciliation when the
modified slice is injected back into the parent document with
newFromDocumentReplace().
ve.dm.InternalList.js:
* Add a method for merging in another InternalList. This provides a
mapping from old to new InternalList indexes so the linear model data
being injected by newFromDocumentReplace() can have its InternalList
indexes remapped.
ve.dm.Transaction.js:
* Replace newFromNodeReplacement() with newFromDocumentReplace()
ve.ui.MWMediaEditDialog.js, ve.ui.MWReferenceDialog.js:
* Use getDocumentSlice/newFromDocumentReplace for editing captions/refs
* Change insertion code path to insert an empty internalItem/caption, then
newFromDocumentReplace into that
* Add empty internalList to new mini-documents
ve/test/dm/ve.dm.Transaction.test.js:
* Replace newFromNodeReplacement tests with newFromDocumentReplace tests
ve-mw/test/dm/ve.dm.Transaction.test.js (new):
* Add tests for newFromDocumentReplace with mwReference nodes
ve.dm.mwExample.js:
* Add data for newFromDocumentReplace with mwReference tests
VisualEditor.hooks.php:
* Add new test file
Bug: 52102
Change-Id: I4aa980780114b391924f04df588e81c990c32983
Code with a similar purpose was added in 568e0e5701 but got lost
when some things were moved from ve.Surface to ve.ce.Surface in
5012ed10.
Initializing the selection at (0,0) was known to cause problems before,
and since 789d0caf09 breaks editing of empty documents: typing in an
empty document begins in an inline slug, but SurfaceObserver doesn't
notice typing in an inline slug unless the ce.Surface pawns it, which
is OK because insertions in slugs are always pawned, but the pawning
logic believes the cursor to be at offset 0 where there is no slug
(it's at offset 1) and so it doesn't pawn.
Bonus: update tests and add descriptions for dm.Surface.change tests
Change-Id: Id72314d0fe650dacc7cdb842f5cea2f3bfba5145
The previous implementation couldn't deal with transactions that
replaced both data and metadata at the same time (rather than replacing
data while moving metadata around), and extending its approach to
deal with that case would have made it much more complex.
So I rewrote the algorithm from scratch. The previous implementation
scheduled deferred moves for existing items, but immediately processed
insertions and removals. This is problematic for replacements and
maintaining the order in the binary search list. So instead, this new
implementation builds an array representing what the new item list
should be, then processes insertions, removals and moves in the correct
order to achieve that state.
It looks like the previous implementation didn't always work correctly,
which was masked because the test suite passed full=false to
assertItemsMatchMetadata(). This rewrite fixes this.
Also remove setMove/applyMove from MetaItem, because we don't need them
anymore and they're evil anyway; and add isAttached(), because the new
algorithm needs it.
Change-Id: I899d2b3c94c2cfa55823879bca95456750f64382
Transactions that replaced metadata twice at the same offset
(with retainMeta-replaceMeta-retainMeta-replaceMeta) were broken
because the processor for replaceMetadata didn't advance the
metadata cursor.
Change-Id: I7ad24e7ffb4c39b40ec9c347db301f8e28f3692d
Add some test cases for documents with trailing metadata, and fix an
off-by-one error in the metadata-mutating transactions (since the
document metadata array is one larger than the document data array).
Change-Id: I8f049466e03ed55010dfcf0a35702536edfa7b0a
This version pushes a `replaceMetadata` operation after a `replace` to
fixup trailing metadata if there is no inserted region and the removed
region includes metadata. This avoids a corner case where the
size of the metadata arrays inserted/removed in `Transaction.pushReplace()`
do not match the size of the data arrays inserted/removed.
We remove the now-unused `Document.getMetadataReplace()` method.
We also adjust `MetaList.onTransact()` so that it continues to work
properly when the number of metadata entries in `replace.insert` is
not the same as the number of metadata entries in `replace.remove`.
Change-Id: I1d600405b855ca1cb569853bb885b0752df47173
The `Transaction.pushReplace` method has a corner case if the removed
region has metadata and the inserted region is empty. This works fine
unless there are two adjacent `pushReplace` operations, which can occur
in `Transaction.newFromUnwrap`. Fix this by having `pushReplace` look
at a preceding replace and correctly merge the two operations if
possible (in particular in the tricky case where the previous case has
a zero-length insertion). Pleasantly, this can be done without a lot of
special-casing code in `pushReplace` or `newFromUnwrap`.
Add test cases verifying the `newFromUnwrap` works correctly (both
in commit and in rollback) when there is metadata present.
Change-Id: I6cfec0d2b1823dad724422f018a3c73dc0c7f186
Doing retainMetadata followed by a replace (not replaceMetadata)
doesn't make any sense and its behavior is undefined, so don't do that
in the test suite.
Change-Id: Ica032b0a5122d24e40e43e3eb43fea940270aece
In ve.dm.Document.getMetadataReplace(), we used to only merge metadata
if the amount removed is larger than the amount inserted. But this
could end up putting metadata in odd positions, for example if you
have Foo[[Category:Bar]]Baz and you delete 'ooBa' and replace it with
{image}xxx{/image}, then the category ends up inside the image.
We should always merge metadata when a segment is deleted, so that it
appears outside any new structure added.
There's a weird corner case here when a segment is removed but no
insertion is made: the removed metadata then needs to get glommed onto
the next element. We extend the insert/replace metadata array
when this happens.
Bug: 53444
Bug: 53445
Change-Id: I51d55fb370b473273f9cf152fdd0f356377d4109
This avoids problems when unnamed references were copy-pasted.
Knowing that key is always non-null simplifies a lot of logic
elsewhere.
Bug: 53365
Change-Id: I3a23123ae732d9583814d38dd880a0cdf691fd5d
Currently ignores all non-element data, but element content (e.g.
images) can be annotated.
Added test cases and updated the test runner to only compare
store indexes for a more readable output.
Bug: 50127
Change-Id: I234586a28072811c8288aab56f6abaaa0da0c88d
This is bit of a hack, as leading whitespace could be
significant if styled with white-space:pre.
Long term VE shouldn't be editing the user's HTML, and
should just highlight potential formatting issues.
We avoid the stripping in preformatted elements as we
expect they will have that styling.
Bug: 51462
Change-Id: I654d98e17dd604cb2a192831ff3f3597f95b9962
These have been pointing to the same method for a while now,
we can safely remove these obsolete aliases and just use it
as generic copy.
* Each file touched by my editor had its new line at EOF fixed
where absent
* Don't copy an otherwise unused empty object
(ve.dm.Converter)
* Use common ve#copy syntax instead to create a link
(ve.dm.Document, ve.dm.example)
* Remove redundant conditionals for isArray/copyArray/copyObject
(ve.dm.example)
Change-Id: If560e658dc1fb59bf01f702c97e3e82a50a8a255
We would dirty-diff "</span>\n<!-- comment -->\n<span>" to
"</span>\n\n<!-- comment --><span>", i.e. the second newline made
a bunny-hop to the left over the comment.
The actual bug turned out to involve a double bunny-hop, with
"</span> <!-- comment -->\n<span>" turning into
"</span>\n <!--comment --><span>", i.e. the newline bunny-hops
both the comment and the space.
This happened because outputWrappedMetaItems() didn't take
wrappedWhitespace into account when restoring meta items and
associated whitespace. I hacked a check for wrappedWhitespace into it,
but we should really just rewrite this pile of hacks into a unified
system for queuing and processing both whitespace and metadata.
Change-Id: I4375f4c07983ffec6877d0371aeaa9bf6e65fd6e
To avoid confusion between IV store indexes and the index
within the set, rename them to storeIndex and offset.
Change-Id: Ic7d741bd5d39240d63fdc04a2df45658a64441de
An empty document is one which contains no 'real' data, so
we should check for meta-only documents when deciding whether
to add in a wrapper paragraph.
Bug: 50289
Change-Id: Ib3ebf0717aa0c6c51fd1d0b14e95de50b2842647
It was being used correctly in the textual replace case, but it was
omitted in the structural replace case. This caused rare, insidious
metadata placement bugs that I haven't been able to reproduce outside
of newFromDocumentInsertion testing, but this might explain some of
the odd category bugs that have been reported.
Change-Id: I1424e482303853f285e4516a93c9609076648eff
Any annotation could have a style attribute which has
a compound effect, and this also prevents roundtrip
errors. Button tool logic should prevent any annotation
from being applied twice which shouldn't be.
Bug: 49755
Change-Id: I8502a55cd2b195d28d0e2ecd63f15de670f80d60
Move all MW-specific files into the ve-mw directory, in preparation
for moving them out into a separate repo.
All MW-specific files were moved into a parallel directory structure
in modules/ve-mw . Files with both generic and MW-specific things were
split up. Files in ve/init/mw/ were moved to ve-mw/init/ rather than
ve-mw/init/mw ; they're still named ve.init.mw.* but we should change
that. Some of the test files for core classes had MW-specific test cases,
so those were split up and the test runner was duplicated; we should
refactor our tests to use data providers so we can add cases more easily.
Split files:
* ve.ce.Node.css
* ve.ce.ContentBranchNode.test.js (MWEntityNode)
* ve.ce.Document.test.js (some core test cases genericized)
* ve.dm.InternalList.test.js (uses mwReference test document)
* ve.dm.SurfaceFragment.test.js, ve.ui.FormatAction.test.js
** Made core tests use heading instead of mwHeading
** Updated core tests because normal headings don't break out of lists
** Moved test runners into ve.test.utils.js
* ve.ui.Icons-*.css
* ve.ui.Dialog.css (MW parts into ve.ui.MWDialog.css)
* ve.ui.Tool.css
* ve.ui.Widget.css (move ve-ui-rtl and ve-ui-ltr to ve.ui.css)
ve.dm.Converter.test.js: Moved runner functions into ve.test.utils.js
ve.dm.example.js:
* Refactored createExampleDocument so mwExample can use it
* Removed wgExtensionAssetsPath detection, moved into mw-preload.js
* Genericized withMeta example document (original version copied to mwExample)
* Moved references example document to mwExample
ve.dm.mwExample.js:
* Move withMeta and references example documents from ve.dm.example.js
* Add createExampleDocument function
ve-mw/test/index.php: Runner for MW-specific tests only
ve-mw/test/mw-preload.js: Sets VE_TESTDIR for Special:JavaScriptTest only
ve.ui.Window.js:
* Remove magic path interpolation in addLocalStyleSheets()
* Pass full(er) paths to addLocalStyleSheets(), here and in subclasses
ve.ui.MWDialog.js: Subclass of Dialog that adds MW versions of stylesheets
ve.ui.MW*Dialog.js:
* Subclass MWDialog rather than Dialog
* Load both core and MW versions of stylesheets that have both
ve.ui.PagedDialog.js: Converted to a mixin rather than an abstract base class
* Don't inherit ve.ui.Dialog
* Rather than overriding initialize(), provide initializePages() which the
host class is supposed to call from its initialize()
* Rename onOutlineSelect to onPageOutlineSelect
ve.ui.MWMetaDialog.js, ve.ui.MWTransclusionDialog.js:
* Use PagedDialog as a mixin rather than a base class, inherit MWDialog
bullet-icon.png: Unused, deleted
Stuff we should do later:
* Refactor tests to use data providers
* Write utility function for SVG compat check
* Separate omnibus CSS files such as ve.ui.Widget.css
* Separate omnibus RL modules
* Use icon classes in ViewPageTarget
Change-Id: I1b28f8ba7f2d2513e5c634927a854686fb9dd5a5
Set a static property on big, small, sup, sub to allow them
to be added multiple times to an annotationSet.
Fix the converter to count out annotations when opening/closing.
Bug: 49755
Change-Id: Ifbede9345a66434022dbd681eada447ab81ab025
We currently change <ref name="foo">Foo</ref> ... <ref name="foo">Foo</ref>
to <ref name="foo">Foo</ref> ... <ref name="foo" /> , because know
that the second ref tag isn't canonical and so we blank it.
Instead, we now preserve the contents of all ref tags that come after
the canonical one.
Change-Id: I45a51a879271890fe46c4184f1029f12d27af678
<table>\n\n</table> round-tripped to <table>\n\n\n\n</table> because
we would store '\n\n' in both the innerPre and innerPost fields.
Fixed by not setting innerPost if the element is empty.
Change-Id: I0393bfaf9793fdebc8fff72c8760113fa69bb2bd
The converter wasn't setting .annotations on meta items created to
represent empty annotations, which meant that HTML like
<i>Foo<b></b></i> would end up as <i>Foo</i><b></b> in the linmod.
Change-Id: I13d7d9820beeee1e8c3673e08051361d6c6ac4cd
If you had <meta /><b>Annotated text</b> in a wrapper paragraph,
the converter would swap them and output the linear model equivalent
of <p wrapper><b>Annotated text</b></p><meta />.
This happened because the meta item was queued, and annotations didn't
trigger metadata queue flushes. The fix is to trigger a metadata queue
flush whenever we're about to write something that isn't itself queued.
Change-Id: I168abc0392fbec5503271d1653ee5c38518f857d
Objectives:
* Split reference dialog (at least for now) an edit and an insert dialog
* Add reference search widget for selecting an existing source, or
choosing to add a new one
* Abstract reference names, don't allow editing them and generate them
when needed
* When editing groups, move the internal item and update all references
to it
* Resolve name conflicts when moving a reference to a new group by
generating a new list key
Bonus:
* Add getNodeGroups method to internal list
* Add getUniqueListKey method to internal list
* Add destroy functionality to ce.node to release events and references
Bug: 49733
Change-Id: Ib244ff6ad9b4cee1decfd9b9e1d3d4e9cdcfb78c
* Port the class preservation logic from BlockImage to InlineImage
* Add preservation for unrecognized classes
* Add the logic for collapsing spaces in the class attribute in more
places
Change-Id: I26faad7e00ab2f0a0f5d076552e56b32c692ae74
