Commit graph

331 commits

Author SHA1 Message Date
Ed Sanders a8d84db0e8 Refactor out data processing from ve.dm.Document constructor
Also make collection of metadata and construction of nodes optional.

Change-Id: I02ba6d2199caccaf9fe9dcfba58eefa7b52c52b1
2013-10-07 17:26:21 +01:00
Ed Sanders 44b1fdebe4 Use FlatLinearData for storing converter results
Previously we returned ElementLinearData from the converter, then
stripped out the MetaLinearData. This meant that before processing
the ElementLinearData from the converter actually contained metadata
which is confusing.

The new document constructor stores the converter results in a
FlatLinearData object and simultaneously populates element and meta
data stores.

Also in this commit I have moved various methods from ElementLinearData
to FlatLinearData, from which ElementLinearData inherits.

Change-Id: I64561bde2c31d8f703c13ac7b0a0c5f7ade9f3d4
2013-10-06 20:27:32 +01:00
jenkins-bot b36c401ec7 Merge "Get rid of 'reversed' flag on transactions" 2013-10-04 06:18:06 +00:00
Roan Kattouw 69ad031bff When cloning the InternalList, pass through properties that aren't rebuilt
InternalList.clone() assumed that all properties are automatically rebuilt
when a new document is built, but that's not true for .nextUniqueNumber
(or for .itemHtmlQueue for that matter). This meant that, in practice,
.nextUniqueNumber was being reset to 0 after auto/N numbers for existing
references had been assigned, but before assigning numbers to newly
created references. This caused all sorts of naming collision fun.

Bug: 54712
Change-Id: I1d087a5f3c23979d7d488e3ab32eb064ebc23e94
2013-10-03 14:43:20 -07:00
Ed Sanders 5c31d3215b Change ve.dm.DocumentSlice to a mixin to ve.dm.LinearData
Document slice only ever contained linear data, with extra functionality
to preserve the range. It pre-dated LinearData, but now we should
refactor it to reflect its purpose.

Change-Id: Ifc908f7526c83a43a51372c8d2494d7260e7facd
2013-10-03 19:38:59 +01:00
Ed Sanders 1957eb3e28 Rename getDocumentSlice to cloneFromRange
We already getSlice which returns a ve.dm.DocumentSlice, so using
the word slice in this method is very confusing. What we are actually
doing is creating a ve.dm.Document from a range. Also remove argument
overloading as it's not particularly helpful and would make the new
name a lie.

Change-Id: I93da3419510410b170396e6765fbe2a87f9795be
2013-10-03 12:48:01 +01:00
Roan Kattouw 6772f92e70 Get rid of 'reversed' flag on transactions
The way we implemented undoing transactions was horrible. We'd process
the original transaction, but with a reversed=true flag. That meant we
had to keep track of the 'reversed' flag everywhere, and use ternaries
like insert = reversed ? op.remove : op.insert; all over the place to
access transaction operations. Redo then worked by reapplying the
transaction. We would verify that this was OK by tracking whether the
transaction was in an applied state or an undone state.

This commit makes it so every transaction can only be applied once. To
undo, you obtain a mirror image of the transaction with tx.reverse(),
then apply that. To redo, you clone the original transaction with
tx.clone() and apply that. All the code that had to use ternaries to
check whether the transaction was being applied in reverse or not is
gone now, because you can only apply a given transaction forwards,
never in reverse.

Bonus:
* Make ve.dm.Document's .completeHistory a simple array of
  transactions, rather than transaction/boolean pairs
* In the protection of double application test, clone the example
  document properly; it modified ve.dm.example.data, which was "fine"
  because it ran .commit() and .rollback() the same number of times

Change-Id: I3050c5430be4a12510f22e20853560b92acebb67
2013-10-02 19:37:08 -07:00
jenkins-bot bea113fee2 Merge "Introduce newFromDocumentReplace() transaction builder" 2013-09-26 20:45:54 +00:00
Roan Kattouw cf17789985 Introduce newFromDocumentReplace() transaction builder
Replaces newFromNodeReplacement(). newFromNodeReplacement was very
simplistic and didn't support metadata or internal list items, so
if you had comments or references inside of the data you were editing
(reference contents or an image caption), they'd get mangled.

With this, you can do:
newDoc = doc.getDocumentSlice( node );
// Edit newDoc
tx = ve.dm.Transaction.newFromDocumentReplace( doc, node, newDoc );
surface.change( newDoc );

and that takes care of metadata, internal list items, and things like
references that reference internal list items.

ve.dm.Document.js:
* In getDocumentSlice(), store a reference to the original document
  and the number of items in its InternalList at the time of slicing
  in the created slice. This is used for reconciliation when the
  modified slice is injected back into the parent document with
  newFromDocumentReplace().

ve.dm.InternalList.js:
* Add a method for merging in another InternalList. This provides a
  mapping from old to new InternalList indexes so the linear model data
  being injected by newFromDocumentReplace() can have its InternalList
  indexes remapped.

ve.dm.Transaction.js:
* Replace newFromNodeReplacement() with newFromDocumentReplace()

ve.ui.MWMediaEditDialog.js, ve.ui.MWReferenceDialog.js:
* Use getDocumentSlice/newFromDocumentReplace for editing captions/refs
* Change insertion code path to insert an empty internalItem/caption, then
  newFromDocumentReplace into that
* Add empty internalList to new mini-documents

ve/test/dm/ve.dm.Transaction.test.js:
* Replace newFromNodeReplacement tests with newFromDocumentReplace tests

ve-mw/test/dm/ve.dm.Transaction.test.js (new):
* Add tests for newFromDocumentReplace with mwReference nodes

ve.dm.mwExample.js:
* Add data for newFromDocumentReplace with mwReference tests

VisualEditor.hooks.php:
* Add new test file

Bug: 52102
Change-Id: I4aa980780114b391924f04df588e81c990c32983
2013-09-25 21:46:38 +00:00
Ed Sanders f001d306ad Node annotation blacklists
Allow nodes to specify a blacklist of annotations which can't be applied
to it.

Bug: 53151
Change-Id: Iac7649f4b38d8bfa94c64f734634afd45a5afc53
2013-09-19 19:09:25 +01:00
Roan Kattouw d3af8b877b dm.Surface: Initialize selection at (1,1) instead of (0,0)
Code with a similar purpose was added in 568e0e5701 but got lost
when some things were moved from ve.Surface to ve.ce.Surface in
5012ed10.

Initializing the selection at (0,0) was known to cause problems before,
and since 789d0caf09 breaks editing of empty documents: typing in an
empty document begins in an inline slug, but SurfaceObserver doesn't
notice typing in an inline slug unless the ce.Surface pawns it, which
is OK because insertions in slugs are always pawned, but the pawning
logic believes the cursor to be at offset 0 where there is no slug
(it's at offset 1) and so it doesn't pawn.

Bonus: update tests and add descriptions for dm.Surface.change tests

Change-Id: Id72314d0fe650dacc7cdb842f5cea2f3bfba5145
2013-09-18 01:06:34 +00:00
jenkins-bot 973d9f999d Merge changes I899d2b3c,I0272fe38,I7ad24e7f
* changes:
  Rewrite MetaList.onTransact
  Allow replace operations to replace metadata as well
  Fix processing of double metadata replacements
2013-09-17 21:01:41 +00:00
C. Scott Ananian 3928ca16fd Transactions: Add trailing retainMetadata when there is trailing metadata.
Ensure that all transactions move the cursor past the trailing
metadata, if present.

Change-Id: I869c75f8cfabe1e14bb66d8e627d0b750bde46af
2013-09-11 16:02:12 -07:00
Roan Kattouw ccef625a08 Rewrite MetaList.onTransact
The previous implementation couldn't deal with transactions that
replaced both data and metadata at the same time (rather than replacing
data while moving metadata around), and extending its approach to
deal with that case would have made it much more complex.

So I rewrote the algorithm from scratch. The previous implementation
scheduled deferred moves for existing items, but immediately processed
insertions and removals. This is problematic for replacements and
maintaining the order in the binary search list. So instead, this new
implementation builds an array representing what the new item list
should be, then processes insertions, removals and moves in the correct
order to achieve that state.

It looks like the previous implementation didn't always work correctly,
which was masked because the test suite passed full=false to
assertItemsMatchMetadata(). This rewrite fixes this.

Also remove setMove/applyMove from MetaItem, because we don't need them
anymore and they're evil anyway; and add isAttached(), because the new
algorithm needs it.

Change-Id: I899d2b3c94c2cfa55823879bca95456750f64382
2013-09-11 15:29:28 -07:00
Roan Kattouw eb90af29e0 Fix processing of double metadata replacements
Transactions that replaced metadata twice at the same offset
(with retainMeta-replaceMeta-retainMeta-replaceMeta) were broken
because the processor for replaceMetadata didn't advance the
metadata cursor.

Change-Id: I7ad24e7ffb4c39b40ec9c347db301f8e28f3692d
2013-09-11 15:26:12 -07:00
C. Scott Ananian 1f5b78ab94 Fix off-by-one error with metadata-mutating transactions.
Add some test cases for documents with trailing metadata, and fix an
off-by-one error in the metadata-mutating transactions (since the
document metadata array is one larger than the document data array).

Change-Id: I8f049466e03ed55010dfcf0a35702536edfa7b0a
2013-09-11 10:07:20 -07:00
jenkins-bot cba2935173 Merge "Add alt attribute to core image nodes" 2013-09-10 01:43:40 +00:00
Ed Sanders dd73b873a8 Add alt attribute to core image nodes
Also update and refactor tests.

Change-Id: I96d08faed42118b713af8011c5d6befb760e65c9
2013-09-07 12:56:17 -07:00
C. Scott Ananian 47545a5d6f Remove no-insertion metadata corner case from ve.dm.Transaction.pushReplace().
This version pushes a `replaceMetadata` operation after a `replace` to
fixup trailing metadata if there is no inserted region and the removed
region includes metadata.  This avoids a corner case where the
size of the metadata arrays inserted/removed in `Transaction.pushReplace()`
do not match the size of the data arrays inserted/removed.

We remove the now-unused `Document.getMetadataReplace()` method.

We also adjust `MetaList.onTransact()` so that it continues to work
properly when the number of metadata entries in `replace.insert` is
not the same as the number of metadata entries in `replace.remove`.

Change-Id: I1d600405b855ca1cb569853bb885b0752df47173
2013-09-06 01:07:37 +00:00
C. Scott Ananian 5830bce72c Correctly preserve metadata in Transaction.newFromUnwrap.
The `Transaction.pushReplace` method has a corner case if the removed
region has metadata and the inserted region is empty.  This works fine
unless there are two adjacent `pushReplace` operations, which can occur
in `Transaction.newFromUnwrap`.  Fix this by having `pushReplace` look
at a preceding replace and correctly merge the two operations if
possible (in particular in the tricky case where the previous case has
a zero-length insertion).  Pleasantly, this can be done without a lot of
special-casing code in `pushReplace` or `newFromUnwrap`.

Add test cases verifying the `newFromUnwrap` works correctly (both
in commit and in rollback) when there is metadata present.

Change-Id: I6cfec0d2b1823dad724422f018a3c73dc0c7f186
2013-09-06 01:06:59 +00:00
Roan Kattouw 73edc2a46f Add group to internal list items in test data
Previously we were just omitting the first parameter to
queueItemHtml()

Change-Id: Ib4b6d2877d6cfe6b8469a300f89a4a54cbec6da3
2013-09-05 11:25:49 -07:00
Roan Kattouw 0e51375aa9 Fix ridiculous MetaList test case
Doing retainMetadata followed by a replace (not replaceMetadata)
doesn't make any sense and its behavior is undefined, so don't do that
in the test suite.

Change-Id: Ica032b0a5122d24e40e43e3eb43fea940270aece
2013-09-05 11:25:49 -07:00
jenkins-bot fc5003aa35 Merge "Collapse metadata on any removal." 2013-09-04 02:22:31 +00:00
C. Scott Ananian 5095e27925 Collapse metadata on any removal.
In ve.dm.Document.getMetadataReplace(), we used to only merge metadata
if the amount removed is larger than the amount inserted.  But this
could end up putting metadata in odd positions, for example if you
have Foo[[Category:Bar]]Baz and you delete 'ooBa' and replace it with
{image}xxx{/image}, then the category ends up inside the image.

We should always merge metadata when a segment is deleted, so that it
appears outside any new structure added.

There's a weird corner case here when a segment is removed but no
insertion is made: the removed metadata then needs to get glommed onto
the next element.  We extend the insert/replace metadata array
when this happens.

Bug: 53444
Bug: 53445
Change-Id: I51d55fb370b473273f9cf152fdd0f356377d4109
2013-09-01 11:00:51 -04:00
Ed Sanders 52de8b76ec Always give references names. Ignore if used once.
This avoids problems when unnamed references were copy-pasted.

Knowing that key is always non-null simplifies a lot of logic
elsewhere.

Bug: 53365
Change-Id: I3a23123ae732d9583814d38dd880a0cdf691fd5d
2013-08-28 12:47:33 -07:00
Ed Sanders 380b368986 getAnnotationsFromRange should only ignore non-content data
Currently ignores all non-element data, but element content (e.g.
images) can be annotated.

Added test cases and updated the test runner to only compare
store indexes for a more readable output.

Bug: 50127
Change-Id: I234586a28072811c8288aab56f6abaaa0da0c88d
2013-08-21 22:32:34 +01:00
Ed Sanders 00c681f0d9 Generic support for multiple tags in TextStyleAnnotation
Bug: 52477
Change-Id: If8e2f541adb045f9b84a913829a5b7430ca83299
2013-08-09 04:15:45 +00:00
Ed Sanders d47cfe7d02 Handle <tt> as textStyle/code
Also store the node name so we round-trip correctly.

Bug: 52352
Change-Id: Id1fc85540804f5724ac7ded649a2d3ed0eb26770
2013-08-02 23:07:40 +01:00
Ed Sanders f153f932a2 Remove inserted leading whitespace
This is bit of a hack, as leading whitespace could be
significant if styled with white-space:pre.

Long term VE shouldn't be editing the user's HTML, and
should just highlight potential formatting issues.

We avoid the stripping in preformatted elements as we
expect they will have that styling.

Bug: 51462
Change-Id: I654d98e17dd604cb2a192831ff3f3597f95b9962
2013-08-02 19:01:22 +00:00
Timo Tijhof 44623c9b2a ve.copy: Remove obsolete copyArray and copyObject
These have been pointing to the same method for a while now,
we can safely remove these obsolete aliases and just use it
as generic copy.

* Each file touched by my editor had its new line at EOF fixed
  where absent
* Don't copy an otherwise unused empty object
  (ve.dm.Converter)
* Use common ve#copy syntax instead to create a link
  (ve.dm.Document, ve.dm.example)
* Remove redundant conditionals for isArray/copyArray/copyObject
  (ve.dm.example)

Change-Id: If560e658dc1fb59bf01f702c97e3e82a50a8a255
2013-07-30 01:44:22 +02:00
jenkins-bot 6355d9937f Merge "Fix the newline bunny-hop bug" 2013-07-27 18:07:07 +00:00
Roan Kattouw 4041d3f342 Fix the newline bunny-hop bug
We would dirty-diff "</span>\n<!-- comment -->\n<span>" to
"</span>\n\n<!-- comment --><span>", i.e. the second newline made
a bunny-hop to the left over the comment.

The actual bug turned out to involve a double bunny-hop, with
"</span> <!-- comment -->\n<span>" turning into
"</span>\n <!--comment --><span>", i.e. the newline bunny-hops
both the comment and the space.

This happened because outputWrappedMetaItems() didn't take
wrappedWhitespace into account when restoring meta items and
associated whitespace. I hacked a check for wrappedWhitespace into it,
but we should really just rewrite this pile of hacks into a unified
system for queuing and processing both whitespace and metadata.

Change-Id: I4375f4c07983ffec6877d0371aeaa9bf6e65fd6e
2013-07-26 16:24:27 -07:00
Ed Sanders f11db48234 Rename index to offset in AnnotationSet
To avoid confusion between IV store indexes and the index
within the set, rename them to storeIndex and offset.

Change-Id: Ic7d741bd5d39240d63fdc04a2df45658a64441de
2013-07-26 23:04:01 +00:00
jenkins-bot 5a5fb6030d Merge "Extend empty document check to check for meta-only data" 2013-07-15 16:18:50 +00:00
Ed Sanders aaf4f5b134 Extend empty document check to check for meta-only data
An empty document is one which contains no 'real' data, so
we should check for meta-only documents when deciding whether
to add in a wrapper paragraph.

Bug: 50289
Change-Id: Ib3ebf0717aa0c6c51fd1d0b14e95de50b2842647
2013-07-15 12:12:00 +01:00
Roan Kattouw bab7689feb Actually use op.retainMetadata in structural replace mode
It was being used correctly in the textual replace case, but it was
omitted in the structural replace case. This caused rare, insidious
metadata placement bugs that I haven't been able to reproduce outside
of newFromDocumentInsertion testing, but this might explain some of
the odd category bugs that have been reported.

Change-Id: I1424e482303853f285e4516a93c9609076648eff
2013-07-12 15:57:26 -07:00
Ed Sanders d2778b1df4 Fix annotated element alien metadata inside wrapped paragraph
Add check that we can close the wrapper to meta items, also
annotate them correctly.

Bug: 51235
Change-Id: Idd4a2304f588ea74b9a9814ce86c1fd0c026e441
2013-07-12 20:13:15 +01:00
Ed Sanders e91cd00a2f Make all annotations additive
Any annotation could have a style attribute which has
a compound effect, and this also prevents roundtrip
errors. Button tool logic should prevent any annotation
from being applied twice which shouldn't be.

Bug: 49755
Change-Id: I8502a55cd2b195d28d0e2ecd63f15de670f80d60
2013-07-05 21:01:20 +00:00
Roan Kattouw 92c38eab85 The great directory split of 2013
Move all MW-specific files into the ve-mw directory, in preparation
for moving them out into a separate repo.

All MW-specific files were moved into a parallel directory structure
in modules/ve-mw . Files with both generic and MW-specific things were
split up. Files in ve/init/mw/ were moved to ve-mw/init/ rather than
ve-mw/init/mw ; they're still named ve.init.mw.* but we should change
that. Some of the test files for core classes had MW-specific test cases,
so those were split up and the test runner was duplicated; we should
refactor our tests to use data providers so we can add cases more easily.

Split files:
* ve.ce.Node.css
* ve.ce.ContentBranchNode.test.js (MWEntityNode)
* ve.ce.Document.test.js (some core test cases genericized)
* ve.dm.InternalList.test.js (uses mwReference test document)
* ve.dm.SurfaceFragment.test.js, ve.ui.FormatAction.test.js
** Made core tests use heading instead of mwHeading
** Updated core tests because normal headings don't break out of lists
** Moved test runners into ve.test.utils.js
* ve.ui.Icons-*.css
* ve.ui.Dialog.css (MW parts into ve.ui.MWDialog.css)
* ve.ui.Tool.css
* ve.ui.Widget.css (move ve-ui-rtl and ve-ui-ltr to ve.ui.css)

ve.dm.Converter.test.js: Moved runner functions into ve.test.utils.js

ve.dm.example.js:
* Refactored createExampleDocument so mwExample can use it
* Removed wgExtensionAssetsPath detection, moved into mw-preload.js
* Genericized withMeta example document (original version copied to mwExample)
* Moved references example document to mwExample

ve.dm.mwExample.js:
* Move withMeta and references example documents from ve.dm.example.js
* Add createExampleDocument function

ve-mw/test/index.php: Runner for MW-specific tests only

ve-mw/test/mw-preload.js: Sets VE_TESTDIR for Special:JavaScriptTest only

ve.ui.Window.js:
* Remove magic path interpolation in addLocalStyleSheets()
* Pass full(er) paths to addLocalStyleSheets(), here and in subclasses

ve.ui.MWDialog.js: Subclass of Dialog that adds MW versions of stylesheets

ve.ui.MW*Dialog.js:
* Subclass MWDialog rather than Dialog
* Load both core and MW versions of stylesheets that have both

ve.ui.PagedDialog.js: Converted to a mixin rather than an abstract base class
* Don't inherit ve.ui.Dialog
* Rather than overriding initialize(), provide initializePages() which the
  host class is supposed to call from its initialize()
* Rename onOutlineSelect to onPageOutlineSelect

ve.ui.MWMetaDialog.js, ve.ui.MWTransclusionDialog.js:
* Use PagedDialog as a mixin rather than a base class, inherit MWDialog

bullet-icon.png: Unused, deleted

Stuff we should do later:
* Refactor tests to use data providers
* Write utility function for SVG compat check
* Separate omnibus CSS files such as ve.ui.Widget.css
* Separate omnibus RL modules
* Use icon classes in ViewPageTarget

Change-Id: I1b28f8ba7f2d2513e5c634927a854686fb9dd5a5
2013-07-02 20:51:38 -07:00
Ed Sanders 37cf529d73 Allow annotations to be additive
Set a static property on big, small, sup, sub to allow them
to be added multiple times to an annotationSet.

Fix the converter to count out annotations when opening/closing.

Bug: 49755
Change-Id: Ifbede9345a66434022dbd681eada447ab81ab025
2013-07-02 21:43:47 +01:00
jenkins-bot 13612910a0 Merge "Preserve the contents of secondary reference definitions" 2013-07-01 17:42:02 +00:00
jenkins-bot c75e379086 Merge "Fix bug where whitespace in an empty node was doubled" 2013-07-01 17:15:58 +00:00
jenkins-bot 74ec334a30 Merge "Fix reordering of metadata bug" 2013-07-01 17:09:30 +00:00
Roan Kattouw 78b402545b Preserve the contents of secondary reference definitions
We currently change <ref name="foo">Foo</ref> ... <ref name="foo">Foo</ref>
to <ref name="foo">Foo</ref> ... <ref name="foo" /> , because know
that the second ref tag isn't canonical and so we blank it.

Instead, we now preserve the contents of all ref tags that come after
the canonical one.

Change-Id: I45a51a879271890fe46c4184f1029f12d27af678
2013-06-30 02:01:04 -07:00
Roan Kattouw f93b051acf Fix bug where whitespace in an empty node was doubled
<table>\n\n</table> round-tripped to <table>\n\n\n\n</table> because
we would store '\n\n' in both the innerPre and innerPost fields.
Fixed by not setting innerPost if the element is empty.

Change-Id: I0393bfaf9793fdebc8fff72c8760113fa69bb2bd
2013-06-30 00:29:38 -07:00
Roan Kattouw 46c3d48ba7 Also annotate empty annotations
The converter wasn't setting .annotations on meta items created to
represent empty annotations, which meant that HTML like
<i>Foo<b></b></i> would end up as <i>Foo</i><b></b> in the linmod.

Change-Id: I13d7d9820beeee1e8c3673e08051361d6c6ac4cd
2013-06-29 23:51:34 -07:00
Roan Kattouw 414709c5cb Fix reordering of metadata bug
If you had <meta /><b>Annotated text</b> in a wrapper paragraph,
the converter would swap them and output the linear model equivalent
of <p wrapper><b>Annotated text</b></p><meta />.

This happened because the meta item was queued, and annotations didn't
trigger metadata queue flushes. The fix is to trigger a metadata queue
flush whenever we're about to write something that isn't itself queued.

Change-Id: I168abc0392fbec5503271d1653ee5c38518f857d
2013-06-29 22:35:21 -07:00
Trevor Parscal 0aac4c7e60 Reference insertion, new and existing
Objectives:

* Split reference dialog (at least for now) an edit and an insert dialog
* Add reference search widget for selecting an existing source, or
  choosing to add a new one
* Abstract reference names, don't allow editing them and generate them
  when needed
* When editing groups, move the internal item and update all references
  to it
* Resolve name conflicts when moving a reference to a new group by
  generating a new list key

Bonus:

* Add getNodeGroups method to internal list
* Add getUniqueListKey method to internal list
* Add destroy functionality to ce.node to release events and references

Bug: 49733
Change-Id: Ib244ff6ad9b4cee1decfd9b9e1d3d4e9cdcfb78c
2013-06-27 18:11:31 -07:00
Roan Kattouw 026102abe3 Fix class attribute preservation for images
* Port the class preservation logic from BlockImage to InlineImage
* Add preservation for unrecognized classes
* Add the logic for collapsing spaces in the class attribute in more
  places

Change-Id: I26faad7e00ab2f0a0f5d076552e56b32c692ae74
2013-06-27 17:39:43 -07:00
Inez Korczyński 8d6f2f9c42 Get rid of mw-figcaption class (Parsoid does not use it anymore)
Change-Id: Id4aed15b7db6a3a42bb75637c3d31b52677a8a4d
2013-06-27 16:13:28 -07:00