Commit graph

669 commits

Author SHA1 Message Date
Ed Sanders 6a52fba643 Only strip style attributes on rich paste from VE
Stripping all HTML atributes (to avoid CE-added styles such as
'font-size: 1em;') also strips data-parsoid which can cause
round trip errors. As an improvement only strip the style
attribute.

Bug: 58136
Change-Id: I34386bd847d1cf0583317a8b07916e43ff7af029
2013-12-09 22:25:43 +00:00
Timo Tijhof cf7f2b141d Set up node-jscs, pass it, and configure in local Gruntfile
Let's experiment with this via our local Gruntfile. If it works
fine we can install it in Jenkins (similar to node-csslint).

Verify through $ npm install && npm test;

Fixed all outstanding violations.

Also:
* Added syntaxhighight to ignore.
* Added imetests (which contain unformatted JSON) to ignore.
* In ve.dm.ModelRegistry#matchTypeRegExps, removed redundant
  !! cast from the [+!!withFunc] statement which was hitting
  a bug in node-jscs. All callers to this local private function
  pass a literal boolean true/false so no need to cast it.
* Removed "/* key .. , value */" from ve.setProp, though this
  wasn't caught by node-jscs, found it when searching for " , ".
* Made npm.devDependencies fixed instead of using tilde-ranges.
  This too often leads to strange bugs or sudden changes. Fixed
  them at the version they were currently ranging to.

Bug: 54218
Change-Id: Ib2630806f3946874c8b01e58cf171df83a28da29
2013-12-06 10:37:27 -08:00
Moriel Schottlender cf237b882e Insert special character tool
A tool to add special characters and diacritics to text.

Also added a new button type ve.ui.GroupButtonWidget that includes a
group of PushButtonWidget objects and returna the individual button's
value upon click

Wikis can edit <visualeditor-specialcharinspector-characterlist-insert>,
a JSON string, to include their own desird special characters to insert
through the tool.

Bug: 50296
Change-Id: I26d1f437feef1c8b61ed3be5f74ef524b33baf49
2013-12-05 22:51:14 -08:00
Ed Sanders 25a6e93ef2 Separate data/metadata split and node tree loops
It's not much of an optimisation to combine these loops but
separating them gives us greater flexibility.

Move the building of the node tree to happen lazily when
getDocumentNode is called.

In the rich paste path we can now create the DM without building
the node tree and remove the metadata.

Change-Id: I10b4bc486ff8ff8037158aa6dfd45aac87557d42
2013-12-05 21:30:04 +00:00
Ed Sanders e01e2e045b Always use #getDocumentNode.
Because if you only sometimes use a getter you're just wasting your time!

Change-Id: Idd38976f77b58e322d527cecca9f9c36e3ad9318
2013-12-04 22:55:26 +00:00
Ed Sanders 6d34e344ee getDataFromDom -> getModelFromDom
Following on from getDomFromModel, this returns a document model
instead of element linear data. The only instance that hasn't been
replaced is in rich paste, where we need to sanitize the converted
data before constructing the document model.

This should be cleaned up in a later commit.

Change-Id: I37a2b641632af2cb515e3409deed5cd1fa358af5
2013-12-04 14:46:34 -08:00
Ed Sanders 86922f8ca8 Change signature of getDomFromData and rename
Currently it takes 4 arguments which are all properties
of the document model, so just pass the model instead and
access the properties later. Rename to getDomFromModel.

Change-Id: I0c378a04dc08b9b90bdc3984f8fa8c4acfe0b667
2013-12-04 16:30:40 +00:00
Ed Sanders 456c97d912 Pressing enter in a list with no nearby content creates a paragraph
Currently we assume getNearestContentOffset will give us something
sane however it can return -1 when there is no nearby content and so
an exception is thrown. In this case we have to create an empty
paragraph to place the cursor in.

Change-Id: Ic6c19da881e47ff6be45cdaa4b71bfcc1c654796
2013-12-02 15:57:43 +00:00
Ed Sanders 519e54d463 ve.ce.Surface.handleEnter tests
Change-Id: Ic30cd4e116029be85822f73dcb84938e641bc4b9
2013-12-02 15:57:41 +00:00
Ed Sanders 96e2313e6e Genericise runSurfaceHandleDeleteTest to work with other 'handle' methods
We will want to reuse this test runner for handleEnter and handleInsertion.

Change-Id: I7489bf2d20fa17b1c164ff20e0cfcd9b9d5102d2
2013-12-02 15:52:44 +00:00
jenkins-bot 91f00b0eaf Merge "Revert model to use simple UTF-16 code units" 2013-11-26 22:53:06 +00:00
jenkins-bot c18534bcad Merge changes If814e178,Id133431e,I38d63e31,I16d575b6
* changes:
  Plain text paste with paste special
  Use rare unicode characters for paste placeholders
  Rich paste
  Add fixUpInsertion to newFromDocumentReplace
2013-11-26 21:07:46 +00:00
David Chan 6f2090aac6 Revert model to use simple UTF-16 code units
This is a prerequisite to browser-based grapheme cluster handling, which
is needed so left/right cursoring and backspace behave as users expect.

modules/ve/ve.js
modules/ve/ce/ve.ce.Document.js
modules/ve/ce/ve.ce.js
* Revert cluster-aware splitting to trivial javascript code unit splitting
* Rewrite ve.splitClusters as a trivial compatibility method (remove soon)
* getClusterOffset/getByteOffset use unicodeJS.graphemebreak.splitClusters

modules/unicodejs/tools/unicodejs-properties.py
modules/unicodejs/unicodejs.graphemebreakproperties.js
modules/unicodejs/unicodejs.js
* Allow grapheme break tests to work with surrogate pairs

demos/ve/pages/minimal.html
demos/ve/pages/multibyte.html
demos/ve/pages/unicode.html
* replace file with more precise tests

modules/ve/test/ve.test.js
* Remove reference to grapheme-based splitting (which is no longer used)
* Correct typo

Bug: 53757
Bug: 51472
Bug: 51596
Bug: 51846
Change-Id: Ife34c87ebe40bc1689298b592eec5c0cdc2f7589
2013-11-26 19:38:14 +00:00
Ed Sanders b1e6dfcda6 Plain text paste with paste special
Register ctrl/cmd+shift+v as a trigger which sets a flag for the
next paste event.

When the paste special flag is set, modify the sanitizeData method
to strip all annotations, and any elements other than paragraphs.

Bug: 53781
Change-Id: If814e1786ffa805b52ab32f4a06f52da743fd9af
2013-11-26 18:23:58 +00:00
Thalia 65aab78d3b Use rare unicode characters for paste placeholders
Helps with bug reporting, like the snowman and pawn.

Change-Id: Id133431eea987442c8b50a05821ebd54649b26b9
2013-11-26 18:23:58 +00:00
Ed Sanders 7cec9ae04a Rich paste
Allow pasting of rich (HTML) content.

ve.ce.Surface
* Use a sliced document clone for converting to DM HTML (copy)
* Add full context to pasteTarget before copying
* Add ve-pasteProtect class to spans to prevent them being dropped
* Implement external paste by converting HTML to data and inserting
  with newFromDocumentInsertion
* Remove clipboard key placeholder after read so they aren't picked
  up by rich paste. Hash no longer includes the placeholder.
* Detect the corruption of important spans and fallback to clipboard
  data HTML if available.

ve.dm.LinearData
* Add clone method for copy

ve.dm.ElementLinearData
* Add compareUnannotated for use by context diffing.
* Add sanitize method for cleaning data according to a set of rules.

ve.dm.Transaction
* Add range parameter for inserting a range of a document only,
  e.g. stripping the paste context.

ve.dm.Document
* Implement sliced document clone creation so that DM HTML
  is generated correctly in onCopy

ve.dm.DocumentSlice
* Replaces LinearDataSlice. Now has two ranges for balanced data
  and data with a full context.

ve.init.Target.js
* Define default, loose, paste rules (just remove aliens).

ve.init.mw.ViewPageTarget
* Define strict MW paste rules:
  + no links, spans, underlines
  + no images, divs, aliens
  + strip extra HTML attribues

ve.init.sa.Target, ve.init.mw.ViewPageTarget, ve.ui.Surface
* Pass through and store paste rules.

Bug: 41193
Bug: 48170
Bug: 50128
Bug: 53828
Change-Id: I38d63e31ee3e3ee11707e3fffed5174e1d633b42
2013-11-26 18:23:12 +00:00
David Chan b4f4eeb4b6 Event Logger: collect IME functionality info
ve/ce/SurfaceObserver.js
* Do not setTimeout if frequency === null

demos/ve/eventLogger.html
* Standalone event logging script

ve/test/ce/imetests/*.js
* JSON event logs for various tests/browsers/IMEs

ve/test/ce/ve.ce.test.js
* Add an IME test

VisualEditor.hooks.php
* Add test files

Change-Id: I50e89d5a289f3fcb4fe2a6835a2ec96fb497242c
2013-11-25 16:56:09 +00:00
David Chan addd1d7b23 Test Surface with IME-like event sequences
* modules/ve/test/ce/ve.ce.TestRunner.js
Class to interact with the CE Surface and document in tests

* modules/ve/ve.EventSequencer.js
Wrap setTimeout/clearTimeout calls (for easy replacement in tests)

Change-Id: I2e2407e2b169ae77237c87bf8857b3026cc7efce
2013-11-25 16:49:49 +00:00
Ed Sanders 24a95b102d Add fixUpInsertion to newFromDocumentReplace
In order to do this we have to separate out the removal
operation from NDFR, so it becomes newFromDocumentInsertion
(again, although actually, for the first time). As NFDI is
an insertion we can just run fixUpInsertion on the data
part of it.

In order for the removal operation to be a proper removal
we have to allow metadata removal (the default is to merge it).

Change-Id: I16d575b61b9796e7e889f2c27cfe02b4a40b7639
2013-11-20 11:24:06 +00:00
Roan Kattouw cde3304864 Recompute auto-numbered reference listKeys when merging internalLists
This fixes some of the problems with pasting references.

It's a bit overzealous in that references get renumbered even when
replacing, which is unnecessary but doesn't actually have any
noticeable effect.

Unfortunately, the internal list state depends so much on the converter
having run that we now need to add yet another hack, to set the counter
to the appropriate value.

Change-Id: I3c6514ce600af4f4c037f419554d34b5a5c86a63
2013-11-20 16:50:48 +05:30
jenkins-bot 137ca9d2d6 Merge "Store inner whitespace of the body and compare it on conversion" 2013-11-20 10:45:04 +00:00
Timo Tijhof de979ff293 Replace raw oojs-ui files with v0.1.0-pre (0267100ab3)
Change-Id: I792bc0335269976ff29433227b6fa562a1026aeb
2013-11-20 10:27:48 +05:30
Trevor Parscal 3dcba7679a Add OO.ui.ToggleWidget
Compact toggle widget with short labels for on/off states.

Change-Id: I582e81126d5f1564302409c1b262e28e4a9505ac
2013-11-12 16:54:35 -08:00
Trevor Parscal 65a2f41268 Cleanup after window refactor
Related to: I82c4fed8bcb3fb5630938c8bc4dd9b2d5f1a8c1d

Change-Id: I4cc280725315fee0c5dd042479d4373cc0cdd0b4
2013-11-12 12:02:28 -08:00
Trevor Parscal 4192cbc4d5 Window refactor
Changes:

* Cleanup the window API to use more consistent and intuitive methods - we
  now use initialize/setup/teardown instead of
  initialize/onSetup/onOpen/onClose as methods which are overridden, and
  use open/close methods to control the window
* Change events around to have opening/open and closing/close events which
  act as before/after points during the opening/closing process
* Make WindowSet and Context respond to windows being opened, rather than
  opening them directly
* Fix a LinkInspector creation mode bug where the initial text doesn't get
  reset
* Move inspector, a VisualEditor concept, back to VE
* Cleanup naming of SurfaceDialog, SurfaceToolbar, etc. to use shorter
  names, they were given Surface* names when the generic ones were also in
  VE, but now the generic ones are in OO, so they can return to their
  original names

Change-Id: I82c4fed8bcb3fb5630938c8bc4dd9b2d5f1a8c1d
2013-11-08 12:33:25 -08:00
Trevor Parscal db9f941fa6 Rename this.$ to this.$element, and this.$$ to this.$
Objectives:
* Rename this.$ to this.$element
* Rename this.$$ to this.$
* Get rid of the need to use this.frame.$$
* Rename OO.ui.Element.get$$ to OO.ui.Element.getJQuery

Changes: (using Sublime Text regex patterns)
* Replace "get$$" with "getJQuery"
* Replace "\.(\$)([^\$a-zA-Z])" with ".$element$2"
* Replace "\.(\$\$)" with ".$"
* Replace "'$$'" with "'$'"
* Set this.$ to null in constructor of OO.ui.Window
* Set this.$ to this.frame.$ in initialize method of OO.ui.Window
* Replace "\.(frame.\$)([^\$a-zA-Z])" with ".\$$2"

Bonus:
* Use this.$() in a bunch of places where $() was erroneously used

Change-Id: If3d870124ab8d10f8223532cda95c2b2b075db94
2013-11-03 23:03:49 -08:00
Ed Sanders 3838498e8b Store inner whitespace of the body and compare it on conversion
Calculate and store the two inner whitespace values of the body in the
dm.Document. When converting back, make sure the first/last nodes
pre/post outer whitespace matches the inner left/right whitespace
of the body.

Bug: 54964
Change-Id: I45f1ffd63669f25a6cae878400bfe21719ed58ee
2013-10-31 17:09:06 +01:00
Ed Sanders 91f6b1d86c Fix: getDataFromDom tests should check for body or head
This wasn't updated when Roan split html into body & head, so
0 tests were running!

Change-Id: I4396b94426b2009f20707d64bf3d0f1ae651dbc8
2013-10-31 09:02:19 +00:00
Ed Sanders 5aa7f6526c Fix: files missing from demo and test runner
Change-Id: I783384a28147c9c7cdfdcb6639fea1f197cffb54
2013-10-31 08:57:39 +00:00
Trevor Parscal a3295f0939 Allow direct opening of pages in the meta dialog
Objectives:

* Hamburger menu in actions area of toolbar
* Add tools that open specific pages in the meta dialog
* Fix support for using setPage in ve.ui.PagedOutlineLayout
* Allow passing setup config objects through window open calls
* Add dialog action, similar to inspector action
* Fix incorrect or missing documentation

Change-Id: I2d2c9b87554fb2a0c90ed6944a58b38a37efa712
2013-10-29 05:58:06 +00:00
Trevor Parscal 0ee35c99d0 Add OO.ui.PopupTool
Tool that launches a popup when clicked.

Change-Id: Ie35b51f8160ad3fb844491e2799656413d4029c0
2013-10-29 05:57:40 +00:00
Trevor Parscal d2dfb9ac4f Split oojs-ui from ve.ui
* Move and rename generic parts of ve.ui to OO.ui
* We now have a UI test suite because ve.Element (outside ve.ui)
  is now part of oojs-ui, so it needs a test suite.
* Added to the MW test run (just like we do for unicodejs).
* Updated csslint config (also added ve-mw and syntaxhighlight
  which were missing).

oojs-ui still depends on the TriggerRegistry in VE, this is addressed
in a follow-up commit.

Change-Id: Iec147155c1ddf20b73a4d15d87b8742207032312
2013-10-28 22:40:08 -07:00
jenkins-bot cc8010e556 Merge changes Ia6ca85bc,I919135eb,Ia18bd8fc,I49dfc81f,Ia234f174,I06425e2c,Ia192331b,I5624ae07
* changes:
  Get rid of dmRendering hack in ve.ce.MWInternalLinkAnnotation
  Render resolved URLs for href and src attributes in CE
  Give ce.Annotations a reference to their ce.ContentBranchNode
  Track the original HTMLDocument in ve.dm.Document
  Create CE nodes and annotations with the correct $$
  Add ve.resolveUrl for URL resolution
  Don't render href as src in MWBlockImageNode
  Rename 'html' to 'body' in converter tests
2013-10-28 18:55:11 +00:00
jenkins-bot 6bfa803fda Merge "Resolve rendered URLs according to the provided <base>" 2013-10-28 18:32:55 +00:00
Roan Kattouw a05703114a Track the original HTMLDocument in ve.dm.Document
Add it as an optional parameter to the constructor, and create a new
one if omitted.

This is going to be used to resolve URLs according to the right <base>,
but really that's a hack and we should come up with a better way to
track metadata from the <head>.

Change-Id: I49dfc81ff793d73e08a20e502d681a15613d23f7
2013-10-28 15:48:55 +00:00
Roan Kattouw 904abe74c8 Create CE nodes and annotations with the correct $$
Most of CE wasn't passing through $$ at all. Also fix CE tests that
weren't passing the required surface parameter to the ce.Document
constructor.

Change-Id: Ia234f174050f4b2666ec20e8acc24c6aa4305202
2013-10-28 15:48:51 +00:00
Roan Kattouw 17192da3a2 Add ve.resolveUrl for URL resolution
Use it to replace fullUrl in the tests

Change-Id: I06425e2c90a08a6f622e083f9297444f6f4672e8
2013-10-28 15:48:45 +00:00
Roan Kattouw 608b8d378d Rename 'html' to 'body' in converter tests
Because that's what it is now since 'head' was added. Also removed
the wrapping <body> tag (now added by the test runner) and renamed
normalizedHtml to normalizedBody.

Change-Id: I5624ae076c5e661d2789e499cd28e8282c885409
2013-10-28 15:46:55 +00:00
Roan Kattouw 74b8807df5 Resolve rendered URLs according to the provided <base>
This is done by using the computed property value rather than the
literal attribute value when rendering href and src attributes.
Helpfully, this provides perfect URL resolution natively in the browser,
which means the document's <base> is respected and all that good stuff.

For GeneratedContentNodes, we also need to find all DOM elements inside
the rendered DOM that have href or src attributes and resolve those.
This is done in the new getRenderedDomElements() function, which the
existing cleanup steps (remove <link>/<meta>/<style>, clone for
correct document) were moved into.

In order to make sure that the computed values are always computed
correctly, we need to make sure that in cases where HTML strings
in data-mw are parsed, they're parsed in the context of the correct
document so the correct <base> is applied.

We still need to solve this problem for models that actually store and
edit an href or src as an attribute. I'll post more about that on
bug 48915.

Bug: 48915
Change-Id: Iaccb9e3fc05cd151a0f5e632c8d3bd3568735309
2013-10-28 15:16:05 +00:00
Ed Sanders 59fa4c1097 Remove ve.ce.Surface.static.textPattern
It hasn't been used for a while.

Change-Id: I7d004f38bc76ef8a794294a7c685ada73ab7855e
2013-10-28 14:44:23 +00:00
jenkins-bot 291e1f3449 Merge changes Iaba76ac1,Ib15c39f3,If5a3b4d4,I44425d26,I8f162894
* changes:
  .change( null, foo ) -> .setSelection( foo )
  Remove SurfaceObserver locking (unused)
  Implement ve.dm.Surface.prototype.undo() and redo() in terms of change()
  Enslave dm.Surface to dm.Document
  Remove dm.Surface's 'change' event
2013-10-25 21:05:32 +00:00
jenkins-bot e180b8c1ca Merge changes Ifeb1a1fc,I521eff00
* changes:
  Migrate away from using the 'change' event in dm.Surface
  Make dm.Surface's 'select' event more useful
2013-10-25 17:59:00 +00:00
Roan Kattouw 534f0bdde7 .change( null, foo ) -> .setSelection( foo )
Change-Id: Iaba76ac17888a7825f5a9cd1c383d4f429454278
2013-10-25 18:29:49 +01:00
Roan Kattouw 1bf58252ce Implement ve.dm.Surface.prototype.undo() and redo() in terms of change()
...or really changeInternal(), so we can avoid adding undo transactions
to the undo stack.

Also get rid of the pattern where undo() and redo() return a selection
which the caller then has to restore, and instead just restore the
selection.

Bug: 53224
Change-Id: If5a3b4d4162e9f0713ee9cd26e79a66efe52770f
2013-10-25 18:29:48 +01:00
Roan Kattouw b7ee30d4e4 Enslave dm.Surface to dm.Document
* Replace surface 'transact' event with 'documentUpdate' event
* Have surface listen for all document transactions and update selection
  as appropriate (as well as emitting 'documentUpdate')
* Implement change() in terms of setSelection()
** Queue 'contextChange' events so contextChange is only emitted once
** Use this.transacting flag to prevent setSelection() (which is called
   because the model emits transact events) from doing too much
** Behavioral change: lock/unlock now emitted separately for
   transaction and selection changes

Change-Id: I44425d260ec70758f99d13f99e41f5c993d260c2
2013-10-25 18:29:48 +01:00
David Chan 04cbbef4ee Remove dm.Surface's 'change' event
ve.dm.Surface.js:
* Stop emitting 'change' and remove its event documentation

ve.ce.Surface.js:
* Listen to 'select' instead of 'change'
* Perform a CE surface update after model-based keydown handling

ve.dm.Surface.test.js:
* Stop asserting that 'change' is emitted

Change-Id: I8f16289493e835d890709c6dfe093d04c18522b6
2013-10-25 18:29:13 +01:00
jenkins-bot 044e2e289c Merge "Split ext.visualEditor.experimental into specific modules" 2013-10-24 14:05:33 +00:00
Roan Kattouw 412eb34de8 Split ext.visualEditor.experimental into specific modules
Math, hiero, language and alienextension are now each in their own
module. Kept the experimental module for backwards compatibility,
it just has all of its constituent modules as dependencies.

MWExperimentalTool.js was split up, and ExperimentalTool.js was
renamed to LanguageInspectorTool.js.

Change-Id: I63b49dfbdb59dc9a494049553cc0c01e89e48826
2013-10-23 16:20:30 -07:00
Ed Sanders 58fba521e0 Use jsdifflib for QUnit diff
Because QUnit's inline diff is terrible for large diffs,
especially when there are block whitespaces changes.

Change-Id: I786fb981b02777ede38c4bee261f9e32f8f908ed
2013-10-23 20:31:17 +00:00
Roan Kattouw 53e4c370ea Migrate away from using the 'change' event in dm.Surface
Instead, listen to 'select', or to 'transact' on the dm.Document.

This commit only fixes uses outside of the dm/ce.Surface ecosystem.
ce.Surface still listens to 'change'.

ve.init.mw.ViewPageTarget.js:
* Rename onSurfaceModelTransact to clearSaveDialogDiff and bind it to
  the document's transact event instead
* Rename onSurfaceModelChange to checkForWikitextWarning and bind it
  to the surface's transact event. This is needed because the function
  inspects the surface's selection, which isn't yet in a consistent
  state when the document's transact event fires

ve.ui.MWReferenceDialog.js:
* Rename onSurfaceChange to onDocumentTransact and rebind accordingly

ve.ce.ProtectedNode.js:
* Get rid of onSurfaceModelChange
* Instead, bind positionPhantoms to the document's transact event
  directly, and only bind it while phantoms are visible

ve.ui.Context.js:
* Rename onChange to onModelSelect and rebind accordingly
* Rename afterChange to afterModelSelect
* Drop check for undefined selection, no longer needed now that we're
  listening to a finer-grained event

ve.ce.Surface.test.js:
* Listen to 'select' instead of 'change'

Change-Id: Ifeb1a1fc5427696f2aae5441d4b54dde366793e0
2013-10-23 11:54:35 -07:00