Specifically, go to [[mw:VisualEditor/Feedback]] rather than [[mw:Visual editor/
Feedback]], as the primary set of pages on MWwiki have been moved some time ago
and right now we're stranding users' feedback messages at the wrong place.
Change-Id: Ia2ba0dad7d1bf0d1633fe2704a2e8354285c66da
This allowed me to move ve.ce.Surface particulars (such as the starting, stopping and clearing of polling) out of the UI code.
Also cleaned up some switch statements.
Change-Id: I7b85e42a4e01f8d76237d995e25275f2424541ea
Previously, we were looking one offset to the left to load the insertAnnotations. This would fail at the beginning of a document that began with a slug, and probably other cases too. Now using getRelativeContent offset.
Change-Id: I31b24e2ccfa9fda2ce7fb19d1221f8708a96083f
Slugs aren't represented in the model at all, but in the DOM they do
contain a single character ( or ), so we have to override
the return value of getDomText() to be empty.
Change-Id: Ic93cb694b0632bc81d8d0749149e4d739232ed48
Iterating forwards while removing from an array is a good way to get half way there and be confused as hell.
Change-Id: I74db84eee87c73d8e035f6dc8a92be0d0b9b3dad
* Added some new icons (to be used soon in the inspector redesign)
* Added right-to-left icons for lists
* Renamed some icons for future RTL use
Change-Id: I10e3e3fcda82786e3064176e7eefe211b88db95c
Was cloning the original set, then adding to it - resulting in always containing the whole set in any match.
Also using test instead of exec since exec returns a string result, which under strange circumstances could return a falsy result, despite the RegExp finding a match.
Change-Id: I09a7cb264521d58f02d6ff2547edad9a740b23b2
Surface model annotate method previously cleared basic tool annotation objects.
Now that bold and other annotations have htmlAttributes from parsoid,
we must gather annotations by type from selection and iterate through
to properly clear.
Change-Id: Id53bf5733078524ae5aac376e01b9679eb8c32df
Rather than each tool requesting annotations, and nodes pertaining to selection,
Emitted event supplies annotations and nodes to each tool's update method.
Using select vs. of traverseLeafNodes for code optimization.
Better documentation for updateTools()
Removed unneeded code.
Change-Id: I7c0baa1cc0f7fb731d6e28b175a76e931e9e2961
* Added documentation for ve.AnnotationSet
* Replaced uses of "// Inheritance" with "// parent Constructor"
* Added "// Mixin constructor" where needed
* Added missing section comments like "/* Static Methods */"
* Cleaned up excessive newlines (matching /\n\n\n/g)
* Put unnecessarily multi-line statements on a single line
Change-Id: I2c9b47ba296f7dd3c9cc2985581fbcefd6d76325
* Commands for Sublime:
Find*: "(\* @[a-z]+) ([^{].*) \{(.*)\}"
Replace: "$1 {$3} $2"
Save all && Close all
Find: " function("
Replace: " function ("
Save all && Close all
Find: "Intialization"
Replace: "Initialization"
Save all && Close all
* Consistent use of types (documented in CODING.rm):
- Merged {Integer} into {Number}.
- Merged {DOM Node} into {DOMElement}.
* Remove work-around /*jshint newcap: false */ from ve.js
Calling Object() as a function to to use the internal
toObject no longer throws a newcap warning in JSHint.
It only does that normal functions now .
(e.g. var a = Cap(); or var a = new uncap();)
* Add missing annotations (@static, @method, ..).
* Remove unused variables
* Remove null-assignments to variables that should just be
undefined. There's a few variables explicitly set to null
whereas they are set a few lines under and not used otherwise
(e.g. 'tx' in ve.ce.Surface.prototype.onPaste)
Change-Id: I0721a08f8ecd93c25595aedaa1aadb0e08b83799
Detecting page status in a similar way as WikiEditor inspector.
Disabled accept button now behaves appropriately.
Accept button status is now evaluated on enter or submit.
Change-Id: Ibfef6ffd87cb9a71e37242d6214d0f8e3af2e2c0
This node type represents <meta> or <link> (transparently, based on the
style attribute). I had to make two node types for this and hack the
toData conversion code directly into ve.dm.Converter, because we don't
have native support for node types that can be both inline and block.
(We should add this in the node API rewrite.)
The CE implementation renders a placeholder (with the same styles as an
alien node) right now. I'm not sure how nice that is, but it's better
than rendering raw <meta>/<link> tags.
This whole thing is a total pile of hacks to make VE deal with
<meta>/<link> tags until we have a proper node types API.
Change-Id: Id6783fcfc35a896db088ff424ff9faaabcaff716
Icon appears when scrolling and resizing window.
Instead of always setting the context on scroll and resize, bind to the
updateContextIcon method.
Icon appears when selecting a non content data offset.
Changed logic to show icon changed to content length vs range difference.
Move Link inspector getSelectionText to ve.dm.document getText.
Rationale, more bits of the code depend on evaluating content.
Added new ve.Range truncate method.
Remove getSelectionText, using truncate range & document.getText instead.
Change-Id: Ibd3e99c923f18d2c96a86d92e74e2e9ebd49c85f
This was broken in three different ways:
* On the way in, we were applying whitespace to an array of elements
rather than the actual element, so the whitespace wasn't stored.
* Whitespace processing on the way out was skipped for aliens because
they had their own code path. Refactored this so alien openings and
regular openings share much more code, including whitespace output.
* Somewhat unrelatedly, innerPost output was broken for paragraphs
containing inline elements, because the inline elements' processing
polluted lastOuterPost. Discovered this because my test with inline
aliens also happened to be the first test of whitespace preservation
in paragraphs with inline content elements. Fixed by explicitly
skipping content nodes when outputting whitespace.
Fixed these issues and added a test case.
Change-Id: I8edb61a008e60ace886b1a841b3417682ec39c32
* For the most common case:
- replace ve.extendClass with ve.inheritClass (chose slightly
different names to detect usage of the old/new one, and I
like 'inherit' better).
- move it up to below the constructor, see doc block for why.
* Cases where more than 2 arguments were passed to
ve.extendClass are handled differently depending on the case.
In case of a longer inheritance tree, the other arguments
could be omitted (like in "ve.ce.FooBar, ve.FooBar,
ve.Bar". ve.ce.FooBar only needs to inherit from ve.FooBar,
because ve.ce.FooBar inherits from ve.Bar).
In the case of where it previously had two mixins with
ve.extendClass(), either one becomes inheritClass and one
a mixin, both to mixinClass().
No visible changes should come from this commit as the
instances still all have the same visible properties in the
end. No more or less than before.
* Misc.:
- Be consistent in calling parent constructors in the
same order as the inheritance.
- Add missing @extends and @param documentation.
- Replace invalid {Integer} type hint with {Number}.
- Consistent doc comments order:
@class, @abstract, @constructor, @extends, @params.
- Fix indentation errors
A fairly common mistake was a superfluous space before the
identifier on the assignment line directly below the
documentation comment.
$ ack "^ [^*]" --js modules/ve
- Typo "Inhertiance" -> "Inheritance".
- Replacing the other confusing comment "Inheritance" (inside
the constructor) with "Parent constructor".
- Add missing @abstract for ve.ui.Tool.
- Corrected ve.FormatDropdownTool to ve.ui.FormatDropdownTool.js
- Add function names to all @constructor functions. Now that we
have inheritance it is important and useful to have these
functions not be anonymous.
Example of debug shot: http://cl.ly/image/1j3c160w3D45
Makes the difference between
< documentNode;
> ve_dm_DocumentNode
...
: ve_dm_BranchNode
...
: ve_dm_Node
...
: ve_dm_Node
...
: Object
...
without names (current situation):
< documentNode;
> Object
...
: Object
...
: Object
...
: Object
...
: Object
...
though before this commit, it really looks like this
(flattened since ve.extendClass really did a mixin):
< documentNode;
> Object
...
...
...
Pattern in Sublime (case-sensitive) to find nameless
constructor functions:
"^ve\..*\.([A-Z])([^\.]+) = function \("
Change-Id: Iab763954fb8cf375900d7a9a92dec1c755d5407e
In preparation of pushing the object-management branch, which
will need more shared variables (for efficiency) creating
a shared closure. To avoid a spegetti of immediately-invoked
functions that return functions (like ve.getObjectValues and
ve.getObjectKeys).
Less duplication of code and faster execution.
First I had the closure around it as-is but then I figured it'd
be faster to have a local reference to ve (instead of having to
go through implied globals for references to other ve.*)
So I made it a local variable and then exposed it. That way
anything inside referring to each other stays within the same
scope.
Review with ignore-whitespace for clarity.
Change-Id: I415d8635db6d82cf239f0364ccc2d63a61bd5a6d
Introduced the ve.AnnotationSet class to manage sets of annotations. This
is a generalization of ve.OrderedHashSet, a class that manages a set
using an array and an object keyed by hash.
Converted everything that stores, tracks or passes around annotations to
use ve.AnnotationSet. In particular, this means the linear model now
contains AnnotationSets instead of hash-keyed objects.
This allows us to maintain the order of annotations in the linear model,
and will help fix bugs with annotation ordering and splitting.
Change-Id: I50975b0a95f4cc33017a0b59fdede9ed1eff0124
Currently this is done in a hacky way because we don't have a real
registry of RDFa types for node types, so we just hardcode the list of
recognized types (only links currently).
Change-Id: I5afcc55701fc6fa0ee2a360dcf5ca62b065292f5
* Also clean up ve.init.mw.ViewPageTarget...setupSaveDialog
Consistently use viewPage instead of 'this' inside this
function. The reason it is locally aliased is because there is
other 'this'es used here. Using them mixed is even more
confusing.
- No need for ve.bind, other handlers in this function also
just use viewPage instead of binding this.
Change-Id: I25e93862a39134961bf80835f46cbf531d8109e0
[jshint]
ce/ve.ce.Surface.js: line 670, col 9, Too many var statements.
ce/ve.ce.Surface.js: line 695, col 6, Missing semicolon.
ce/ve.ce.Surface.js: line 726, col 22, Expected '===' and instead saw '=='.
ce/ve.ce.Surface.js: line 726, col 41, Expected '===' and instead saw '=='.
ce/ve.ce.Surface.js: line 733, col 13, Too many var statements.
ce/ve.ce.Surface.js: line 734, col 24, Expected '===' and instead saw '=='.
ce/ve.ce.Surface.js: line 1013, col 13, Too many var statements.
ce/ve.ce.Surface.js: line 1019, col 17, Too many var statements.
ce/ve.ce.Surface.js: line 1023, col 18, Too many ar statements.
ce/ve.ce.Surface.js: line 1027, col 13, Too many var statements.
dm/annotations/ve.dm.LinkAnnotation.js: line 70, col 52, Insecure '.'.
dm/ve.dm.Converter.js: line 383, col 29, Empty block.
dm/ve.dm.Converter.js: line 423, col 33, Empty block.
Commands:
* jshint .
* ack '(if|else|function|switch|for|while)\('
* Sublime Text 2:
Find(*): (if|else|function|switch|for|while)\(
Replace: $1 (
* ack ' ' -Q # double spaces, except in certain comments
Change-Id: I8e34bf2924bc8688fdf8acef08bbc4f6707e93be
MultiSuggest:
Added CSS Ellipsis config option which provides CSS to suggestion items.
LinkInspector:
-Configured multiSuggest for page suggestions with CSS ellipsis off and removed CSS definitions.
Instead, using jquery.autoEllpisis plugin for centered position ellipsis. (Bug 39591)
-Temporary tweak to link input padding to prevent text from overlapping down arrow.
Down arrow soon to be implemented differently.
RTL Fixes:
-MultiSuggest overlay positions correctly by setting a width.
-MultiSuggest background image position fix.
Change-Id: I806ead5a2c2621589f76cfb2b03805cbd0b0a18a
* Moved icons into Illustrator (used to be in Photoshop)
* Added SVG icons too
* Added support for devices with pixel ratio > 1 (they use SVG)
* Cleaned up icons (little rendering errors here and there)
* Organized icons into their own folder
* Increased the horizontal margin of the down arrows in the formatting (in the toolbar) and location (in the link inspector) drop down menus
Change-Id: I29b7084c9b1145051b2a76f514cfca9826d53ddb
This will cause ve.dm.SurfaceFragment.prototype.insertContent() to place
the selection after the insertion as well.
Change-Id: Ifa7e627daceb90408422eb58c110d475f34ba1e2
* Separated DOM changes from creation of elements
* Always using parsing for element creation with known attributes
* Always using attr or addClass for variable attributes
Change-Id: Id101f56594014786892d382d06c658f416224a9c
* Switched a lot of classes from es-* to ve-ui-*
* Removed all the DOM structure left over from the old sandbox demo
* Got rid of transparent backgrounds
* Added menu font-size rule to stand-alone target
* Moved some rules around that were in the wrong places
* Got rid of some unused/unneeded methods in the mw target (attach and detach surface methods)
* Added active class to context icon with shallower shadow effect so it doesn't break your spacial perception when you click on it
* Renamed the iframe and iframe wrapper elements so it's easier to see where they came from
* Removed unused CSS rules
* Fixed some uses of prop( 'class', … ) to addClass
Change-Id: I54a660ca0baf0baa4463faca7a1edcf648130b6b
* function() -> function ()
* ){ -> ) {
* Quoted JSON keys (Look the other way Timo!)
* Using more descriptive group names, which also avoid using "new" as a key
* Line breaks at 100 columns using 4 spaces/tab
* Not setting classes on suggestion items that have no effect (such as 'external' on a div - even on an <a> tag this isn't enough to get the style because it's not inside #content)
Change-Id: I37032fa8ba93adb7b719d9797f9b1b806359cc13
* Replaces c8b4a28936
* Use Object() casting to detect objects instead of .constructor
(or instanceof). Both .constructor and instanceof compare by reference
the type "Object" which means if the object comes from another window
(where there is a different "Object" and "Object.prototype") it will
drop out of the system and go freewack.
Theory: If a variable casted to an object returns true when strictly compared
to the original, the input must be an object.
Which is true. It doesn't change the inheritance, it doesn't make it inherit
from this window's Object if the object is from another window's object. All it
does is cast to an object if not an object already.
So e.g. "Object(5) !== 5" because 5 is a primitive value as opposed to an instance
of Number.
And contrary to "typeof", it doesn't return true for "null".
* .constructor also has the problem that it only works this way if the
input is a plain object. e.g. a simple construtor function that creates
an object also get in the wrong side of the if/else case since it is
an instance of Object, but not directly (rather indirectly via another
constructor).
* Added unit tests for basic getHash usage, as well as regression tests
against the above two mentioned problems (these tests fail before this commit).
* While at it, also improved other utilities a bit.
- Use hasOwnProperty instead of casting to boolean
when checking for presence of native support.
Thanks to Douglas Crockford for that tip.
- Fix documentation for ve.getHash: Parameter is not named "obj".
- Add Object-check to ve.getObjectKeys per ES5 Object.keys spec (to match native behavior)
- Add Object-check to ve.getObjectValues to match ve.getObjectKeys
- Improved performance of ve.getObjectKeys shim. Tried several potential optimizations
and compared with jsperf. Using a "static" reference to hasOwn improves performance
(by not having to look it up 4 scopes up and 3 property levels deep).
Also using [.length] instead of .push() shared off a few ms.
- Added unit tests for ve.getObjectValues
Change-Id: If24d09405321f201c67f7df75d332bb1171c8a36
This commit fully utilizes all four positions in the internal.whitespace
array. Outer whitespace is now preserved as well, and is duplicated
either in the adjacent sibling (one node's outerPost is the next
sibling's outerPre) or in the parent (a branch node's innerPre is its
first child's outerPre, and its innerPost is its last child's
outerPost). Before restoring saved whitespace, we check that these two
agree with each other, and if they disagree we assume the user has been
moving stuff around and don't restore any whitespace in that spot. The
whitespace at the very beginning and the very end of the document (i.e.
the first node's outerPre and the last node's outerPost) isn't
duplicated anywhere, nor is inner whitespace in content nodes.
The basic outline of the implementation is:
* When we encounter whitespace, strip it and store it in the previous
node's outerPost. Also store it in nextWhitespace so we can put it in
the next node's outerPre once we encounter that node.
* When we encounter whitespace in wrapped bare text, we don't know in
advance if it's gonna be succeeded by more non-whitespace (in which
case it needs to be output verbatim), or not (in which case it's
leading whitespace and needs to be stripped and stored). The fact that
annotations are nodes in HTML makes this trickier. So we write the
whitespace to the temporary linmod and store it in wrappedWhitespace,
then if it turns out to be trailing whitespace we take it back out of
the data array and record it the usual way.
* Because text nodes can contain any combination of leading whitespace
actual text and trailing whitespace, and because we may or may not
already have opened a wrapping paragraph, there are a lot of different
combinations to handle. We handle all of them but the resulting code
is pretty dense and verbose.
More low-level list of changes:
In getDataFromDom():
* Added helper function addWhitespace() for storing whitespace for an
element
* Added helper function processNextWhitespace() for processing any
whitespace passed on from the previous node via the nextWhitespace var
* Rename paragraph to wrappingParagraph. Make wrapping default to
alreadyWrapped so we can simplify wrapping||alreadyWrapped and
!wrapping&&!alreadyWrapped. Add wrappingIsOurs to track whether the
wrapping originated in this recursion level (needed for deciding when
to close the wrapper).
* Add prevElement to track the previous element so we can propagate
whitespace to it, and nextWhitespace so we can propagate whitespace to
the next element.
* Remove previous newline stripping hacks
* Integrate the logic for wrapping bare content with the outer
whitespace preservation code
* Remove wrapperElement, no longer needed because we have a dedicated
variable for the wrapping paragraph now and what was previously inner
whitespace preservation for wrapper paragraphs is now covered by the
outer whitespace preservation code.
In getDomFromData():
* Reinsert whitespace where appropriate
** outerPre is inserted when opening the element
** This covers outerPost as well except for the last child's outerPost,
which is handled as the parent's innerPost when closing the parent.
** innerPre and innerPost are inserted when closing the element. Care is
taken not to insert these if they're duplicates of something else.
* Propagate each node's outerPost to the next node (either the next
sibling or the parent) using parentDomElement.lastOuterPost. We can't
get this using .lastChild because we will have destroyed that child's
.veInternal by then, and we can't tell whether a node will be its
parent's last child when we process it (all other processing,
including first child handling is done when processing the node itself,
but this cannot be).
* Special handling is needed for the last node's outerPost, which ends
up in the container's .lastOuterPost property.
Tests:
* Allow .html to be null in data<->DOM converter tests. This indicates
that the test is a one-way data->DOM test, not a DOM->data->DOM
round-trip test. The data will be converted to HTML and checked
against .normalizedHtml
* Update existing tests as needed
* Add tests for outer whitespace preservation and storage
* Add test for squashing of whitespace in case of disagreement (this
requires .html=null)
Change-Id: I4db4fe372a421182e80a2535657af7784ff15f95
Pressing enter in an empty last list item should remove the list item and add a paragraph. Transaction was not being committed.
Change-Id: I0b173125d48f629eab923e051648d2036bb8956c
In current implementation ve.ce.TextNode can be represented by more then one DOM node (TEXT_NODE and or ELEMENT_NODE) - for instance "Lorem <b>Ipsum</b> est" (TEXT, ELEMENT, TEXT).
Change-Id: I6530cbf1273ad1b1de3599c3eb929eb7abc2d59e
* Added oldid param to the API
* Added oldId argument to ve.init.mw.ViewPageTarget
* Added redirect when saving an oldid page (which adds a new revision)
* Added venotify param to allow notifications even when redirected
* Added creation notification
* Added page title to saved/created notification
Change-Id: I9e957e6f5bc7920093481ffe3c33e299f87ce50a
Got rid of some commented out code we aren't going to use and fixed the order of arguments to background position to x, y
Change-Id: I4e638c33b053dd916681dc91c8c0d7cff224375d
* Added style for monobook
* Added monobook to whitelisted skins
* Made ve.init.mw.ViewPageTarget resilliant to vector and monobook-like skins
Change-Id: Ia39e034229204ae5f1f70dce5d74c6c4db42277b
data-mw-gc is ancient and unused. We do need to detect and alienate
generated nodes, but that is now based on RDFa types. Removing the
data-mw-gc stuff for now because it doesn't work anyway, will replace it
with proper detection later.
Replaced instances of data-mw-gc in the test suite with unregistered
node types.
Change-Id: If3f5898d382a436fa57929013264c53af5e840ba
* Added more support for making dynamic skin changes in vector (based on media queries) get applied to the toolbar while it's floating
** Added animation for toolbar wrapper margins
** Added resize handler to update left and right positions on the toolbar when it's floated
* Changed from using generic "float" and "bottom" class names which are likely going to get re-used elsewhere and cause issues
Change-Id: Ic596b2b8aceb8a2d81539e197ef4d6e17326a87a
Some of the HTML structure in the suggestions might need to be changed to pull off pixel-perfect reproduction of the original design, but this gets us very close.
Change-Id: Ie574577af4815e4f0fc9f8a03e57759dd9dfff84
There is a TODO about this but these were never handled. This lack of
handling caused weird offset issues when editing paragraphs containing
inline aliens, because the text of the inline alien was returned.
Instead, return an appropriate number of characters so the offset logic
keeps working.
Removed the textContent||innerText code because it would bypass the
uneditable node logic: paragraphNode.textContent would return the text
of the entire paragraph (including any aliens), and we'd never descend
into the paragraph to reach the alien node.
Change-Id: Ia05755755dd5380cb9a0b5a0334c6721b4e0d3b7
Don't try to restore the selection after the removal part of
onContentChange, because the removal may have caused that selection to
now be out of bounds. We also don't need to set the selection there at
all, because the insertion does it anyway (where it is safe).
Change-Id: I7b340b18dd8efc77d1df78007161720c29866cb6
Make ve.dm.Surface.change accept array of transactions as a parameter (instead of just one) and use it in complex content removal (handled in Surface view).
Change-Id: I453b3606cefe140db206f5a2d2c9036bcbd639c9
The error occurred after pressing backspace in a paragraph that
contained an alienInline node (or other inline node). It happened
because ve.ce.BranchNode.clean() called this.$.empty(), which destroyed
all element data in the paragraph's children, including the
.data('node') property which associates the DOM node with the CE tree
node. Use .detach() instead.
Change-Id: I563c936a706a1d1711e6333404c6ca8e6ebf8eeb
-Using keydown on arrows allows preventDefault which stops the cursor movement in the input.
-Consolidated Keydown bindings, no longer unbinding keydown, instead checking for visible state first.
-Using case insensitive comparison for selected item.
-Changed use of ve.inArray to ve.indexOf because method name was changed.
-Moved clear break into suggestion container rather than after it.
Added margin bottom on suggestion container for category separation.
Change-Id: I2bd1db049a948db189194037dc8e38dfe884c197
Created jQuery plugin MultiSuggest which builds a categorized dropdown
under specified input box.
Revised inspector to no longer be an iframe but to contain an Iframe.
This reduces xbrowser issues with positioning and toggling inspector
container.
Added Inspector overlay element for positioning arbitrary elements
over the iFrame. This prevents growing the iframe to arbitrary lenghts.
Change-Id: I8efbbd091b0b24a19a4b73aa122d21a329cf97e4
When the document is empty, there is no last branch, and it causes an error because the code was assuming there was always going to be a last branch.
Change-Id: I371dce89db6258d30a11022c1bdb11830f59505d
* Summary label is now placeholder text
* Edit summary and options are visually connected
* Added summary length count-down label
* Removed check-mark icons from toolbar and dialog save buttons
* Made toolbar and dialog save buttons the same size
* Reduced line-height and margins of license information
* Changed from X icon for close (which might be confused with cancel) to ^ icon for collapse
Change-Id: Ib1711f49af8929be12796aecdea49467b726856e
* When you visit the page with url# (empty target) it scrolls down - which made setting the URL of the namespace tab to # really bad, plus it was useless since we already handle that click using JS
* Found that this.$toolbarWrapper wasn't pointing to the right place because of a missing .end() call at the end of a chain of jQuery madness
Change-Id: I7dc6953e39d4081f1b91a351bb830e4c18f7b988
Also:
* Made a fragment with a null range become a null fragment
* Fixed incorrect order of arguments for binding a handler to transact event
* Added getters for surface, document and range
* Fixed several instances of passing a document instead of a surface into the constructor of a new surface fragment
* Fixed closest mode in expandRange - needed to check if parent existed before checking for it's type
* Fixed uses of ve.Transaction (doesn't exist) that were supposed to be uses of ve.dm.Transaction (does exist)
Change-Id: Ide13d9d2d1637399188c98c2e8b6e0826caeecc4
When a document is created, it should take it upon itself to make sure it has a new reference to the data using slice, not place this on the caller. Callers that do not use slice will often find strange and mysterious things going on and not know why. The real reason is that multiple documents sharing a reference to the same data array leads to seriously messed up behavior.
Change-Id: Ic4e25fcd9bf3f41a805003520a8f38e2768f5dbf
domToData wraps bare content in paragraph elements, which were then
converted to <p> tags by domToData. With this fix, HTML with "missing"
<p> tags actually round-trips through the editor correctly now, rather
than having <p> tags added wherever VE believes they should exist.
* Mark generated paragraph elements with .internal.generated = 'wrapper'
** This signifies the wrapper was generated but its contents were not,
so the right thing to do when converting back to HTML is to remove
the wrapper and keep the contents. We might want to use other values
of generated in the future.
* Unwrap nodes with generated=wrapper when converting to HTML
Tests:
* Add 'generated': 'wrapper' as appropriate. Only affects 1 test
* Remove 'normalizedHtml' for this test because it is no longer needed
** Need to keep 'normalizedHtml' for now because we normalize hrefs
* Eventually the main example should test bare content, but that
requires touching a lot of stuff. The main example could use some
beefing up anyway.
Change-Id: I277ad5fe3f64e07c1bbf49007d6bbaecc90b7466
This allows us to put other internal data in there in the future. Also
passing it through the Node constructor properly now.
* ve.dm.Node
** Rename fringeWhitespace property to internal
** Add internal parameter to constructor
** Remove setFringeWhitespace()
* Increase the number of parameters passed through by ve.Factory to 3
* Pass through .internal from linmod to nodeFactory in ve.dm.Document
* ve.dm.Converter
** Rename .fringeWhitespace to .internal.whitespace and make it an array
** Store a temporary reference to .internal in domElement.veInternal
* Add internal to all node constructors except TextNode
Tests:
* Update for fringeWhitespace->internal rename
* Add third parameter to ve.Factory tests
* Add .internal to getNodeTreeSummary
Change-Id: If20c0bb78fee3efa55f72e51e7fc261283358de7
This makes a lot more sense when you start making a surface fragment from a new surface, because a null range would seem to have unpredictable behavior.
Change-Id: I85210878deca3067960fa4a14e2a760e55f67e4e
* Made vector specific styles only active in the vector skin
* Added apex specific styles
* Removed override of text size for document node
* Added stylesheet for stand-alone to specify text size for document node
Change-Id: I8a57918912499f9453a5692ff45a04a16ed34cde
To do this, we are using the replacer callback of JSON.stringify, which is supported by all of our target browsers including IE8. We are also leaning on Object.keys and Array.reduce, the latter of which required adding a new fallback implementation for some browsers which do not support it yet.
Change-Id: Ifa285ca3da4d94d962464f09414591532bbea79c
Because the Parsoid prefix format changed from /mw:Foo to /mw/Foo , the
href format for internal links has changed from "/Foo" to "Foo". So the
href is now simply the title, except that it may be preceded by one or
more "../" if the title of the page we're on contains a '/'.
So instead of stripping the leading slash from internal link hrefs and
putting it back on the way out, only strip any leading "../"s and dump
the titles directly into the hrefs on the way out.
Also update the link test case for this, and add a test case for the ../
stripping.
Change-Id: I3e0bdde20f22cda34eb45fc351de5e780419b6a2
Annotation types with more than one slash such as 'link/ExtLink/URL'
weren't being processed correctly because .split( '/', 2 ) throws away
everything after the second slash. Instead, don't pass a limit to
.split(); the code for reconstructing a slash-separated string from
multiple components was already in place.
Also add test cases for URL links and numbered links.
(Do you like the lines-of-code to lines-of-test ratio in this commit,
Trevor? ;) )
Change-Id: I7add87396447a01b1c23a4f9bfd63d2e8fd861ce
Refactor:
* ve.indexOf
Renamed from ve.inArray.
This was named after the jQuery method which in turn has a longer
story about why it is so unfortunately named. It doesn't return
a boolean, but an index. Hence the native method being called
indexOf as well.
* ve.bind
Renamed from ve.proxy.
I considered making it use Function.prototype.bind if available.
As it performs better than $.proxy (which doesn't use to the native
bind if available). However since bind needs to be bound itself in
order to use it detached, it turns out with the "call()" and
"bind()" it is slower than the $.proxy shim:
http://jsperf.com/function-bind-shim-perf
It would've been like this:
ve.bind = Function.prototype.bind ?
Function.prototype.call.bind( Function.prototype.bind ) :
$.proxy;
But instead sticking to ve.bind = $.proxy;
* ve.extendObject
Documented the parts of jQuery.extend that we use. This makes it
easier to replace in the future.
Documentation:
* Added function documentation blocks.
* Added annotations to functions that we will be able to remove
in the future in favour of the native methods.
With "@until + when/how".
In this case "ES5". Meaning, whenever we drop support for browsers
that don't support ES5. Although in the developer community ES5 is
still fairly fresh, browsers have been aware for it long enough
that thee moment we're able to drop it may be sooner than we think.
The only blocker so far is IE8. The rest of the browsers have had
it long enough that the traffic we need to support of non-IE
supports it.
Misc.:
* Removed 'node: true' from .jshintrc since Parsoid is no longer in
this repo and thus no more nodejs files.
- This unraveled two lint errors: Usage of 'module' and 'console'.
(both were considered 'safe globals' due to nodejs, but not in
browser code).
* Replaced usage (before renaming):
- $.inArray -> ve.inArray
- Function.prototype.bind -> ve.proxy
- Array.isArray -> ve.isArray
- [].indexOf -> ve.inArray
- $.fn.bind/live/delegate/unbind/die/delegate -> $.fn.on/off
Change-Id: Idcf1fa6a685b6ed3d7c99ffe17bd57a7bc586a2c
This makes things like
== Foo ==
* Bar
render without the leading and trailing spaces, while still
round-tripping those spaces.
* Added a .fringeWhitespace property to the linear model and ve.dm.Node
** Object containing innerPre, innerPost, outerPre, outerPost
** Only inner* are used right now, outer* are planned for future use
** Like .attributes , it's suppressed if it's an empty object
* In getDataFromDom():
** Store the stripped whitespace in .fringeWhitespace
** Move emptiness check up: empty elements with .fringeWhitespace have
to be preserved
** Move paragraph wrapping up: .fringeWhitespace has to be applied to
the generated paragraph, not its parent
** Add wrapperElement to keep track of the element .fringeWhitespace has
to be added to; this is either dataElement or the generated paragraph
or nothing, but we can't modify dataElement because it's used later
* In getDomFromData():
** When processing an opening, store the fringeWhitespace data in the
generated DOM node
** When processing a closing, add the stored whitespace back in
* In the ve.dm.Document constructor, pass through .fringeWhitespace from
the linear model data to the generated nodes
Tests:
* Change one existing test case to account for this change
* Add three new test cases for this behavior
* Add normalizedHtml field so I can test behavior with bare content
Change-Id: I0411544652dd72b923c831c495d69ee4322a2c14
It was replacing spaces with s in its copy of a slice of linmod
data, but the copy was a shallow copy, so when modifying annotated
spaces it would modify the actual data. Fixed by detecting this case and
cloning the character's array before modifying it.
Change-Id: Ic96ed34605a302af18f274a0faa6d0f650f5b771
Also added some checks in content branch conversion to make sure that converting from and to the same thing results in a no-op
Change-Id: Ie47520d666e45a77d12c7ebb9457aef7ab6b8097
* Per Krinkle's comment, be tolerant of missing .htmlAttributes
* Drop .htmlAttributes if no attributes, fixes some tests
Change-Id: I65589a8b489e19a7c8a41ba2f4a57e68fc52684c
* Restricting "camelcase":
No changes, we were passing all of these already
* Explicitly unrestricting "forin" and "plusplus"
These are off by default in node-jshint, but some distro of jshint
and editors that use their own wrapper around jshint instead of
node-jshint (Eclipse?) may have different defaults. Therefor
setting them to false explicitly. This also serves as a reminder
for the future so we'll always know we don't pass that, in case
we would want to change that.
* Fix order ("quotemark" before "regexp")
* Restricting "unused"
We're not passing all of this, which is why I've set it to false
for now. But I did put it in .jshintrc as placeholder.
I've fixed most of them, there's some left where there is no clean
solution.
* While at it fix a few issues:
- Unused variables ($target, $window)
- Bad practices (using jQuery context for find instead of creation)
- Redundant /*global */ comments
- Parameters that are not used and don't have documentation either
- Lines longer than 100 chars @ 4 spaces/tab
* Note:
- ve.ce.Surface.prototype.onChange takes two arguments but never
uses the former. And even the second one can be null/undefined.
Aside from that, the .change() function emits
another event for the transaction already. Looks like this
should be refactored a bit, two more separated events probably
or one that is actually used better.
- Also cleaned up a lot of comments, some of which were missing,
others were incorrect
- Reworked the contentChange event so we are no longer using the
word new as an object key; expanded a complex object into multiple
arguments being passed through the event to make it easier to work
with and document
Change-Id: I8490815a508c6c379d5f9a743bb4aefd14576aa6
Right now this means things like headings and list items are rendered
nicer (without the whitespace), but also get their whitespace normalized
when saving back. I'll submit code tomorrow that preserves this
whitespace.
Submitting this now because it's needed to make <br>s look reasonable
Change-Id: I4b5e5ad8ee1bbe2f1eaf0fb860dd59f6e401dc3d
The new annotation API will do this too; this is a temporary hack to fix
the bugs caused by stripping attributes.
This code doesn't actually render the attributes, but the new annotation
API will.
Change-Id: Ic0ddf822fe02f101f2e825080c6bcc2a03115974
Stack traces, line numbers, etc. All the approaches I've seen are bad hacks. This is the best way to go.
Change-Id: Ib12e9d2ecfe610bcc89d046005e35cc13efa3d99
Throwing strings is bad because it doesn't include a lot of important
information that an error object does, such as a stack trace or where
the error was actually thrown from.
ve.Error inherits directly from Error. In the future we may create
more specific subclasses and/or do custom stuff.
Some interesting reading on the subject:
* http://www.devthought.com/2011/12/22/a-string-is-not-an-error/
Change-Id: Ib7c568a1dcb98abac44c6c146e84dde5315b2826
* Use 'href' of #ca-edit instead of constructing it manually.
MediaWiki's output of #ca-edit already takes care of all needed
queries, including "oldid" (bug 38125).
* Misc clean up:
- Use .get() instead of accessing the array directly.
* Clean up setupSkinTabs and add more inline documentation.
Change-Id: I7d702a3eb1f9ce23a5e3c9e846b00da5cead386e
* There were only 3 files with single quotes, fixed them all.
* Added option to .jshintrc (be sure to use the latest version of
node-jshint since this is a fairly new addition to the library).
Change-Id: I8bf8895ce56bf86e3bed244279a9d32269e44763
This just cleans up a few places where single quotes should have been used instead of double quotes.
Change-Id: I6c53652e71ab96842ed5bb41fb1e0b8c923eb25d
When closing annotation nodes, we weren't popping them off
annotationStack. Not sure where this came from, but the code was
definitely bad and this fixes it.
Change-Id: I6d805e9aca3778666212135f76ff34c6baacbbc8
* Default value of wgVisualEditorParsoidURL is broken.
Slash is needed, else Api will request to
http://hostnamePageName
Roan says double slashes are okay, and look cleaner than string
search checks etc.
* Use .clone() for mw.Uri instead of converting to string
and letting mw.Uri parse it, again. Clone creates a basic
instance and copies over properties internally (deep copy,
no references).
* No need for hasOwnProperty (and its potential issues)
* Code clean up
- Whitespace consistency
- Variable hosting
- Remove redundant `return false;` statements in event handlers
e.preventDefault() is a jQuery.Event method that takes care
of cross-browser issues.
- Same for e.keyCode||e.which thing, this is already normalized
by jQuery.Event
- Add missing parameter to setTimeout
- Consistent order in success/error handlers in $.ajax options
Change-Id: I5bc24e0cbdf01b3704d4ccb0b45b3052e3b58694
Also:
* Removed a lot of dead code in Surface that was used in the now dead and gone sandbox.
* Changed from throwing an exception when calling getBalancedData on a range that produces no results from selectNodes to just returning []
Change-Id: Icf27094724eae5b90eec21308f9e26afe877e3ee
When editor surface loses focus, now checking to see if an inspector
is set or a menu is open prior to closing the context.
This ruling ensures more rational context icon / child menu behavior.
Change-Id: Ic4b74bb51e811e264a2109a3c1c5a8ae503a8c49
* Moved all constants to a singleton constants object
* Moved couple methods to mediawiki.Util.js
* Fixed url regexp bug -- relative urls weren't being matched
but, this bug escaped through because the previous url regexp
had a typo (which means url sanitization wasn't being done by
the previous code). This also means we have to either find
sanitizer tests or add new ones so these bugs are caught. Maybe
parserTests.txt is not the right place for this?
Change-Id: Ia05e1d1596bb9bc4a9eb21d7c77248f5626a710e
* Dynamically select piped / simple wikilinks in the serializer
* Use generic attribute shadowing, drop sHref
* Actually handle modifications to non-piped wikilinks
* Properly escape anything that looks like a percent-encoded char
* Add stripSuffix utility method
* Fix a bunch of JSHint warnings in the serializer in particular
Change-Id: I2d8954f6b665093676ccc5dd5437ea9b37c014ad
* Classicifation (JS)
Use addClass instead of attr( 'class' ) whenever possible.
addClass will manipulate the properties directly instead of
(re-)setting an attribute which (most) browsers then sync
with the properties.
Difference between:
elem.className
and
elem.setAttribute( 'class', .. );
Just like .checked, .value, .disabled and other interactive
properties, the HTML attributes should only be used for initial
values from the html document. When in javascript, only set
properties. Attributes are either ignored or slow.
* Styling (JS)
Use .css() instead of attr( 'style' ).
Again, setting properties instead of attributes is much faster,
easier and safer. And this way it takes care of cross-browser
issues where applicable, and less prone to error due to dealing
with key-value pairs instead of css strings.
Difference between:
elem.style.foo = 'bar';
and
elem.setAttribute( 'style', 'foo: bar;' );
* Finding (JS)
Use .find( 'foo bar' ) instead of .find( 'foo' ).find( 'bar' ).
It is CSS!
* Vendor prefixes (CSS)
It is important to always list newer (standards-compliant) versions
*after* the older/prefixed variants.
See also http://css-tricks.com/ordering-css3-properties/
So the following three:
-webkit-gradient (Chrome, Safari 4)
-webkit-linear-gradient (Chrome 10, Safari 5+)
linear-gradient (CSS3 standard)
... must be in that order.
Notes:
- "-moz-opacity" is from before Mozilla 1.7 (Firefox < 0.8)
Has not been renamed to "opacity" since Firefox 0.9.
- Removed redundant "-moz-opacity"
- Added "filter: alpha(opacity=**);" where missing
- Fixed order of css3 properties (old to new)
- Add standardized css3 versions where missing
(some 'border-radius' groups didn't have the non-prefixed version)
- Spacing
- @embed
- Shorten hex colors where possible (#dddddd -> #ddd)
$ ack '#([0-9a-f])\1{5}' --css
$ ack '#([0-9a-f])\1{2};' --css
Change-Id: I386fedb9058c2567fd0af5f55291e9859a53329d
* Need auditing to see if the whitelist is complete.
* Added <gallery> and <tag> to the whitelist since they seem to
come from some extensions and parser tests depend on them.
May need to use parsoid config or parser hooks/extension info
to extend sanitizer whitelist. For now, probably okay.
Change-Id: Id9ecdd96843e8f6ca65e8666807dec1015443d49
* Also update test/index.html with latest minimalistic format
* Update test suite hardcoded paths to match the definition in
VisualEditor.php for ResourceLoader
* Issues:
- 'jquery/jquery.json.js' should not be loaded directly,
using a dependency instead.
- Load scripts from the <head> in test/index.html so that
code that depends on document being ready is catches instead of
silently being ignored.
Change-Id: I5ad7390137f4d17c153a1bf69f19c4869c08e323
-Bug 38042
Save dialog description field doesn't respond to 'Enter' or 'Return' keys
-Bug 38621
Pressing 'Esc' in Save-dialog should exit saving, return to editor
Change-Id: I9c43c6c9f2f2b538becc4fbbce1eda6e918d4879
'''Kranitor commits''' are commits by Krinkle with his janitor hat on.
Must never contain functional changes mixed with miscellaneous changes.
.gitignore:
* Add .DS_Store to the ignore list so that browsing the directories
on Mac OS X, will not add these files to the list of untracked
files.
* Fix missing newline at end of file
.jshintrc
* raises -> throws
* +module (QUnit.module)
* remove 'Node' (as of node-jshint 1.7.2 this is now part of
'browser:true', as it should be)
Authors:
* Adding myself
MWExtension/VisualEditor.php
* Fix default value of wgVisualEditorParsoidURL to not
point to the experimental instance in WMF Labs.
Issues:
* ve.ce.TextNode:
- Fix TODO: Don't perform a useless clone of an already-jQuerified object.
- Use .html() to set html content instead of encapsulating between
two strings. This is slightly faster but more importantly safer,
and prevents situations where the resulting jQuery collection
actually contains 2 elements instead of 1, thus messing up
what .contents() is iterating over.
* ve.ce.Document.test.js
- Fix: ReferenceError: assert is not defined
* ve.dm.Document.test.js
- Fix: ReferenceError: assert is not defined
* ve.dm.Transaction.test.js
- Fix: ReferenceError: assert is not defined
* ve.dm.TransactionProcessor.test.js
- Fix: ReferenceError: assert is not defined
* ext.visualEditor.viewPageTarget
- Missing dependency on 'mediawiki.Title'
Code conventions / Misc cleanup
* Various JSHint warnings.
* Whitespace
* jQuery(): Use '<tag>' for element creation,
use '<valid><xml/></valid>' for parsing
* Use the default operator instead of ternary when the condition and
first value are the same.
x = foo ? foo : bar; -> x = foo || bar;
Because contrary to some programming language (PHP...), in JS the
default operator does not enforce a boolean result but returns the
original value, hence it being called the 'default' operator, as
opposed to the 'or' operator.
* No need to call addClass() twice, it takes a space-separated list
(jQuery splits by space and adds if needed)
* Use .on( event[, selector], fn ) instead of the deprecated
routers to it such as .bind(), .delegate() and .live().
All these three are now built-in and fully compatible with .on()
* Add 'XXX:' comments for suspicious code that I don't want to change
as part of a clean up commit.
* Remove unused variables (several var x = this; where x was not
used anywhere, possibly from boilerplate copy/paste)
* Follows-up Trevor's commit that converts test suites to the new
QUnit format. Also removed the globals since we no longer use those
any more.
Change-Id: I7e37c9bff812e371c7f65a6fd85d9e2af3e0a22f
This should make it much simpler to keep MediaWiki specifics out of VisualEditor, which will in turn make it easier to integrate VisualEditor into another platform.
Change-Id: I073e9737b37c28af889f2457d10b082cefd0d63b
* CSS sanitization code now ported.
* Could use non-port review another day to see if the ported code makes
sense and has any gotchas.
* More sanitizer parser tests now green.
* Could use a lot more aggressive addition of parser tests.
Change-Id: I9df003540bd31f327f5307472c9f7dcbbe7b4342
Convert underscores in the href attribute to spaces in the linear model,
and back to underscores when going back to HTML. This ensures the link
targets displayed to and edited by the user look nice
Change-Id: I4855fce28ad8b724284c53881abc7b99b59b9079
Clearing context icon when editor loses focus.
-Reproduce problem by selecting text, then click outside of
the editor. Selection is lost, and context icon is stuck.
Change-Id: I4b321f16cea73ec0e51540c0e71f265ab47514e9
This means we don't have to rely on data-rt.sHref. It also means that
we'll now be showing the canonical link target in the link inspector
rather than the link target as entered by the user, but that's fine.
Also change test to have href differ from sHref to show that we use
href.
Change-Id: Idabdbf2579663ef1efb47d6a73f39743c9f64f3b
This is ugly but makes things work again. I intend to clean this up once
we have a better attribute API
* Recognize mw:WikiLink, mw:SimpleWikiLink, mw:ExtLink,
mw:NumberedExtLink and mw:UrlLink
* Support is incomplete because we can't get to the annotation text with
the current API
* Preserve all unhandled attributes rather than special-casing data-mw
* Update remaining code using data-mw (sHref and stx extraction) to
account for the data-mw -> data-rt rename
* Update tests accordingly
Change-Id: Ia13d3008a6d4cdc8828f9acda5aa797566bc597f
The recent directory move broke parserTests, fix it for now. Will need
refixing once me migrate to our own repo.
Change-Id: I014001cd6904d1dea3f9417c9cde9c80ab079232
* Quite unique according to google, and more obvious
* Also adjust parserTests to ignore mw:Nowiki and mw:Placeholder spans
Change-Id: I340e85092b60a65b4053a40bf8c238e26cb49c96
- Attaching save dialog to toolbar wrapper vs toolbar itself.
- Attaching surface specific toolbar wrapper vs all toolbar wrappers
in the case of multiple editors on the page.
Change-Id: Ic81f5a680f5593c71c27b7d47fe246487eebd4a3
* Ported attribute sanitization code (and related functions) from
core/includes/Sanitizer.php
* Added dummy flags and set to true (use of rdfa, microdata attrs,
and html5 mode).
* Removed couple whitelisted sanitizer tests.
* A few sanitizer tests now pass.
* More work to be done.
Change-Id: I19c92bbfcb57f3e97a7af1b7c5f63772e427dae4
(or this shouldn't be allowed)
-Revised method for for returning all link annotations in a
selection. Now properly clearning all selected links.
-Trimming whitespace from selection
-Modifying selection if it doesn't contain annotated range
-Disabled link creation only if target is blank. This allows
Existing link text to be modified while having the same target.
Change-Id: I7255dcf1c88fa1cd6e7edbc3baa82cd4c72a95d1
-Hide / show inspector with visibility vs display property
for iframe cross browser compatibility.
Change-Id: Ibdd0250872c42d74d6ff7d22abdf9d838962acc1
This was caused by a bug in fixupInsertion that caused it to believe
that inserting something like "a</p><p>b</p><p>c" into the middle of an
empty paragraph was invalid.
This commit fixes the fixupInsertion bug, which fixes the
select-all-cut-paste behavior in Chrome. It's still broken in Firefox
because of selection-related issues, but I'll split that out into a
different bug report.
Change-Id: I767f5d37ec7e511778ae9ca8283ec4b26c728298
* addAttribute and getAttribute do the obvious and simple thing
* addNormalizedAttribute remembers an unnormalized version of an attribute and
supports change detection for the normalized attribute
* getAttributeSource retrieves the original attribute if there was a
normalized version which was not changed, or the current value (potentially
based on the normalized version) otherwise. For use by the
WikitextSerializer.
Change-Id: I72533cf6cfff1ddb88be2501653c7c47d270898c
In preparation for the big extraction of Parsoid out of VisualEditor,
we'll start by moving the tests into the parsoid location.
Change-Id: I4a926ee4aad1490d4f769d44e91af80842b881f0
* Got rid of mergeProperties monkey-patch from core-upgrade.
* Reformatted class defns in mediawiki.parser.defines.js.
* Protected unconditional tokenization of list handler output with an
env.trace check.
* Other minor formatting fixes to respect 80-100 column code width
guideline.
Change-Id: Ida769e0e239b01a813b2d30a65aba60216262a43
* the used RDFa types for links are now identical to those listed in
http://www.mediawiki.org/wiki/Parsoid/RDFa_vocabulary, and are supported for
serialization
* Editors are responsible for adjusting the type when converting between link
types. Adding a caption to an mw:UrlLink for example should convert it into
an mw:ExtLink.
Update: rebased on top of trace patches
Change-Id: Ie1b882e2b3fbad08be94769e1167dccd8dfea65d
* Source-based round-tripping now uses typeof="mw:Placeholder" instead of
data-gen.
* mw:Image is supported for round-tripping, but not yet for modifications as
it is still source-based
Change-Id: Ie5cf4e54de0163168c25c2b5c09380657a15970f
* changes:
Additional work on readable tokenizer debug output
Added 'href' key to anonymous KV wikilink and isbn attribute.
Removed utility functions from mediawiki.parser.environment
Added utility methods to ext.Util.js
Added missing var keyword
In trace mode, wrap transform to output trace info
Output chunk tokens to console only in trace mode
Further refinement of readable pretty-printing of tokens.
More fixes on the way to readable debug/trace output.
Added mergeProperties function to Object.prototype
In RTL interface the drop-down arrow overlaps with the text.
The arrow was positioned explicitly in relation to the left margin, and this
can't work in a flipped RTL environment. I changed the position to "right"
and modified the arrow image a bit.
Some visual tweaks to the arrow may still be needed. Another option can be
to convert the image to SVG or to use a character like ▼.
Change-Id: Ib09a2a20b150de6e8a9531fc0db7dfffe4e95525
* This makes wikilink attrs more similar to ext links.
* Added 'content' key to ISBN links, but couldn't add it to regular
wikilinks yet because of complexity of how they are handled in
the rest of the pipeline. Changing this requires fixing up other
parts down the pipeline -- something for later.
* Fixed up wikilink handler to use named lookup for 'href' and
'tail' rather than positional lookup. Content lookup is still
positional as before.
Change-Id: I657b1f338d38df3cfdfa99f27ac46e7fe1c9fd65
* these functions have already been added to ext.Util.js
* removed a couple jshint warnings.
* minor code restructuring in tokensToString and comments
to better indicate what is going on.
Change-Id: I9d6a03cc35075e1a64d8fac9e167a3ce4ccd9424
* Copied over utility methods from mediawiki.parser.environment.js
to ext.Util.js.
* Moved over utility method from mediawiki.parser.defines.js to
ext.Util.js.
* Converted Util to be a singleton object rather than an allocatable
class. There is no reason to allocate a new utility class everywhere
since this utility object has no useful state.
* Fixed up use of utility methods to use Util rather than env.
Change-Id: Ib81f96b894f6528f2ccbe36e1fd4c3d50cd1f6b7
- Added extra debug_name parameter to addTransform which is
used in addTransform to output useful trace info.
Change-Id: I160ba0c45f681149375e32ab19f97baa439b09a8
- Added limits to toolbar float, Toolbar will not go past the last node in editor.
- Added bottom mode to allow toolbar to stick above the last node until the scroll position
is above the last node.
- Actually checking toolbar config now and setting float when flag is set.
- Gave float method for top toolbar a better name.
Change-Id: Ic39c5402fa7a05e13c5e81722d8729d93776d7e9
But still put slugs before them. Done by overriding canHaveSlugAfter()
in ve.ce.ListNode.
Eventually this should be configurable and MediaWiki-specific
Change-Id: I5ad15ca4085a2d730add4954acbea358819b3986
These determine whether a node can have a slug before and after,
respectively. The default implementation in ve.ce.Node is to use the
same rules for both, but individual node types can override this.
I'll need this to suppress slugs after nested lists but not before them.
Change-Id: Id88c0fc98aca7c7f52ce990ed9b8c42181ef6d18
For pressing Enter in an empty list item at the end of a top-level list,
this has the same result as the previous code, but if you're in a nested
list it has the effect of jumping down a level. A previous incarnation
of this change just made Enter insert more list items ad infinitum if
you were in a nested list, but I think this is better.
This fixes a bug where pressing at the end of a nested list inserted a
paragraph in an invalid location
Change-Id: I9c7dbaf29a98f84926ed3a05e71c6294926dfce2
Fix the commented-out code: it caused unindent to be triggered by just
pressing Shift. ASCII 16 is "data link escape", no idea where that came
from, so I removed it and used e.shiftKey instead.
Also check whether indent/outdent is even possible before doing it.
Currently this is done in a very hacky way (by checking the state of the
indent button), ideally we'd refactor things such that toolbar tools can
listen for keydowns and intercept them, that would make the code much
cleaner and we wouldn't have this problem.
Change-Id: I99885ee4b8a79cd24c4958c188addfc2b0453b03
After indenting or outdenting a list item in a numbered list, the
numbering wasn't updated. So if you had:
1. One
2. Two
3. Three
and you indented "Two", you'd get:
1. One
1. Two
3. Three
Adding or removing items in the list using the keyboard, or even
inspecting the list in Firebug (!), would trigger a renumbering and fix
the list to display "2. Three". But then the same issue would occur in
in reverse when outdenting "Two" (either using undo or using the
outdent button):
1. One
2. Two
2. Three
The workaround is to force a reflow by requesting the height (thank you
Timo). Implemented this in an override of onSplice() in ve.ce.ListNode, so
the list is detached and reattached every time children are spliced into
or out of it.
I haven't managed to come up with a minimal test case for this, not even
by putting a list in a contentEditable div and doing the same DOM
operations that ve.ce does from a setTimeout callback.
Change-Id: I93b2a309034c411a7b4e4b6c6bd4ef9d473999eb
This works pretty well, the only problems I found are:
* when selecting multiple list items, only the first is in/outdented
* there's no special handling for child lists, so the behavior for
in/outdenting list items that have a child list can look weird, but
it's consistent
Also needs more documentation
Change-Id: I6f4f3725e57a590196d7d638a77b87ea85586dc8
* When ascending back up the stack, check for a start between two closings
* Also check for an end between two openings
* This introduces code duplication but selectNodes() is full of that
already. I'll have to do a duplication cleanup soon
* Add test case for </li><li>
* Update existing test case that covered a </li>
Change-Id: Ifc80585ce0e0d6988bc54228602c69f0d519200a
For nested lists, this function would return multiple groups where one
was wholly contained in the other, use offsets to prevent that from
happening.
Change-Id: Ib03bb1c81712d805cc263c2975cc3942de63d2ed
to be added. Create inspector elements in the propper document
scope. Restore inspector css classnames to camel case for proof
that inspectors are being created in the correct document scope.
Previously, inspector elements created in the wrong document scope
would have css rules applied only if class names were lowercase.
Issue only surfaced in Webkit browsers. Though, this implementation
is more future proof and will help prevent future inspector bugs.
Patch 3) Fixed global variable definition and mistake with
inspectorDoc
Change-Id: I36c0d078aea10d919689768878004a19f7f89b55
-Selection of part of a link now modifies selection to entire link
range on inspection.
-Retaining selection direction on new range
Only partial fix to bug as previous link annotation is not
yet properly cleared.
Bug 33053 - VisualEditor: Link creation should not include trailing
spaces, and should provide a suggestion based on selected text
-Created method to return a new range without outer spaces.
-Retaining selection direction on new range.
-Enhancement needed for link suggestion.
Bug 33108 - VisualEditor: Highlighted trailing whitespace should
not have styles applied
-Modified trim method to retain selection, added call to trim
range on annotate method.
Change-Id: I92f264e19350c62b7c2ac3cd9e78af0071afef5c
Now that we have access to the contents we can more easily compare the content
with link targets. This is still to do- this commit only converts the link
handler to work on the collected tokens.
* Start to implement latest RDFa spec from
http://www.mediawiki.org/wiki/Parsoid/RDFa_vocabulary
* Capitalize types, add mw:Entity type for html entities
* Detect changes to entities using tokenCollector and srcContent
Change-Id: I45429f4b930858a16e166ef8377c8f6f5114c414
This license change is aimed at maximizing the reusability of this code
in other projects. VisualEditor is more than just an awesome editor for
MediaWiki, it's the new editor for the entire internet.
Added license and author files, plus mentions of the license to all
VisualEditor PHP, JavaScript and CSS files. Parser files have not been
modified but are effectively re-licensed since there's no overriding
license information. 3rd party libraries are not changed, but are all
already MIT licensed.
Change-Id: I895b256325db7c8689756edab34523de4418b0f2
* "onevar" warning sometimes solved by just merging var statements
other times solved by making it a function declaration instead
of a function expression.
* Also fixed several '_this' variable names in ve.es.Surface to
more descriptive names, and enabled warnings for dangling _
in identifiers.
Change-Id: I7d411881e3e06cf9a7fe56d689c29375881a81de
This is just to avoid re-licensing along with VE. We want to be compatible
with MediaWiki core to make sure a closely-integrated C port is still
GPL-compatible. We could consider adding MIT to the JS implementation after
porting to C.
Change-Id: Ia83e8620e26c95625793438c4c5e8ddcf2702368
This is work in progress, but committed for now so I can use it for links and
tweak it while doing so.
Change-Id: I757277f6efacda6d9432ca57542a957f597a98de
* This code change is an attempt to address the FIXME about constant
resorting of transformations in _getTransforms. This caches sorted
transformations and selectively clears/updates the cache on add/remove.
Change-Id: If24a807b84d494aa4e5597339039a5573a30905e
This hopefully makes it clearer that data-rt contains private round-trip info
instead of semantically interesting data.
Change-Id: I03b476ed112a4b627c9871ee3677c450f943429a
* Arbitrary predicate support for the termination of collection mode
* tokens as property of the collector instead of a state-global thing
Change-Id: Ibcb342bc64a76fece9b04a760ea56c7878e67cad
* Fixed image serializer to deal with missing 'v' value in a k-v pair
representing an image attribute.
* Added fix to deal with bare <li>'s (without surrounding <ul> tags)
NOTE: The second fix is required currently to deal with bugs in the parser
as it deals with complex cases. But, in the future, we could deal with
this in one of the following ways:
(a) The serializer expects a well-formed DOM and all cleanup will be
done as part of external tools/passes.
(b) The serializer supports a small set of exceptional cases and bare
list items could be one of them
(c) The serializer ought to handle any DOM that is thrown at it.
Yet to be resolved.
Change-Id: Ib585e5c9f2a8a80854740ce0211bde705f9fd6f4
This makes it clear that setInterwiki can modify existing mappings, and adds a
similar method for the removal of existing mappings.
Change-Id: Ic603a4b2ccec35d086513fa7cf711702bfb2baa0
* Strips the first paragraph tag in a list item or table cell context
if there are no attributes on it and stx:html is not set
Change-Id: I74988645fe505c662f86488e32d0f11d464ffe41
* Looks like I misled myself in commit 88fc91 -- that wikitext
roundtripped perfectly because it went through the 'src' route
because it was a thumbnail with an explicit image which doesn't
go through renderThumb -- so, the serializer simply spit out the
original 'src' string and hence perfect rt :).
* More whitespace preserving fixes in LinkHandler.
* Also changed resource value in the img tag to use the original
filename rather than the normalized capitalized filename.
* 2 more parsertests rt -- now upto 400.
Change-Id: I144a6486dd9d07da8a74a68700fe96c78d192826
* Something to be said for code alignment - easier on the eye!
* Maybe a good case for breaking mediawiki coding guidelines.
* But, happy to abandon commit if not useful. :)
Change-Id: I1133af488f572ac7f8727be9108e08e14c4e6420
* Changed PrefixImageOptions so that thumb and thumbnail are
distinct key-value pairs. Without this fix, cannot distinguish
between thumb=foo.jpg and thumbnail=foo.jpg
* Fixed link handler so whitespace is preserved around prefixed image
options.
* Fixed figure handler to process the 3 different kind of image options:
size, simple image options, and prefixed image options.
* There is a hack/fixme for "upright: aspect" prefixed image option
which needs to be looked into.
* Still need to fix uppercasing of the image resource name.
With these fixes, the following wikitext roundtrips perfectly
(after newline breaks are removed)
[[Image:Foo.jpg|thumbnail = 'baby.jpg'|100x100px|center| alt =bbbbb|
upright=true|bottom|link='http://foo.bar'|
This is a [[Linked Caption]] in the image]]
Change-Id: I6606df56874c2b97f00f08cb6bbeaec9878167d3
* For now, extracted image markup options out of the link handler.
* This info will also be used by the serializer.
* More properties/global constants can be moved into this structure
over time.
Change-Id: I4cfbfd703f42e93fbad52b38b435f68d8a5c22ee
* Minor refactoring
* Cleared src in dataAttribs in renderThumb since we can serialize
thumbs now (or at least we can once all bugs are fixed and missing
pieces are handled).
Change-Id: If18865801cdd3d89c1477e68bfa3e13107c45b40
Anything with data-gen="both" and dataAttribs.src defined serializes to
dataAttribs.src and drops its contents (if any). We can use this to round-trip
elements we don't properly parse or serialize yet. Without RDFa info, the
editor will not touch the contents after encountering data-gen="both".
Change-Id: Ia39e5fdd765c2c9b36f26313455685d29f118839
* Don't consider them for auto-numbered links
* Don't insert a trailing space if the content is empty
These links are still wrapped in nowiki on round-tripping since the
valid/invalid url determination is done in the LinkHandler and not the
Tokenizer as it is configuration-dependent. Not incorrect for rendering (and
perhaps easier to understand for humans too), but might still introduce a
dirty diff. We'll still need reconciliation / damage tracking in the end ;)
Change-Id: I959ebc1b7f81d110a1141bb38ba5ee97f52ebf96
This only applies to newly created headings, so headings with a single newline
preceding them will be round-tripped that way.
Change-Id: Ic09972bbd25c3934b53f6fd3b5be5a0c3185c2af
* Collect all figure tokens and process them as a chunk
* This effectively mimics context-sensitive DOM walking,
but since we need serialization supported on a token stream,
we cannot use real DOM walking. The current technique should
also work on a token stream.
* There is a FIXME about the image filename being capitalized.
This needs fixing in the parser or some other way of recognizing
original unnormalized filenam.
Amended by gwicke:
* Build option list and join it with pipe to avoid stray trailing pipe
* Satisfy JSHint's weird preference to have '&&' and '||' at the end of the line
Change-Id: I1e5f6600f297fcdf81e3227a82ca3b71d4e97fc3
This is a zero-length tsr for now (and thus not 100% correct), but will do the
job for starttag / endtag range establishment
Change-Id: Iedd50ad319aa8d5916434fb6744deb04e031e456
* Removed dead commented out code.
* Cleaned up newline handling in serializer some more.
* Now, onNewLine and onStartOfLine reflect serializer state
more accurately.
* No implicit new lines for explicit html tags.
* 9 more roundtrip tests now green.
Change-Id: I9f640de2ae769c7472538fa687400dc8a40c2b2d
297 round-trip tests are passing with this patch.
TODO:
* generalize data-mw-gc handling in the serializer for any tag
* use data-mw-gc="both" and data-mw.src: 'the wikitext' for round-tripping of
wikitext structures, optionally with some presentational (but read-only)
content
* use span and data-mw-gc="both" for nowiki
Change-Id: I700142a56818977c20c8c06e6a5f2e77a708d25e
This makes sure that we escape start-of-line syntax when needed, since
onNewline is often not yet set.
Discussion / background:
[19:18] <subbu> this will fix it, but, i think this is asking for another
minor refactoring of these flags ... because this is a subtle fix which means
it might be possible to make it clearer. onNewline is one true in on
direction, i.e. if true, we are in a new line state, but if we are in a
newline context, onNewline is not true, which is why this new method is
needed.
[19:19] <subbu> i dont know if it is possible, but it seems like it shoudl be
possible. but, something for later.
[19:20] <subbu> badly phraed. "onNewline" ==> in new line context, but if in
new line context, onNewline may be false.
[19:20] <gwicke> we should perhaps update it as early as possible instead
[19:21] <subbu> i cannot today, but possible monday. i am heading out in
about 15-30 mins.
[19:22] <gwicke> will need to check all conditions depending on it in
_serializeToken
[19:22] <subbu> oh, i misunderstood you :)
[19:22] <gwicke> and if there are cases where the onNewline / onStartOfLine
state could be reverted later
[19:23] <subbu> you were referring to the flag, i thought you meant we should
fix this sooner than later.
[19:23] <gwicke> yes, I wasn't terribly clear
[19:23] <gwicke> you wrote something about following productions swallowing
newlines, but I think we don't actually do that any more
[19:24] <gwicke> I'm quite optimistic that updating those flags much earlier
would work
[19:25] <subbu> yes, it could fix it.
[19:26] <subbu> you might be right reg. swallowing. it was happening earlier.
but, not right now, after single-line mode and other fixes.
Change-Id: Ic1d8141c04eb54a59977d0ba87bcf06bafd421e0
This should not really be needed if the tokenizer did not decode html entities
on the fly. It is still a quick way to make sure no htmlish content can be
inserted even with the current decoding.
The next step and proper fix is to make entity decoding either optional in the
tokenizer (flag-controlled), or move it to a later stage in the token
processing pipeline.
Change-Id: Ife093dcfb95113763dab5635b098c795d3550586
* Renamed defaultOptions to initialState
* Got rid of unused state property
* Added comments explaining how state attributes
and tag handler flags are used
* Refactored listItemHandler check into functions and
added FIXME possible rewriting of that check.
* Protected serializeDOM in a try-catch handler to
catch exceptions and output the exception to the console.
Change-Id: I3d351c06e4b86baeb5a55243b11dbfa9baca5bb7
This fixes a bug Trevor reported where selecting from a list item across
a heading and into a paragraph, pressing backspace, then clicking undo
caused an exception.
Change-Id: Id2851271529e10548f6979a030a198054aa1c48f
ve.ce.TextNode listed textStyle annotations that didn't actually exist,
and failed to recognize some that did exist (such as span; bug 37808).
Added all annotations to both places. <span> tags are now tolerated by
the editor in that it doesn't crash anymore, but they're displayed (and
saved!) without any attributes, so <span style="color:yellow;">y</span>
doesn't show a yellow 'y' in the editor and is saved back as
<span>y</span> .
Change-Id: Iaae11ad5044150fa904010983ff83579cb37733d
Fixed by adding the specialMessages module which is only loaded once the
editor loads. Then after it's loaded we use the summary message from
there to update the (possibly broken) summary message in the save
dialog.
Change-Id: I67f5c59501cdf7c66c925cef8d4dd42b0f2cfde3
* Removed murky ' :' -> ' :' replacement in tokenizer. This breaks four
parser tests, and should be fixed in a token stream transformer or DOM
postprocessor. This replacement clashes with round-tripping, and is not
terribly important visually.
* Added stx:row annotation to single-line dt/dd pairs and use it to preserve
single-line syntax in the serializer. There is no attempt yet to support the
addition of nested lists in an originally single-line dd. We'd need to look
ahead in the serializer to support this. Perhaps the editor can simply drop
data-mw in that case.
* Switched default dt/dd serialization to multi-line. This supports all nested
lists and multiple dds.
* Don't close dls when switching from dt to dd or back in the token stream
ListHandler.
Overall 290 round-trip tests are passing now (up from 284, some due to ,
some due to lists). The number of passing parser tests dropped slightly from
303 to 297 (or 301/295 on weekdays other than Thursday).
Change-Id: I85ff40571833713388c6523e6a4ba2e94daa3807
Basically only prefix all bullets if the serialization output is going to be
in start-of-line context. The test for that is currently inline, but should
perhaps be factored out to a method or state flag instead.
We could alternatively consider to return the start-of-line prefix and let it
be used in _serializeToken in case we end up in start-of-line context.
This patch also fixes a newline issue on input like this:
:d1
::: d3
Both the list and list item handlers now set the startsNewline flag
dynamically depending on the context, so that we don't depend on the
suppression of newlines from list syntax by the singleLineMode any more.
There is still an extra newline inserted between list items in the following
example:
;t1 :d1
;;t2 ::d2
This looks like a bug in the produced DOM and not in the serializer, since the
outer definition list is closed and re-opened between d1 and t2.
Change-Id: I78e3a1ef34cf9159d5a1e86fb64c774ff111e71d
* changes:
Got rid of iteration to get the surface
Removed attach and detach methods from ve.ce.Node
Track adjustments in DocumentSynchronizer and apply them to oldRange
This is needed because oldRange is relative to the state of the model before any changes were made, but when we call selectNodes() it's gonna operate on a partially updated model tree.
This is a genuine bug in DocumentSynchronizer proper, which means I owe the entire team lunch
Change-Id: Ia6510de19df02e961c7f25fb8e7833abceb8d25b
* Adjust both start and end for preceding operations
* Adjust end for the current operation as well
Change-Id: I2f96d609bddf3788aa5700ad1f0b46208f3517d7
text for a link is the selected text.
patchset 2 - add case for if data offset is an object, be sure it
is a string prior to adding it. truncate to 255 chars.
patchset 3 - actually add the patched file
Change-Id: Ibddf942c2a0ba3412d93cf9730f74eb858025fad
Also fixed unlisting to not break when unlisting a range that includes things that are not list nodes
Change-Id: Ib9d4ea851c3ed9bf72a93aa87e470ce40c308453
contained by a list node the button is off. Button is now only toggled on
if all nodes in selection have a parent list node.
Patchset 2 - cleanup whitespace
Change-Id: Ia9adc39c0d5c75e2e96580f0e172f5b602540ac3
The main issue is that the bullets from dd/dt were not stored on the stack. I
added a separate field for it in each stack entry, which now fixes the basic
indent case without (afaik) breaking anything else.
There are still some newline issues, and the need to handle the single-line
dd/dt vs. the multi-line variant.
Change-Id: I65939c05e2c5dde0789bf8aefd7651161a2f137c
to update save dialog checkbox with correct watched state.
Call mw.page.watch.updateWatchLink onSave to refresh icon
with watched state.
Patchset 2- updated event name
Change-Id: I23ef1aad9c8ace13df1b9a6bf0bfeddb9d8bcb37
* Don't escape html-syntax pre content for now; Should parse this with a new
pre content production later (which needs to be split out of the regular pre
production in the tokenizer)
* Protect indent-pre content from start-of-line syntax escaping
* Preserve extra leading spaces in the tokenizer
* Two more (now 284) round-trip tests are passing
Change-Id: I199b89c0ee7fae12546df10c1b5117c97caccac5
Queued newlines and new trailing newlines were not cleanly separated so far,
which caused some trailing newlines to be consumed for needed leading
newlines. This change fixes several newline bugs, taking the number of passing
round-trip tests from 276 back up to 282.
Change-Id: Idb4706e15ce71e63085033e3f3f29557915c11a8
Fixed a bug in the list handler for multiple dds in a definition list. Also
fixed a few JSHint warnings.
Change-Id: I3e883786698a9521347fc2a5e6420646318813a7
Because of the end>=left condition, the loop was exiting right before
hitting the startBefore case, so use end>=left-1 instead.
Fixing this exposed another bug that caused nodeRange and nodeOuterRange
to be off by one: we need to increment left after storing it in
startOffset, not before
Change-Id: I54e18fb2119c8caefb4f7a7f2be43c6129afc4c0
Known issue: breaks round-tripping of :;;;::. That test is normally disabled
anyway, so we can fix it later.
Change-Id: I7954271311bfb7e71caae59d8177e3f04a9ebbca
This handles internal and external links separately, only setting
'title' for the former and only 'href' for the latter.
On the way out, detect external links using a simple regex (needs
improvement). Also don't write sHref any more per Gabriel's request.
Change-Id: Ibc2436d0a12de1d027e116f181db640cbcf3d522
* Started to add more complete tag source range (tsr) annotations to most
start / empty tags. These replace the old sourcePos and sourceTagPos
annotations, and look more promising for general round-tripping than block
source ranges (bsr). See
http://www.mediawiki.org/wiki/User:GWicke/Parsoid_source_ranges for some
notes on this.
* Added an escapeWikitext method in the serializer that tokenizes supposedly
text-only content from the DOM with the tokenizer and wraps runs of returned
non-text tokens into nowiki tags. The source corresponding to non-text
tokens is retrieved using the tsr annotations.
* Removed old (unused) table productions to avoid confusion.
* 276 round-trip tests are passing, vs. 283 without escaping.
Known issues:
* harmless for now, can be improved later: urllinks in external link captions
are wrapped in nowiki. Example HTML:
<a rel='mw:extLink' href="http://example.com">http://example2.com</a>
* some start-of-line syntax in wiki-syntax preformatted blocks might be
wrapped into nowiki when that would not really be needed. Example HTML DOM:
<pre>
* foo
* bar
</pre>
Change-Id: I01c34aedd5c566614d36924add47a6a960e91987
Now that the message keys are showing in the demo we can see how bad things look if the drop-down label is too big and wraps
Change-Id: I776a7e480a3f6240c0929f2f50a865c2de4ec0a5
Various parts of DM and CE choke on completely empty documents, so
return an empty paragraph instead.
Change-Id: I67062b66a44efe53a1bdaf60907653f0cc55dd25
Made edit tabs rendered on non view pages work correctly by routing them to the view page with an extra param that auto-initializes the editor
Change-Id: I4fd9106c8b45c6fc79af9ccb44e18944e9b9d8b9
* Added a newlineTransparent flag to handlers that prevents changes to the
onNewline status, so that content following it is still considered to be in
start-of-line context. This fixes a few rt tests where a comment or nowiki
tag is at the start of the line, and following content should end up on the
same line.
* 283 rt parser tests are now passing.
Change-Id: Ie58dcb9e5e9af9000fff61c2e1db5d8649ffc3f6
This solves lots of issues in the integration work, and also makes it much easier to extend this class to integrate it into other skins
Change-Id: I3b3c5b22a5664e6cf37e429cc0ac3be2e75b630f
* tokens are not modified any more (they are supposed to be immutable)
* handler info is now split in start / end objects and potentially a 'make'
method; added more flags to govern the newline behavior of different tags
* added a generic singleLine mode for single-line syntactical environments
* switched the web service to line-based diffs to avoid issues when diffing
the round-trip results of [[:en:Programming language]]
* 280 round-trip tests are passing now
Change-Id: I74b8ffbf69643c5d6e5ec852ec58e680c9018901
HTML5 defines space characters as [ \r\n\t\f] in
http://www.whatwg.org/specs/web-apps/current-work/multipage/common-microsyntaxes.html#space-character.
It treats these specially in a few contexts. As an example, the foster
parenting algorithm does not apply to space characters.
As a result, this change fixes the round-tripping of spaces between table
tags, which were previously moved before the table.
Change-Id: I32ab29275a9f824fc66d8286638eb42748cfc9a5
from Parsoid HTML output as well as VE HTML output. There are still
some newline related failures from parser tests that needs fixing, but
this is getting close. So committing for now so other eyes can make the
bugs shallow :).
Change-Id: Ia6a218ee9fb3e18fe0573c89ff3a4236779e1e64
- Check if href for links has the wgScriptPath prefix before
attempting to strip it from the href.
Change-Id: I844151ef7317476668d1306b96a2aec5a56fd0f1
- Something like this:
<ul><li>1</li><li>2<ul><li>2.1</li><li>2.2<ul><li>2.2.1</li><li>2.2.2</li></ul></li><li>2.3</li></ul></li><li>3</li></ul>
now serializes properly to:
*1
*2
**2.1
**2.2
***2.2.1
***2.2.2
**2.3
*3
So does this form which is what the above wikitext parses to:
<ul><li>1
</li><li>2
<ul><li>2.1
</li><li>2.2
<ul><li>2.2.1
</li><li>2.2.2
</li></ul></li><li>2.3
</li></ul></li><li>3
</li></ul>
- Lists (and nested lists) are not entirely newline-insensitive.
They still depend on newlines *between* lists. The opening
<ul> tag for non-nested lists should always start on a new line.
So, for example,
<ul><li>foo</li></ul><ul><li>bar</li></ul>
will serialize to:
*foo
*bar
which is incorrect. But,
<ul><li>foo</li></ul>
<ul><li>bar</li></ul>
will correctly serialize to:
*foo
*bar
Change-Id: I13a0290368574865957bcf57aebab488fbbb7026
- More pieces are now simplified and all(?) newline handling
is now centralized in the serializeToken function.
- This commit fixes bugs in rt-ing some code snippets
----------
Ex 1: foo<p>bar</p>baz
----------
- This commit fixes bugs serializing VE generated html
----------
Ex 2: <p>foo</p><pre>bar</pre> ==> foo\n bar
----------
- But, this round of fixes introduces RT failures for certain
code examples in parserTests.txt. In all these failing cases,
inline text/html is embedded within a generated <p> tag during
parsing. If these generated <p> tags can have a "gc:1" attribute
added to them, we can properly serialize them to the original
form.
----------
Ex 3: foo<pre>bar</pre>
Parsed HTML: <p>foo</p><pre>bar</pre>
----------
Note how this parsed HTML is identical to what the VE outputs
in Example 2 above. So, without the gc:1 attribute, we now
have conflicting requirements on the example same HTML.
This increases confidence in the correctness of my commit here.
Change-Id: I86beadec91c445a7f8a6d36a639b406697daa0a2
- Eliminated newline handling from several places in code and
mostly isolated it to serializeToken thus simplifying newline
handling logic.
- Fixing some bugs in the process: # of green roundtrip tests
went up by 5 (294 --> 299) but actually introduced failures on
a few originally succeeding tests (additional leading/trailing
newlines on the entire test output).
- Added bonus: made list serializing (mostly) insensitive to
newlines between tags. So, all the following DOM serialize
identically to the following wikitext:
*foo
*bar
----------
<ul><li>foo</li><li>bar</li></ul>
----------
<ul>
<li>foo</li>
<li>bar</li>
</ul>
----------
<ul>
<li>
foo
</li>
<li>
bar</li>
</ul>
----------
Change-Id: I76be56c4b2789039dff5f47de4659746882e45d6
HTML5 defines space characters as [ \r\n\t\f] in
http://www.whatwg.org/specs/web-apps/current-work/multipage/common-microsyntaxes.html#space-character.
It treats these specially in a few contexts. As an example, the foster
parenting algorithm does not apply to space characters.
As a result, this change fixes the round-tripping of spaces between table
tags, which were previously moved before the table.
Change-Id: I32ab29275a9f824fc66d8286638eb42748cfc9a5
from Parsoid HTML output as well as VE HTML output. There are still
some newline related failures from parser tests that needs fixing, but
this is getting close. So committing for now so other eyes can make the
bugs shallow :).
Change-Id: Ia6a218ee9fb3e18fe0573c89ff3a4236779e1e64
* Parsoid outputs bare newlines after a heading unless it's followed by
a <p>, so strip leading and trailing newlines in all bare text
* Adding a leading newline in <p>s is only needed if preceded by a
heading, don't add it otherwise
* Headings need a bare newline after them unless followed by a <p>
* Headings also need a bare newline before them if preceded by a <pre>
Change-Id: Ib02f800b26453541604e920fbb3845c51cdc6dea
This strips certain newlines added by Parsoid so they don't end up in
the linear model, and puts them back in on the way out so Parsoid
doesn't freak out and produce invalid wikitext
Change-Id: I256aaded4229c915868dc868ec6eaa1a73e00be1
I know this code is still being worked on but I felt like I should put
this in anyway, it might save the person working on it some work
Change-Id: I1535399b3798cd8de2fc5334cd1eac64b71e8821
This is needed because there are onTransact event handlers that use the
selection and expect it to be up-to-date. The previous behavior caused a
bug when pressing backspace at the end of the document, because the old
selection (at the end) was invalid in the context of the updated
document.
Change-Id: I159e37894d14d437f46495604c14804c0a13e84e
Empty annotation objects are unexpected by the rest of the data model
and cause weird breakage in the converter, resulting in inserted text
being in its own paragraph
Change-Id: I63de37c3c5e19ac650e7c7f2d1a0bfab21d45da9
I forgot to adjust a range based on this.cursor for this.adjustment .
This indirectly caused Rob to get an exception when trying to wrap
the last node in the document, because the unadjusted range was past the
end of the document.
Change-Id: If9d5b76568fae25ba2c0f405f1c4fcdd8d879e4f
- Check if href for links has the wgScriptPath prefix before
attempting to strip it from the href.
Change-Id: I844151ef7317476668d1306b96a2aec5a56fd0f1
This fixed a bug where an <h2> element with HTML attributes would be
converted to a 'heading' element with those HTML attributes but without
the 'level' attribute, which indirectly caused an exception somewhere in
ve.ce
Change-Id: I8bf32ff0d8e0f9d016b2abc6cb31824df05bdfc2
- Something like this:
<ul><li>1</li><li>2<ul><li>2.1</li><li>2.2<ul><li>2.2.1</li><li>2.2.2</li></ul></li><li>2.3</li></ul></li><li>3</li></ul>
now serializes properly to:
*1
*2
**2.1
**2.2
***2.2.1
***2.2.2
**2.3
*3
So does this form which is what the above wikitext parses to:
<ul><li>1
</li><li>2
<ul><li>2.1
</li><li>2.2
<ul><li>2.2.1
</li><li>2.2.2
</li></ul></li><li>2.3
</li></ul></li><li>3
</li></ul>
- Lists (and nested lists) are not entirely newline-insensitive.
They still depend on newlines *between* lists. The opening
<ul> tag for non-nested lists should always start on a new line.
So, for example,
<ul><li>foo</li></ul><ul><li>bar</li></ul>
will serialize to:
*foo
*bar
which is incorrect. But,
<ul><li>foo</li></ul>
<ul><li>bar</li></ul>
will correctly serialize to:
*foo
*bar
Change-Id: I13a0290368574865957bcf57aebab488fbbb7026
- More pieces are now simplified and all(?) newline handling
is now centralized in the serializeToken function.
- This commit fixes bugs in rt-ing some code snippets
----------
Ex 1: foo<p>bar</p>baz
----------
- This commit fixes bugs serializing VE generated html
----------
Ex 2: <p>foo</p><pre>bar</pre> ==> foo\n bar
----------
- But, this round of fixes introduces RT failures for certain
code examples in parserTests.txt. In all these failing cases,
inline text/html is embedded within a generated <p> tag during
parsing. If these generated <p> tags can have a "gc:1" attribute
added to them, we can properly serialize them to the original
form.
----------
Ex 3: foo<pre>bar</pre>
Parsed HTML: <p>foo</p><pre>bar</pre>
----------
Note how this parsed HTML is identical to what the VE outputs
in Example 2 above. So, without the gc:1 attribute, we now
have conflicting requirements on the example same HTML.
This increases confidence in the correctness of my commit here.
Change-Id: I86beadec91c445a7f8a6d36a639b406697daa0a2
Previously, data-mw-gc (generated content) elements were unconditionally
converted to alienInline nodes, and unrecognized elements were
unconditionally converted to alienBlock nodes. This is wrong and
produced weird results when I started experimenting with <code> tags.
Instead, I made both gc and unknown element trigger alienation, but the
decision of whether we generate an alienInline or an alienBlock node is
separate and is based only on whether we're inside a content node.
Change-Id: I12335337c3fa60c725ae7bcfbfb52a1dda153fb5
- Eliminated newline handling from several places in code and
mostly isolated it to serializeToken thus simplifying newline
handling logic.
- Fixing some bugs in the process: # of green roundtrip tests
went up by 5 (294 --> 299) but actually introduced failures on
a few originally succeeding tests (additional leading/trailing
newlines on the entire test output).
- Added bonus: made list serializing (mostly) insensitive to
newlines between tags. So, all the following DOM serialize
identically to the following wikitext:
*foo
*bar
----------
<ul><li>foo</li><li>bar</li></ul>
----------
<ul>
<li>foo</li>
<li>bar</li>
</ul>
----------
<ul>
<li>
foo
</li>
<li>
bar</li>
</ul>
----------
Change-Id: I76be56c4b2789039dff5f47de4659746882e45d6
Parsoid ignores sHref when converting back to wikitext, so we have to
set the href attribute to "/$title"
Change-Id: I1068116c0be72197619d0df3b4d1231a3879fa14
And made it not start on it's own, but be started by ve.Surface - this makes it so it's not polling in the unit tests, for instance
Change-Id: I940df04d392fd134d18847949efe0e2232328323
* As part of an earlier fix, I had changed default value of 'res'
to null instead of ''. But, this was potentially buggy because
the previous check was (res !== '') which could be triggered
by return values of handlers. By changing the check to null,
I was effectively changing the code paths for those handlers that
returned ''.
Change-Id: I2302023be7422ce4fb384ff5a50fe53fa7732855
Fix Inspector bug which prevented applying a link annotation to data
already containing annotations.
Change-Id: I6f315d50805c8c71f2155f955ea5674a7ce98656
This was causing some issues when you started typing there, and it's not clear that the white text is really the best way to go anyways
Change-Id: I8a9d6571ea204603729e96b7ff77184279a31a95
The offset map was broken from the start because it wasn't updated when
adjusting the length of a text node, and if we fix that bug it's
doubtful whether the costs of updating the offset map outweigh the
benefits, especially considering that adjusting the length of a text
node is something we do for almost every keypress. If it turns out
having an offset map does make sense, we can always reintroduce it
later.
Change-Id: I59e8bc154f7d07aa1bab2f473c13ff466d0e463f
paragraphs in lists.
* We need to look at other special-case handling requirements of
html tags in lists (and other contexts like tables).
Change-Id: I84b8402d90a186c9075c2d45263c94377312927a
Parsoid outputs rel="mw:wikiLink" or rel="mw:extLink", so we convert that to link/wikiLink and wiki/extLink respectively.
Also preserve the data-mw attribute; we probably need to do this more generally but this'll do for now.
Change-Id: I32e570bffa5a73a733a120d52cfd8b75d3191e02
Adding a 'style' attribute which is set to either 'data' or 'header'
This breaks even more tests because of missing style attributes
Change-Id: I0a75d8c1578b4414eeae8c484f6c4d6f8a59472a
When a text node was closed (either by encountering a non-text node or by reaching the end of the document), the fixupStack would not be processed. This led to attributes being dropped when nodes were split because of text being inserted into them
Change-Id: I41f6d20e0c1bfc8d8689b7e6325e724dd8156ab1
* Add code to handle elements and annotations
* Drop support for aliens from getDomElementFromDataElement() and move it into getDomFromData()
* Implement getDomElementFromDataAnnotation()
* Document a few functions
Change-Id: Ic6a418cbf9d7d1ad96299d7d3633970a876c6103
it does not include Alien nodes and Images - because we are not going to
support them for June release).
Change-Id: I229e4b5f2881714252699f23aef164655fa8bcf6
* Moved wikipedia default prefixes to environment
* Added 'addInterwiki' method
* Adjusted link handling normalizeTitle to reflect this
Change-Id: If5b2314cc36346b6da8649ed410457a612d80a22
* mw:Foo now loads pages from mediawiki.org
* The default prefix still is 'en'. You can switch this to 'mw' in ParserService.js.
Change-Id: I1208667e6114bd711b7988a8b3adb32ffab70969
- Three bugs that were messing up quote transformations.
- Now, the following cases are handled properly:
* ''foo'''
* '''foo''
* ''foo''''
* ''''foo''
These tests (and other quote tests) have to be added to core parser
tests file.
- One more parser test green.
Change-Id: I4f93e8910639f546bfc9304becab17d26d5529de
Created new document method to determine if a specific annotation
object is inside an annotation array.
Change-Id: Id645929cbf31030b8b0fcacb8dfb36e61aaad129
This fixes a bug where the second replace operation in a transaction
would cause the rebuild of the wrong range, or the adjustment of the
wrong text node.
Change-Id: I9b1c68d84999d538fe10bb193f4dfdd694121d2a
This is needed to make the results of certain transactions' tree sync
round-trip cleanly through the ve.dm.Document constructor
Change-Id: I2ab0758ec6bd7afba5b6645c7330f9fa2d45205d
new static method looks for annotation in annotation object.
ve2 Cleanup on annotate method and surface model.
Partially revive UI tools by exchaning old method usage
for ve2 methods.
Change-Id: Id0ac58330292d76801bbcf1d71a919b493f8ab9e
An improvement, but there still are some extra newlines inserted after
paragraphs. Example input:
-------
Foo:
{|
|foo
|}
-------
Extra newlines are inserted after the Foo: and the foo in the table. They are
not fed as tokens or text to the tree builder, so there is likely a bug in the
html5 library or JSDom.
Change-Id: I83eb6180e3cd1c4e7f9b15b31d339e1d32bccd3f
* Possibly more efficient under heavy GC load -- untested.
* No change in time and memory use for single file parsing.
Change-Id: Id2f3f65cc0e5f38ed968bbda60b97e46523e700e
* Moved the tail attribute to the second attribute (a bit cleaner)
* Disallowed newlines in the tail production
* Improved the selection of round-tripped href vs. generated content vs. href
in the serializer
* renamed state.linkTail to state.dropTail
Change-Id: I5d98c704b6ea566011e22237786f8da17548570f
Pages titles with a wikipedia interwiki prefix now load the page from
corresponding Wikipedia. Links in a page then stay within the given language.
Note that Parsoid currently makes no effort to recognize localized namespaces,
so it won't render media files, categories etc correctly.
Change-Id: I7bc4102e81a402772ea23231170734d580ea15b9
Functional changes (fixes):
* Make writeElement() also update parentNode and parentType for openings
* Also add to fixupStack when opening a wrapper for a text node
Non-functional changes (cleanup&docs):
* Document all variables at the beginning of the function
* Group variables according to where/how they're used
* Move expectedType into writeElement()
* Kill node, duplicates parentNode unnecessarily
* Kill paragraphOpened, was misnamed and unnecessary
* Rename closedElements to reopenElements
Change-Id: Ie5b4e4f30b267943048fdc170accb29139039192
* Push entire elements onto openingStack rather than type strings
* When closing an element, build a clone of the opening and push it onto
closedElements, then insert that clone when reopening the element
Change-Id: I8b0fb44394aed6c471dc6dacaab03e44c2333733
* Don't explicitly add the newline in the pre, as we preserve newline tokens
now. This avoids doubling of newlines when round-tripping.
* Use the sHref attribute even if the href contains spaces.
Change-Id: I8bec8fbfd6a7836bf2e5eec20869a0edd95c93b6
Lists interrupted by non-empty lines would not close the list properly.
Register for any token instead of just for newlines and close the list if no
listItem follows the newline.
Change-Id: I1743901e3db541bbeda78d17707db943e6ceb9b9
If the href would not denormalize, add a copy of the original href in data-mw
and use it to preserve non-conventional capitalization etc.
Change-Id: Ifef50eec7343b0e6b0ba66b6d19a8a3e8c9f8001
A tail containing regexp syntax (a ? in [[:en:Main Page]]) would crash the
serializer. Use substr instead.
Change-Id: I8519aec9c07dfe31893d676b1c936a42d2af74a0
- Added a tail json attribute for wikiLinks
- During serialization, this attribute is used to strip the tail from
the link target and render it after the link
[[hen]]s ==> <a ... data-mw="{gc:1, tail: 's'}" ...>hens</a>
==> [[hen]]s
- 2 more roundtrip tests green
Change-Id: I84f3dabaf0271f7a67641a00148467daa8310eb0
This allows us to check the watchlist checkbox on save dialog.
Added watchlist toggling to ve save api.
Added some i18n messages to core integration.
Change-Id: Ibed8edb2c59ad49e1738c937c3bea518238d0845
* The state of syntax stops is now properly included in the cache key for the
tokenizer-internal backtracking cache. This fixes some mis-parses when
re-parsing a bit of text with different flags.
* Clear the backtracking cache after each toplevelblock. This drops the peak
memory usage when expanding [[:en:Barack Obama]] from ~380M to ~110M.
Change-Id: Icdb879cae5907e4595903dd6acba2e686e8c2e4b
* Added converters to all relevant node implementations
* Added new annotation objects with their own factory
Change-Id: I9870d6d5eac45083929d74d2e58917d0939ca917
Also:
* Refactored tests
* Added tests for ve.dm.Transaction.newFromInsertion
* Added tests for ve.dm.Transaction.newFromRemoval
* Fixed problems with ve.dm.Transaction.newFromInsertion
* Added ve.dm.Node.canBeMergedWith which is partially a port of ve.Node.getCommonAncestorPaths merged with canMerge from within ve.dm.DocumentNode.prepareRemoval from the old ve codebase
Change-Id: Ibbc3887d08286d8ab33fd6296487802d65b319fa
* This routine attempts to rewrite the DOM to maximize tag overlap
and thus minimize tag uses.
* This takes as input a set of tags which participate in the
minimization.
* Tested on the following example
<b><i><u><s>BIUS</s></u></i></b><b><i><s>BIS</s></i></b><b><u><s>BUS</s></u></b><u><i>UI</i></u>
with multiple combinations of the 2^4 possible variations of i,b,u,s
tags: [], ['i','b','u','s'], ['i'], ['b','s'], ['i','b','u']
- But, I am not fully sure if this implements the right behavior when
only a subset of inline tags are provided. Needs discussion and tweaking
as necessary.
* Also tested on few others:
<b>B</b><b><i>BI</i></b><b><i><u>BIU</u></i></b><b><i><u><s>BIUS</s></u></i></b>
<s><i><b>SIB</s></i></b><s><i><u>SIU</u></i></s><i><u>IU</u></i><i>I</i>
* The previous pairwise tag rewriting version fails on several of these
examples, so this new version is a definite improvement.
* No change in parserTests run (203 passing before and after).
* Possible improvements that could/should be undertaken:
- get rid of useless/idempotent add/remove of nodes that don't change
the DOM.
- ensure that node attributes post-restructuring are correct.
Change-Id: Ib4a8b39583fa96a2be880a77021ca81cefa06484
Copy-pasting things like "text<IMAGE>moretext" failed spectacularly,
this commit fixes that.
* Check for content rather than structure in the inserted/removed data
* In the content case
** Run selectNodes() over the removal range, rather than just the cursor
*** i.e. no longer assume that content replacements only affect one node
** If there is structure involved, rebuild all affected nodes
Change-Id: I80e40b5b7c514a3fb105d57e4a17770d0fefaaea
Some of the replacement code was assuming that "does not contain
elements" and "is content" were the same. They're not any more, because
we have content nodes (like image) now, so I need a separate function
to distinguish between these cases.
Change-Id: I206ccdf082b7baddf99d382eb3cdd77ea34fb479
If the last element of the input data array was text, the resulting text
node would have length=0 rather than the expected length value.
Change-Id: I3d089a80b8a447a12ba411b2e11c1b84f14f2959
To allow non sysops to save via VE, refactored ve save api
to use doEdit which bypasses namespace protection.
Add edit link in view nav for non sysop so that they may edit
Add View source link in dropdown for non sysops
Add Edit source link in dropdown for sysops
Cleaned up some of the integration core code
UI tweaks
Change-Id: Ib4249bc5fb7ffa6410e4f2d278aafbb871800981
WARNING: This is not as fast as the implementation of getNodeFromOffset in dm
Change-Id: I5fbe9b6edc66169b9caaa6751fde1b7b752814d1
NOTE: ve.ce.getNodeFromOffset and ve.dm.getNodeFromOffset should be renamed to getBranchNodeFromOffset to clarify that they only return branch nodes.
This patch fixes a tokenizer syntax error encountered on
[[:en:Template:JacksonvilleWikiProject-Member]] and [[:en:Template:Infobox
former country]] by allowing optional whitespace before start-of-line template
syntax.
Change-Id: Ic214a731de58bf766e51f23d5e24ea2ce6788f58
254 round-trip tests (up from 184) are now passing.
Also:
* tweaked runtests.sh slightly (use less -R instead of -r).
* made sure the EOFTk is preserved in phase 3 transforms
Change-Id: I1de22186bdb78e52019370e43f096877005b8f5a
- This is implemented as a post-processing pass.
- Might require additional checks to verify rewriteability.
- Implemented as a pair-wise tag DOM minimization strategy,
i.e. it takes tag pairs (B, I) for ex, and attempts to
normalize the tree just for those tag pairs. Normalizing
across multiple tags is implemented as pairwise rewriting
across all pairs: Ex:(b,i), (b,u),(i,u) for (b,i,u)
- Copied over attributes as part of rewriting, but some of the
attributes lose their meaning on rewriting since tags are
reordered (ex: sourcePosn, sourceTagPosn). How do we handle this?
Output examples and possible issues to fix:
<i><b><u>biu</u></b></i><b><u>bu</u></b><u>u</u>
gets rewritten to:
<u><b><i>biu</i>bu</b>u</u>
But, the equivalent wikitext form:
'''''<u>biu</u>''''''''<u>bu</u>'''<u>u</u>
does not get rewritten because of parsing differences.
This wikitext gets parsed into:
<i><b><u>biu</u>'''</b></i><u>bu<b>u</b></u>
The extra ''' token in the middle thwarts DOM rewriting.
However, a slightly different version:
"'''''<u>biu</u>''<u>bu</u>'''<u>u</u>"
gets properly normalized to:
<u>'''''biu''bu'''u</u>
An alternative, but fun strategy to play with is to use the following
two normalization primitives: S(wap) and M(erge).
- S rewrites T1(T2(x)) into T2(T1(x))
(ex: <b><i>foo</i></b> ==> <i><b>foo</b></i>)
- M rewrites (T(x),T(y)) into (T(x,y)).
(ex: <b>foo</b><b>bar</b> ==> <b>foobar</b>)
The current rewriting strategy could possibly be re-implemented as S-M
rewriting. The problem to solve there would be to find an efficient
rewriting strategy that is guaranteed to lead to a normal form. I may
not play with it now, but just documenting it for later (to play with
in my spare time).
This commit is just as a record of fun/experimental code where I get to
learn details of JS, wikitext, parsing, and DOM manipulation. Next
version of this code will attempt to introduce minimal DOM restructuring
across multiple tags at once which can be more efficient.
gwicke: Removed now passing test from whitelist, and updated another whitelist
entry which is now improved.
Change-Id: Ie97bcb164eb62c34ba61aa76ba2f4c232aa713d8
Save action
1) posts html to parsoid service in exchange for wikitext
2) saves wikitext
3) returns parsed content.
On save, VE is hidden and page content is replaced.
Demoing save in toolbar, followup commit will redesign save options
Change-Id: Ibfbe52de08e3483e1a33f0740c03f96ec2b7f90a
This was caused by the fact that a non-structural leaf can not have children, which makes it appear incompatible as a sibling to an arbitrary structural element (like a paragraph) but since it can not contain content we can check that instead.
Change-Id: Ie3c58b4b43f2aa6921f8f82aa82511e231207854
* Added a generic stx_v 'syntax variant' round-trip attribute
* For pre, use stx:'html' vs. no syntax annotation. This might not be 100%
safe for arbitrary html input, so we might want to flip this to stx:'wiki'
later.
* 181 round-trip tests passing
Change-Id: If6080917a3a7c069066db3db60efe59b1f6c28d8
* Very basic support attribute key-value pairs emitted from templates
* Add TALKPAGENAME stub implementation
* Only show 'no revisions' message for top-level pages
Change-Id: I4b4ac0c7b2c0531ac4b39f0f49f4217302576ab9
Using this argument will only return true if the offset is a place you can add any element to (hence the unrestricted part of it). This is good for testing if a paragraph could potentially be inserted there.
Change-Id: I6cc91da437c52493de03eb687b28966198270fea
This code still needs a lot of work, but it seems to work for most
cases. Things that still need to be done:
* Documentation and comments
* Handling of content and text nodes
** Use Trevor's isContent/canContainContent code which I don't have yet
* Preserve attributes when reopening closed elements
* Tests :)
Change-Id: I3bc16c964ef158693490a61ce12beb21e6fe2a9d
The 'insert' and 'remove' operations weren't implemented in the
transaction processor and were a holdover from the old DM
implementation.
Also migrated the tests, especially those that asserted that consecutive
insert/remove operations were combined (this is no longer the case now
that they are replace operations)
Change-Id: I2379fe92b331c5316f70f4b695397da41581cce9
Removed hard-coding of alien nodes, now aliens are automatically used for anything unknown, and block or inline aliens are selected based on whether the parent element can contain content or not.
Change-Id: I5d2a521ead4f4c96cb44d084a5c160cc20d8048e
* After installing Parsoid (sudo npm install -g in modules/parser), run 'node
server.js' from the api directory and navigate to http://localhost:8000/ and
follow the directions. You can start to navigate the English wikipedia at
http://localhost:8000/Main_Page, or manually enter wikitext or HTML DOM to
convert.
* Uses the express framework, could also use just connect
* Uses the cluster module to manage workers per-core and restart those on
failure
Change-Id: I443f2996ed3df00826b038b7476a2f966ab0c425
* Changed RDFa for links according to
http://www.mediawiki.org/wiki/Parsoid/RDFa_vocabulary
* Added basic support for internal/external link serialization
* Moved numbering of external links from tokenizer to LinkHandler
* Added round-tripping for generic HTML tags
* Replaced nowiki tag with <meta typeOf="mw:tag" content="nowiki"> and <meta
typeOf="mw:tag" content="/nowiki"> for now.
* 154 round-trip tests passing (node parserTests.js --roundtrip).
Change-Id: I16c4db21b1b543ee57c73e569c83025b64664542
Create context instance in surface.
Move over getSelectionRect into ce.surface
Cleanup ve.surface contstructor class
Move linmod/html test objects to sandbox.js
Change-Id: I0cf602ef991100bf6128c68750b02a00566911dc
Switch sandbox demo to use new ui modules
Update VisualEditor.php to use ve2 modules
SpecialPageSandbox working
Change-Id: I8261d6bf6ceb6ae7b7bfa5f61aec6a0121906765
* added stx: 'html' round-trip information for html tags
* added t_stx: 'row' info for row-wise table wiki syntax, and support for it
in the serializer
* the first table row is implicit in wikitext
* renamed lastToken to prevToken in serializer
* strip first newline in an initial chunkCB
Change-Id: I014b046539d1b674d830551c5fd1b74a67f81993
in the new DM. Change method name getAnnotationRange from offset to
getAnnotatedRangeFromOffset. Write tests
Change-Id: I7028803065409e271ceced73e4803954d4a956dc
Lists are a bit tricky, as nested lists are not wrapped in a separate list
item. Should work now though.
Change-Id: I2e5f29f6afa6bdd2d5e5c0c5d019b70c611b73d1
Splits and merges now work, or at least the tests for it pass
The strategy I used is to gather the affected ranges for each of the
following:
* removed stuff
* the entirety of each node touched by a non-zero removal
* if the inserted data busts out of its parent, the entirety of that
parent node (the 'scope')
then get the covering range of all those ranges, and rebuild that.
Change-Id: I7c3b421abc0ba134157ac8b59042675bb1b5073c
getAnnotationRangeFromOffset and offsetContainsAnnotation
which deprecated getAnnotationBoundaries, and getIndexOfAnnotation
write unit tests for proof
Change-Id: I6c0d4e3ca96dd569b1909cd22fce68c3a6fe382c
This fix only affects following transforms, of which there are few right now.
Also removed a stray token mutation in QuoteTransformer.
Change-Id: Id6d4adce944b06fc1a3651cfbf63fc2670125225
* Tokens are now immutable. The progress of transformations is tracked on
chunks instead of tokens. Tokenizer output is cached and can be directly
returned without a need for cloning. Transforms are required to clone or
newly create tokens they are modifying.
* Expansions per chunk are now shared between equivalent frames via a cache
stored on the chunk itself. Equivalence of frames is not yet ideal though,
as right now a hash tree of *unexpanded* arguments is used. This should be
switched to a hash of the fully expanded local parameters instead.
* There is now a vastly improved maybeSyncReturn wrapper for async transforms
that either forwards processing to the iterative transformTokens if the
current transform is still ongoing, or manages a recursive transformation if
needed.
* Parameters for parser functions are now wrapped in abstract Params and
ParserValue objects, which support some handy on-demand *value* expansions.
Keys are always expanded. Parser functions are converted to use these
interfaces, and now properly expand their values in the correct frame.
Making this expansion lazier is certainly possible, but would complicate
transformTokens and other token-handling machinery. Need to investigate if
it would really be worth it. Dead branch elimination is certainly a bigger
win overall.
* Complex recursive asynchronous expansions should now be closer to correct
for both the iterative (transformTokens) and recursive (maybeSyncReturn
after transformTokens has returned) code paths.
* Performance degraded slightly. There are no micro-optimizations done yet
and the shared expansion cache still has a low hit rate. The progress
tracking on chunks is not yet perfect, so there are likely a lot of unneeded
re-expansions that can be easily eliminated. There is also more debug
tracing right now. Obama currently expands in 54 seconds on my laptop.
Change-Id: I4a603f3d3c70ca657ebda9fbb8570269f943d6b6
This means inserting things like </p><p> are now synced correctly and
split the paragraph in the model tree. Merges (removing e.g. </p><p>)
aren't supported yet.
Also, this needs tests, Trevor tells me he's working on porting replace
tests from the old ve/ directory
Change-Id: Ic5050849d7d007a1696dc36548654979aedb53a8
The tree sync for content replacements was adjusting the parent of the
text node affected, rather than the text node itself. This was because
it called getNodeFromOffset(), which returns branch nodes. Switched it
to use selectNodes() in leaves mode
Change-Id: I50a9be18151a1b75815ab19b787b16b6be385bf9
This was because the while loop was never entered as end >= left was
true from the start. Convert the while loop to a do-while loop to make
sure it runs at least once
Change-Id: I9c6436a7b296e65a36b8301095b6edd00507d321
Now returning an empty array when a non annotated character is found
in the range. No longer looping through each annotation, simply
comparing to previous characters annotations and trimming differences.
Write additional test.
Change-Id: I41d2422a931a74325693edca409aed6d5da20ba8
This makes TransactionProcessor work for regular replacements, as well
as insertions and deletions of self-contained pieces of data. This does
NOT yet work for inserting and deleting unbalanced data
(splitting/merging nodes).
I've tested this from the console for insertions and deletions and
simple replacements, but I haven't tested wrappings. We should write a
bunch of unit tests for this some time :)
Change-Id: Ic2fd75d1cf2e127bc9ae58debce67576be2c912f
This is for the case where we have a zero-length range in between two
siblings, and we need to know what index that corresponds to in order to
be able to insert nodes there (rebuildNodes() will use it for this
purpose)
Change-Id: I357d1cd665667a76f955a10b8d9d2810976cdbd7
* Initialize startOffset to 0 not 1, don't know what I was thinking
* Use currentFrame for nodeRange instead of parentFrame, don't know what
I was thinking there either
* If the returned node has no parent (is the document node), don't
attempt to access parentFrame and don't set index
Change-Id: Iad969a7c29436cdf4151ead7e9d3d8e2a30befb3
Selecting a zero-length range at the start or end of a text node
(e.g. (1,1)) would return the text node's parent instead of the text
node
Change-Id: I7fe089bf66b93185dd3415eff53aa7e04e3ffdb2
* Needs to be initialized to 1, not 0
* Needs to be stored *after* left is incremented to account for an
opening
Change-Id: I7978ae241578a8a17120e494684e6e93626a8529
* Text nodes do not have a wrapper to set classes on
* Use CSS class names that are equivalent to JS class names, swapping . with -
Change-Id: I49c877dd5c9b5dd2a9afad3137f12b14883043a1
Trying something trivial like echo 'Hello world' | node parse.js
would throw TypeError: Function.prototype.apply: Arguments list has wrong type
Change-Id: Ia0a1154b0f3edbfb1f228a1d2072fced1b147141
* Setting the rank on tokens is still used currently, but will be phased out
in favor of setting it on chunks. Tokens will be immutable to allow sharing
and caching without a need for cloning.
* Only register for newline and end tokens in QuoteTransformer when active.
Change-Id: I2c45bc7e4a105219a1404ab221eed7f242128f1e
I was using data.length to check if the range was out of bounds, but
this is a problem when using selectNodes() inside of tree sync code
(which happens when performing rebuilds). While tree sync is in
progress, the model tree and the linear model don't match, so we
shouldn't be looking at the linear model for information about the model
tree. Instead, get the length of the DocumentNode and use that.
`
Change-Id: I11a378544ce1281a89cdcd4363c5cb1bf56f3434
This makes it possible to transclude list items from a template.
Note: "5 quotes" test is broken by this patch, it appears that ListHandler
newline processing is changing some state which mysteriously affects the
QuoteTransformer. This is ominous, hopefully there's a simple explanation...
gwicke: fix a bug in tokenizer triggered by definition lists like this:
**; foo : bar
Change-Id: I4e3a86596fe9bffcbfc4bf22895362c3bf742bad
Currently only implements mode=='leaves', i.e. traverse all leaf nodes.
Seems to work from casual testing, but is missing unit tests. See also
other TODO comments in this commit
Change-Id: I41292c21c627a18af7985e8ef9e23c7b14252b21
This gets rid of the length == outerLength-2 hack in getDataFromNode()
and will make it easier to implement similar logic in selectNodes()
Change-Id: I1294350b67ca3eefde2b7fe9fea0bc6d8b90f772
* Moved implementation of getting and updating a DOM wrapper to ve.ce.BranchNode
* Updated ve.ce.BranchNode tests
* Renamed ve.NodeFactory.createNode to ve.NodeFactory.create
* Added ve.NodeFactory.lookup which gets the constructor of a type
* Added attribute pass-through to ve.dm.BranchNodeStub
Change-Id: I8f5b7d3d3ae616cc5f39828b24b655163d782ae5
The bubbling update events are not needed with ce or dm, but were once upon a time useful for es, this just eliminates some unused cruft that was costing extra function call overhead.
Change-Id: Ia16d0f4cd74c84cded5caecada33ee83d0882f30
* Makes it simpler in the linear model because we don't have to use style: "item" for regular list items and style: "definition" for definition lists
* Enforces correct nesting through existing node rules systems
* Updates tests accordingly
Change-Id: I64d80af938e325f1961226505bdc386bb35ccdda
* Also fixed calls to addListenerMethod
* Also routed adding children in the constructor of ve.dm.BranchNode to the splice method
* Renamed types of ve.dm stub nodes to avoid collisions (since we have to register ce nodes by the same names for them to be generated by onSplice)
Change-Id: Ia2e75cf0a62186cc0e214683feb25c619590318a
* Also renamed convertDomElement to replaceDomWrapper in ve.ce.BranchNode
* Also added extra documentation for node rules
Change-Id: Ia8ac6be34e2b021be96974ac1ba9119bd8077d60
By using this.$ as a selection of contiguous top-level nodes (text or otherwise) we can avoid wrapping each text node in a span
Change-Id: I141c2df8f13646db3fff0da93d218c2dcf154c8a
* Fixed constructor of ve.ce.BranchNode which was calling the wrong method to perform an onSplice and with the wrong arguments
* Removed/renamed events emitted from ve.ce.BranchNode.onSplice
* Reintroduced .$ to all ce nodes
* Ported over functionality for DOM node type variance used in headings, lists and list items
* Moved the old ve.ce.Content guts to ve.ce.TextNode
* Added getOffsetFromNode and getDataFromNode to ve.dm.DocumentFragment
* Added setDocument and getDocument to dm nodes
Change-Id: I185423ba2f1a858dde562cb2f5bc3852aec930db
* Add .process() and its helpers from ve1
* Fix applyAnnotations()
** Use data[i] rather than data[j]
** Don't add empty annotation objects for no reason
** Remove empty annotation objects
*** I thought this wasn't needed, but it is needed for clean rollbacks
* Remove unused second parameter in applyAnnotations() call
Change-Id: Ia338f62d2eaf2a76f8ef653eead05bc44757a122
* Also removed beforeSplice and afterSplice in favor of just plain splice which is the same as afterSplice used to be - beforeSplice was never used and it was making things more complex looking than needed
Change-Id: Icbbc57eac73a2a206ba35409ab57b3d1a49ab1a5
They are used by ve.dm.factory so this might make it easier for people to understand what's going on.
Change-Id: I490627e3bfc55ca9c96fdc4f5d047737b6a3db8c
* Added support for asking if a given node type can have children or grandchildren and what types of nodes can be it's parent or child
* Removed canHaveChildren methods from leaf and branch nodes and converted use of them to depend on factory to read static rules from constructor lookup by type
Change-Id: I9769f95647066576416bacb791c4b68dd0285b35
* Moved node tree assertion to ve.dm.example
* Added rebuildNodes test
* Fixed some typos in rebuildNodes
Change-Id: I4853ded4b062aaa3758435093368bc23667ca3bf
Image nodes are leafs, so providing an empty array to their children/length "contents" constructor argument ends up setting the numeric length to [], which casts to an empty string in arithmetic, causing all further calculations to be concatenations instead of additions.
Change-Id: I40e1ea2295f6095318bc4c24185cadfdfb684557
Within ve.dm.DocumentFragment it makes more sense to call the root node (which is always a document node) a document node, especially since there may be a different node used as a root.
This commit also adds test for getDocumentNode and getNodeFromOffset which uses the offset map.
Change-Id: Ic4609233cedc41f7e5a5f8fdb0e6178652c95554
And fixed ve.dm.DocumentFragment constructor to generate a correct offset map which creates references to branch nodes only
Change-Id: If9e515be0c63d272bfed9bf4da625a48edd36f48
* parentCB (if set) is called with { async: true } if expansion is going to be
asynchronous.
* Strings are handled efficiently
* all value parameter chunks can now be converted using .to().
Change-Id: Ib013e1bc3d8e7f692009038209db6a056887326e
Now setting up multiple toolbars per config
Tools & Modes are now configurable per toolbar per instance
Base elements are created on demand and no longer id specific
Note: There are some bugs with multiple instances.
Change-Id: Id0bbbca2d1b76fd2db3f3b0f9abd90194930b610
* [[:en:Barack Obama]] can now be expanded in 77 seconds using 330MB RAM,
while it would prevously run out of RAM after ~30 minutes. Wohoooo!
The token transform framework rework really paid off.
* 303 parser tests are passing in the new record time of 5.5 seconds. Two more
tests are passing since these tests expect the day of the week to be
Thursday. Won't be the case tomorrow.
Change-Id: I56e850838476b546df10c6a239c8c9e29a1a3136
* All parser pipelines including tokenizer and DOM stuff are now constructed
from a 'recipe' data structure in a ParserPipelineFactory.
* All sub-pipelines of these can now be cached
* Event registrations to a pipeline are directly forwarded to the last
pipeline member to save relatively expensive event forwarding.
* Some APIs for on-demand expansion / format conversion of parameters from
parser functions are added:
param.to('tokens/expanded', cb)
param.to('text/wiki', cb) (this does not work yet)
All parameters are additionally wrapped into a Param object that provides
method for positional parameter naming (.named() or conversion to a dict
(.dict()).
* The async token transform manager is now separated from a frame object, with
the frame holding arguments, an on-demand expansion method and loop checks.
* Only keys of template parameters are now expanded. Parser functions or
template arguments trigger an expansion on-demand. This (unsurprisingly)
makes a big performance difference with typical switch-heavy template
systems.
* Return values from async transforms are no longer used in favor of plain
callbacks. This saves the complication of having to maintain two code paths.
A trick in transformTokens still avoids the construction of unneeded
TokenAccumulators.
* The results of template expansions are no longer buffered.
* 301 parser tests are passing
Known issues:
* Cosmetic cleanup remains to do
* Some parser functions do not support async expansions yet, and need to be
modified.
Change-Id: I1a7690baffbe8141cadf67270904a1b2e1df879a
Renamed Selection method to more suitable name.
Misc cleanup
Patchset 2, whitespace cleanup
Patchset 3: Change values used with selection direction to -1 or 1
1 for left to right (normal)
-1 for right to left (opposite)
Change-Id: If9ecc721ace1c7550903170f92395947f1ccc22c
It's not being used at all, and it's broken because
this.lengthDifference is set to zero regardless of what length
difference the operations passed into the constructor might cause
Change-Id: I3b7a312a1920347e7bf34df88a05bf6f2ff11f7d
* Changed splice to check all elements about to be inserted are allowed before inserting any of them so that catching an exception leaves you in a sane state
* Fixed the order of execution of parent class constructors in ve.dm.LeafNode and ve.dm.TwigNode so that canHaveChildren and canHaveGrandchildren produce correct values and added tests to ensure these methods are correctly inherited in subclasses
* Added tests that check for exceptions when adding nodes that can have children to nodes that can not have grandchildren
* Added test that check for events being emitted before and after splicing, including that beforeSplice should be emitted even in cases where a splice fails and throws an exception because the nodes are incompatible (but afterSplice is not called in this case) since beforeSplice might modify the nodes in some way before the compatibility tests are run
Change-Id: Id12aea995a42c26ff63a74ae3d31f2bf455759e3
* Moved getParent and getRoot from ve.dm.Node back to ve.Node
* Fixed use of getElementLength that should have been changed to getOuterLength, but was changed to getLength (oops)
Change-Id: Ibe5b855aef533dcd493f762a8a02c6a11ce6e7de
In this commit several methods (child node add/remove and parent/root modification) were also moved to ve.dm.BranchNode ve.dm.Node respectively. ve.Node and ve.BranchNode are immutable. ve.dm.Node and ve.dm.BranchNode are mutable. Other subclasses of ve.Node and ve.BranchNode should implement functionality to mimic changes made to a data model.
Change-Id: Ia9ff78764f8f50f99fc8f9f9593657c0a0bf287e
Ground-up rewrite of the data model. Putting this in the ve2 directory for now so we still have the old code floating around.
Main changes so far in this rewrite:
* Renamed hasChildren() to canHaveChildren()
* Added canHaveGrandchildren()
* Added a new node type TwigNode that can have children but not grandchildren (so all of its children must be LeafNodes)
* Implemented push/pop/shift/unshift as wrappers around splice()
* Renamed getElementType() to getType(). Nodes now take a string as a type, and the element stuff is gone and won't be back
* Removed clearRoot(), replaced it with setRoot( this ) where needed
Change-Id: I23f3bb1b4a2473575e5446e87fdf17af107bacf6
This is a bit better than cloning tokens wholesale, but not by much. There is
a lot of potential for much better per-token caching with reduced token
cloning. Need to map out all dependencies besides token attributes expanded
from template parameters or other scoped state. Even if tokens themselves
don't need transformation, they might still need to be considered for other
token transformers, so simply keeping the final rank won't quite work even if
the token itself is fully transformed. As a minimum, a shallow clone would
need to be made and the rank reset (as in env.cloneTokens).
Change-Id: I4329113bb21750bae9a635229ed1b08da75dc614
* Added an LRU cache (using the lru-cache node module) for tokenizer output
* Mutation of nested attributes now replaces the containers. A shallow copy of
tokens is sufficient to isolate token transformations. Need to investigate
if we can actually get away without isolation and re-transformation for most
ordinary tokens.
Change-Id: I9136b1d7a1fbcc538183a319d4ecaa290d616fdf
* Ignore safesubst for now
* Remove an unneeded whitelist entry
* Make sure the caption is not lost for thumbs (fix to last commit) and remove
debug print
Change-Id: I243584ed0838cf7c3b4110fe9cdf869272477312
The HTML5 parser we are using to normalize expected HTML output in parserTests
reverses the order of attributes (see
https://github.com/aredridel/html5/pull/53 for the fix). Remove whitelist
entries concerned with this and use the proper order in external image
attributes.
Change-Id: If1868cae05396a150757c85a20473ab756cbcd97
* less verbose logging in noinclude processing and template expansion
* Give priority to the processing of templates transcluded from transclusions
to get closer to depth-first processing. This serves to minimize memory
usage from queued-up tokens.
* Increase the maximum outstanding requests per template retrieval. 10000
amazingly proved too low a limit on some big pages.
* Only process a single template request callback at a time for now
* Add a debug print in the treebuilder wrapper
* Don't treat multiple comments on a single line as a single comment to match
the PHP parser's behavior
Change-Id: I9a86b6d7bec3b9e1f17415daf1bf74170240721a
This has some TODOs still but I want to land it now anyway, and fix the
TODOs later.
* Add this.offsetMap which maps each linear model offset to a model tree node
* Refactor createNodesFromData()
** Rename it to buildSubtreeFromData()
** Have it build an offset map as well as a node subtree
** Have it set the root on the fake root node so that when the subtree
is attached to the main tree later, we don't get a rippling root
update all the way down
** Normalize the way the loop processes content, that way adding offsets
for content is easier
* Add rebuildNodes() which uses buildSubtreeFromData() to rebuild stuff
* Use rebuildNodes() in DocumentSynchronizer
* Use pushRebuild() in TransactionProcessor
* Optimize setRoot() for the case where the root is already set correctly
Change-Id: I8b827d0823c969e671615ddd06e5f1bd70e9d54c
In experiments this dropped the memory consumption further, and reduces the
queuing overhead in the node reactor.
Change-Id: I9409b6ca863b43b7557663bbec9572365059c078
Only call back a few callbacks per reactor iteration from the template fetch
request queue. This changes the expansion pattern from a (memory intensive)
breadth-first expansion to something quite close to depth-first expansion.
Additionally, retrieved pages are quickly added to the page cache so that a
lot of request queuing is avoided in favor of synchronous expansion from the
cache. On pages like Barack Obama that previously ran out of memory after
consuming node's 1.6G heap limit, expansion now runs in relatively constant
100-300M resident (so far, still running).
Change-Id: Ie34a1eeff00d868416de45ef8d289898258f560c
Eat unbalanced external link parts within template parameters. This does not
produce the same output as the PHP parser
(try echo '{{YouTube}}' | node parse.js), but preserves a level of sanity.
Need to check how common this is for external links. If it is rare enough,
moving the ']' after the parser function manually would fix the rendering for
the YouTube case.
Change-Id: I597d808efff36baa22191e7946a0061cc31120e8
Effectively stopping & starting polling prior to conversion
Getting Selection from model
Reselecting after conversion (TODO: modify selection to entire block ?)
Change-Id: I9ba331b5393bf568cc8d137646b43244ae2640a8
* add past paths for empty arguments etc
* cache attribute token transform pipelines
* fix bugs in TokenCollector and NoIncludeOnly handler, and improve its
efficiency by only registering for 'end' tokens on demand
* Remove empty reset methods from a few handlers
* Add a simple 'ap' debug print function that makes it easy to only print some
debug prints by temporarily changing 'dp' to 'ap'
* Improvements and bug fixes in AttributeExpander
Change-Id: Ie69729c8f62d48bba922712e44ebce484c621c50
Non-include attribute pipelines are not cached for now. Adding separate
caching for non-include attribute pipelines is very likely worth it, but
deferred for now.
Change-Id: I13f949d9f0a04536f9ccfcb73a2be69c5c08be01
This was an artifact from experimentation with multiple cursors long long ago in a land far far away
Change-Id: I14491c4adbd40bb8df4b1c31725cb1621351bef2
* Convert isNoInclude logic to positive isInclude throughout and set it
properly on attribute pipelines. Also don't cache non-include pipelines.
* Add a --pagename parameter to parse.js, which sets the page name in the
environment. This is then returned by {{PAGENAME}}. Not the final solution,
but useful for taxobox testing as taxons are selected based on PAGENAME.
* Add rudimentary pagenamebase parser function
Change-Id: If9c0be4c255200d0f2a30f02e5619437b4fd8f12
* DOM based on Wikia's thumb output: HTML5, clean caption without magnify
icon.
* basic RDFa annotations, but most options additionally in data-mw object-
might want to move more (or all?) of those into RDFa data using meta tags.
* no support yet for framed or other formats, image scaling etc
* also tweaked some config options in the environment
Change-Id: Ie461fcdce060cfc2dec65cc057709ae650ef3368
This makes it possible to get identical rendering in the editor, but may make other things more complex. The Wikitext serializer is no longer compatible for rendering lists so it's been stubbed out. Also the way the toolbar works with lists is broken, so that's been disabled. The HTML serializer has been fixed to work correctly and no-longer-used styles have been removed.
Change-Id: If156f55068b1f6d229b3fa789164f28b2e3dfc76
Also:
* Simplified ve.ce.Surface.getLeafNode, which may be better to just be removed and be used inline in the few places it's being used.
* Removed method wrapper for static function ve.ce.Surface.getLeafNode
Change-Id: I1d4cf0bb7ecc8f07f030753e40a13ebef7d02daa
behavior switches are converted to tokens which set parser.environment flags during the async transformation stage.
The next step would be for handlers in the sync23 stage to generate the TOC, section edit links, and so on according to these directives.
No tests written, because the switches are consumed and don't appear in rendered html. We can test the magic word layout controls individually, once they're implemented.
Another small change was to store option flags directly in the environment object, not that it makes much difference.
Change-Id: I863fbf4be1a17d2f6c31158298dd301f19ae1137
Explained in the README how to use npm to load the dependencies and run tests. Too bad about NODE_PATH...
Don't try to find parserTests.txt in assorted places--if it isn't present, fetch from gerrit. You can symlink from core if you're developing on both parsers, and the fetch script will not overwrite.
Use __dirname in parserTests.js to allow the script to run independent of current working directory.
Change-Id: I4c8b884e91f4fdeae385c7697aff768bdd199dd5
Match pairs of {{!}} or | for template productions, but not a mix of the two.
Example:
{{#if:1|{{!}}-
{{!}} {{#if:1|style="color: red"{{!}}|}}
}}
Note that the style parameter ends up as the *key* of an empty-valued
attribute on the table cell currently.
Change-Id: I5f9357dd1645ef97b0af89f32e8d92ae49218c72
Parser functions which only accept positional arguments now return both the
key and value of arguments. Complete attributes (key and value) for templates
and the like from parser functions are not yet supported though.
Change-Id: I3f81bb35acd27186222ce6d5217e820042527c01
Instead of a proliferation of data-mw-* attributes, it should be easier to
stash all private / non-semantic round-trip information in a JSON object
stored in data-mw.
Change-Id: Id200a6a8789fa152f29ea530e5a24b6ee7b4b285
* This high level surface object is responsible for creating & managing editor instances
* Revised Sandbox demo to invoke in this way.
Change-Id: I4043779af9a2ab964deaf26079a992e82ebeef27
* Configured VisualEditorSandbox to use es
* Reconfigured the ce demo to share the sandbox module
* Removed es demo
* Renamed ce demo to ve (es is broken anyways)
Patchset 2: squashed in https://gerrit.wikimedia.org/r/3953
Change-Id: If8d13bf7011616d222be78899b23186859d5ed70
Also, in ParserPipeline:
* Import the LM converter and expose it through getLinearModel()
* Fix getWikiDom() to actually work (still unused)
In parse.js:
* Add --help option that prints usage information (was unreachable)
* Add --linearmodel option to output linear model JSON instead of HTML
Change-Id: Ic534e03ff40a7c9117bb63f0c635a4213d5e3406
To handle replace operations that are not themselves consistent (these
are common, for instance when replacing an opening element in one place,
then replacing the closing element somewhere else), we process
subsequent replace operations inside the first one until things are
balanced again, then issue a single rebuild for the whole thing.
Change-Id: Ide4613f046fabfeeef383138c39e350b1b710033
gets a bit closer to supporting table fragments passed through template
arguments. Next, we'll need a way to indicate start-of-line position to
enable sol block-levels in template parameters.
Example:
{|
{{#if: true|{{!}}Table cell|}}
|}
re-processing in a phase is wanted. By default, after a token type change or
the return of multiple tokens only the remaining transforms with higher ranks
are applied.
Updated a few comments as well.
to maximize IO concurrency. Signal that all tokens are fully transformed to
callbacks called from TokenAccumulator._returnTokens. The result should be a
single re-transformation when entering the callback chain, and only if the
transform does not signal that it took care of full transformation itself.
Template expansion would set this flag, as the nested transform pipeline
processes all tokens to the end of phase async12.
to callback which lets transforms indicate if their returned tokens are fully
processed for their phase. If not, the callback re-processes them so that any
remaining transforms are applied.
wgUploadPath configurable. Also change the hard-coded fall-back image sizes to
sensible defaults. This breaks three parser tests until image size retrieval
from the wiki is implemented.
construction' part of the HTML5 spec:
http://www.whatwg.org/specs/web-apps/current-work/multipage/urls.html#url-manipulation-and-creation
Removed a few whitelisted test cases that are now passing directly.
The encoding canonicalization could also be moved to the Sanitizer. Doing this
early in token stream processing however has the advantage of providing further
transformations uniform data to work with. We could even consider to move this
even further into the tokenizer.
possible to support template / template argument expansion in image options,
and causes little trouble for wikilinks. Non-image wikilinks with multiple
text pipes are quite rare in the dumps, and concatenating description tokens
with a plain '|' is quite easy. 261 parser tests passing.
mediawiki.tokenizer.js module, and pass a reference to parse(). Faster
inline_breaks production using a JS function which seems to be generally
correct, but still breaks five tests when enabled. Seems to be some weird
interaction with peg.js, possibly something to do with caching.
* Convert all attributes into strings in Sanitizer
* Use strict comparison against empty string in tokenizer
* Add very simple sitename parserfunction
* 138 tests passing
wrapper. HTML ist now the only supported format. The DOMConverter is now no
longer used. Roan, feel free to remove / butcher it for direct HTML to linear
model conversion.
serialized into a single data-mw-rt attribute if present. Update parserTests
to ignore this attribute for comparisons with expected parser output.
A few more tweaks and notes are thrown into this commit too. 233 tests are
passing now.
only expand used branches selected by parser functions. Template (and
-argument) expansion is simply registered before general expansion.
Additionally, a few more simple time-based magic words are added in
ParserFunctions.
values. This includes comments, templates and template arguments.
This also replaces the specialized expansion logic in the TemplateHandler. The
removal of link validation lets one more parser test fail for now. External
link target validation will need to be implemented in the token stream handler
for links. This is noted as TODO in
https://www.mediawiki.org/wiki/Future/Parser_development#Token_stream_transforms.
functionality (comments, templates, template arguments) in arbitrary
attributes. The grammar for this is still quite rough, will need to
consolidate that area.
other tokens. This is only the first half of the conversion. The next step is
to drop the type attribute on most tokens and match on the constructor in the
token transform machinery.
improvements to parser functions on the way to support the cite extensions.
Preparation for generic template and template arg in attribute support. 222
parser tests now passing.
* Fix getScope()
** Drop the -1 which caused the result to be off by one level
** Prevent JS errors from occurring if bad input causes the loop to try to traverse up above the root node
* insert()
** Detect the case where the input data tries to close the containing element; in that case, we'll get scope != node
** Move the getNodeFromOffset() and getScope() calls up and out of the conditionals
** Remove unnecessary parent==model conditional, no longer needed now that getScope() can safely handle things that try to traverse too far up
** Add some comments to explain what's going on. I'll restructure this function a bit more shortly
page like this:
cd extensions/VisualEditor/modules/parser
echo '{{:Main Page}}' | node parse.js
echo '{{:Main Page}}' | node parse.js --html
echo '{{:Main Page}}' | node parse.js --debug
Even the date-based includes work somewhat, although they don't yet accept
passed-in dates.
directly to WikiDom from enwiki using a commandline like this:
echo '{{User:GWicke/Test}}' | node parse.js
Wohoo!
Complex pages with templates won't render properly yet, as noinclude /
includeonly and parser functions are not yet implemented. As a result, the
parser will run out of memory or hit the currently low expansion depth limit
as it tries to expand documentation for all templates.
disable it by default in parserTests as it tries to fetch all sorts of parser
functions and is not yet fully supported in parserTests. The next step will be
to build a list of parser functions (to avoid fetching them as templates) and
pushing the event interface into parserTests.
characters from host portions of links hrefs for now. This module needs to be
filled up with pretty much everything Sanitizer.php does, including tag and
attribute whitelists and attribute value sanitation (especially for style
attributes).
We'll also need to think about round-tripping of sanitized tokens.
* Add handler for post-expand paragraph wrapping on token stream, to handle
things like comments on its own line post-expand
* Add general Util module
* Fix self-closing tag handling in HTML5 tree builder
* Created AttributeTokenTransformManager for generic attribute conversion, and
removed { title, template argument {key, value} } expansion from
TemplateHandler.
* Added caching for attribute and input sub-pipelines. Especially attribute
pipelines would otherwise be recreated for each attribute value and key.
* TokenTransformDispatcher is now renamed to TokenTransformManager, and is
also turned into a base class
* SyncTokenTransformManager and AsyncTokenTransformManager subclass
TokenTransformManager and implement synchronous (phase 1,3) and asynchronous
(phase 2) transformation stages.
* Communication between stages uses the same chunk / end events as all the
other token stages.
* The AsyncTokenTransformManager now supports the creation of nested
AsyncTokenTransformManagers for template expansion.
The AsyncTokenTransformManager object takes on the responsibilities of a
preprocessor frame. Transforms are newly created (or potentially resurrected
from a cache), so that transforms do not have to worry about concurrency.
* The environment is pushed through to all transform managers and the
individual transforms.
are now merged with specific registrations by rank. Not yet clear if that is a
good idea overall, need to check use cases when implementing template expansion
and other functionality.
183 parser test now passing.
The TokenTransformDispatcher now actually implements an asynchronous, phased
token transformation framework as described in
https://www.mediawiki.org/wiki/Future/Parser_development/Token_stream_transformations.
Additionally, the parser pipeline is now mostly held together using events.
The tokenizer still emits a lame single events with all tokens, as block-level
emission failed with scoping issues specific to the PEGJS parser generator.
All stages clean up when receiving the end tokens, so that the full pipeline
can be used for repeated parsing.
The QuoteTransformer is not yet 100% fixed to work with the new interface, and
the Cite extension is disabled for now pending adaptation. Bold-italic related
tests are failing currently.
tests now passing.
Link trails depend on language-dependent positive character classes in the PHP
parser. These classes all seem to disallow punctuation implicitly and list
differing plain text characters instead, so it might be possible to get away
with identifying a common class of non-trail punctuation instead. This would
help to keep the tokenizer independent of configurations, which is very
desirable for caching and simplified external parsing.
start / row / table end). The old productions are not deleted yet to make it
easy to compare the output on more complex articles. 181 tests passing after
adding two table tests with whitespace-only differences to the whitelist.
is in its early stages and nowhere near deployment, please Be Bold and just
commit things like this directly! IMHO it makes more sense to fully review this
once it settles down a bit.
This required a few further additions to the TokenTransformDispatcher. In
particular, there is now an 'any' token match whose callbacks are executed
before more specific callbacks. This is used by the Cite extension to eat all
tokens between ref and /ref tags. This need is very common, so should be
broken out to an intermediate layer in the future.
In general, the requirements for the TokenTransformDispatcher API are now
clearer, and the API should likely be cleaned up / simplified.
token stream. This is the first token transformation exercising the
TokenTransformer class as its dispatcher. Template expansions, wiki link
formatting, tag sanitation and extensions should be able to use the same
dispatcher by registering for specific token types.
The parser performance is very slightly improved as the token stream is only
traversed once.
token type, and supports asynchronous token expansion (for example for async
template expansion). This code is not yet tested or used. The interface for
token insertion from transformation functions will be expanded as needed.
