When closing annotation nodes, we weren't popping them off
annotationStack. Not sure where this came from, but the code was
definitely bad and this fixes it.
Change-Id: I6d805e9aca3778666212135f76ff34c6baacbbc8
* Default value of wgVisualEditorParsoidURL is broken.
Slash is needed, else Api will request to
http://hostnamePageName
Roan says double slashes are okay, and look cleaner than string
search checks etc.
* Use .clone() for mw.Uri instead of converting to string
and letting mw.Uri parse it, again. Clone creates a basic
instance and copies over properties internally (deep copy,
no references).
* No need for hasOwnProperty (and its potential issues)
* Code clean up
- Whitespace consistency
- Variable hosting
- Remove redundant `return false;` statements in event handlers
e.preventDefault() is a jQuery.Event method that takes care
of cross-browser issues.
- Same for e.keyCode||e.which thing, this is already normalized
by jQuery.Event
- Add missing parameter to setTimeout
- Consistent order in success/error handlers in $.ajax options
Change-Id: I5bc24e0cbdf01b3704d4ccb0b45b3052e3b58694
Also:
* Removed a lot of dead code in Surface that was used in the now dead and gone sandbox.
* Changed from throwing an exception when calling getBalancedData on a range that produces no results from selectNodes to just returning []
Change-Id: Icf27094724eae5b90eec21308f9e26afe877e3ee
When editor surface loses focus, now checking to see if an inspector
is set or a menu is open prior to closing the context.
This ruling ensures more rational context icon / child menu behavior.
Change-Id: Ic4b74bb51e811e264a2109a3c1c5a8ae503a8c49
* Moved all constants to a singleton constants object
* Moved couple methods to mediawiki.Util.js
* Fixed url regexp bug -- relative urls weren't being matched
but, this bug escaped through because the previous url regexp
had a typo (which means url sanitization wasn't being done by
the previous code). This also means we have to either find
sanitizer tests or add new ones so these bugs are caught. Maybe
parserTests.txt is not the right place for this?
Change-Id: Ia05e1d1596bb9bc4a9eb21d7c77248f5626a710e
* Dynamically select piped / simple wikilinks in the serializer
* Use generic attribute shadowing, drop sHref
* Actually handle modifications to non-piped wikilinks
* Properly escape anything that looks like a percent-encoded char
* Add stripSuffix utility method
* Fix a bunch of JSHint warnings in the serializer in particular
Change-Id: I2d8954f6b665093676ccc5dd5437ea9b37c014ad
* Classicifation (JS)
Use addClass instead of attr( 'class' ) whenever possible.
addClass will manipulate the properties directly instead of
(re-)setting an attribute which (most) browsers then sync
with the properties.
Difference between:
elem.className
and
elem.setAttribute( 'class', .. );
Just like .checked, .value, .disabled and other interactive
properties, the HTML attributes should only be used for initial
values from the html document. When in javascript, only set
properties. Attributes are either ignored or slow.
* Styling (JS)
Use .css() instead of attr( 'style' ).
Again, setting properties instead of attributes is much faster,
easier and safer. And this way it takes care of cross-browser
issues where applicable, and less prone to error due to dealing
with key-value pairs instead of css strings.
Difference between:
elem.style.foo = 'bar';
and
elem.setAttribute( 'style', 'foo: bar;' );
* Finding (JS)
Use .find( 'foo bar' ) instead of .find( 'foo' ).find( 'bar' ).
It is CSS!
* Vendor prefixes (CSS)
It is important to always list newer (standards-compliant) versions
*after* the older/prefixed variants.
See also http://css-tricks.com/ordering-css3-properties/
So the following three:
-webkit-gradient (Chrome, Safari 4)
-webkit-linear-gradient (Chrome 10, Safari 5+)
linear-gradient (CSS3 standard)
... must be in that order.
Notes:
- "-moz-opacity" is from before Mozilla 1.7 (Firefox < 0.8)
Has not been renamed to "opacity" since Firefox 0.9.
- Removed redundant "-moz-opacity"
- Added "filter: alpha(opacity=**);" where missing
- Fixed order of css3 properties (old to new)
- Add standardized css3 versions where missing
(some 'border-radius' groups didn't have the non-prefixed version)
- Spacing
- @embed
- Shorten hex colors where possible (#dddddd -> #ddd)
$ ack '#([0-9a-f])\1{5}' --css
$ ack '#([0-9a-f])\1{2};' --css
Change-Id: I386fedb9058c2567fd0af5f55291e9859a53329d
* Need auditing to see if the whitelist is complete.
* Added <gallery> and <tag> to the whitelist since they seem to
come from some extensions and parser tests depend on them.
May need to use parsoid config or parser hooks/extension info
to extend sanitizer whitelist. For now, probably okay.
Change-Id: Id9ecdd96843e8f6ca65e8666807dec1015443d49
* Also update test/index.html with latest minimalistic format
* Update test suite hardcoded paths to match the definition in
VisualEditor.php for ResourceLoader
* Issues:
- 'jquery/jquery.json.js' should not be loaded directly,
using a dependency instead.
- Load scripts from the <head> in test/index.html so that
code that depends on document being ready is catches instead of
silently being ignored.
Change-Id: I5ad7390137f4d17c153a1bf69f19c4869c08e323
-Bug 38042
Save dialog description field doesn't respond to 'Enter' or 'Return' keys
-Bug 38621
Pressing 'Esc' in Save-dialog should exit saving, return to editor
Change-Id: I9c43c6c9f2f2b538becc4fbbce1eda6e918d4879
'''Kranitor commits''' are commits by Krinkle with his janitor hat on.
Must never contain functional changes mixed with miscellaneous changes.
.gitignore:
* Add .DS_Store to the ignore list so that browsing the directories
on Mac OS X, will not add these files to the list of untracked
files.
* Fix missing newline at end of file
.jshintrc
* raises -> throws
* +module (QUnit.module)
* remove 'Node' (as of node-jshint 1.7.2 this is now part of
'browser:true', as it should be)
Authors:
* Adding myself
MWExtension/VisualEditor.php
* Fix default value of wgVisualEditorParsoidURL to not
point to the experimental instance in WMF Labs.
Issues:
* ve.ce.TextNode:
- Fix TODO: Don't perform a useless clone of an already-jQuerified object.
- Use .html() to set html content instead of encapsulating between
two strings. This is slightly faster but more importantly safer,
and prevents situations where the resulting jQuery collection
actually contains 2 elements instead of 1, thus messing up
what .contents() is iterating over.
* ve.ce.Document.test.js
- Fix: ReferenceError: assert is not defined
* ve.dm.Document.test.js
- Fix: ReferenceError: assert is not defined
* ve.dm.Transaction.test.js
- Fix: ReferenceError: assert is not defined
* ve.dm.TransactionProcessor.test.js
- Fix: ReferenceError: assert is not defined
* ext.visualEditor.viewPageTarget
- Missing dependency on 'mediawiki.Title'
Code conventions / Misc cleanup
* Various JSHint warnings.
* Whitespace
* jQuery(): Use '<tag>' for element creation,
use '<valid><xml/></valid>' for parsing
* Use the default operator instead of ternary when the condition and
first value are the same.
x = foo ? foo : bar; -> x = foo || bar;
Because contrary to some programming language (PHP...), in JS the
default operator does not enforce a boolean result but returns the
original value, hence it being called the 'default' operator, as
opposed to the 'or' operator.
* No need to call addClass() twice, it takes a space-separated list
(jQuery splits by space and adds if needed)
* Use .on( event[, selector], fn ) instead of the deprecated
routers to it such as .bind(), .delegate() and .live().
All these three are now built-in and fully compatible with .on()
* Add 'XXX:' comments for suspicious code that I don't want to change
as part of a clean up commit.
* Remove unused variables (several var x = this; where x was not
used anywhere, possibly from boilerplate copy/paste)
* Follows-up Trevor's commit that converts test suites to the new
QUnit format. Also removed the globals since we no longer use those
any more.
Change-Id: I7e37c9bff812e371c7f65a6fd85d9e2af3e0a22f
This should make it much simpler to keep MediaWiki specifics out of VisualEditor, which will in turn make it easier to integrate VisualEditor into another platform.
Change-Id: I073e9737b37c28af889f2457d10b082cefd0d63b
* CSS sanitization code now ported.
* Could use non-port review another day to see if the ported code makes
sense and has any gotchas.
* More sanitizer parser tests now green.
* Could use a lot more aggressive addition of parser tests.
Change-Id: I9df003540bd31f327f5307472c9f7dcbbe7b4342
Convert underscores in the href attribute to spaces in the linear model,
and back to underscores when going back to HTML. This ensures the link
targets displayed to and edited by the user look nice
Change-Id: I4855fce28ad8b724284c53881abc7b99b59b9079
Clearing context icon when editor loses focus.
-Reproduce problem by selecting text, then click outside of
the editor. Selection is lost, and context icon is stuck.
Change-Id: I4b321f16cea73ec0e51540c0e71f265ab47514e9
This means we don't have to rely on data-rt.sHref. It also means that
we'll now be showing the canonical link target in the link inspector
rather than the link target as entered by the user, but that's fine.
Also change test to have href differ from sHref to show that we use
href.
Change-Id: Idabdbf2579663ef1efb47d6a73f39743c9f64f3b
This is ugly but makes things work again. I intend to clean this up once
we have a better attribute API
* Recognize mw:WikiLink, mw:SimpleWikiLink, mw:ExtLink,
mw:NumberedExtLink and mw:UrlLink
* Support is incomplete because we can't get to the annotation text with
the current API
* Preserve all unhandled attributes rather than special-casing data-mw
* Update remaining code using data-mw (sHref and stx extraction) to
account for the data-mw -> data-rt rename
* Update tests accordingly
Change-Id: Ia13d3008a6d4cdc8828f9acda5aa797566bc597f
The recent directory move broke parserTests, fix it for now. Will need
refixing once me migrate to our own repo.
Change-Id: I014001cd6904d1dea3f9417c9cde9c80ab079232
* Quite unique according to google, and more obvious
* Also adjust parserTests to ignore mw:Nowiki and mw:Placeholder spans
Change-Id: I340e85092b60a65b4053a40bf8c238e26cb49c96
- Attaching save dialog to toolbar wrapper vs toolbar itself.
- Attaching surface specific toolbar wrapper vs all toolbar wrappers
in the case of multiple editors on the page.
Change-Id: Ic81f5a680f5593c71c27b7d47fe246487eebd4a3
* Ported attribute sanitization code (and related functions) from
core/includes/Sanitizer.php
* Added dummy flags and set to true (use of rdfa, microdata attrs,
and html5 mode).
* Removed couple whitelisted sanitizer tests.
* A few sanitizer tests now pass.
* More work to be done.
Change-Id: I19c92bbfcb57f3e97a7af1b7c5f63772e427dae4
(or this shouldn't be allowed)
-Revised method for for returning all link annotations in a
selection. Now properly clearning all selected links.
-Trimming whitespace from selection
-Modifying selection if it doesn't contain annotated range
-Disabled link creation only if target is blank. This allows
Existing link text to be modified while having the same target.
Change-Id: I7255dcf1c88fa1cd6e7edbc3baa82cd4c72a95d1
-Hide / show inspector with visibility vs display property
for iframe cross browser compatibility.
Change-Id: Ibdd0250872c42d74d6ff7d22abdf9d838962acc1
This was caused by a bug in fixupInsertion that caused it to believe
that inserting something like "a</p><p>b</p><p>c" into the middle of an
empty paragraph was invalid.
This commit fixes the fixupInsertion bug, which fixes the
select-all-cut-paste behavior in Chrome. It's still broken in Firefox
because of selection-related issues, but I'll split that out into a
different bug report.
Change-Id: I767f5d37ec7e511778ae9ca8283ec4b26c728298
* addAttribute and getAttribute do the obvious and simple thing
* addNormalizedAttribute remembers an unnormalized version of an attribute and
supports change detection for the normalized attribute
* getAttributeSource retrieves the original attribute if there was a
normalized version which was not changed, or the current value (potentially
based on the normalized version) otherwise. For use by the
WikitextSerializer.
Change-Id: I72533cf6cfff1ddb88be2501653c7c47d270898c
In preparation for the big extraction of Parsoid out of VisualEditor,
we'll start by moving the tests into the parsoid location.
Change-Id: I4a926ee4aad1490d4f769d44e91af80842b881f0
* Got rid of mergeProperties monkey-patch from core-upgrade.
* Reformatted class defns in mediawiki.parser.defines.js.
* Protected unconditional tokenization of list handler output with an
env.trace check.
* Other minor formatting fixes to respect 80-100 column code width
guideline.
Change-Id: Ida769e0e239b01a813b2d30a65aba60216262a43
* the used RDFa types for links are now identical to those listed in
http://www.mediawiki.org/wiki/Parsoid/RDFa_vocabulary, and are supported for
serialization
* Editors are responsible for adjusting the type when converting between link
types. Adding a caption to an mw:UrlLink for example should convert it into
an mw:ExtLink.
Update: rebased on top of trace patches
Change-Id: Ie1b882e2b3fbad08be94769e1167dccd8dfea65d
* Source-based round-tripping now uses typeof="mw:Placeholder" instead of
data-gen.
* mw:Image is supported for round-tripping, but not yet for modifications as
it is still source-based
Change-Id: Ie5cf4e54de0163168c25c2b5c09380657a15970f
* changes:
Additional work on readable tokenizer debug output
Added 'href' key to anonymous KV wikilink and isbn attribute.
Removed utility functions from mediawiki.parser.environment
Added utility methods to ext.Util.js
Added missing var keyword
In trace mode, wrap transform to output trace info
Output chunk tokens to console only in trace mode
Further refinement of readable pretty-printing of tokens.
More fixes on the way to readable debug/trace output.
Added mergeProperties function to Object.prototype
In RTL interface the drop-down arrow overlaps with the text.
The arrow was positioned explicitly in relation to the left margin, and this
can't work in a flipped RTL environment. I changed the position to "right"
and modified the arrow image a bit.
Some visual tweaks to the arrow may still be needed. Another option can be
to convert the image to SVG or to use a character like ▼.
Change-Id: Ib09a2a20b150de6e8a9531fc0db7dfffe4e95525
* This makes wikilink attrs more similar to ext links.
* Added 'content' key to ISBN links, but couldn't add it to regular
wikilinks yet because of complexity of how they are handled in
the rest of the pipeline. Changing this requires fixing up other
parts down the pipeline -- something for later.
* Fixed up wikilink handler to use named lookup for 'href' and
'tail' rather than positional lookup. Content lookup is still
positional as before.
Change-Id: I657b1f338d38df3cfdfa99f27ac46e7fe1c9fd65
* these functions have already been added to ext.Util.js
* removed a couple jshint warnings.
* minor code restructuring in tokensToString and comments
to better indicate what is going on.
Change-Id: I9d6a03cc35075e1a64d8fac9e167a3ce4ccd9424
* Copied over utility methods from mediawiki.parser.environment.js
to ext.Util.js.
* Moved over utility method from mediawiki.parser.defines.js to
ext.Util.js.
* Converted Util to be a singleton object rather than an allocatable
class. There is no reason to allocate a new utility class everywhere
since this utility object has no useful state.
* Fixed up use of utility methods to use Util rather than env.
Change-Id: Ib81f96b894f6528f2ccbe36e1fd4c3d50cd1f6b7
- Added extra debug_name parameter to addTransform which is
used in addTransform to output useful trace info.
Change-Id: I160ba0c45f681149375e32ab19f97baa439b09a8
- Added limits to toolbar float, Toolbar will not go past the last node in editor.
- Added bottom mode to allow toolbar to stick above the last node until the scroll position
is above the last node.
- Actually checking toolbar config now and setting float when flag is set.
- Gave float method for top toolbar a better name.
Change-Id: Ic39c5402fa7a05e13c5e81722d8729d93776d7e9
But still put slugs before them. Done by overriding canHaveSlugAfter()
in ve.ce.ListNode.
Eventually this should be configurable and MediaWiki-specific
Change-Id: I5ad15ca4085a2d730add4954acbea358819b3986
These determine whether a node can have a slug before and after,
respectively. The default implementation in ve.ce.Node is to use the
same rules for both, but individual node types can override this.
I'll need this to suppress slugs after nested lists but not before them.
Change-Id: Id88c0fc98aca7c7f52ce990ed9b8c42181ef6d18
For pressing Enter in an empty list item at the end of a top-level list,
this has the same result as the previous code, but if you're in a nested
list it has the effect of jumping down a level. A previous incarnation
of this change just made Enter insert more list items ad infinitum if
you were in a nested list, but I think this is better.
This fixes a bug where pressing at the end of a nested list inserted a
paragraph in an invalid location
Change-Id: I9c7dbaf29a98f84926ed3a05e71c6294926dfce2
Fix the commented-out code: it caused unindent to be triggered by just
pressing Shift. ASCII 16 is "data link escape", no idea where that came
from, so I removed it and used e.shiftKey instead.
Also check whether indent/outdent is even possible before doing it.
Currently this is done in a very hacky way (by checking the state of the
indent button), ideally we'd refactor things such that toolbar tools can
listen for keydowns and intercept them, that would make the code much
cleaner and we wouldn't have this problem.
Change-Id: I99885ee4b8a79cd24c4958c188addfc2b0453b03
After indenting or outdenting a list item in a numbered list, the
numbering wasn't updated. So if you had:
1. One
2. Two
3. Three
and you indented "Two", you'd get:
1. One
1. Two
3. Three
Adding or removing items in the list using the keyboard, or even
inspecting the list in Firebug (!), would trigger a renumbering and fix
the list to display "2. Three". But then the same issue would occur in
in reverse when outdenting "Two" (either using undo or using the
outdent button):
1. One
2. Two
2. Three
The workaround is to force a reflow by requesting the height (thank you
Timo). Implemented this in an override of onSplice() in ve.ce.ListNode, so
the list is detached and reattached every time children are spliced into
or out of it.
I haven't managed to come up with a minimal test case for this, not even
by putting a list in a contentEditable div and doing the same DOM
operations that ve.ce does from a setTimeout callback.
Change-Id: I93b2a309034c411a7b4e4b6c6bd4ef9d473999eb
This works pretty well, the only problems I found are:
* when selecting multiple list items, only the first is in/outdented
* there's no special handling for child lists, so the behavior for
in/outdenting list items that have a child list can look weird, but
it's consistent
Also needs more documentation
Change-Id: I6f4f3725e57a590196d7d638a77b87ea85586dc8
* When ascending back up the stack, check for a start between two closings
* Also check for an end between two openings
* This introduces code duplication but selectNodes() is full of that
already. I'll have to do a duplication cleanup soon
* Add test case for </li><li>
* Update existing test case that covered a </li>
Change-Id: Ifc80585ce0e0d6988bc54228602c69f0d519200a
For nested lists, this function would return multiple groups where one
was wholly contained in the other, use offsets to prevent that from
happening.
Change-Id: Ib03bb1c81712d805cc263c2975cc3942de63d2ed
to be added. Create inspector elements in the propper document
scope. Restore inspector css classnames to camel case for proof
that inspectors are being created in the correct document scope.
Previously, inspector elements created in the wrong document scope
would have css rules applied only if class names were lowercase.
Issue only surfaced in Webkit browsers. Though, this implementation
is more future proof and will help prevent future inspector bugs.
Patch 3) Fixed global variable definition and mistake with
inspectorDoc
Change-Id: I36c0d078aea10d919689768878004a19f7f89b55
-Selection of part of a link now modifies selection to entire link
range on inspection.
-Retaining selection direction on new range
Only partial fix to bug as previous link annotation is not
yet properly cleared.
Bug 33053 - VisualEditor: Link creation should not include trailing
spaces, and should provide a suggestion based on selected text
-Created method to return a new range without outer spaces.
-Retaining selection direction on new range.
-Enhancement needed for link suggestion.
Bug 33108 - VisualEditor: Highlighted trailing whitespace should
not have styles applied
-Modified trim method to retain selection, added call to trim
range on annotate method.
Change-Id: I92f264e19350c62b7c2ac3cd9e78af0071afef5c
Now that we have access to the contents we can more easily compare the content
with link targets. This is still to do- this commit only converts the link
handler to work on the collected tokens.
* Start to implement latest RDFa spec from
http://www.mediawiki.org/wiki/Parsoid/RDFa_vocabulary
* Capitalize types, add mw:Entity type for html entities
* Detect changes to entities using tokenCollector and srcContent
Change-Id: I45429f4b930858a16e166ef8377c8f6f5114c414
This license change is aimed at maximizing the reusability of this code
in other projects. VisualEditor is more than just an awesome editor for
MediaWiki, it's the new editor for the entire internet.
Added license and author files, plus mentions of the license to all
VisualEditor PHP, JavaScript and CSS files. Parser files have not been
modified but are effectively re-licensed since there's no overriding
license information. 3rd party libraries are not changed, but are all
already MIT licensed.
Change-Id: I895b256325db7c8689756edab34523de4418b0f2
* "onevar" warning sometimes solved by just merging var statements
other times solved by making it a function declaration instead
of a function expression.
* Also fixed several '_this' variable names in ve.es.Surface to
more descriptive names, and enabled warnings for dangling _
in identifiers.
Change-Id: I7d411881e3e06cf9a7fe56d689c29375881a81de
This is just to avoid re-licensing along with VE. We want to be compatible
with MediaWiki core to make sure a closely-integrated C port is still
GPL-compatible. We could consider adding MIT to the JS implementation after
porting to C.
Change-Id: Ia83e8620e26c95625793438c4c5e8ddcf2702368
This is work in progress, but committed for now so I can use it for links and
tweak it while doing so.
Change-Id: I757277f6efacda6d9432ca57542a957f597a98de
* This code change is an attempt to address the FIXME about constant
resorting of transformations in _getTransforms. This caches sorted
transformations and selectively clears/updates the cache on add/remove.
Change-Id: If24a807b84d494aa4e5597339039a5573a30905e
This hopefully makes it clearer that data-rt contains private round-trip info
instead of semantically interesting data.
Change-Id: I03b476ed112a4b627c9871ee3677c450f943429a
* Arbitrary predicate support for the termination of collection mode
* tokens as property of the collector instead of a state-global thing
Change-Id: Ibcb342bc64a76fece9b04a760ea56c7878e67cad
* Fixed image serializer to deal with missing 'v' value in a k-v pair
representing an image attribute.
* Added fix to deal with bare <li>'s (without surrounding <ul> tags)
NOTE: The second fix is required currently to deal with bugs in the parser
as it deals with complex cases. But, in the future, we could deal with
this in one of the following ways:
(a) The serializer expects a well-formed DOM and all cleanup will be
done as part of external tools/passes.
(b) The serializer supports a small set of exceptional cases and bare
list items could be one of them
(c) The serializer ought to handle any DOM that is thrown at it.
Yet to be resolved.
Change-Id: Ib585e5c9f2a8a80854740ce0211bde705f9fd6f4
This makes it clear that setInterwiki can modify existing mappings, and adds a
similar method for the removal of existing mappings.
Change-Id: Ic603a4b2ccec35d086513fa7cf711702bfb2baa0
* Strips the first paragraph tag in a list item or table cell context
if there are no attributes on it and stx:html is not set
Change-Id: I74988645fe505c662f86488e32d0f11d464ffe41
* Looks like I misled myself in commit 88fc91 -- that wikitext
roundtripped perfectly because it went through the 'src' route
because it was a thumbnail with an explicit image which doesn't
go through renderThumb -- so, the serializer simply spit out the
original 'src' string and hence perfect rt :).
* More whitespace preserving fixes in LinkHandler.
* Also changed resource value in the img tag to use the original
filename rather than the normalized capitalized filename.
* 2 more parsertests rt -- now upto 400.
Change-Id: I144a6486dd9d07da8a74a68700fe96c78d192826
* Something to be said for code alignment - easier on the eye!
* Maybe a good case for breaking mediawiki coding guidelines.
* But, happy to abandon commit if not useful. :)
Change-Id: I1133af488f572ac7f8727be9108e08e14c4e6420
* Changed PrefixImageOptions so that thumb and thumbnail are
distinct key-value pairs. Without this fix, cannot distinguish
between thumb=foo.jpg and thumbnail=foo.jpg
* Fixed link handler so whitespace is preserved around prefixed image
options.
* Fixed figure handler to process the 3 different kind of image options:
size, simple image options, and prefixed image options.
* There is a hack/fixme for "upright: aspect" prefixed image option
which needs to be looked into.
* Still need to fix uppercasing of the image resource name.
With these fixes, the following wikitext roundtrips perfectly
(after newline breaks are removed)
[[Image:Foo.jpg|thumbnail = 'baby.jpg'|100x100px|center| alt =bbbbb|
upright=true|bottom|link='http://foo.bar'|
This is a [[Linked Caption]] in the image]]
Change-Id: I6606df56874c2b97f00f08cb6bbeaec9878167d3
* For now, extracted image markup options out of the link handler.
* This info will also be used by the serializer.
* More properties/global constants can be moved into this structure
over time.
Change-Id: I4cfbfd703f42e93fbad52b38b435f68d8a5c22ee
* Minor refactoring
* Cleared src in dataAttribs in renderThumb since we can serialize
thumbs now (or at least we can once all bugs are fixed and missing
pieces are handled).
Change-Id: If18865801cdd3d89c1477e68bfa3e13107c45b40
Anything with data-gen="both" and dataAttribs.src defined serializes to
dataAttribs.src and drops its contents (if any). We can use this to round-trip
elements we don't properly parse or serialize yet. Without RDFa info, the
editor will not touch the contents after encountering data-gen="both".
Change-Id: Ia39e5fdd765c2c9b36f26313455685d29f118839
* Don't consider them for auto-numbered links
* Don't insert a trailing space if the content is empty
These links are still wrapped in nowiki on round-tripping since the
valid/invalid url determination is done in the LinkHandler and not the
Tokenizer as it is configuration-dependent. Not incorrect for rendering (and
perhaps easier to understand for humans too), but might still introduce a
dirty diff. We'll still need reconciliation / damage tracking in the end ;)
Change-Id: I959ebc1b7f81d110a1141bb38ba5ee97f52ebf96
This only applies to newly created headings, so headings with a single newline
preceding them will be round-tripped that way.
Change-Id: Ic09972bbd25c3934b53f6fd3b5be5a0c3185c2af
* Collect all figure tokens and process them as a chunk
* This effectively mimics context-sensitive DOM walking,
but since we need serialization supported on a token stream,
we cannot use real DOM walking. The current technique should
also work on a token stream.
* There is a FIXME about the image filename being capitalized.
This needs fixing in the parser or some other way of recognizing
original unnormalized filenam.
Amended by gwicke:
* Build option list and join it with pipe to avoid stray trailing pipe
* Satisfy JSHint's weird preference to have '&&' and '||' at the end of the line
Change-Id: I1e5f6600f297fcdf81e3227a82ca3b71d4e97fc3
This is a zero-length tsr for now (and thus not 100% correct), but will do the
job for starttag / endtag range establishment
Change-Id: Iedd50ad319aa8d5916434fb6744deb04e031e456
* Removed dead commented out code.
* Cleaned up newline handling in serializer some more.
* Now, onNewLine and onStartOfLine reflect serializer state
more accurately.
* No implicit new lines for explicit html tags.
* 9 more roundtrip tests now green.
Change-Id: I9f640de2ae769c7472538fa687400dc8a40c2b2d
297 round-trip tests are passing with this patch.
TODO:
* generalize data-mw-gc handling in the serializer for any tag
* use data-mw-gc="both" and data-mw.src: 'the wikitext' for round-tripping of
wikitext structures, optionally with some presentational (but read-only)
content
* use span and data-mw-gc="both" for nowiki
Change-Id: I700142a56818977c20c8c06e6a5f2e77a708d25e
This makes sure that we escape start-of-line syntax when needed, since
onNewline is often not yet set.
Discussion / background:
[19:18] <subbu> this will fix it, but, i think this is asking for another
minor refactoring of these flags ... because this is a subtle fix which means
it might be possible to make it clearer. onNewline is one true in on
direction, i.e. if true, we are in a new line state, but if we are in a
newline context, onNewline is not true, which is why this new method is
needed.
[19:19] <subbu> i dont know if it is possible, but it seems like it shoudl be
possible. but, something for later.
[19:20] <subbu> badly phraed. "onNewline" ==> in new line context, but if in
new line context, onNewline may be false.
[19:20] <gwicke> we should perhaps update it as early as possible instead
[19:21] <subbu> i cannot today, but possible monday. i am heading out in
about 15-30 mins.
[19:22] <gwicke> will need to check all conditions depending on it in
_serializeToken
[19:22] <subbu> oh, i misunderstood you :)
[19:22] <gwicke> and if there are cases where the onNewline / onStartOfLine
state could be reverted later
[19:23] <subbu> you were referring to the flag, i thought you meant we should
fix this sooner than later.
[19:23] <gwicke> yes, I wasn't terribly clear
[19:23] <gwicke> you wrote something about following productions swallowing
newlines, but I think we don't actually do that any more
[19:24] <gwicke> I'm quite optimistic that updating those flags much earlier
would work
[19:25] <subbu> yes, it could fix it.
[19:26] <subbu> you might be right reg. swallowing. it was happening earlier.
but, not right now, after single-line mode and other fixes.
Change-Id: Ic1d8141c04eb54a59977d0ba87bcf06bafd421e0
This should not really be needed if the tokenizer did not decode html entities
on the fly. It is still a quick way to make sure no htmlish content can be
inserted even with the current decoding.
The next step and proper fix is to make entity decoding either optional in the
tokenizer (flag-controlled), or move it to a later stage in the token
processing pipeline.
Change-Id: Ife093dcfb95113763dab5635b098c795d3550586
* Renamed defaultOptions to initialState
* Got rid of unused state property
* Added comments explaining how state attributes
and tag handler flags are used
* Refactored listItemHandler check into functions and
added FIXME possible rewriting of that check.
* Protected serializeDOM in a try-catch handler to
catch exceptions and output the exception to the console.
Change-Id: I3d351c06e4b86baeb5a55243b11dbfa9baca5bb7
This fixes a bug Trevor reported where selecting from a list item across
a heading and into a paragraph, pressing backspace, then clicking undo
caused an exception.
Change-Id: Id2851271529e10548f6979a030a198054aa1c48f
ve.ce.TextNode listed textStyle annotations that didn't actually exist,
and failed to recognize some that did exist (such as span; bug 37808).
Added all annotations to both places. <span> tags are now tolerated by
the editor in that it doesn't crash anymore, but they're displayed (and
saved!) without any attributes, so <span style="color:yellow;">y</span>
doesn't show a yellow 'y' in the editor and is saved back as
<span>y</span> .
Change-Id: Iaae11ad5044150fa904010983ff83579cb37733d
Fixed by adding the specialMessages module which is only loaded once the
editor loads. Then after it's loaded we use the summary message from
there to update the (possibly broken) summary message in the save
dialog.
Change-Id: I67f5c59501cdf7c66c925cef8d4dd42b0f2cfde3
* Removed murky ' :' -> ' :' replacement in tokenizer. This breaks four
parser tests, and should be fixed in a token stream transformer or DOM
postprocessor. This replacement clashes with round-tripping, and is not
terribly important visually.
* Added stx:row annotation to single-line dt/dd pairs and use it to preserve
single-line syntax in the serializer. There is no attempt yet to support the
addition of nested lists in an originally single-line dd. We'd need to look
ahead in the serializer to support this. Perhaps the editor can simply drop
data-mw in that case.
* Switched default dt/dd serialization to multi-line. This supports all nested
lists and multiple dds.
* Don't close dls when switching from dt to dd or back in the token stream
ListHandler.
Overall 290 round-trip tests are passing now (up from 284, some due to ,
some due to lists). The number of passing parser tests dropped slightly from
303 to 297 (or 301/295 on weekdays other than Thursday).
Change-Id: I85ff40571833713388c6523e6a4ba2e94daa3807
Basically only prefix all bullets if the serialization output is going to be
in start-of-line context. The test for that is currently inline, but should
perhaps be factored out to a method or state flag instead.
We could alternatively consider to return the start-of-line prefix and let it
be used in _serializeToken in case we end up in start-of-line context.
This patch also fixes a newline issue on input like this:
:d1
::: d3
Both the list and list item handlers now set the startsNewline flag
dynamically depending on the context, so that we don't depend on the
suppression of newlines from list syntax by the singleLineMode any more.
There is still an extra newline inserted between list items in the following
example:
;t1 :d1
;;t2 ::d2
This looks like a bug in the produced DOM and not in the serializer, since the
outer definition list is closed and re-opened between d1 and t2.
Change-Id: I78e3a1ef34cf9159d5a1e86fb64c774ff111e71d
* changes:
Got rid of iteration to get the surface
Removed attach and detach methods from ve.ce.Node
Track adjustments in DocumentSynchronizer and apply them to oldRange
This is needed because oldRange is relative to the state of the model before any changes were made, but when we call selectNodes() it's gonna operate on a partially updated model tree.
This is a genuine bug in DocumentSynchronizer proper, which means I owe the entire team lunch
Change-Id: Ia6510de19df02e961c7f25fb8e7833abceb8d25b
* Adjust both start and end for preceding operations
* Adjust end for the current operation as well
Change-Id: I2f96d609bddf3788aa5700ad1f0b46208f3517d7
text for a link is the selected text.
patchset 2 - add case for if data offset is an object, be sure it
is a string prior to adding it. truncate to 255 chars.
patchset 3 - actually add the patched file
Change-Id: Ibddf942c2a0ba3412d93cf9730f74eb858025fad
Also fixed unlisting to not break when unlisting a range that includes things that are not list nodes
Change-Id: Ib9d4ea851c3ed9bf72a93aa87e470ce40c308453
contained by a list node the button is off. Button is now only toggled on
if all nodes in selection have a parent list node.
Patchset 2 - cleanup whitespace
Change-Id: Ia9adc39c0d5c75e2e96580f0e172f5b602540ac3
The main issue is that the bullets from dd/dt were not stored on the stack. I
added a separate field for it in each stack entry, which now fixes the basic
indent case without (afaik) breaking anything else.
There are still some newline issues, and the need to handle the single-line
dd/dt vs. the multi-line variant.
Change-Id: I65939c05e2c5dde0789bf8aefd7651161a2f137c
to update save dialog checkbox with correct watched state.
Call mw.page.watch.updateWatchLink onSave to refresh icon
with watched state.
Patchset 2- updated event name
Change-Id: I23ef1aad9c8ace13df1b9a6bf0bfeddb9d8bcb37
* Don't escape html-syntax pre content for now; Should parse this with a new
pre content production later (which needs to be split out of the regular pre
production in the tokenizer)
* Protect indent-pre content from start-of-line syntax escaping
* Preserve extra leading spaces in the tokenizer
* Two more (now 284) round-trip tests are passing
Change-Id: I199b89c0ee7fae12546df10c1b5117c97caccac5
Queued newlines and new trailing newlines were not cleanly separated so far,
which caused some trailing newlines to be consumed for needed leading
newlines. This change fixes several newline bugs, taking the number of passing
round-trip tests from 276 back up to 282.
Change-Id: Idb4706e15ce71e63085033e3f3f29557915c11a8
Fixed a bug in the list handler for multiple dds in a definition list. Also
fixed a few JSHint warnings.
Change-Id: I3e883786698a9521347fc2a5e6420646318813a7
Because of the end>=left condition, the loop was exiting right before
hitting the startBefore case, so use end>=left-1 instead.
Fixing this exposed another bug that caused nodeRange and nodeOuterRange
to be off by one: we need to increment left after storing it in
startOffset, not before
Change-Id: I54e18fb2119c8caefb4f7a7f2be43c6129afc4c0
Known issue: breaks round-tripping of :;;;::. That test is normally disabled
anyway, so we can fix it later.
Change-Id: I7954271311bfb7e71caae59d8177e3f04a9ebbca
This handles internal and external links separately, only setting
'title' for the former and only 'href' for the latter.
On the way out, detect external links using a simple regex (needs
improvement). Also don't write sHref any more per Gabriel's request.
Change-Id: Ibc2436d0a12de1d027e116f181db640cbcf3d522
* Started to add more complete tag source range (tsr) annotations to most
start / empty tags. These replace the old sourcePos and sourceTagPos
annotations, and look more promising for general round-tripping than block
source ranges (bsr). See
http://www.mediawiki.org/wiki/User:GWicke/Parsoid_source_ranges for some
notes on this.
* Added an escapeWikitext method in the serializer that tokenizes supposedly
text-only content from the DOM with the tokenizer and wraps runs of returned
non-text tokens into nowiki tags. The source corresponding to non-text
tokens is retrieved using the tsr annotations.
* Removed old (unused) table productions to avoid confusion.
* 276 round-trip tests are passing, vs. 283 without escaping.
Known issues:
* harmless for now, can be improved later: urllinks in external link captions
are wrapped in nowiki. Example HTML:
<a rel='mw:extLink' href="http://example.com">http://example2.com</a>
* some start-of-line syntax in wiki-syntax preformatted blocks might be
wrapped into nowiki when that would not really be needed. Example HTML DOM:
<pre>
* foo
* bar
</pre>
Change-Id: I01c34aedd5c566614d36924add47a6a960e91987
Now that the message keys are showing in the demo we can see how bad things look if the drop-down label is too big and wraps
Change-Id: I776a7e480a3f6240c0929f2f50a865c2de4ec0a5
Various parts of DM and CE choke on completely empty documents, so
return an empty paragraph instead.
Change-Id: I67062b66a44efe53a1bdaf60907653f0cc55dd25
Made edit tabs rendered on non view pages work correctly by routing them to the view page with an extra param that auto-initializes the editor
Change-Id: I4fd9106c8b45c6fc79af9ccb44e18944e9b9d8b9
* Added a newlineTransparent flag to handlers that prevents changes to the
onNewline status, so that content following it is still considered to be in
start-of-line context. This fixes a few rt tests where a comment or nowiki
tag is at the start of the line, and following content should end up on the
same line.
* 283 rt parser tests are now passing.
Change-Id: Ie58dcb9e5e9af9000fff61c2e1db5d8649ffc3f6
This solves lots of issues in the integration work, and also makes it much easier to extend this class to integrate it into other skins
Change-Id: I3b3c5b22a5664e6cf37e429cc0ac3be2e75b630f
* tokens are not modified any more (they are supposed to be immutable)
* handler info is now split in start / end objects and potentially a 'make'
method; added more flags to govern the newline behavior of different tags
* added a generic singleLine mode for single-line syntactical environments
* switched the web service to line-based diffs to avoid issues when diffing
the round-trip results of [[:en:Programming language]]
* 280 round-trip tests are passing now
Change-Id: I74b8ffbf69643c5d6e5ec852ec58e680c9018901
HTML5 defines space characters as [ \r\n\t\f] in
http://www.whatwg.org/specs/web-apps/current-work/multipage/common-microsyntaxes.html#space-character.
It treats these specially in a few contexts. As an example, the foster
parenting algorithm does not apply to space characters.
As a result, this change fixes the round-tripping of spaces between table
tags, which were previously moved before the table.
Change-Id: I32ab29275a9f824fc66d8286638eb42748cfc9a5
from Parsoid HTML output as well as VE HTML output. There are still
some newline related failures from parser tests that needs fixing, but
this is getting close. So committing for now so other eyes can make the
bugs shallow :).
Change-Id: Ia6a218ee9fb3e18fe0573c89ff3a4236779e1e64
- Check if href for links has the wgScriptPath prefix before
attempting to strip it from the href.
Change-Id: I844151ef7317476668d1306b96a2aec5a56fd0f1
- Something like this:
<ul><li>1</li><li>2<ul><li>2.1</li><li>2.2<ul><li>2.2.1</li><li>2.2.2</li></ul></li><li>2.3</li></ul></li><li>3</li></ul>
now serializes properly to:
*1
*2
**2.1
**2.2
***2.2.1
***2.2.2
**2.3
*3
So does this form which is what the above wikitext parses to:
<ul><li>1
</li><li>2
<ul><li>2.1
</li><li>2.2
<ul><li>2.2.1
</li><li>2.2.2
</li></ul></li><li>2.3
</li></ul></li><li>3
</li></ul>
- Lists (and nested lists) are not entirely newline-insensitive.
They still depend on newlines *between* lists. The opening
<ul> tag for non-nested lists should always start on a new line.
So, for example,
<ul><li>foo</li></ul><ul><li>bar</li></ul>
will serialize to:
*foo
*bar
which is incorrect. But,
<ul><li>foo</li></ul>
<ul><li>bar</li></ul>
will correctly serialize to:
*foo
*bar
Change-Id: I13a0290368574865957bcf57aebab488fbbb7026
- More pieces are now simplified and all(?) newline handling
is now centralized in the serializeToken function.
- This commit fixes bugs in rt-ing some code snippets
----------
Ex 1: foo<p>bar</p>baz
----------
- This commit fixes bugs serializing VE generated html
----------
Ex 2: <p>foo</p><pre>bar</pre> ==> foo\n bar
----------
- But, this round of fixes introduces RT failures for certain
code examples in parserTests.txt. In all these failing cases,
inline text/html is embedded within a generated <p> tag during
parsing. If these generated <p> tags can have a "gc:1" attribute
added to them, we can properly serialize them to the original
form.
----------
Ex 3: foo<pre>bar</pre>
Parsed HTML: <p>foo</p><pre>bar</pre>
----------
Note how this parsed HTML is identical to what the VE outputs
in Example 2 above. So, without the gc:1 attribute, we now
have conflicting requirements on the example same HTML.
This increases confidence in the correctness of my commit here.
Change-Id: I86beadec91c445a7f8a6d36a639b406697daa0a2
- Eliminated newline handling from several places in code and
mostly isolated it to serializeToken thus simplifying newline
handling logic.
- Fixing some bugs in the process: # of green roundtrip tests
went up by 5 (294 --> 299) but actually introduced failures on
a few originally succeeding tests (additional leading/trailing
newlines on the entire test output).
- Added bonus: made list serializing (mostly) insensitive to
newlines between tags. So, all the following DOM serialize
identically to the following wikitext:
*foo
*bar
----------
<ul><li>foo</li><li>bar</li></ul>
----------
<ul>
<li>foo</li>
<li>bar</li>
</ul>
----------
<ul>
<li>
foo
</li>
<li>
bar</li>
</ul>
----------
Change-Id: I76be56c4b2789039dff5f47de4659746882e45d6
HTML5 defines space characters as [ \r\n\t\f] in
http://www.whatwg.org/specs/web-apps/current-work/multipage/common-microsyntaxes.html#space-character.
It treats these specially in a few contexts. As an example, the foster
parenting algorithm does not apply to space characters.
As a result, this change fixes the round-tripping of spaces between table
tags, which were previously moved before the table.
Change-Id: I32ab29275a9f824fc66d8286638eb42748cfc9a5
from Parsoid HTML output as well as VE HTML output. There are still
some newline related failures from parser tests that needs fixing, but
this is getting close. So committing for now so other eyes can make the
bugs shallow :).
Change-Id: Ia6a218ee9fb3e18fe0573c89ff3a4236779e1e64
* Parsoid outputs bare newlines after a heading unless it's followed by
a <p>, so strip leading and trailing newlines in all bare text
* Adding a leading newline in <p>s is only needed if preceded by a
heading, don't add it otherwise
* Headings need a bare newline after them unless followed by a <p>
* Headings also need a bare newline before them if preceded by a <pre>
Change-Id: Ib02f800b26453541604e920fbb3845c51cdc6dea
This strips certain newlines added by Parsoid so they don't end up in
the linear model, and puts them back in on the way out so Parsoid
doesn't freak out and produce invalid wikitext
Change-Id: I256aaded4229c915868dc868ec6eaa1a73e00be1
I know this code is still being worked on but I felt like I should put
this in anyway, it might save the person working on it some work
Change-Id: I1535399b3798cd8de2fc5334cd1eac64b71e8821
This is needed because there are onTransact event handlers that use the
selection and expect it to be up-to-date. The previous behavior caused a
bug when pressing backspace at the end of the document, because the old
selection (at the end) was invalid in the context of the updated
document.
Change-Id: I159e37894d14d437f46495604c14804c0a13e84e
Empty annotation objects are unexpected by the rest of the data model
and cause weird breakage in the converter, resulting in inserted text
being in its own paragraph
Change-Id: I63de37c3c5e19ac650e7c7f2d1a0bfab21d45da9
I forgot to adjust a range based on this.cursor for this.adjustment .
This indirectly caused Rob to get an exception when trying to wrap
the last node in the document, because the unadjusted range was past the
end of the document.
Change-Id: If9d5b76568fae25ba2c0f405f1c4fcdd8d879e4f
- Check if href for links has the wgScriptPath prefix before
attempting to strip it from the href.
Change-Id: I844151ef7317476668d1306b96a2aec5a56fd0f1
