- Added limits to toolbar float, Toolbar will not go past the last node in editor.
- Added bottom mode to allow toolbar to stick above the last node until the scroll position
is above the last node.
- Actually checking toolbar config now and setting float when flag is set.
- Gave float method for top toolbar a better name.
Change-Id: Ic39c5402fa7a05e13c5e81722d8729d93776d7e9
But still put slugs before them. Done by overriding canHaveSlugAfter()
in ve.ce.ListNode.
Eventually this should be configurable and MediaWiki-specific
Change-Id: I5ad15ca4085a2d730add4954acbea358819b3986
These determine whether a node can have a slug before and after,
respectively. The default implementation in ve.ce.Node is to use the
same rules for both, but individual node types can override this.
I'll need this to suppress slugs after nested lists but not before them.
Change-Id: Id88c0fc98aca7c7f52ce990ed9b8c42181ef6d18
For pressing Enter in an empty list item at the end of a top-level list,
this has the same result as the previous code, but if you're in a nested
list it has the effect of jumping down a level. A previous incarnation
of this change just made Enter insert more list items ad infinitum if
you were in a nested list, but I think this is better.
This fixes a bug where pressing at the end of a nested list inserted a
paragraph in an invalid location
Change-Id: I9c7dbaf29a98f84926ed3a05e71c6294926dfce2
Fix the commented-out code: it caused unindent to be triggered by just
pressing Shift. ASCII 16 is "data link escape", no idea where that came
from, so I removed it and used e.shiftKey instead.
Also check whether indent/outdent is even possible before doing it.
Currently this is done in a very hacky way (by checking the state of the
indent button), ideally we'd refactor things such that toolbar tools can
listen for keydowns and intercept them, that would make the code much
cleaner and we wouldn't have this problem.
Change-Id: I99885ee4b8a79cd24c4958c188addfc2b0453b03
After indenting or outdenting a list item in a numbered list, the
numbering wasn't updated. So if you had:
1. One
2. Two
3. Three
and you indented "Two", you'd get:
1. One
1. Two
3. Three
Adding or removing items in the list using the keyboard, or even
inspecting the list in Firebug (!), would trigger a renumbering and fix
the list to display "2. Three". But then the same issue would occur in
in reverse when outdenting "Two" (either using undo or using the
outdent button):
1. One
2. Two
2. Three
The workaround is to force a reflow by requesting the height (thank you
Timo). Implemented this in an override of onSplice() in ve.ce.ListNode, so
the list is detached and reattached every time children are spliced into
or out of it.
I haven't managed to come up with a minimal test case for this, not even
by putting a list in a contentEditable div and doing the same DOM
operations that ve.ce does from a setTimeout callback.
Change-Id: I93b2a309034c411a7b4e4b6c6bd4ef9d473999eb
This works pretty well, the only problems I found are:
* when selecting multiple list items, only the first is in/outdented
* there's no special handling for child lists, so the behavior for
in/outdenting list items that have a child list can look weird, but
it's consistent
Also needs more documentation
Change-Id: I6f4f3725e57a590196d7d638a77b87ea85586dc8
* When ascending back up the stack, check for a start between two closings
* Also check for an end between two openings
* This introduces code duplication but selectNodes() is full of that
already. I'll have to do a duplication cleanup soon
* Add test case for </li><li>
* Update existing test case that covered a </li>
Change-Id: Ifc80585ce0e0d6988bc54228602c69f0d519200a
For nested lists, this function would return multiple groups where one
was wholly contained in the other, use offsets to prevent that from
happening.
Change-Id: Ib03bb1c81712d805cc263c2975cc3942de63d2ed
to be added. Create inspector elements in the propper document
scope. Restore inspector css classnames to camel case for proof
that inspectors are being created in the correct document scope.
Previously, inspector elements created in the wrong document scope
would have css rules applied only if class names were lowercase.
Issue only surfaced in Webkit browsers. Though, this implementation
is more future proof and will help prevent future inspector bugs.
Patch 3) Fixed global variable definition and mistake with
inspectorDoc
Change-Id: I36c0d078aea10d919689768878004a19f7f89b55
-Selection of part of a link now modifies selection to entire link
range on inspection.
-Retaining selection direction on new range
Only partial fix to bug as previous link annotation is not
yet properly cleared.
Bug 33053 - VisualEditor: Link creation should not include trailing
spaces, and should provide a suggestion based on selected text
-Created method to return a new range without outer spaces.
-Retaining selection direction on new range.
-Enhancement needed for link suggestion.
Bug 33108 - VisualEditor: Highlighted trailing whitespace should
not have styles applied
-Modified trim method to retain selection, added call to trim
range on annotate method.
Change-Id: I92f264e19350c62b7c2ac3cd9e78af0071afef5c
Now that we have access to the contents we can more easily compare the content
with link targets. This is still to do- this commit only converts the link
handler to work on the collected tokens.
* Start to implement latest RDFa spec from
http://www.mediawiki.org/wiki/Parsoid/RDFa_vocabulary
* Capitalize types, add mw:Entity type for html entities
* Detect changes to entities using tokenCollector and srcContent
Change-Id: I45429f4b930858a16e166ef8377c8f6f5114c414
This license change is aimed at maximizing the reusability of this code
in other projects. VisualEditor is more than just an awesome editor for
MediaWiki, it's the new editor for the entire internet.
Added license and author files, plus mentions of the license to all
VisualEditor PHP, JavaScript and CSS files. Parser files have not been
modified but are effectively re-licensed since there's no overriding
license information. 3rd party libraries are not changed, but are all
already MIT licensed.
Change-Id: I895b256325db7c8689756edab34523de4418b0f2
* "onevar" warning sometimes solved by just merging var statements
other times solved by making it a function declaration instead
of a function expression.
* Also fixed several '_this' variable names in ve.es.Surface to
more descriptive names, and enabled warnings for dangling _
in identifiers.
Change-Id: I7d411881e3e06cf9a7fe56d689c29375881a81de
This is just to avoid re-licensing along with VE. We want to be compatible
with MediaWiki core to make sure a closely-integrated C port is still
GPL-compatible. We could consider adding MIT to the JS implementation after
porting to C.
Change-Id: Ia83e8620e26c95625793438c4c5e8ddcf2702368
This is work in progress, but committed for now so I can use it for links and
tweak it while doing so.
Change-Id: I757277f6efacda6d9432ca57542a957f597a98de
* This code change is an attempt to address the FIXME about constant
resorting of transformations in _getTransforms. This caches sorted
transformations and selectively clears/updates the cache on add/remove.
Change-Id: If24a807b84d494aa4e5597339039a5573a30905e
This hopefully makes it clearer that data-rt contains private round-trip info
instead of semantically interesting data.
Change-Id: I03b476ed112a4b627c9871ee3677c450f943429a
* Arbitrary predicate support for the termination of collection mode
* tokens as property of the collector instead of a state-global thing
Change-Id: Ibcb342bc64a76fece9b04a760ea56c7878e67cad
* Fixed image serializer to deal with missing 'v' value in a k-v pair
representing an image attribute.
* Added fix to deal with bare <li>'s (without surrounding <ul> tags)
NOTE: The second fix is required currently to deal with bugs in the parser
as it deals with complex cases. But, in the future, we could deal with
this in one of the following ways:
(a) The serializer expects a well-formed DOM and all cleanup will be
done as part of external tools/passes.
(b) The serializer supports a small set of exceptional cases and bare
list items could be one of them
(c) The serializer ought to handle any DOM that is thrown at it.
Yet to be resolved.
Change-Id: Ib585e5c9f2a8a80854740ce0211bde705f9fd6f4
This makes it clear that setInterwiki can modify existing mappings, and adds a
similar method for the removal of existing mappings.
Change-Id: Ic603a4b2ccec35d086513fa7cf711702bfb2baa0
* Strips the first paragraph tag in a list item or table cell context
if there are no attributes on it and stx:html is not set
Change-Id: I74988645fe505c662f86488e32d0f11d464ffe41
* Looks like I misled myself in commit 88fc91 -- that wikitext
roundtripped perfectly because it went through the 'src' route
because it was a thumbnail with an explicit image which doesn't
go through renderThumb -- so, the serializer simply spit out the
original 'src' string and hence perfect rt :).
* More whitespace preserving fixes in LinkHandler.
* Also changed resource value in the img tag to use the original
filename rather than the normalized capitalized filename.
* 2 more parsertests rt -- now upto 400.
Change-Id: I144a6486dd9d07da8a74a68700fe96c78d192826
* Something to be said for code alignment - easier on the eye!
* Maybe a good case for breaking mediawiki coding guidelines.
* But, happy to abandon commit if not useful. :)
Change-Id: I1133af488f572ac7f8727be9108e08e14c4e6420
* Changed PrefixImageOptions so that thumb and thumbnail are
distinct key-value pairs. Without this fix, cannot distinguish
between thumb=foo.jpg and thumbnail=foo.jpg
* Fixed link handler so whitespace is preserved around prefixed image
options.
* Fixed figure handler to process the 3 different kind of image options:
size, simple image options, and prefixed image options.
* There is a hack/fixme for "upright: aspect" prefixed image option
which needs to be looked into.
* Still need to fix uppercasing of the image resource name.
With these fixes, the following wikitext roundtrips perfectly
(after newline breaks are removed)
[[Image:Foo.jpg|thumbnail = 'baby.jpg'|100x100px|center| alt =bbbbb|
upright=true|bottom|link='http://foo.bar'|
This is a [[Linked Caption]] in the image]]
Change-Id: I6606df56874c2b97f00f08cb6bbeaec9878167d3
* For now, extracted image markup options out of the link handler.
* This info will also be used by the serializer.
* More properties/global constants can be moved into this structure
over time.
Change-Id: I4cfbfd703f42e93fbad52b38b435f68d8a5c22ee
* Minor refactoring
* Cleared src in dataAttribs in renderThumb since we can serialize
thumbs now (or at least we can once all bugs are fixed and missing
pieces are handled).
Change-Id: If18865801cdd3d89c1477e68bfa3e13107c45b40
Anything with data-gen="both" and dataAttribs.src defined serializes to
dataAttribs.src and drops its contents (if any). We can use this to round-trip
elements we don't properly parse or serialize yet. Without RDFa info, the
editor will not touch the contents after encountering data-gen="both".
Change-Id: Ia39e5fdd765c2c9b36f26313455685d29f118839
* Don't consider them for auto-numbered links
* Don't insert a trailing space if the content is empty
These links are still wrapped in nowiki on round-tripping since the
valid/invalid url determination is done in the LinkHandler and not the
Tokenizer as it is configuration-dependent. Not incorrect for rendering (and
perhaps easier to understand for humans too), but might still introduce a
dirty diff. We'll still need reconciliation / damage tracking in the end ;)
Change-Id: I959ebc1b7f81d110a1141bb38ba5ee97f52ebf96
This only applies to newly created headings, so headings with a single newline
preceding them will be round-tripped that way.
Change-Id: Ic09972bbd25c3934b53f6fd3b5be5a0c3185c2af
* Collect all figure tokens and process them as a chunk
* This effectively mimics context-sensitive DOM walking,
but since we need serialization supported on a token stream,
we cannot use real DOM walking. The current technique should
also work on a token stream.
* There is a FIXME about the image filename being capitalized.
This needs fixing in the parser or some other way of recognizing
original unnormalized filenam.
Amended by gwicke:
* Build option list and join it with pipe to avoid stray trailing pipe
* Satisfy JSHint's weird preference to have '&&' and '||' at the end of the line
Change-Id: I1e5f6600f297fcdf81e3227a82ca3b71d4e97fc3
This is a zero-length tsr for now (and thus not 100% correct), but will do the
job for starttag / endtag range establishment
Change-Id: Iedd50ad319aa8d5916434fb6744deb04e031e456
* Removed dead commented out code.
* Cleaned up newline handling in serializer some more.
* Now, onNewLine and onStartOfLine reflect serializer state
more accurately.
* No implicit new lines for explicit html tags.
* 9 more roundtrip tests now green.
Change-Id: I9f640de2ae769c7472538fa687400dc8a40c2b2d
297 round-trip tests are passing with this patch.
TODO:
* generalize data-mw-gc handling in the serializer for any tag
* use data-mw-gc="both" and data-mw.src: 'the wikitext' for round-tripping of
wikitext structures, optionally with some presentational (but read-only)
content
* use span and data-mw-gc="both" for nowiki
Change-Id: I700142a56818977c20c8c06e6a5f2e77a708d25e
This makes sure that we escape start-of-line syntax when needed, since
onNewline is often not yet set.
Discussion / background:
[19:18] <subbu> this will fix it, but, i think this is asking for another
minor refactoring of these flags ... because this is a subtle fix which means
it might be possible to make it clearer. onNewline is one true in on
direction, i.e. if true, we are in a new line state, but if we are in a
newline context, onNewline is not true, which is why this new method is
needed.
[19:19] <subbu> i dont know if it is possible, but it seems like it shoudl be
possible. but, something for later.
[19:20] <subbu> badly phraed. "onNewline" ==> in new line context, but if in
new line context, onNewline may be false.
[19:20] <gwicke> we should perhaps update it as early as possible instead
[19:21] <subbu> i cannot today, but possible monday. i am heading out in
about 15-30 mins.
[19:22] <gwicke> will need to check all conditions depending on it in
_serializeToken
[19:22] <subbu> oh, i misunderstood you :)
[19:22] <gwicke> and if there are cases where the onNewline / onStartOfLine
state could be reverted later
[19:23] <subbu> you were referring to the flag, i thought you meant we should
fix this sooner than later.
[19:23] <gwicke> yes, I wasn't terribly clear
[19:23] <gwicke> you wrote something about following productions swallowing
newlines, but I think we don't actually do that any more
[19:24] <gwicke> I'm quite optimistic that updating those flags much earlier
would work
[19:25] <subbu> yes, it could fix it.
[19:26] <subbu> you might be right reg. swallowing. it was happening earlier.
but, not right now, after single-line mode and other fixes.
Change-Id: Ic1d8141c04eb54a59977d0ba87bcf06bafd421e0
This should not really be needed if the tokenizer did not decode html entities
on the fly. It is still a quick way to make sure no htmlish content can be
inserted even with the current decoding.
The next step and proper fix is to make entity decoding either optional in the
tokenizer (flag-controlled), or move it to a later stage in the token
processing pipeline.
Change-Id: Ife093dcfb95113763dab5635b098c795d3550586
* Renamed defaultOptions to initialState
* Got rid of unused state property
* Added comments explaining how state attributes
and tag handler flags are used
* Refactored listItemHandler check into functions and
added FIXME possible rewriting of that check.
* Protected serializeDOM in a try-catch handler to
catch exceptions and output the exception to the console.
Change-Id: I3d351c06e4b86baeb5a55243b11dbfa9baca5bb7
This fixes a bug Trevor reported where selecting from a list item across
a heading and into a paragraph, pressing backspace, then clicking undo
caused an exception.
Change-Id: Id2851271529e10548f6979a030a198054aa1c48f
ve.ce.TextNode listed textStyle annotations that didn't actually exist,
and failed to recognize some that did exist (such as span; bug 37808).
Added all annotations to both places. <span> tags are now tolerated by
the editor in that it doesn't crash anymore, but they're displayed (and
saved!) without any attributes, so <span style="color:yellow;">y</span>
doesn't show a yellow 'y' in the editor and is saved back as
<span>y</span> .
Change-Id: Iaae11ad5044150fa904010983ff83579cb37733d
Fixed by adding the specialMessages module which is only loaded once the
editor loads. Then after it's loaded we use the summary message from
there to update the (possibly broken) summary message in the save
dialog.
Change-Id: I67f5c59501cdf7c66c925cef8d4dd42b0f2cfde3
* Removed murky ' :' -> ' :' replacement in tokenizer. This breaks four
parser tests, and should be fixed in a token stream transformer or DOM
postprocessor. This replacement clashes with round-tripping, and is not
terribly important visually.
* Added stx:row annotation to single-line dt/dd pairs and use it to preserve
single-line syntax in the serializer. There is no attempt yet to support the
addition of nested lists in an originally single-line dd. We'd need to look
ahead in the serializer to support this. Perhaps the editor can simply drop
data-mw in that case.
* Switched default dt/dd serialization to multi-line. This supports all nested
lists and multiple dds.
* Don't close dls when switching from dt to dd or back in the token stream
ListHandler.
Overall 290 round-trip tests are passing now (up from 284, some due to ,
some due to lists). The number of passing parser tests dropped slightly from
303 to 297 (or 301/295 on weekdays other than Thursday).
Change-Id: I85ff40571833713388c6523e6a4ba2e94daa3807
Basically only prefix all bullets if the serialization output is going to be
in start-of-line context. The test for that is currently inline, but should
perhaps be factored out to a method or state flag instead.
We could alternatively consider to return the start-of-line prefix and let it
be used in _serializeToken in case we end up in start-of-line context.
This patch also fixes a newline issue on input like this:
:d1
::: d3
Both the list and list item handlers now set the startsNewline flag
dynamically depending on the context, so that we don't depend on the
suppression of newlines from list syntax by the singleLineMode any more.
There is still an extra newline inserted between list items in the following
example:
;t1 :d1
;;t2 ::d2
This looks like a bug in the produced DOM and not in the serializer, since the
outer definition list is closed and re-opened between d1 and t2.
Change-Id: I78e3a1ef34cf9159d5a1e86fb64c774ff111e71d
* changes:
Got rid of iteration to get the surface
Removed attach and detach methods from ve.ce.Node
Track adjustments in DocumentSynchronizer and apply them to oldRange
This is needed because oldRange is relative to the state of the model before any changes were made, but when we call selectNodes() it's gonna operate on a partially updated model tree.
This is a genuine bug in DocumentSynchronizer proper, which means I owe the entire team lunch
Change-Id: Ia6510de19df02e961c7f25fb8e7833abceb8d25b
* Adjust both start and end for preceding operations
* Adjust end for the current operation as well
Change-Id: I2f96d609bddf3788aa5700ad1f0b46208f3517d7
text for a link is the selected text.
patchset 2 - add case for if data offset is an object, be sure it
is a string prior to adding it. truncate to 255 chars.
patchset 3 - actually add the patched file
Change-Id: Ibddf942c2a0ba3412d93cf9730f74eb858025fad
Also fixed unlisting to not break when unlisting a range that includes things that are not list nodes
Change-Id: Ib9d4ea851c3ed9bf72a93aa87e470ce40c308453
contained by a list node the button is off. Button is now only toggled on
if all nodes in selection have a parent list node.
Patchset 2 - cleanup whitespace
Change-Id: Ia9adc39c0d5c75e2e96580f0e172f5b602540ac3
The main issue is that the bullets from dd/dt were not stored on the stack. I
added a separate field for it in each stack entry, which now fixes the basic
indent case without (afaik) breaking anything else.
There are still some newline issues, and the need to handle the single-line
dd/dt vs. the multi-line variant.
Change-Id: I65939c05e2c5dde0789bf8aefd7651161a2f137c
to update save dialog checkbox with correct watched state.
Call mw.page.watch.updateWatchLink onSave to refresh icon
with watched state.
Patchset 2- updated event name
Change-Id: I23ef1aad9c8ace13df1b9a6bf0bfeddb9d8bcb37
* Don't escape html-syntax pre content for now; Should parse this with a new
pre content production later (which needs to be split out of the regular pre
production in the tokenizer)
* Protect indent-pre content from start-of-line syntax escaping
* Preserve extra leading spaces in the tokenizer
* Two more (now 284) round-trip tests are passing
Change-Id: I199b89c0ee7fae12546df10c1b5117c97caccac5
Queued newlines and new trailing newlines were not cleanly separated so far,
which caused some trailing newlines to be consumed for needed leading
newlines. This change fixes several newline bugs, taking the number of passing
round-trip tests from 276 back up to 282.
Change-Id: Idb4706e15ce71e63085033e3f3f29557915c11a8
Fixed a bug in the list handler for multiple dds in a definition list. Also
fixed a few JSHint warnings.
Change-Id: I3e883786698a9521347fc2a5e6420646318813a7
Because of the end>=left condition, the loop was exiting right before
hitting the startBefore case, so use end>=left-1 instead.
Fixing this exposed another bug that caused nodeRange and nodeOuterRange
to be off by one: we need to increment left after storing it in
startOffset, not before
Change-Id: I54e18fb2119c8caefb4f7a7f2be43c6129afc4c0
Known issue: breaks round-tripping of :;;;::. That test is normally disabled
anyway, so we can fix it later.
Change-Id: I7954271311bfb7e71caae59d8177e3f04a9ebbca
This handles internal and external links separately, only setting
'title' for the former and only 'href' for the latter.
On the way out, detect external links using a simple regex (needs
improvement). Also don't write sHref any more per Gabriel's request.
Change-Id: Ibc2436d0a12de1d027e116f181db640cbcf3d522
* Started to add more complete tag source range (tsr) annotations to most
start / empty tags. These replace the old sourcePos and sourceTagPos
annotations, and look more promising for general round-tripping than block
source ranges (bsr). See
http://www.mediawiki.org/wiki/User:GWicke/Parsoid_source_ranges for some
notes on this.
* Added an escapeWikitext method in the serializer that tokenizes supposedly
text-only content from the DOM with the tokenizer and wraps runs of returned
non-text tokens into nowiki tags. The source corresponding to non-text
tokens is retrieved using the tsr annotations.
* Removed old (unused) table productions to avoid confusion.
* 276 round-trip tests are passing, vs. 283 without escaping.
Known issues:
* harmless for now, can be improved later: urllinks in external link captions
are wrapped in nowiki. Example HTML:
<a rel='mw:extLink' href="http://example.com">http://example2.com</a>
* some start-of-line syntax in wiki-syntax preformatted blocks might be
wrapped into nowiki when that would not really be needed. Example HTML DOM:
<pre>
* foo
* bar
</pre>
Change-Id: I01c34aedd5c566614d36924add47a6a960e91987
Now that the message keys are showing in the demo we can see how bad things look if the drop-down label is too big and wraps
Change-Id: I776a7e480a3f6240c0929f2f50a865c2de4ec0a5
Various parts of DM and CE choke on completely empty documents, so
return an empty paragraph instead.
Change-Id: I67062b66a44efe53a1bdaf60907653f0cc55dd25
Made edit tabs rendered on non view pages work correctly by routing them to the view page with an extra param that auto-initializes the editor
Change-Id: I4fd9106c8b45c6fc79af9ccb44e18944e9b9d8b9
* Added a newlineTransparent flag to handlers that prevents changes to the
onNewline status, so that content following it is still considered to be in
start-of-line context. This fixes a few rt tests where a comment or nowiki
tag is at the start of the line, and following content should end up on the
same line.
* 283 rt parser tests are now passing.
Change-Id: Ie58dcb9e5e9af9000fff61c2e1db5d8649ffc3f6
This solves lots of issues in the integration work, and also makes it much easier to extend this class to integrate it into other skins
Change-Id: I3b3c5b22a5664e6cf37e429cc0ac3be2e75b630f
* tokens are not modified any more (they are supposed to be immutable)
* handler info is now split in start / end objects and potentially a 'make'
method; added more flags to govern the newline behavior of different tags
* added a generic singleLine mode for single-line syntactical environments
* switched the web service to line-based diffs to avoid issues when diffing
the round-trip results of [[:en:Programming language]]
* 280 round-trip tests are passing now
Change-Id: I74b8ffbf69643c5d6e5ec852ec58e680c9018901
HTML5 defines space characters as [ \r\n\t\f] in
http://www.whatwg.org/specs/web-apps/current-work/multipage/common-microsyntaxes.html#space-character.
It treats these specially in a few contexts. As an example, the foster
parenting algorithm does not apply to space characters.
As a result, this change fixes the round-tripping of spaces between table
tags, which were previously moved before the table.
Change-Id: I32ab29275a9f824fc66d8286638eb42748cfc9a5
from Parsoid HTML output as well as VE HTML output. There are still
some newline related failures from parser tests that needs fixing, but
this is getting close. So committing for now so other eyes can make the
bugs shallow :).
Change-Id: Ia6a218ee9fb3e18fe0573c89ff3a4236779e1e64
- Check if href for links has the wgScriptPath prefix before
attempting to strip it from the href.
Change-Id: I844151ef7317476668d1306b96a2aec5a56fd0f1
- Something like this:
<ul><li>1</li><li>2<ul><li>2.1</li><li>2.2<ul><li>2.2.1</li><li>2.2.2</li></ul></li><li>2.3</li></ul></li><li>3</li></ul>
now serializes properly to:
*1
*2
**2.1
**2.2
***2.2.1
***2.2.2
**2.3
*3
So does this form which is what the above wikitext parses to:
<ul><li>1
</li><li>2
<ul><li>2.1
</li><li>2.2
<ul><li>2.2.1
</li><li>2.2.2
</li></ul></li><li>2.3
</li></ul></li><li>3
</li></ul>
- Lists (and nested lists) are not entirely newline-insensitive.
They still depend on newlines *between* lists. The opening
<ul> tag for non-nested lists should always start on a new line.
So, for example,
<ul><li>foo</li></ul><ul><li>bar</li></ul>
will serialize to:
*foo
*bar
which is incorrect. But,
<ul><li>foo</li></ul>
<ul><li>bar</li></ul>
will correctly serialize to:
*foo
*bar
Change-Id: I13a0290368574865957bcf57aebab488fbbb7026
- More pieces are now simplified and all(?) newline handling
is now centralized in the serializeToken function.
- This commit fixes bugs in rt-ing some code snippets
----------
Ex 1: foo<p>bar</p>baz
----------
- This commit fixes bugs serializing VE generated html
----------
Ex 2: <p>foo</p><pre>bar</pre> ==> foo\n bar
----------
- But, this round of fixes introduces RT failures for certain
code examples in parserTests.txt. In all these failing cases,
inline text/html is embedded within a generated <p> tag during
parsing. If these generated <p> tags can have a "gc:1" attribute
added to them, we can properly serialize them to the original
form.
----------
Ex 3: foo<pre>bar</pre>
Parsed HTML: <p>foo</p><pre>bar</pre>
----------
Note how this parsed HTML is identical to what the VE outputs
in Example 2 above. So, without the gc:1 attribute, we now
have conflicting requirements on the example same HTML.
This increases confidence in the correctness of my commit here.
Change-Id: I86beadec91c445a7f8a6d36a639b406697daa0a2
- Eliminated newline handling from several places in code and
mostly isolated it to serializeToken thus simplifying newline
handling logic.
- Fixing some bugs in the process: # of green roundtrip tests
went up by 5 (294 --> 299) but actually introduced failures on
a few originally succeeding tests (additional leading/trailing
newlines on the entire test output).
- Added bonus: made list serializing (mostly) insensitive to
newlines between tags. So, all the following DOM serialize
identically to the following wikitext:
*foo
*bar
----------
<ul><li>foo</li><li>bar</li></ul>
----------
<ul>
<li>foo</li>
<li>bar</li>
</ul>
----------
<ul>
<li>
foo
</li>
<li>
bar</li>
</ul>
----------
Change-Id: I76be56c4b2789039dff5f47de4659746882e45d6
HTML5 defines space characters as [ \r\n\t\f] in
http://www.whatwg.org/specs/web-apps/current-work/multipage/common-microsyntaxes.html#space-character.
It treats these specially in a few contexts. As an example, the foster
parenting algorithm does not apply to space characters.
As a result, this change fixes the round-tripping of spaces between table
tags, which were previously moved before the table.
Change-Id: I32ab29275a9f824fc66d8286638eb42748cfc9a5
from Parsoid HTML output as well as VE HTML output. There are still
some newline related failures from parser tests that needs fixing, but
this is getting close. So committing for now so other eyes can make the
bugs shallow :).
Change-Id: Ia6a218ee9fb3e18fe0573c89ff3a4236779e1e64
* Parsoid outputs bare newlines after a heading unless it's followed by
a <p>, so strip leading and trailing newlines in all bare text
* Adding a leading newline in <p>s is only needed if preceded by a
heading, don't add it otherwise
* Headings need a bare newline after them unless followed by a <p>
* Headings also need a bare newline before them if preceded by a <pre>
Change-Id: Ib02f800b26453541604e920fbb3845c51cdc6dea
This strips certain newlines added by Parsoid so they don't end up in
the linear model, and puts them back in on the way out so Parsoid
doesn't freak out and produce invalid wikitext
Change-Id: I256aaded4229c915868dc868ec6eaa1a73e00be1
I know this code is still being worked on but I felt like I should put
this in anyway, it might save the person working on it some work
Change-Id: I1535399b3798cd8de2fc5334cd1eac64b71e8821
This is needed because there are onTransact event handlers that use the
selection and expect it to be up-to-date. The previous behavior caused a
bug when pressing backspace at the end of the document, because the old
selection (at the end) was invalid in the context of the updated
document.
Change-Id: I159e37894d14d437f46495604c14804c0a13e84e
Empty annotation objects are unexpected by the rest of the data model
and cause weird breakage in the converter, resulting in inserted text
being in its own paragraph
Change-Id: I63de37c3c5e19ac650e7c7f2d1a0bfab21d45da9
I forgot to adjust a range based on this.cursor for this.adjustment .
This indirectly caused Rob to get an exception when trying to wrap
the last node in the document, because the unadjusted range was past the
end of the document.
Change-Id: If9d5b76568fae25ba2c0f405f1c4fcdd8d879e4f
- Check if href for links has the wgScriptPath prefix before
attempting to strip it from the href.
Change-Id: I844151ef7317476668d1306b96a2aec5a56fd0f1
This fixed a bug where an <h2> element with HTML attributes would be
converted to a 'heading' element with those HTML attributes but without
the 'level' attribute, which indirectly caused an exception somewhere in
ve.ce
Change-Id: I8bf32ff0d8e0f9d016b2abc6cb31824df05bdfc2
- Something like this:
<ul><li>1</li><li>2<ul><li>2.1</li><li>2.2<ul><li>2.2.1</li><li>2.2.2</li></ul></li><li>2.3</li></ul></li><li>3</li></ul>
now serializes properly to:
*1
*2
**2.1
**2.2
***2.2.1
***2.2.2
**2.3
*3
So does this form which is what the above wikitext parses to:
<ul><li>1
</li><li>2
<ul><li>2.1
</li><li>2.2
<ul><li>2.2.1
</li><li>2.2.2
</li></ul></li><li>2.3
</li></ul></li><li>3
</li></ul>
- Lists (and nested lists) are not entirely newline-insensitive.
They still depend on newlines *between* lists. The opening
<ul> tag for non-nested lists should always start on a new line.
So, for example,
<ul><li>foo</li></ul><ul><li>bar</li></ul>
will serialize to:
*foo
*bar
which is incorrect. But,
<ul><li>foo</li></ul>
<ul><li>bar</li></ul>
will correctly serialize to:
*foo
*bar
Change-Id: I13a0290368574865957bcf57aebab488fbbb7026
- More pieces are now simplified and all(?) newline handling
is now centralized in the serializeToken function.
- This commit fixes bugs in rt-ing some code snippets
----------
Ex 1: foo<p>bar</p>baz
----------
- This commit fixes bugs serializing VE generated html
----------
Ex 2: <p>foo</p><pre>bar</pre> ==> foo\n bar
----------
- But, this round of fixes introduces RT failures for certain
code examples in parserTests.txt. In all these failing cases,
inline text/html is embedded within a generated <p> tag during
parsing. If these generated <p> tags can have a "gc:1" attribute
added to them, we can properly serialize them to the original
form.
----------
Ex 3: foo<pre>bar</pre>
Parsed HTML: <p>foo</p><pre>bar</pre>
----------
Note how this parsed HTML is identical to what the VE outputs
in Example 2 above. So, without the gc:1 attribute, we now
have conflicting requirements on the example same HTML.
This increases confidence in the correctness of my commit here.
Change-Id: I86beadec91c445a7f8a6d36a639b406697daa0a2
Previously, data-mw-gc (generated content) elements were unconditionally
converted to alienInline nodes, and unrecognized elements were
unconditionally converted to alienBlock nodes. This is wrong and
produced weird results when I started experimenting with <code> tags.
Instead, I made both gc and unknown element trigger alienation, but the
decision of whether we generate an alienInline or an alienBlock node is
separate and is based only on whether we're inside a content node.
Change-Id: I12335337c3fa60c725ae7bcfbfb52a1dda153fb5
- Eliminated newline handling from several places in code and
mostly isolated it to serializeToken thus simplifying newline
handling logic.
- Fixing some bugs in the process: # of green roundtrip tests
went up by 5 (294 --> 299) but actually introduced failures on
a few originally succeeding tests (additional leading/trailing
newlines on the entire test output).
- Added bonus: made list serializing (mostly) insensitive to
newlines between tags. So, all the following DOM serialize
identically to the following wikitext:
*foo
*bar
----------
<ul><li>foo</li><li>bar</li></ul>
----------
<ul>
<li>foo</li>
<li>bar</li>
</ul>
----------
<ul>
<li>
foo
</li>
<li>
bar</li>
</ul>
----------
Change-Id: I76be56c4b2789039dff5f47de4659746882e45d6
Parsoid ignores sHref when converting back to wikitext, so we have to
set the href attribute to "/$title"
Change-Id: I1068116c0be72197619d0df3b4d1231a3879fa14
And made it not start on it's own, but be started by ve.Surface - this makes it so it's not polling in the unit tests, for instance
Change-Id: I940df04d392fd134d18847949efe0e2232328323
* As part of an earlier fix, I had changed default value of 'res'
to null instead of ''. But, this was potentially buggy because
the previous check was (res !== '') which could be triggered
by return values of handlers. By changing the check to null,
I was effectively changing the code paths for those handlers that
returned ''.
Change-Id: I2302023be7422ce4fb384ff5a50fe53fa7732855
Fix Inspector bug which prevented applying a link annotation to data
already containing annotations.
Change-Id: I6f315d50805c8c71f2155f955ea5674a7ce98656
This was causing some issues when you started typing there, and it's not clear that the white text is really the best way to go anyways
Change-Id: I8a9d6571ea204603729e96b7ff77184279a31a95
The offset map was broken from the start because it wasn't updated when
adjusting the length of a text node, and if we fix that bug it's
doubtful whether the costs of updating the offset map outweigh the
benefits, especially considering that adjusting the length of a text
node is something we do for almost every keypress. If it turns out
having an offset map does make sense, we can always reintroduce it
later.
Change-Id: I59e8bc154f7d07aa1bab2f473c13ff466d0e463f
paragraphs in lists.
* We need to look at other special-case handling requirements of
html tags in lists (and other contexts like tables).
Change-Id: I84b8402d90a186c9075c2d45263c94377312927a
Parsoid outputs rel="mw:wikiLink" or rel="mw:extLink", so we convert that to link/wikiLink and wiki/extLink respectively.
Also preserve the data-mw attribute; we probably need to do this more generally but this'll do for now.
Change-Id: I32e570bffa5a73a733a120d52cfd8b75d3191e02
Adding a 'style' attribute which is set to either 'data' or 'header'
This breaks even more tests because of missing style attributes
Change-Id: I0a75d8c1578b4414eeae8c484f6c4d6f8a59472a
When a text node was closed (either by encountering a non-text node or by reaching the end of the document), the fixupStack would not be processed. This led to attributes being dropped when nodes were split because of text being inserted into them
Change-Id: I41f6d20e0c1bfc8d8689b7e6325e724dd8156ab1
* Add code to handle elements and annotations
* Drop support for aliens from getDomElementFromDataElement() and move it into getDomFromData()
* Implement getDomElementFromDataAnnotation()
* Document a few functions
Change-Id: Ic6a418cbf9d7d1ad96299d7d3633970a876c6103
it does not include Alien nodes and Images - because we are not going to
support them for June release).
Change-Id: I229e4b5f2881714252699f23aef164655fa8bcf6
* Moved wikipedia default prefixes to environment
* Added 'addInterwiki' method
* Adjusted link handling normalizeTitle to reflect this
Change-Id: If5b2314cc36346b6da8649ed410457a612d80a22
* mw:Foo now loads pages from mediawiki.org
* The default prefix still is 'en'. You can switch this to 'mw' in ParserService.js.
Change-Id: I1208667e6114bd711b7988a8b3adb32ffab70969
- Three bugs that were messing up quote transformations.
- Now, the following cases are handled properly:
* ''foo'''
* '''foo''
* ''foo''''
* ''''foo''
These tests (and other quote tests) have to be added to core parser
tests file.
- One more parser test green.
Change-Id: I4f93e8910639f546bfc9304becab17d26d5529de
Created new document method to determine if a specific annotation
object is inside an annotation array.
Change-Id: Id645929cbf31030b8b0fcacb8dfb36e61aaad129
This fixes a bug where the second replace operation in a transaction
would cause the rebuild of the wrong range, or the adjustment of the
wrong text node.
Change-Id: I9b1c68d84999d538fe10bb193f4dfdd694121d2a
This is needed to make the results of certain transactions' tree sync
round-trip cleanly through the ve.dm.Document constructor
Change-Id: I2ab0758ec6bd7afba5b6645c7330f9fa2d45205d
new static method looks for annotation in annotation object.
ve2 Cleanup on annotate method and surface model.
Partially revive UI tools by exchaning old method usage
for ve2 methods.
Change-Id: Id0ac58330292d76801bbcf1d71a919b493f8ab9e
An improvement, but there still are some extra newlines inserted after
paragraphs. Example input:
-------
Foo:
{|
|foo
|}
-------
Extra newlines are inserted after the Foo: and the foo in the table. They are
not fed as tokens or text to the tree builder, so there is likely a bug in the
html5 library or JSDom.
Change-Id: I83eb6180e3cd1c4e7f9b15b31d339e1d32bccd3f
* Possibly more efficient under heavy GC load -- untested.
* No change in time and memory use for single file parsing.
Change-Id: Id2f3f65cc0e5f38ed968bbda60b97e46523e700e
* Moved the tail attribute to the second attribute (a bit cleaner)
* Disallowed newlines in the tail production
* Improved the selection of round-tripped href vs. generated content vs. href
in the serializer
* renamed state.linkTail to state.dropTail
Change-Id: I5d98c704b6ea566011e22237786f8da17548570f
Pages titles with a wikipedia interwiki prefix now load the page from
corresponding Wikipedia. Links in a page then stay within the given language.
Note that Parsoid currently makes no effort to recognize localized namespaces,
so it won't render media files, categories etc correctly.
Change-Id: I7bc4102e81a402772ea23231170734d580ea15b9
Functional changes (fixes):
* Make writeElement() also update parentNode and parentType for openings
* Also add to fixupStack when opening a wrapper for a text node
Non-functional changes (cleanup&docs):
* Document all variables at the beginning of the function
* Group variables according to where/how they're used
* Move expectedType into writeElement()
* Kill node, duplicates parentNode unnecessarily
* Kill paragraphOpened, was misnamed and unnecessary
* Rename closedElements to reopenElements
Change-Id: Ie5b4e4f30b267943048fdc170accb29139039192
* Push entire elements onto openingStack rather than type strings
* When closing an element, build a clone of the opening and push it onto
closedElements, then insert that clone when reopening the element
Change-Id: I8b0fb44394aed6c471dc6dacaab03e44c2333733
* Don't explicitly add the newline in the pre, as we preserve newline tokens
now. This avoids doubling of newlines when round-tripping.
* Use the sHref attribute even if the href contains spaces.
Change-Id: I8bec8fbfd6a7836bf2e5eec20869a0edd95c93b6
Lists interrupted by non-empty lines would not close the list properly.
Register for any token instead of just for newlines and close the list if no
listItem follows the newline.
Change-Id: I1743901e3db541bbeda78d17707db943e6ceb9b9
If the href would not denormalize, add a copy of the original href in data-mw
and use it to preserve non-conventional capitalization etc.
Change-Id: Ifef50eec7343b0e6b0ba66b6d19a8a3e8c9f8001
A tail containing regexp syntax (a ? in [[:en:Main Page]]) would crash the
serializer. Use substr instead.
Change-Id: I8519aec9c07dfe31893d676b1c936a42d2af74a0
- Added a tail json attribute for wikiLinks
- During serialization, this attribute is used to strip the tail from
the link target and render it after the link
[[hen]]s ==> <a ... data-mw="{gc:1, tail: 's'}" ...>hens</a>
==> [[hen]]s
- 2 more roundtrip tests green
Change-Id: I84f3dabaf0271f7a67641a00148467daa8310eb0
This allows us to check the watchlist checkbox on save dialog.
Added watchlist toggling to ve save api.
Added some i18n messages to core integration.
Change-Id: Ibed8edb2c59ad49e1738c937c3bea518238d0845
* The state of syntax stops is now properly included in the cache key for the
tokenizer-internal backtracking cache. This fixes some mis-parses when
re-parsing a bit of text with different flags.
* Clear the backtracking cache after each toplevelblock. This drops the peak
memory usage when expanding [[:en:Barack Obama]] from ~380M to ~110M.
Change-Id: Icdb879cae5907e4595903dd6acba2e686e8c2e4b
* Added converters to all relevant node implementations
* Added new annotation objects with their own factory
Change-Id: I9870d6d5eac45083929d74d2e58917d0939ca917
