* The initialization of nestedRefCollectionMap[references-id]
in handleReferences was the wrong place to initialize it since
on pages with multiple <references /> tags, the map can get
reset after a references tag is processed. So, for later
references tags, the entry for those will get cleared which
led to crashes.
Ex: es:Abderramán_I and couple others found in RT testing.
* This patch fixes the crasher.
* Added a new parser test with multiple references tags that
captures this scenario. Regenerated selser changes file.
Change-Id: I64402e77996486b099acda114f2e250258b0048e
* Track <ref>s birthed in <references> tags and capture their
HTML and set <references>.data-mw.body.html to the concatenated
HTML for all those child <ref> tags.
* No change in parser-tests because our test normalization code
strips data-mw before comparing results. Since this patch
only changes data-mw output for "References: 5. ..." test,
there is no change. In any case, this patch updates the
data-mw output for the test in question so that if we do
modify our normalization routines, the test will still pass.
Change-Id: Iaea752dec03e5e7f0d36baf3b7a9973f1b85e8a5
* This patch now supports any template that generates nested refs
where the top-level token is just a single-ref.
Ex: {{efn|New timetable{{sfn|Vallance|1991|p=31}}}}
* The solution in this patch removes the hack from the previous
version (53bbcbb3). As documented in the bug report, this
solution essentially "trusts" template authors and lets
nested refs through when they come from templates.
But, top-level nested refs are still not allowed.
<ref>foo<ref>bar</ref>baz</ref> in the toplevel page
will still generate a single reference with content
"foo<ref>bar</ref>baz"
* Updated the nested ref test output
- Regenerated selser changes because of the changed output.
- Regenerated blacklist because of the changed selser changes.
* This now handles:
- https://en.wikipedia.org/wiki/User:Edgepedia/VE/GNoSR
- https://en.wikipedia.org/wiki/Phellinus ellipsoideus
Change-Id: I627955f0be1c5e2bafc49647c94c2be68ce711a8
Problem:
* VisualEditor needs to be able to more reliably target Cite errors.
Solution:
* Add a more specific class to cite errors
Grumble:
* Ideally all extensions would use mw-ext-{extension name} as a prefix to
their CSS classes (or something similar).
Bug: 51337
Change-Id: If4f5360cc1c7b765ad896b14901a9b024782cc93
* There is another unrelated issue on that page ==> It has
a couple instances of <ref>text<ref> which slurps the entire
rest of the page into a 10K string (on which the regexp failed).
Change-Id: Ia4c498b28042de5bfe9b0945c0efaa8c6cb81ea0
* source and group attributes were being added as HTML attrs.
on the ol-node which is invalid. Plus, after refererences
generation, source attr was being deleted which causes crashers
in production if references output is reused from cache.
* Moved both attributes to data.parsoid
* Removed group="" from parserTests that I erroneously added
in that previous commit.
* Minor cleanup in DOMPostProcessor.
Change-Id: I85f4ad0a311d653d7ba3e010c7a5db028b643e9f
* So far, references wikitext was being replaced with a meta marker
token that participated as an inline token in p-wrapping. As a
result, in some cases, a p-tag was wrapping the references ol-node
which is buggy HTML. When VE loaded this DOM, the p-node was split
around the ol-node and duplicated DSR which led to duplicated
output when selser ran on this.
* One way of fixing this would have been to add a special case in
the paragraph wrapper to treat the mw:Extensions/References/Marker
meta tag as a block token.
* A better solution with an eye towards the longer-term is to emit
mw:DOMFragment wrapper for the references tag with the content
being an ol-node. The dom-fragment unpacking code then takes care
of p-wrapping issues for the ol-node. This fixes the bug in this
case.
* Also fixed parser tests output where group attribute was missing
on some tests (which indicated that the older code was not emitting
the group attribute on references tag).
* No change in parser tests.
Change-Id: I073b2e68667d577c75cad07d19cea2b19d0e89fe
* After trying various hacks, came up with a relatively simple
fix/hack to support nested refs.
The fix looks for {{#tag:ref..}} and short-circuits full
pipeline expansion and converts that to an extension tag
in place.
* Tested with the following wikitext which parses and RTs correctly
A <ref name='foo' />
B {{#tag:ref|nested ref <ref>bar</ref> |name=foo}}
<references />
* Also tested on en:Fomitiporia_ellipsoidea from the bug report
and verified correct parse and round tripping.
* Verified that the nested ref in <ref> foo <ref>bar</ref> </ref>
continues to be parsed as plain text.
* No change in parser test results -- have to make another
round of updates to parser tests.
Change-Id: I43bb8b710bd10a9ddbea27818ff8aaf97ddb3fdc
* See example below that clarifies the problem before this patch
-------------------
$ echo "{{#tag:ref|foo}} {{echo|<references />}}" | node parse --extensions ref,references --wt2wt --editMode true
<ref>foo</ref> <references />
-------------------
* The problem is that references are generated after dom fragments
are unpacked in the DOM which lost the mw:Transclusion typeof and
data-mw that had been set on the ref and references tags.
* This is a quick fix to prevent some dirty diffs with #tag:ref being
reported on en-wp. Longer term, we do have a plan to use DOMFragment
encapsulation for refs and references as well and splitting up
references processing differently than is being done currently.
Change-Id: I4186cae93b9882d367c7d4efecc092607fe17c61
* While debugging reports of dirty diffs on some pages (en:Bleak House
specifically), it took me a while to notice that some mw:Transclusion
had about="mwt5" style ids (introduced during template expansion reuse)
whereas all other about ids had about="#mwt5" (note the # char) style ids.
While this by itself shouldn't cause dirty diffs since DOM-diff ignores
about ids, this could potentially introduce introduce other bugs elsewhere
if we start using/comparing about ids.
Fixed all uses of "#" + env.newObjectId() with env.newAboutId and let
env prefix the "#" key.
Change-Id: I74d50ae155f5d24af95c07da15b14eb990cf2891
* Nested ref tags are not supported anymore.
* Turned off pre and p-wrap handlers on ref content since the native
cite extension seems to not do any of this on ref content.
* No change in parser test results (because there are no tests yet).
Other cleanup:
* Removed the 'inBlockToken' hack from Cite since this is not
necessary anymore.
TODO: The use of this flag in TemplateHandler may not be needed
either. Verify and get rid of it.
* Leading whitespace in ref-content is still removed but this may
not be strictly necessary.
Change-Id: I3406236032abe36099a1e420f443277a95fe597b
* The page in question "es:Estadio_Deportivo_Cali" had a ref
with name "constructor", and this name was used as an object
property key and this clashed with the predefined property
constructor.
* The reliable solution here is to prefix ref-name and ref-group
with a string and use it to prevent clashes.
Change-Id: Ib5cf7cce6fa4acd88e3d49ca9d4390a61bfddd7e
* Temporarily hacked sanitizer to pass through typeof attribute
so that mw:DOMFragment wrapper for extension tags can get to
the DOM post processor and get unwrapped.
* Implemented getArgDict for the extension handler since data-mw
for extensions has a different form than that for templates.
* Extracted common functionality into Util.js and used it in Cite.js
and ExtensionHandler.js
* Tested with timeline extension (test snippet below) and verified
that it parses and RTs both with editMode true and false.
TODO: Long overdue. Extension testing.
--------
<timeline>
ImageSize = width:250 height:200
PlotArea = left:40 right:10 top:10 bottom:20
TimeAxis = orientation:horizontal
AlignBars = justify
Colors =
id:gray1 value:gray(0.9)
DateFormat = yyyy
Period = from:1960 till:2010
ScaleMajor = unit:year increment:10 start:1960
PlotData =
bar:3000 color:gray1 width:1
from:start till:end
bar:2000 color:gray1
from:start till:end
bar:1000 color:gray1
from:start till:end
bar:0 color:gray1
LineData =
layer:front
points:(48,96)(84,111) color:blue width:2 #1962 tot 1968. Inwonertal 1962: 1348 1968: 1610
points:(84,111)(100,112) color:blue width:2 #1975: 1627
points:(100,112)(128,116) color:blue width:2 #1982: 1699
points:(128,116)(160,135) color:blue width:2 #1990: 2036
points:(160,135)(196,146) color:blue width:2 #1999: 2217
points:(196,146)(228,158) color:blue width:2 #2004/5
</timeline>
--------
Change-Id: Ia8d2f82e893047e2447cf809e04cc7f508f5899b
* data-mw wasn't being emitted for references -- there was a FIXME
for it.
* Tested fixes on example from 3c88b310.
* Removed meta-placeholder that was being emitted for <references />
tags without any refs to emit since VE might add references and
this wont be valid anymore. Serializer can also handle references
output without any content. So, no need for that hack anymore.
Verified by testing with "<references />" input
Change-Id: I3d2852f2c6a88bf22145add9b2173fd99d152775
* Serializer was complaining about missing source for these tags.
Tweaked code to serialize these back to self-closed tag form.
* Tested with which rts this with the regular serializer in edit mode.
echo "one <ref name='foo'>bar</ref> two<ref name='foo' />\n<references/>" | node parse --extensions ref,references --wt2wt
Change-Id: Iaac63b26660d8b03af937d00d04540c3b8c0c86c
This patch adds the capability to expand individual transclusions all the way
to DOM, which enforces properly nested templates. This is also needed for
DOM-based template re-expansion, which the VE folks would like to use for
template editing.
In normal parsing, DOM-based expansion is currently disabled by default for
transclusions, and enabled by default for extensions. The decision whether to
balance a particular transclusion can be based on a statistics-based
classification of templates into balanced and unbalanced ones. The advantage
of this approach is consistency in behavior for old revisions. Another
alternative is to wrap unbalanced transclusions into <domparse> tags which
disable DOM-based parsing for all transclusions in its content. This has the
advantage that all special cases can be handled after tag insertion, and that
balancing could also be enforced in the PHP parser using an extension.
Other major changes:
* Renamed transclusion content typeof from mw:Object/Template to
mw:Transclusion
* Renamed extension typeof from mw:Object/Extensions/foo to mw:Extension/foo
and switched Cite to use lower-case ref/references too
Other minor changes:
* Only apply template encapsulation algorithm on DOM to mw:Transclusion and
mw:Param objects, and no longer to all mw:Object/* elements. We should
probably switch to a more explicit encapsulation type instead. Maybe
something like mw:EncapStart/Transclusion and mw:EncapEnd/Transclusion
instead of the current mw:Transclusion and mw:Transclusion/End, so that
stripping those metas out selectively is easy?
* Changed the DOMTraverser logic to let handlers explicitly return the next
element to handle. Useful when several siblings are handled, as is the case
for the fragment unwrapper for example, and avoids some cases where deleted
nodes were still being processed.
* Changed Cite to use mw:Extension/Ref{,erences} as all other extensions.
* Make sure we round-trip gallery when the PHP preprocessor is not used
Five parsoid-specific wt2html tests are failing due to the typeof change.
They will be fine once those tests are adjusted. For now, adding them
to parser tests blacklist.
TODO:
* Switch the Cite extension to the generic DOMFragment mechanism instead of
marker tokens.
Change-Id: I64b560a12695256915d2196d0646347381400e80
* PHP parser normalizes whitespace in attribute values of all
extension tags. This patch mimics that behavior.
* Removed special case whitespace stripping for references.group
attribute since that is now handled by the general case.
* As a result, <ref name='foo' /> and <ref name=' foo ' /> are
and treated as being identical.
Change-Id: I34009ab6662d05453fe46379c58d6e989f296958
* Extensions were being serialized back to original wikitext src.
* This patch now uses data-mw content to serialize to extension
wikitext.
* Fixed Cite to add attributes for ref to data-mw so that attributes
roundtrip back (WS and quotes are normalized).
* TODO:
1. add data-mw for <references> in Cite.js
2. add data-mw for all other extensions in the extension handler
Change-Id: If3c35398a409f4775204407b3dcf2442ec2fb4e5
* Updated references output DOM to match spec better.
* Also refactored some repetitive DOM code into helper functions.
* Turned on tpl encapsulation inside extensions.
* Verified output on the example in the spec and the example in
commit 3c88b310.
Change-Id: I1b894759d2cd89b6453156e6b0246f6cc8f0d60b
As per RoanKattouw's suggestions, changed the loading position to top
and changed CSS loading to static (addModuleStyles).
Bug: 48185
Change-Id: I319285d23a4f0a0522cc793f1a6661cda31a1ebf
This is a quick-fix to support proper rendering of <sup>[1]</sup> elements
when the sentence contains a right-to-left word.
Amended: Changed from working through jQuery 'wrap' to CSS unicode-bidi.
Amended: Dos2Unix conversion to get rid of pesky hidden characters, also
corrected all of Amire80's comments.
Bug: 36032
Change-Id: If85efe5ff1e0a3c57ee2523656702db89df07145
'jshint --show-non-errors' shows unused variables. Some of these are unused
function arguments, which are rarely bugs. But try to fix the ones which
are forgotten/dead code and unnecessary require()s.
Change-Id: I685ccd8e388fd3acf75053a07e2f729398fa2855
