Commit graph

5265 commits

Author SHA1 Message Date
libraryupgrader 00d3977424 build: Updating mediawiki/mediawiki-codesniffer to 33.0.0
Change-Id: I385c3993f6fb8def81b82c35d6ef3fe510c24eef
2020-11-03 10:06:13 +00:00
Translation updater bot a1d3d21eec Localisation updates from https://translatewiki.net.
Change-Id: I964aafb8c34af5273564498967b3810a14fb333d
2020-11-03 08:43:49 +01:00
Arlo Breault 1dda4cdc8a Consolidate adding ref errors at references insertion
Change-Id: I01ce55989fb7b822320c63ddad19c2edf7e03bf9
2020-10-29 15:54:30 -04:00
libraryupgrader 4f08772c6b build: Updating mediawiki/mediawiki-codesniffer to 32.0.0
The following sniffs are failing and were disabled:
* MediaWiki.Commenting.PropertyDocumentation.MissingDocumentationProtected

Additional changes:
* Dropped .inc files from .phpcs.xml (T200956).

Change-Id: Ia1461485a1a0122a5b830dc2badb542ae7ec9a1c
2020-10-29 07:00:27 +00:00
sbailey 02fb17d102 Refs in group with no <references group="xxx" />
* Generating error message in data-mw in affected refs

 * Cite test validates correctness of adding error to afflicted
   refs

 * Autogenerated references aren't considered erroneous
   (the extension to the legacy parser also generates them)
   and are not suppressed when serializing because apparently
   that's the behaviour Parsoid clients want.  However, in
   this patch we're marking up autogenerated references
   *with group attributes* as errors (the legacy extension
   doesn't generate them at all) and are choosing to suppress
   them when serializing since we considered them an error
   while parsing and don't want them to persist in the content.

Bug: T51538
Change-Id: Ia651b10449dc41c2cb439b33a361e8c8e482f502
2020-10-28 22:28:13 +00:00
Arlo Breault b5b3475d28 Set a top level doc when serializing
First step in using a single document for the serializing direction as
well.

Bug: T265061
Change-Id: Ia845faed693bf4c6f74facc7576d01a9f5a875be
2020-10-28 18:48:17 +00:00
Thiemo Kreuz d44754168f Fix coverage report for ErrorReporter
The structure of this class changed a lot in If2fe5f5
(T239572). This was never reflected in the @covers tags. I
suggest to go with a trivial @covers for the entire class
and let PHPUnit figure it out. The alternative is to kind
of repeat many private implementation details, and this
doesn't feel right.

Change-Id: Ie414876489133ab9aca934c19a5e403cd339abf1
2020-10-27 09:49:56 +00:00
Arlo Breault 568034a00c Traverse with inEmbeddedContent for ref in ref
While, for the most part, content nested in refs will end up in the
references section and should be fine to maintain a linkback.  However,
if content differs from a previously named ref (ie. !== cachedHtml), it
ends up being serialized and nested in data-mw, so is also unsafe.

Bug: T266294
Bug: T266356
Change-Id: Ia92f42e06353c411b986d0665cbe6338052555fa
2020-10-23 17:52:51 -04:00
Arlo Breault 8e237b4e34 Make $inEmbeddedContent an explicit stack
Change-Id: I48ff2f7be352fdec72b2c5e0eeee843330ec3872
2020-10-23 11:42:45 -04:00
Arlo Breault b8d4bb1f37 Set inEmbeddedContent when running processRefsInReferences
In 47506af, serializing references content to be added to data-mw was
delayed until we were inserting the reference into the dom, to give us a
chance to mark up errors about not finding named ref content.  After
which, the content is cleaned out of references to make room for list
items that belong there.

In 6bd0594, we noted that we can't store linkback from embeded content
since the content is released after serializing and won't be available
when it comes time to mark it up with errors.

Similarly, the linkbacks added to other groups from the references
content won't be available after inserting that reference and so we
shouldn't be holding on to them either.  This means that we won't be
marking them up for the named error above but with the benefit that we
won't crash when trying to access these linkbacks from another group.

A test case is added which crashes trying to access a linkback from a
group that has already been inserted.

Note that the extension to the legacy parser considers referencing
another group from group content to be
"cite_error_references_group_mismatch".  If we choose to also do that,
we may be able to reverse this change.

Change-Id: Idf0e49fa07dc3614068793c72a30ce3de1e2392c
2020-10-23 11:42:25 -04:00
sskenner f0c77eb4c5 Selenium: Add selenium logs to .gitignore
Bug: T265463
Change-Id: I5fbb39b3437bc9e823d04f23f72d991b677676c5
2020-10-20 14:19:24 -04:00
Translation updater bot ba21f9438d Localisation updates from https://translatewiki.net.
Change-Id: Ice3f894e335b0bd0babd6053f379c83fc471740f
2020-10-19 08:35:07 +02:00
sbailey a8e299b21e Added cite error checks for numeric digit in name
* Note: the <ref name="1"> numeric is flagged as an error, but
   <ref follow="1"> is not flagged as well as it would be duplicitous.
   We are proposing that Numeric digits as a name be changed to
   a warning as Parsoid Cite handles them properly without error.

   Background: The core Cite extension started off not allowing numeric
   names because the data structure they used made it inconvenient,
   but Parsoid Cite has a separate index for named refs.
   The core Cite devs thought there was potential for a conflict in
   the ids if numeric names were allowed. The ids that core uses
   follow the pattern:
     for no name defined:  cite_ref-1             cite_ref-2
     for name="refname":   cite_ref-refname_3-0   cite_ref-refname_3-1
   so for name="1" at worst you might see ids like:
     cite_ref-1  or  cite_ref-1_2-0
   so that does not produce conflicting IDs and isn't a concern,
   and that is the pattern that Parsoid Cite uses.

 * Error case of numeric in name and follow and two tests that
   validate errors.

Bug: T51538
Change-Id: I95d725d0f77abadc1ddb2dd6939762b7d322e4f2
2020-10-15 23:57:16 +00:00
Translation updater bot 6c464becc9 Localisation updates from https://translatewiki.net.
Change-Id: If6d7e98b56e7d1f078d652ecfc04e4470c2f7292
2020-10-14 08:27:38 +02:00
jenkins-bot 33b6cdf0b6 Merge "Compact trivial arrays in test providers" 2020-10-13 13:39:34 +00:00
Translation updater bot 7741785df8 Localisation updates from https://translatewiki.net.
Change-Id: Ib2826d14ae7c8bb4179d838c2a7e91bc6f7371cc
2020-10-13 08:41:42 +02:00
Translation updater bot fda5b43f89 Localisation updates from https://translatewiki.net.
Change-Id: I5414dccfaa974239c3db39ee5fada87d640dc05e
2020-10-12 08:34:25 +02:00
Thiemo Kreuz 27c5632ebd Compact trivial arrays in test providers
This is meant to make the code easier to read.

Change-Id: Ib684bf64acfbac9b458d110d9ccff2d071b993be
2020-10-08 11:07:35 +02:00
Translation updater bot aef34e22ab Localisation updates from https://translatewiki.net.
Change-Id: Iad0d4ae190a5dbc5a2d831060b16ef90fa41c107
2020-10-08 08:29:36 +02:00
sbailey aadef667d4 Added cite error checks for invalid text direction
* Error case of dir= not ltr or rtl is caught and a cite tests
   validates the new check.

Bug: T51538
Change-Id: I8c7e088416f0a000e638771a3fe5e8e0c58bcc23
2020-10-07 13:22:46 -07:00
Arlo Breault 1ac603ff1c Always wrap follows content
Avoids crashers from trying to serialize the named content twice if
there's a valid follow with some other error, as expected in the FIXME.

This introduces a backwards incompatibility for invalid follows, which
will result in the contents of the ref being dropped.

A test is added to assert that selser will save us for the most part.

Change-Id: I1f572f996a7c2b3b852752f5348ebb60d8e21c47
2020-10-07 11:42:23 -04:00
Translation updater bot 18b5b1488f Localisation updates from https://translatewiki.net.
Change-Id: I1f20b4d058b016c60ae7a67569d6c59fb4ae34d4
2020-10-07 08:35:14 +02:00
Translation updater bot 387cfc8fcf Localisation updates from https://translatewiki.net.
Change-Id: I823ef8e09bdb2a129fb5a4554f602d4bb2e1ec68
2020-10-05 08:41:04 +02:00
Arlo Breault 6bd0594f28 Don't keep pointers to nodes from embedded content
Since the fragment they're subtrees of goes out of scope.

Follow up to 2f09cdb

Previous to that patch this wasn't an issue because we were creating a
whole document which is retained by the environment.

Fixes the warnings from,
"PHP Warning: DOMElement::getAttribute(): Couldn't fetch DOMElement"
https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-2020.10.02/parsoid-tests?id=AXTqaLL12lgCwKx7fVYz&_g=h@a06543d

Tested on scandium with,
node bin/roundtrip-test.js --proxyURL http://scandium.eqiad.wmnet:80 --parsoidURL http://DOMAIN/w/rest.php --domain vi.wikipedia.org "Vua_Việt_Nam"

Change-Id: I74bc7de79b18054e19b77af25e978d3ab3a505e4
2020-10-02 15:57:33 -04:00
Translation updater bot aa953ae1ed Localisation updates from https://translatewiki.net.
Change-Id: Ie095393b655e3c9a9446b27a13c7ed3cfa102f92
2020-10-02 08:23:49 +02:00
Subramanya Sastry f00325d6cc Introduce preprocessing in the HTML -> WT direction
* This patch introduces a preprocessing step on the edited DOM.

* Existing preprocessing code has been extracted into the
  preprocessDOM method.

  Any registered extensions preprocessors are invoked on the DOM.
  So, this assumes that the htmlPreprocess extension listener is only
  applicable to the edited DOM. If we want to expose the concept of
  selective serialization through the API, we may want to add an
  additional interface method / listener to the DOMProcessor class.

  As of this patch, this is somewhat theoretical since there are no
  such extension handlers registered on either DOM. Future patches
  can clarify this better as specific needs arise.

* The handler also calls the serializer's custom preprocessing steps.
  This step is applicable to both the original as well as edited DOM
  (since DOM Diff is impacted by the results). If a need arises,
  in the future, we may introduce a new extension DOM processor method
  that applies to both original and edited DOMs.

* Right now, only selser strips section tags and non-selser wts
  doesn't need to. So, preprocessDOM there is empty. Additional
  selser-only DOM preprocessing will show up in later patches.

* Moved a stub HTML->WT preprocessor in Cite extension to RefProcessor.

Bug: T254501
Change-Id: I0c12afb2ea82617406d72ad872ac4f33678fa5f2
2020-09-30 17:12:45 -05:00
Translation updater bot 697dd38e82 Localisation updates from https://translatewiki.net.
Change-Id: I6023806ff386e6cf9a1e3a7dd52756e553b7fe1b
2020-09-30 08:41:16 +02:00
Arlo Breault 0667a01637 Release fragments once they're no longer needed
These should all be empty at this point anyways.

Bug: T230861
Change-Id: If53d12c8b8e83cea2e86cb8d6107c22aa5724dd6
2020-09-29 22:37:21 +00:00
Arlo Breault 2f09cdb732 One document to rule them all
The description in T179082 suggests that by using one document for the
entire parse, we'd probably see some performance gains from not having
to import nodes when we get to the top level pipeline and we'd avoid the
validation errors from 19a9c3c.

However, the spec seems to suggest creating a new document when parsing
an HTML fragment,
https://html.spec.whatwg.org/#html-fragment-parsing-algorithm

And, indeed, domino implements it that way,
12a5f67136/lib/htmlelts.js (L84-L96)

So, the request in T217705 may be a little misguided.

What then is this patch good for?  In T221790 the ask is that
sub-pipelines produce DocumentFragment which make for cleaner interfaces
and less confusion when migrating children.

The general outline here is that a document is created when the
environment is constructed that gives us the 1-1 correspondence.
Sub-pipelines do create their own documents for the purpose of tree
building, as in the fragment parsing algorithm, but are then immediately
imported to DocumentFragments to be used for the rest of the
post-processing passes.

Bug: T221790
Bug: T179082
Bug: T217705
Change-Id: Idf856d4e071d742ca38486c8ab402e39b3c8949f
2020-09-29 22:36:33 +00:00
Translation updater bot c12150a98c Localisation updates from https://translatewiki.net.
Change-Id: I7c25628ce266c8914947f318b4519e9e06c1a42c
2020-09-28 08:28:48 +02:00
jenkins-bot b3d83799cb Merge "Downstream images for VE EducationPopup" 2020-09-25 09:11:19 +00:00
Translation updater bot e8335bc0f1 Localisation updates from https://translatewiki.net.
Change-Id: Ide55d4d70fbe1b9cc6144063425c0429be7d1fce
2020-09-25 08:21:06 +02:00
libraryupgrader 2caefdeb92 build: Updating mediawiki/mediawiki-phan-config to 0.10.3
Additional changes:
* Dropped .php5 files from .phpcs.xml (T200956).

Change-Id: Idce735669fabb48e81146dd9be643d7cf7c65163
2020-09-24 22:05:47 +00:00
Ed Sanders 8e3bc8cb2c Match line-breaking whitespace in reference names
Bug: T263698
Change-Id: I436b0fbf1756e08fe9ff9013aff7479d5ec653f6
2020-09-24 20:29:00 +02:00
Ed Sanders 21b9cc6eb4 Downstream images for VE EducationPopup
The messages for the popup lives here, so should the image.

Change-Id: I02041246dda1b3d3ad1bcc0b014fa022e8259b62
2020-09-24 17:51:18 +01:00
Translation updater bot 8ee6c996e7 Localisation updates from https://translatewiki.net.
Change-Id: Ia3d6197723410c1d7ac052e7bea4d0fdff4ca9fd
2020-09-23 08:24:28 +02:00
Translation updater bot 146c26e78d Localisation updates from https://translatewiki.net.
Change-Id: If42063e4a26e89ee1123860179a777def6a16047
2020-09-22 08:34:02 +02:00
Translation updater bot ece0c86ff9 Localisation updates from https://translatewiki.net.
Change-Id: Ia5778b30d9f5b0132dd04b72c5888579a3d1f05c
2020-09-21 08:29:00 +02:00
jenkins-bot e5d29aa8c0 Merge "eslint: Fix and enable the 'no-shadow' rule" 2020-09-18 15:22:51 +00:00
Ed Sanders 309645a7ca eslint: Fix and enable the 'no-shadow' rule
Change-Id: I887003beaa3ee2e984b6f854b777485ca9e49049
2020-09-18 14:37:07 +00:00
Arlo Breault e3484acfc6 Highlight when we have a valid follow
Rather than using no errors as proxy.

Change-Id: I78c445838de2d4f5f6a0f17e5bb38996674ca999
2020-09-17 18:13:08 +00:00
sbailey deb9451c15 Removed parsoid cite extension follow ref id's
* Remove the id's from follow refs because they were
   duplicating the same key value erroneously and also
   did not provide useful info. Fixed all tests accordingly.

 * Added FIXME which refers to a new Phab ticket about
   removing the code which adds style = display-none
   that will be moved to CSS at some point.

Bug: T262986
Change-Id: Ib59f5eec951aa83a02357de865df8ab3dd8d2f67
2020-09-17 16:50:29 +00:00
Translation updater bot 3e8154e801 Localisation updates from https://translatewiki.net.
Change-Id: If47cb323efd543d96cde357f84359cd6bd5fcd70
2020-09-14 09:13:21 +02:00
Arlo Breault bb34d30839 Follow up to "follow" functionality for Cite
These refs get a `style="display: none;"` since they're
not intended to be user visible.

Follow refs with errors conform to the proposed spec in T251842

Bug: T51538
Change-Id: Ie4ea28e7f9afde24614874bb4b8e07c5cabafa12
2020-09-10 12:41:06 -04:00
sbailey 467b82701b Adding "follow" functionality to the Cite extension
* Interim state commit with experimental code.

 * Updates to citeParserTests.txt to check now valid follow
   functionality and newly passing tests.

 * Added to follow refs, <sup style="display: none;" about=...
   to suppress display of hidden sups needed for VE to use
   in editing follow refs.

 * Added code to implemented follow functionality and catch
   invalid usage.

Bug: T51538
Change-Id: Ic3ac8237fd2c490cfaf2fe799759742f72f10686
2020-09-09 19:25:14 -04:00
libraryupgrader 085aaf49e7 build: Updating bl to 4.0.3
* https://npmjs.com/advisories/1555

Change-Id: Ia65cdd7239933482838f96865daed12113399e3c
2020-09-08 19:02:47 +00:00
Translation updater bot 45dace7113 Localisation updates from https://translatewiki.net.
Change-Id: I3b0525238f30b01bbf23e7fc549f186f239ccba2
2020-09-06 20:55:06 +02:00
libraryupgrader 36659b995f build: Updating npm dependencies
* eslint-config-wikimedia: 0.16.2 → 0.17.0
  The following rules are failing and were disabled:
  * no-shadow

* grunt: 1.2.1 → 1.3.0

Change-Id: I2ee910e2940c3db520531b91329955d01f2a9076
2020-09-05 10:14:03 +00:00
Arlo Breault 46a9900f69 Implement DOMCompat::replaceChildren()
Change-Id: Id2597e403dc2cda0804005d5e615f94c965a6196
2020-09-03 12:01:02 -04:00
Thiemo Kreuz 7ce27b432f Stop using Language::formatNum to localize separators
This is done to make the discussion in If3dcfd7 easier.

When we introduced this code we actually used it to format
entire numbers. We had to change this later to *not* localize
digits, but only separators. Language::formatNum is and always
was able to do this, so we just continued to use it.

This is discussed now.

It turns out there is only a single place left where we use
formatNum, and it does nothing but localizing the decimal
point. There is another way to do the same.

Bug: T237467
Change-Id: I89b17a9e11b3afc6c653ba7ccc6ff84c37863b66
2020-09-02 09:40:33 +02:00