Commit graph

4745 commits

Author SHA1 Message Date
Arlo Breault 1ac603ff1c Always wrap follows content
Avoids crashers from trying to serialize the named content twice if
there's a valid follow with some other error, as expected in the FIXME.

This introduces a backwards incompatibility for invalid follows, which
will result in the contents of the ref being dropped.

A test is added to assert that selser will save us for the most part.

Change-Id: I1f572f996a7c2b3b852752f5348ebb60d8e21c47
2020-10-07 11:42:23 -04:00
Translation updater bot 18b5b1488f Localisation updates from https://translatewiki.net.
Change-Id: I1f20b4d058b016c60ae7a67569d6c59fb4ae34d4
2020-10-07 08:35:14 +02:00
Translation updater bot 387cfc8fcf Localisation updates from https://translatewiki.net.
Change-Id: I823ef8e09bdb2a129fb5a4554f602d4bb2e1ec68
2020-10-05 08:41:04 +02:00
Arlo Breault 6bd0594f28 Don't keep pointers to nodes from embedded content
Since the fragment they're subtrees of goes out of scope.

Follow up to 2f09cdb

Previous to that patch this wasn't an issue because we were creating a
whole document which is retained by the environment.

Fixes the warnings from,
"PHP Warning: DOMElement::getAttribute(): Couldn't fetch DOMElement"
https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-2020.10.02/parsoid-tests?id=AXTqaLL12lgCwKx7fVYz&_g=h@a06543d

Tested on scandium with,
node bin/roundtrip-test.js --proxyURL http://scandium.eqiad.wmnet:80 --parsoidURL http://DOMAIN/w/rest.php --domain vi.wikipedia.org "Vua_Việt_Nam"

Change-Id: I74bc7de79b18054e19b77af25e978d3ab3a505e4
2020-10-02 15:57:33 -04:00
Translation updater bot aa953ae1ed Localisation updates from https://translatewiki.net.
Change-Id: Ie095393b655e3c9a9446b27a13c7ed3cfa102f92
2020-10-02 08:23:49 +02:00
Subramanya Sastry f00325d6cc Introduce preprocessing in the HTML -> WT direction
* This patch introduces a preprocessing step on the edited DOM.

* Existing preprocessing code has been extracted into the
  preprocessDOM method.

  Any registered extensions preprocessors are invoked on the DOM.
  So, this assumes that the htmlPreprocess extension listener is only
  applicable to the edited DOM. If we want to expose the concept of
  selective serialization through the API, we may want to add an
  additional interface method / listener to the DOMProcessor class.

  As of this patch, this is somewhat theoretical since there are no
  such extension handlers registered on either DOM. Future patches
  can clarify this better as specific needs arise.

* The handler also calls the serializer's custom preprocessing steps.
  This step is applicable to both the original as well as edited DOM
  (since DOM Diff is impacted by the results). If a need arises,
  in the future, we may introduce a new extension DOM processor method
  that applies to both original and edited DOMs.

* Right now, only selser strips section tags and non-selser wts
  doesn't need to. So, preprocessDOM there is empty. Additional
  selser-only DOM preprocessing will show up in later patches.

* Moved a stub HTML->WT preprocessor in Cite extension to RefProcessor.

Bug: T254501
Change-Id: I0c12afb2ea82617406d72ad872ac4f33678fa5f2
2020-09-30 17:12:45 -05:00
Translation updater bot 697dd38e82 Localisation updates from https://translatewiki.net.
Change-Id: I6023806ff386e6cf9a1e3a7dd52756e553b7fe1b
2020-09-30 08:41:16 +02:00
Arlo Breault 0667a01637 Release fragments once they're no longer needed
These should all be empty at this point anyways.

Bug: T230861
Change-Id: If53d12c8b8e83cea2e86cb8d6107c22aa5724dd6
2020-09-29 22:37:21 +00:00
Arlo Breault 2f09cdb732 One document to rule them all
The description in T179082 suggests that by using one document for the
entire parse, we'd probably see some performance gains from not having
to import nodes when we get to the top level pipeline and we'd avoid the
validation errors from 19a9c3c.

However, the spec seems to suggest creating a new document when parsing
an HTML fragment,
https://html.spec.whatwg.org/#html-fragment-parsing-algorithm

And, indeed, domino implements it that way,
12a5f67136/lib/htmlelts.js (L84-L96)

So, the request in T217705 may be a little misguided.

What then is this patch good for?  In T221790 the ask is that
sub-pipelines produce DocumentFragment which make for cleaner interfaces
and less confusion when migrating children.

The general outline here is that a document is created when the
environment is constructed that gives us the 1-1 correspondence.
Sub-pipelines do create their own documents for the purpose of tree
building, as in the fragment parsing algorithm, but are then immediately
imported to DocumentFragments to be used for the rest of the
post-processing passes.

Bug: T221790
Bug: T179082
Bug: T217705
Change-Id: Idf856d4e071d742ca38486c8ab402e39b3c8949f
2020-09-29 22:36:33 +00:00
Translation updater bot c12150a98c Localisation updates from https://translatewiki.net.
Change-Id: I7c25628ce266c8914947f318b4519e9e06c1a42c
2020-09-28 08:28:48 +02:00
jenkins-bot b3d83799cb Merge "Downstream images for VE EducationPopup" 2020-09-25 09:11:19 +00:00
Translation updater bot e8335bc0f1 Localisation updates from https://translatewiki.net.
Change-Id: Ide55d4d70fbe1b9cc6144063425c0429be7d1fce
2020-09-25 08:21:06 +02:00
libraryupgrader 2caefdeb92 build: Updating mediawiki/mediawiki-phan-config to 0.10.3
Additional changes:
* Dropped .php5 files from .phpcs.xml (T200956).

Change-Id: Idce735669fabb48e81146dd9be643d7cf7c65163
2020-09-24 22:05:47 +00:00
Ed Sanders 8e3bc8cb2c Match line-breaking whitespace in reference names
Bug: T263698
Change-Id: I436b0fbf1756e08fe9ff9013aff7479d5ec653f6
2020-09-24 20:29:00 +02:00
Ed Sanders 21b9cc6eb4 Downstream images for VE EducationPopup
The messages for the popup lives here, so should the image.

Change-Id: I02041246dda1b3d3ad1bcc0b014fa022e8259b62
2020-09-24 17:51:18 +01:00
Translation updater bot 8ee6c996e7 Localisation updates from https://translatewiki.net.
Change-Id: Ia3d6197723410c1d7ac052e7bea4d0fdff4ca9fd
2020-09-23 08:24:28 +02:00
Translation updater bot 146c26e78d Localisation updates from https://translatewiki.net.
Change-Id: If42063e4a26e89ee1123860179a777def6a16047
2020-09-22 08:34:02 +02:00
Translation updater bot ece0c86ff9 Localisation updates from https://translatewiki.net.
Change-Id: Ia5778b30d9f5b0132dd04b72c5888579a3d1f05c
2020-09-21 08:29:00 +02:00
jenkins-bot e5d29aa8c0 Merge "eslint: Fix and enable the 'no-shadow' rule" 2020-09-18 15:22:51 +00:00
Ed Sanders 309645a7ca eslint: Fix and enable the 'no-shadow' rule
Change-Id: I887003beaa3ee2e984b6f854b777485ca9e49049
2020-09-18 14:37:07 +00:00
Arlo Breault e3484acfc6 Highlight when we have a valid follow
Rather than using no errors as proxy.

Change-Id: I78c445838de2d4f5f6a0f17e5bb38996674ca999
2020-09-17 18:13:08 +00:00
sbailey deb9451c15 Removed parsoid cite extension follow ref id's
* Remove the id's from follow refs because they were
   duplicating the same key value erroneously and also
   did not provide useful info. Fixed all tests accordingly.

 * Added FIXME which refers to a new Phab ticket about
   removing the code which adds style = display-none
   that will be moved to CSS at some point.

Bug: T262986
Change-Id: Ib59f5eec951aa83a02357de865df8ab3dd8d2f67
2020-09-17 16:50:29 +00:00
Translation updater bot 3e8154e801 Localisation updates from https://translatewiki.net.
Change-Id: If47cb323efd543d96cde357f84359cd6bd5fcd70
2020-09-14 09:13:21 +02:00
Arlo Breault bb34d30839 Follow up to "follow" functionality for Cite
These refs get a `style="display: none;"` since they're
not intended to be user visible.

Follow refs with errors conform to the proposed spec in T251842

Bug: T51538
Change-Id: Ie4ea28e7f9afde24614874bb4b8e07c5cabafa12
2020-09-10 12:41:06 -04:00
sbailey 467b82701b Adding "follow" functionality to the Cite extension
* Interim state commit with experimental code.

 * Updates to citeParserTests.txt to check now valid follow
   functionality and newly passing tests.

 * Added to follow refs, <sup style="display: none;" about=...
   to suppress display of hidden sups needed for VE to use
   in editing follow refs.

 * Added code to implemented follow functionality and catch
   invalid usage.

Bug: T51538
Change-Id: Ic3ac8237fd2c490cfaf2fe799759742f72f10686
2020-09-09 19:25:14 -04:00
libraryupgrader 085aaf49e7 build: Updating bl to 4.0.3
* https://npmjs.com/advisories/1555

Change-Id: Ia65cdd7239933482838f96865daed12113399e3c
2020-09-08 19:02:47 +00:00
Translation updater bot 45dace7113 Localisation updates from https://translatewiki.net.
Change-Id: I3b0525238f30b01bbf23e7fc549f186f239ccba2
2020-09-06 20:55:06 +02:00
libraryupgrader 36659b995f build: Updating npm dependencies
* eslint-config-wikimedia: 0.16.2 → 0.17.0
  The following rules are failing and were disabled:
  * no-shadow

* grunt: 1.2.1 → 1.3.0

Change-Id: I2ee910e2940c3db520531b91329955d01f2a9076
2020-09-05 10:14:03 +00:00
Arlo Breault 46a9900f69 Implement DOMCompat::replaceChildren()
Change-Id: Id2597e403dc2cda0804005d5e615f94c965a6196
2020-09-03 12:01:02 -04:00
Thiemo Kreuz 7ce27b432f Stop using Language::formatNum to localize separators
This is done to make the discussion in If3dcfd7 easier.

When we introduced this code we actually used it to format
entire numbers. We had to change this later to *not* localize
digits, but only separators. Language::formatNum is and always
was able to do this, so we just continued to use it.

This is discussed now.

It turns out there is only a single place left where we use
formatNum, and it does nothing but localizing the decimal
point. There is another way to do the same.

Bug: T237467
Change-Id: I89b17a9e11b3afc6c653ba7ccc6ff84c37863b66
2020-09-02 09:40:33 +02:00
Translation updater bot bc919e6d78 Localisation updates from https://translatewiki.net.
Change-Id: I0e6fcab7bbf62e392e18f665215b360d5d3f7c7f
2020-08-24 11:57:29 +02:00
C. Scott Ananian 6b813c6874 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit d9a3e14dfcb422e95de7a79f0eb662fd43f9354f

Change-Id: Iae34d2107bfb47304819da7f7c715dec83da1a48
2020-08-18 16:54:33 -04:00
Translation updater bot 5326a844a1 Localisation updates from https://translatewiki.net.
Change-Id: I8ae62784f043909630d3d8f5772d0c2671d58ee9
2020-08-17 08:41:53 +02:00
Translation updater bot 7a3028ad56 Localisation updates from https://translatewiki.net.
Change-Id: I673a9353c36c892dd595eb6b1a050f37500863df
2020-08-13 08:39:04 +02:00
Translation updater bot 7b2c411738 Localisation updates from https://translatewiki.net.
Change-Id: I04e0a2329dbba988921beefcce166ed394d4be6a
2020-08-11 08:20:41 +02:00
sbailey 5e5e360ffd Fix for missing content check where ..body->extsrc is undefined
* Bug fix for accessing undefined extsrc member variable in edge
  case. See T260082 for deeper explanation of the WT that caused a
  case where empty flag is not set and extsrc is also missing, but
  since either case including extsrc being unset indicates no
  content, this additional check is safe for now.

Bug: T259676
Change-Id: I20750c6977883668c83bdae78fbeb171f899e1ab
2020-08-10 22:02:24 +00:00
Translation updater bot 418b173ba3 Localisation updates from https://translatewiki.net.
Change-Id: I745d5b735aa28f5e0e90d02cd3b1711143a63c1d
2020-08-10 09:00:17 +02:00
Subramanya Sastry 542bd7fb99 Extension config option: Rename sealFragment to unpackOutput
* The latter feels more readable and intuitive.
* The option defaults to true.
* To make for simpler code, I ensured that the option value is always
  set before it is accessed.
* But, the typeof value still uses the "/sealed/" qualifier.
  Alternatively, I could use "/packed/" if we want to adhere to the
  config value name more closely.
* Tangentially related changes:
  - made getWrapperTokens a private method since it is only used
    internal to PipelineUtils.
  - remove default value for $opts in encapsulateExpansionHTML since
    the value is always passed in everywhere.

Change-Id: I86c4e5adf11e3151f51f2623e5ed85282a2e1298
2020-08-06 18:33:06 +00:00
Translation updater bot b8cb8d3b55 Localisation updates from https://translatewiki.net.
Change-Id: I43d5721fffba1f82b3823ba0c4b0f4c962c566e0
2020-08-04 08:31:57 +02:00
Translation updater bot 94b181ed37 Localisation updates from https://translatewiki.net.
Change-Id: I05b284b771d20fa121fe55f8226fd3b8089345f4
2020-08-03 08:31:34 +02:00
Arlo Breault d6bcc0ef14 Prefer nullable types in comments
This was done with a custom sniff in,
MediaWiki/Sniffs/Commenting/FunctionCommentSniff.php

`$singleType === 'null' && count( $explodedType ) === 2`

since there's some ambiguity with,

`what|type|null`

but also a case like the following is left out,

`string[]|null`

Change-Id: I1bd50a4486d7ef4974280b476fd03d3ee53232b3
2020-07-29 14:24:32 -04:00
C. Scott Ananian a0faedb942 Sync up with Parsoid citeParserTests.txt
This now aligns with Parsoid commit 77db605163990ad851e3da0fb4fa7eca2081f379

Change-Id: I19ad9f1b6845e5557c2f5f87ac435db0ad87000d
2020-07-28 15:53:08 -04:00
jenkins-bot 66eab58739 Merge "Restore preview of a <references> section alone" 2020-07-28 13:19:01 +00:00
Translation updater bot 099b526f57 Localisation updates from https://translatewiki.net.
Change-Id: Idb61cddad363fdf5001393e2fde9156fab819216
2020-07-28 08:24:15 +02:00
Arlo Breault cdf8ac149b Wrap extension token so that it won't be exposed
Alternative to I6ea271a5d5c7b12a13bb12a682c39bcfd7b1f116

We can follow this up by passing the ExtensionTag to the
ExtensionTagHandler constructor.

Change-Id: I5b1b191bc85968ad617eb3ebcdd7721c55006af2
2020-07-23 14:51:24 +00:00
Thiemo Kreuz 3de3bf4545 Remove unused Cite::$isPagePreview flag
This is dead code we forgot to remove in I7088f25.

Bug: T238195
Change-Id: Iff1d3668beb9d530348ec9f90124774248e59407
2020-07-23 11:46:42 +02:00
Thiemo Kreuz 41149d8072 Restore preview of a <references> section alone
We broke this feature in December 2019 because it was never covered
by any tests. Full explanation in T245376.

All the features we care about are covered by tests. If all existing
tests succeed, that should be proof enough that this patch does not
introduce any new regression.

Bug: T245376
Change-Id: I1a447884bdc507ac762d212466496b4591c18090
2020-07-21 13:19:38 +02:00
Translation updater bot f347863eab Localisation updates from https://translatewiki.net.
Change-Id: If8e268476129ad0bf04bb0803dcd8b413872c875
2020-07-21 09:10:42 +02:00
libraryupgrader d52f8cf5c2 build: Updating grunt to 1.2.1
Change-Id: I64ccb005689bd13caa12aadeca85cc3454bea318
2020-07-18 02:08:25 +00:00
libraryupgrader 063cc59e83 build: Updating eslint-config-wikimedia to 0.16.2
Change-Id: Ic73890c4398c3f08946b4c9677d7e24c634ae7ba
2020-07-15 21:59:30 +00:00