Commit graph

15 commits

Author SHA1 Message Date
Subramanya Sastry 5e256b48aa Start untangling Parsoid internals from extensions
* In this patch, toDOM, fromDOM, and DOM postprocessor extension
  methods all get a ParsoidExtensionAPI object. These API objects
  are constructed at the appropriate times in the wt2html and html2wt
  pipelines.

* Got rid of direct references to SerializerState from fromDOM
  methods in extensions.

* Exposed generic serialization and wikitext escaping methods
  in ParsoidExtensionAPI for extensions to leverage. The implementation
  of these methods is partial and only supports current usage
  of extensions in Parsoid's repo. This will need to be fully
  fleshed out going forward.

* Stopped exposing wt2html options in toto and provided more specific
  convenience methods.

* Reduced direct access to the Env object in a few more places.

* Cite has code to inspect embedded HTML in data attributes of a node.
  Moved this code out of Cite into ParsoidExtensionAPI which reduces
  knowledge that extensions need. Unlike the other cleanups, this one
  is more of a convenience method since this code only requires
  knowledge of a publicly published spec. But, nevertheless an useful
  cleanup since it simplifies Cite's complexity just a bit.

* More followup work is needed.
  - before/after methods should be eliminated in favour of a config flag
    that implements the inline/block layout option. Once this is done,
    extensions will no longer need direct access to the SerializerState
    internal object.
  - Env exposure should be reduced.
  - Provide access to Sanitizer via ParsoidExtensionAPI instead of
    needing extensions to directly import it.
  - It should be possible to eliminate the need for extensions to know
    about DSR / DSR-shifting and do it automatically via some high-level
    conceptual flag.
  - It might also be possible to infer source offsets directly via args
    instead of passing that explicitly.
  - Should we provide a convenience helper class with access to all the
    src/Utils/* methods?

Bug: T242746
Change-Id: I7ffb5aa52a84854a9d363a0e8f1ce650241f1c41
2020-02-06 20:55:27 -05:00
C. Scott Ananian 5d200e0bf0 Move all code from Parsoid to Wikimedia\Parsoid namespace
This matches core conventions.

Bug: T240054
Change-Id: I5feb8a6b41503accd01a740195256e9092609272
2020-02-03 21:34:49 +00:00
Subramanya Sastry 787e6b1cfe No need to explicitly pass 'inTemplate' flag from extension code
* ParsoidExtensionAPI has this info already.

Bug: T242746
Change-Id: I60c393716f31d6f288c54910399bef6e4a42f3dc
2020-01-22 12:56:43 +05:30
Subramanya Sastry ddbca68066 Rename DOM handling methods toDOM/fromDOM to reflect reality
* For extensions, the fromHTML --> fromDOM renaming mirrors the
  toDOM extension method.

Change-Id: Ic361bd0b5a8849c3033f55cfcae6e1cb36f68a10
2020-01-16 20:25:36 +05:30
Subramanya Sastry 1ec3fdbe33 Minor: Rename a function arg
Change-Id: I689eef42414749c2cf47be55268b5bb2575d8078
2019-11-22 20:54:03 +00:00
Arlo Breault b202964b5a Fix dropping nested refs
The extension handler expects a `null` for this, `false` is an
indication not to use the toDOM handler.

php bin/parserTests.php tests/citeParserTests.txt --filter "CircularRef" --wt2html

Change-Id: I849a9aca1133f8a793c9d77e05f192a6af5d78f9
2019-08-09 16:48:03 -04:00
Subramanya Sastry 8757e89e21 Assorted fixes after running PHP parser tests in different modes
* Fixes crashers, notices emitted during parser test runs.

Change-Id: I0e337f22594f6cd36a7dff21afaa7a9dc9c862cf
2019-07-15 17:00:45 -05:00
Subramanya Sastry 0f0d6e0ed7 Followup to 7b6839ac: Fix crashers during serialization
* Don't unconditionally run fromHTML and before handlers without
  checking if we have a native extension handler.

* Remove unintential implementation of `before` in Ref.php

* Hybrid tests for Cite now passes again.

* Discovered while running native parser tests and isolated those
  crashers to this.

Change-Id: I45b48b595a5aee2b8b8d00b4ebcf73a5ea7bc8a3
2019-07-12 13:48:50 -04:00
C. Scott Ananian 7b6839ac59 Remove SerialHandler interface in favor of default methods on ExtensionTag
Change-Id: I245c4b9393720982654d5f4e944329c9d764e04e
2019-07-11 18:51:55 +00:00
C. Scott Ananian 7e444d6364 All extensions implement Extension; tags all implement ExtensionTag
Restructure ExtensionTag as an abstract class with default do-nothing
implementations of all methods.  So instead of Translate and LST
not implementing ExtensionTag::toDOM, they inherit the default
implementation which returns false, and that has the same effect.

The intention is to move SerialHandler::fromHTML and SerialHandler::before
into this framework as well.  Every "optional" method should have a default
implementation in the base class which returns false.

Change-Id: I0ad5c714601c0cf0b5189d4d282c67c6b53fc760
2019-07-11 18:46:14 +00:00
C. Scott Ananian b84b71af22 Gallery: shift TSRs in the DOM, rather than fibbing about srcOffset
Passing srcOffsets which don't actually correspond to actual regions of
the source wikitext cause problems in the token offset conversion code.
Instead, parse the wikitext as itself, then adjust the TSRs in the DOM
tree.

Since Gallery isn't ported to PHP (yet), update the
automatically-generated Gallery/index.php.  The newly-added
ContentUtils::shiftDSR() was ported, however.

Change-Id: I28f3d3398930733ae2bcf9759e49c45f93bc7190
2019-06-28 14:10:16 +00:00
Subramanya Sastry 720f1db084 Fix exception handling: Don't catch exceptions and suppress them
* Now that we are in sync land, we don't need to catch exceptions
  and log error messages at multiple places. Let them bubble up
  to the top.

* I noticed this was actually getting in the way of debugging because
  with $env->log unimplemented, I was only getting very generic
  failures instead of the root cause that was being suppressed
  and unlogged.

* There are still a couple of places where we have generic Exception
  catching in place where it does make sense currently. For example,
  we aren't interested in what caused a templatedata fetch to fail.
  We simply fall back to regular serialization - the rationale here is
  that it is better to emit a transclusion without the preferred formatting
  (but syntactically correct) instead of losing the edit altogether.

* Minor unrelated fix in Cite/Ref.php: Use !isset() instead of empty()

Change-Id: Iebff6f37dcd8278185c4a74b72a99b528efa20ff
2019-06-26 15:50:49 -05:00
Pavel Astakhov 005176a355 Port Cite extension
* All wt2wt, html2wt, and all but one html2html tests pass in
  hybrid mode when entire html2wt code is run in PHP

  Set "Serializer: true" in the html2wt section of phpconfig.yaml

* The single failing html2html test is a <gallery> test which is
  presumably related to the unported <gallery> extension code, but
  not sure. Not investigating it now.

* Update Parsoid Extension API to provide access to extension source
  without exposing internals.

Change-Id: I6d6e21ad2324acfc4306b32c9055d6c088708c48
2019-06-21 16:23:42 -05:00
C. Scott Ananian 320d045ee8 Update automatically-generated PHP files w/ latest js2php
Mostly comment formatting improvements, some significant code changes
to the JS side.

Change-Id: I7a8f2105173df74dc09f2024d68268f5dc6fa632
2019-06-05 17:13:34 -04:00
Arlo Breault 20c627e3f4 Convert cite extension to es6 class structure
Also, runs js2php on these files.

Change-Id: Id8ee13ad536d75f63e0045a21fdfdb667a0df65d
2019-04-03 12:20:41 -04:00