Commit graph

149 commits

Author SHA1 Message Date
Brad Jorsch 7f94d88733 LuaStandalone: Fix signal handling
I252ec046 noticeably broke things by adding a dependency on the pcntl
functions, which tend not to be present under Apache.

It also subtly broke exit handling by using proc_close()'s return value,
which PHP mangles in such a way that we can't tell the difference
between an actual XCPU kill and exit( SIGXCPU ). This one wasn't noticed
because the pcntl functions interpret everything proc_close() is going
to return as a signal kill and we didn't test the 'exited' code path.

I'm not sure what was going on in I57cdf8aa since it provides no details
about what it was trying to fix, but that would have broken signal
handling in the other way: Ibf5f4656 worked because proc_open() on Linux
executes the command by passing it to /bin/sh -c, and that shell is
going to turn any signal that kills Lua (e.g. the SIGXCPU) into an exit
status of 128+signum.

To avoid proc_close()'s broken return value while also avoiding the
race, we can loop on proc_get_status() until $status['running'] is
false.

To have signals that kill Lua actually be interpreted as signals, we
have two options: add an "exec" in front of the command so proc_open()'s
/bin/sh -c is execed away, or detect shell-style signal reporting and
convert it. We may as well do both.

Bug: T128048
Change-Id: I8a62e1660fe1694e9ba5de77d01960c1ab4580aa
2017-03-09 23:16:28 +00:00
Antoine Musso 1cf6339a1b test: change interwiki to a meaningful name
When investigating a test failure on T155600 I found the title
'scribuntotitletest:Module:TestFramework' to be rather obscure.  Had to
browse the code to findout 'scribuntotitletest' is an interwiki prefix.

Update TitleLibraryTest php/lua and change the interwiki prefix to the
more meaningful 'interwikiprefix'.

Bug: T155600
Change-Id: Iec63734c82b0835a7f2444ea9af35617876831d5
2017-01-19 09:48:55 +01:00
jenkins-bot ae677fbc0d Merge "Ustring: Let gcodepoint work with moderately long strings" 2016-12-16 00:42:02 +00:00
Brad Jorsch db07787390 Cleanup backwards-compatibility code
https://www.mediawiki.org/wiki/Extension:Scribunto says that master
requires 1.25+, so let's remove checks for stuff that was added before
that.

* PPFrame::getTTL() was in 1.24.
* PPFrame::setTTL() was in 1.24.
* PPFrame::isVolatile() was in 1.24.
* Parser::fetchCurrentRevisionOfTitle() was in 1.24.
* ObjectCache::getLocalServerInstance() was added in 1.27, so restore the call to ObjectCache::newAccelerator() as BC.

This also removes BC with the php-luasandbox extension older than 1.6, which
was released before MediaWiki 1.22.

Bug: T148012
Change-Id: I36e37f3b65d0f167e1d28b00e0842d9721feee31
2016-10-13 11:07:44 -04:00
Brad Jorsch 629f11d0dd Fix pure-Lua ustring and empty patterns
An empty pattern isn't "safe" since it could match in between the
bytes of a UTF-8 character.

Also, it turns out there's a bug in PHP <5.6.9 preg_replace() that we
need to work around too.

Change-Id: I282e5909e4663461d60c5386693db182de2fd44c
2016-10-05 14:32:27 -04:00
jenkins-bot c48bda0698 Merge "Add handling for PCRE errors in ustringGsub" 2016-10-05 18:15:10 +00:00
Kunal Mehta 48748a6046 Improve validation of ScribuntoContent
Implement Content::prepareSave() to ensure that any content
directly passed to WikiPage::doEditContent() that doesn't run edit
filters will still be validated. We have to use prepareSave() instead of
Content::isValid() because validation depends upon the current Title.

Create a ScribuntoContent::validate() convenience function to hold the
logic for that and add a todo to use it in the EditFilterMerged hook.

Also, remove a parser test that depended upon being able to save invalid
modules directly, as what it is testing is no longer possible (unless it
pre-dates making valid syntax a requirement).

Bug: T145548
Change-Id: Ie57eff36100963f02899d669df7375577f7375e1
2016-09-14 11:50:07 -07:00
Marius Hoch 0f4db74148 Add mw.hash to Scribunto
Provides a simple wrapper for PHP's hash() and
hash_algos() functions.

I will add docs to the Lua reference manual once
this is merged.

Bug: T142585
Change-Id: I6697463974a175e99f9b77428a1085247165ebc9
2016-08-18 04:39:04 +02:00
Brad Jorsch ba19a82c06 Add handling for PCRE errors in ustringGsub
Bug: T130823
Change-Id: I6fab71c82ddab92daf6b369cb9857d9892f2d246
2016-07-15 15:43:58 -04:00
Brad Jorsch d643f40de9 Ustring: Let gcodepoint work with moderately long strings
For the PHP implementation, return the codepoints as a table instead of
multiple return values that get table-ified in Lua, to avoid hitting
too-many-values stack limits.

For the pure-Lua version, inline most of ustring.codepoint instead of
calling it to avoid what's effectively "{ unpack( stuff ) }".

Bug: T118687
Change-Id: I105f388cc23ab55d4124739700ef89d5354b7dbc
2016-07-15 19:35:58 +00:00
Brad Jorsch c9de00aeff SECURITY: Don't escape strip markers when escaping attributes in mw.html
Core strip markers were changed in T110143 to include characters that
are normally encoded in attributes, however we want to pass them through
here so they can be unstripped correctly in the output wikitext.

This fix makes "Strip markers in CSS" parser test pass again.

Bug: T110143
Bug: T135961
Change-Id: I1353931a53c668d8a453dfa2300a99f59fdb01c5
2016-05-22 21:40:32 -04:00
Brad Jorsch aa4d72e3ff Fix uncontroversial phpcs errors
The following continue to be ignored:
* Generic.Arrays.DisallowLongArraySyntax.Found, because I'm not sure
  Scribunto is ready to abandon old version support in master.
* MediaWiki.ControlStructures.AssignmentInControlStructures.AssignmentInControlStructures,
  because it's overly strict for its purpose.

Squiz.Classes.ValidClassName.NotCamelCaps isn't ignored globally, we
just ignore it explicitly every place it's needed.

Change-Id: I307668da6ef7b3e23da19b1fd1e08914239b99b3
2016-05-18 16:31:28 -04:00
jenkins-bot c753698eaa Merge "Provide a standard way to get the target of a redirect page" 2016-05-12 19:32:17 +00:00
Brad Jorsch b3da8a698d Add toNFKC and toNFKD to mw.ustring
This also makes some updates to make-normalization-table.php to handle
the move of UtfNormal to a separate library.

Bug: T126427
Change-Id: Id4985c3ca441cf92f08ba1f1af85c762ba43d7d2
2016-04-02 15:22:42 +00:00
aude 399b94fa24 Update LanguageLibraryTests per changes in core
appears the test depends on kk-cyrl translation of
limitreport-cputime-value, which has changed:

https://gerrit.wikimedia.org/r/#/c/278597/1/languages/i18n/kk-cyrl.json

Change-Id: Iedf847b7dd0dde094ce0c1c2af4f44b71b9c477f
2016-03-20 18:02:22 -04:00
Jackmcbarn b82ed4aa7d Restrict cached results to their original frame
When caching results from frame:preprocess and frame:expandTemplate,
restrict the scope of the cache to the frame object that was used. This
allows the integrity of the empty-frame expansion cache to be maintained
while also allowing parent frame access. This change is the equivalent of
I621e9075 in core.

Change-Id: Iae4c00e7e19ba12cfdaac135be16c991d9d0cea1
2016-03-09 11:27:23 -05:00
Ricordisamoa 1573bee81a Provide a standard way to get the target of a redirect page
The new Scribunto_LuaTitleLibrary::redirectTarget() method is
used by mw.title objects as read-only attribute 'redirectTarget'.

If the page does not exist or it is not a redirect, the value
of the attribute is `false`; otherwise, it is the target of the
redirect page, as mw.title object.

This is a proper alternative to parsing wikitext as it is done in:
https://en.wikipedia.org/wiki/Module:Redirect

Bug: T68974
Change-Id: Id4d9b0f8c1cd09ebc42c031d4d3fc0c33eea44aa
2016-03-01 14:30:22 +01:00
Brad Jorsch 1365e35d89 Waste CPU cycles, not wall clock time
Apparently microtime is a bit unreliable for this test for some reason,
so let's use getrusage() to measure actual CPU time instead.

Bug: T125045
Change-Id: Ia1ab6b043b99abb4fc6c2989ad09a24d97dd09c3
2016-02-11 14:00:03 -05:00
Jan Zerebecki 1517a1ba2e Fix typo in comment and missing spaces
Change-Id: Ifda05edd9b488768814d3251a93d71e9b55ea5a6
2015-11-11 23:08:15 +00:00
Jan Zerebecki fa0e32ce99 Temporarily skip a test that fails on HHVM.
HHVM has a bug in json decode https://github.com/facebook/hhvm/issues/5813 .
Skip the test until there is a fix upstream.

Bug: T103346
Change-Id: I7e44d98c29ba9b9f5443665fc046382f696193c9
2015-11-11 21:49:12 +01:00
Brad Jorsch cd618c7a92 ustring: Handle "empty" charset like Lua does (part 2)
Lua actually treats a close-bracket at the start of a bracketed
character class as a literal, rather than using it to close the
character class. Probably unintended behavior, but it happens.

Also, have the pure-lua version throw our more informative errors on
error even when falling back to string.find and the like, and fix some
other weird edge cases that came up in testing.

Bug: T95958
Bug: T115686
Change-Id: Iab783d4a3e58b1514cc09729d4a71c2cb1242ee8
2015-10-16 09:26:55 -04:00
Jan Berkel fb20934b16 Fix a problem with simple pattern detection
A string with a dot pattern is only "simple" if
followed by +, - or *. The end of string condition was not checked
properly.

Change-Id: Ia10b9164caeabe464c76441cc82eef37a7013048
2015-10-07 10:27:45 -04:00
Jan Berkel 7c5454b36c Fix off-by one error in gsub
Change-Id: I49c0386970e007271d23087fd112580af7b21c9c
2015-09-23 17:41:15 +01:00
Jackmcbarn 828c6cf513 Prevent leaking title fragments across invokes
Bug: T106951
Change-Id: Iace5d75deac3d8ffde6f3dec6a4f910dcb77d1e2
2015-07-27 10:46:23 -04:00
Jackmcbarn ca7a84b5b2 Fix some PHPCS issues
Change-Id: I5a44d07553d45bc01db070c99856b35a3d275bd1
2015-06-30 13:14:58 -04:00
Ori Livneh d426627c9b lint: 'if(' => 'if ('
Change-Id: I056ff6bbc5f992bddfd7e3bd82803de107651b80
2015-06-20 21:38:56 -07:00
Brad Jorsch 58d722bcdf Allow nil in mw.text.jsonEncode
If it somehow gets in there (e.g. via a crafty __pairs), let it through.

Change-Id: I9f79dbb1a09cd62b2a8f4b6beb84a3e2f1c85560
2015-06-16 16:36:30 +00:00
Brad Jorsch e55c5ee946 Fix unit test for core change I31d4556b
Replace deprecated $parser->uniqPrefix() with Parser::MARKER_PREFIX.

Change-Id: I53721e1aa9022ad6bbfdb08b1a07feac81806514
2015-06-16 10:22:10 -04:00
mjbmr 7ab1348e94 Add namespace aliases for mzn
Change-Id: I871fd627e7d3d492daae3f608453cf0df2330695
2015-04-22 21:18:26 +00:00
Brad Jorsch 4669e43135 ustring: Handle empty charset like Lua does
Both '[]' and '[^]' give a rather odd error, but it's probably best to
follow suit.

Bug: T95958
Change-Id: I3310da55f655537c9082fc9039003f6b2d31eff4
2015-04-13 18:20:33 -04:00
Brad Jorsch 0c4e9ecf10 Fix parser tests, YET AGAIN
And now Ie9e334e broke the tests again. Why can't anyone be bothered to
fix things they should *know* by now are going to need fixing?

Bug: T90018
Change-Id: I7f71dda4af38b23f89c64b6116c1145c44f5c523
2015-03-18 10:42:57 -04:00
Brad Jorsch cb4cf733f2 Fix parser tests, again
I736cb7f5 broke them again.

Bug: T90018
Change-Id: If77f5b40e2576d18ad908d8fd995cfa8eb8f8324
2015-03-16 16:05:42 -04:00
Vivek Ghaisas bdec6124b5 Update Scribunto tests after I562e437e
The parser's output was changed in I562e437e to add clickable anchors
for section headers which made Scribunto parser tests fail.

Bug: T90018
Change-Id: Ia74877e0641f2763d42be4a8fad1aa8ecffe449e
2015-02-21 12:08:09 +05:30
Marius Hoch 53ff0bf246 Revert "Revert "Make tests work when $wgLanguageCode != 'en'""
Should be good to apply now after Wikibase has been adopted to
this.

This reverts commit 06f03fe09b.

Bug: T69483
Change-Id: I28073a891b8847fc6aa5ad026bf702e4015f7d4a
2015-02-19 22:30:47 +00:00
Legoktm 06f03fe09b Revert "Make tests work when $wgLanguageCode != 'en'"
Broke Wikibase tests and hoo needs to do backports today.

This reverts commit 499665c36f.

Change-Id: I5151ce963bcd17ed6d3b98a0f332f95c99596bff
2015-02-19 17:41:19 +00:00
Brad Jorsch 499665c36f Make tests work when $wgLanguageCode != 'en'
* For most tests, subclassing MediaWikiLangTestCase instead of
  MediaWikiTestCase works.
* The Site library needs to not reuse its namespace cache if the
  language changed.
* ScribuntoExceptions are still being localized for some reason; test
  the message and args of the exception instead of using getMessage().

Bug: T69483
Change-Id: I1c0ca3b11b6228654b47ceb92c1b1be7bda6709a
2015-02-19 16:05:17 +00:00
Brad Jorsch 10bc0f7316 Adds support for JSON encoding and decoding
Provides methods to encode and decode JSON in the mw.text module.

Bug: T47470
Change-Id: I274f2ff13adb616e50600ee30e29b35327f3251e
2015-01-26 15:13:22 -08:00
Brad Jorsch ccfcbfc17f Update unit tests for recent core changes
* I0ede3169 caused Language::fetchLanguageNames() to use 'en' rather than
  null when given an invalid language code. This didn't break Jenkins
  since, without cldr, it always uses null anyway.
* I4bead5f5 caused Language::listToText() to start encoding
  single-quotes in messages such as 'and'.

Bug: T85854
Change-Id: Ic2807be8dad3ef029652e4b0db05e4dd8a6631b9
2015-01-08 23:23:59 +00:00
Mr. Stradivarius c58c528d28 Add mw.site.interwikiMap
This makes the interwiki map available to Lua modules. The code is
based on the API interwiki map code in core (the appendInterwikiMap
method of includes/api/ApiQuerySiteInfo.php.) Everything that the
API includes is added, apart from iw_api and iw_wikiid, which I
couldn't think of a use for from Lua modules.

Accessing the interwiki map would be useful for modules like
enwiki's Module:InterwikiTable,[1] as it would stop module writers
having to duplicate the data.

[1] https://en.wikipedia.org/wiki/Module:InterwikiTable

Change-Id: Ie8ad2582aaf5e422824f7da51714a347bb4041d1
2014-12-24 01:17:48 +09:00
Jackmcbarn 4002f43ef2 Use a metatable when os.date("*t") is called
When os.date("*t") or ("!*t") is called, instead of just setting the TTL
to 1 second, create a metatable that sets TTLs as the values are looked
at.

Change-Id: Id1e2df731f182f21cf19708738f9907fa927185c
2014-12-19 03:46:23 +00:00
Jackmcbarn ce5ac6611d Avoid unnecessary database queries
Currently, mw.title.new always results in a database query, which holds up
the parse until it finishes. This changes it to not require a database
query if it's not actually necessary.

Bug: T68328
Change-Id: I62f347d4cd9176bd0440215dcbe804c1dc3d4c99
2014-12-11 13:34:06 -05:00
Mr. Stradivarius 98f25aa9a1 Improve error messages in mw.html
Add more information to error messages in mw.html. This includes the
error level, the function name, and the position of the argument in the
argument list. Where possible, use the functions in libraryUtil.lua to
do this.

Some functions in mw.html accept multiple types, so add a checkTypeMulti
function to libraryUtil.lua to make these kinds of functions easy to check.
And while we're at it, add test cases for libraryUtil.lua as well.

Change-Id: If9cf9a52bd4b1bb42cc7f9f1f1096828710cbc52
2014-12-08 17:01:31 +00:00
jenkins-bot 6f7349dcc8 Merge "Add mw.text.unstripNoWiki, mw.text.killMarkers, fix mw.text.unstrip" 2014-12-01 21:42:17 +00:00
Thiemo Mättig 55fe0b69d0 Add missing visibility keywords everywhere
Change-Id: I270d1dd9b6545e15398c2f8b8e9ae533844cc998
2014-11-14 10:10:23 +01:00
Thiemo Mättig 61af67b819 Fix visibility of setUp/tearDown
Required for Idf44d31.

Change-Id: If6d270549290bed2d1c7617da0fedbd385f3e96c
2014-11-12 12:28:58 +01:00
Brad Jorsch e5564cf942 Add mw.text.unstripNoWiki, mw.text.killMarkers, fix mw.text.unstrip
mw.text.unstrip is too broad, it's allowing for unstripping things that
cause problems when unstripped (e.g. bug 61268). Since the original
request was only for unstripping <nowiki>, let's add a function that
does only that.

We should also add an interface to StripState::killMarkers(), instead of
requiring everyone to roll their own work-alike.

Then, to fix the bug, we can make mw.text.unstrip be the combination of
the two. This is the most like the original behavior of mw.text.unstrip
(removes all strip markers, replacing them with text where applicable)
without causing issues.

Bug: 61268
Change-Id: I3a151fd678b365d629b71b4f1cb0d5d284b98555
2014-11-05 12:32:35 -05:00
Brad Jorsch df38a296bf Allow for dynamically-loaded PHP libraries
Scribunto currently supports libraries with PHP callbacks that are
loaded on startup, and pure-Lua libraries that may be loaded from the
module with require().

This change allows for libraries with PHP callbacks to also be loaded
with require().

Change-Id: Ibdc1f4ef51b1c8644c3d4c98d57755b5c06447a5
2014-10-03 09:27:23 -04:00
Jackmcbarn 634f75f53e Don't escape the delete character
Escaping the delete character breaks strip markers, so don't do it.

Bug: 68011
Change-Id: Ica97c898209c59c0084bf700d891b28603f79dd1
2014-09-21 22:59:52 -04:00
Jackmcbarn c5959b56e7 Display the full error message in wikitext
Instead of just displaying "Script error" in wikitext, display the actual
error string. Currently, many users don't see meaningful error messages
generated by modules, and instead just assume that "Script error" means
the module itself has a bug.

Bug: 71038
Change-Id: Ie5358049975352e0fef92088a06b97ad94717000
2014-09-19 12:11:38 +00:00
Jackmcbarn b970046f2e Don't output a semicolon at the end of CSS
It's not necessary, it makes the output bigger, and some pages have enough
elements with CSS that it does make an actual difference.

Change-Id: I80d471899c7e04a8a4876c205198a8c0d0b1f281
2014-09-10 19:08:34 -04:00