When caching results from frame:preprocess and frame:expandTemplate,
restrict the scope of the cache to the frame object that was used. This
allows the integrity of the empty-frame expansion cache to be maintained
while also allowing parent frame access. This change is the equivalent of
I621e9075 in core.
Change-Id: Iae4c00e7e19ba12cfdaac135be16c991d9d0cea1
The new Scribunto_LuaTitleLibrary::redirectTarget() method is
used by mw.title objects as read-only attribute 'redirectTarget'.
If the page does not exist or it is not a redirect, the value
of the attribute is `false`; otherwise, it is the target of the
redirect page, as mw.title object.
This is a proper alternative to parsing wikitext as it is done in:
https://en.wikipedia.org/wiki/Module:Redirect
Bug: T68974
Change-Id: Id4d9b0f8c1cd09ebc42c031d4d3fc0c33eea44aa
Apparently microtime is a bit unreliable for this test for some reason,
so let's use getrusage() to measure actual CPU time instead.
Bug: T125045
Change-Id: Ia1ab6b043b99abb4fc6c2989ad09a24d97dd09c3
HHVM has a bug in json decode https://github.com/facebook/hhvm/issues/5813 .
Skip the test until there is a fix upstream.
Bug: T103346
Change-Id: I7e44d98c29ba9b9f5443665fc046382f696193c9
Lua actually treats a close-bracket at the start of a bracketed
character class as a literal, rather than using it to close the
character class. Probably unintended behavior, but it happens.
Also, have the pure-lua version throw our more informative errors on
error even when falling back to string.find and the like, and fix some
other weird edge cases that came up in testing.
Bug: T95958
Bug: T115686
Change-Id: Iab783d4a3e58b1514cc09729d4a71c2cb1242ee8
A string with a dot pattern is only "simple" if
followed by +, - or *. The end of string condition was not checked
properly.
Change-Id: Ia10b9164caeabe464c76441cc82eef37a7013048
And now Ie9e334e broke the tests again. Why can't anyone be bothered to
fix things they should *know* by now are going to need fixing?
Bug: T90018
Change-Id: I7f71dda4af38b23f89c64b6116c1145c44f5c523
The parser's output was changed in I562e437e to add clickable anchors
for section headers which made Scribunto parser tests fail.
Bug: T90018
Change-Id: Ia74877e0641f2763d42be4a8fad1aa8ecffe449e
Should be good to apply now after Wikibase has been adopted to
this.
This reverts commit 06f03fe09b.
Bug: T69483
Change-Id: I28073a891b8847fc6aa5ad026bf702e4015f7d4a
* For most tests, subclassing MediaWikiLangTestCase instead of
MediaWikiTestCase works.
* The Site library needs to not reuse its namespace cache if the
language changed.
* ScribuntoExceptions are still being localized for some reason; test
the message and args of the exception instead of using getMessage().
Bug: T69483
Change-Id: I1c0ca3b11b6228654b47ceb92c1b1be7bda6709a
* I0ede3169 caused Language::fetchLanguageNames() to use 'en' rather than
null when given an invalid language code. This didn't break Jenkins
since, without cldr, it always uses null anyway.
* I4bead5f5 caused Language::listToText() to start encoding
single-quotes in messages such as 'and'.
Bug: T85854
Change-Id: Ic2807be8dad3ef029652e4b0db05e4dd8a6631b9
This makes the interwiki map available to Lua modules. The code is
based on the API interwiki map code in core (the appendInterwikiMap
method of includes/api/ApiQuerySiteInfo.php.) Everything that the
API includes is added, apart from iw_api and iw_wikiid, which I
couldn't think of a use for from Lua modules.
Accessing the interwiki map would be useful for modules like
enwiki's Module:InterwikiTable,[1] as it would stop module writers
having to duplicate the data.
[1] https://en.wikipedia.org/wiki/Module:InterwikiTable
Change-Id: Ie8ad2582aaf5e422824f7da51714a347bb4041d1
When os.date("*t") or ("!*t") is called, instead of just setting the TTL
to 1 second, create a metatable that sets TTLs as the values are looked
at.
Change-Id: Id1e2df731f182f21cf19708738f9907fa927185c
Currently, mw.title.new always results in a database query, which holds up
the parse until it finishes. This changes it to not require a database
query if it's not actually necessary.
Bug: T68328
Change-Id: I62f347d4cd9176bd0440215dcbe804c1dc3d4c99
Add more information to error messages in mw.html. This includes the
error level, the function name, and the position of the argument in the
argument list. Where possible, use the functions in libraryUtil.lua to
do this.
Some functions in mw.html accept multiple types, so add a checkTypeMulti
function to libraryUtil.lua to make these kinds of functions easy to check.
And while we're at it, add test cases for libraryUtil.lua as well.
Change-Id: If9cf9a52bd4b1bb42cc7f9f1f1096828710cbc52
mw.text.unstrip is too broad, it's allowing for unstripping things that
cause problems when unstripped (e.g. bug 61268). Since the original
request was only for unstripping <nowiki>, let's add a function that
does only that.
We should also add an interface to StripState::killMarkers(), instead of
requiring everyone to roll their own work-alike.
Then, to fix the bug, we can make mw.text.unstrip be the combination of
the two. This is the most like the original behavior of mw.text.unstrip
(removes all strip markers, replacing them with text where applicable)
without causing issues.
Bug: 61268
Change-Id: I3a151fd678b365d629b71b4f1cb0d5d284b98555
Scribunto currently supports libraries with PHP callbacks that are
loaded on startup, and pure-Lua libraries that may be loaded from the
module with require().
This change allows for libraries with PHP callbacks to also be loaded
with require().
Change-Id: Ibdc1f4ef51b1c8644c3d4c98d57755b5c06447a5
Instead of just displaying "Script error" in wikitext, display the actual
error string. Currently, many users don't see meaningful error messages
generated by modules, and instead just assume that "Script error" means
the module itself has a bug.
Bug: 71038
Change-Id: Ie5358049975352e0fef92088a06b97ad94717000
It's not necessary, it makes the output bigger, and some pages have enough
elements with CSS that it does make an actual difference.
Change-Id: I80d471899c7e04a8a4876c205198a8c0d0b1f281
If a function name contains an equals sign, it doesn't increase the index,
so indexOffset=1 makes Lua see arguments as starting with 0. This change
fixes that, so that indexOffset is only set to 1 if the function name has
an index.
Change-Id: Ifa2dfad4d2f7228c3fd5721ccfc4c235576b663d
Add a way to fetch cascading protection information from Lua without
needing to call the CASCADINGSOURCES parser function.
Change-Id: I1b3ac18af11d3066f78d27b31da8d6709a6a2631
The pure-Lua ustring pattern matching functions short-circuit to the
much faster string library when the pattern would match the same against
the raw bytes.
A pattern like "[^a-z]" can match a partial UTF-8 character when applied
bytewise, and so must be detected as unsafe.
Let's also directly test the pure-Lua module, instead of me having to
comment out lines in Scribunto_LuaUstringLibrary::register() whenever I
want to test them.
Change-Id: I91ed3374aadfea379b9db2e13b4248ab20df509e
Clean up trailing whitespace from all of our code, and add comments
indicating that apparently unused variables are ScopedCallbacks.
Change-Id: I8e5997797cc7b1c64c5351ec112a18f30edc8fef
Two similar bugs are handled here:
* mw.getCurrentFrame() doesn't work when the module is loaded (only when
a function is called), which breaks os.date and os.time at module
scope since I59ad364d.
* mw.getCurrentFrame() gives access to frame args from inside
mw.loadData, which allows for data leakage between #invokes.
Bug: 67498
Bug: 65687
Change-Id: I82dde43e2601b59c03c6ed4b9365829c40a953a5
From the PHP subclass of Scribunto_LuaEngineTestBase, Lua tests run by
Scribunto_LuaEngineTestBase::testLua() may be skipped by adding an entry
for the test name to $this->skipTests.
From the Lua code run by Scribunto_LuaEngineTestBase::testLua(), tests
may be skipped by calling the "markTestSkipped" function exported by the
TestFramework module.
Also, use this new mechanism to skip certain mw.language tests if
Extension:CLDR or a similar extension is not available.
Bug: 67343
Change-Id: I4f0b15073a84bf2f9d8a5d905c3c960941dd2b71
Some functions in mw.html accept numbers as arguments, but later fail when
constructing the string. This disallows numbers in attribute names, since
they aren't valid anyway, and fixes the remainder of the cases to properly
build the string.
Bug: 67201
Change-Id: Ie7bcbb9d8df580dd8793681f78a8b0719d8a287a
Lua's string functions tend to auto-convert numbers to strings. We
should do the same in mw.ustring.
Bug: 67201
Change-Id: Icd3c5e93bac19dafd78d737ec9b315daba9f1729
Certain wikitext, such as that containing Cite.php <ref> or <references>
or the #tag versions of the same, should not be cached. This uses the
isVolatile method added to PPFrame in I95b3cf87 to avoid caching the
preprocessed output of such wikitext from frame:preprocess and similar
methods.
Bug: 46815
Change-Id: I1084f87fd863eb22f2f3f3d3ff308b24e20a08ef
When os.date, os.time, or mw.language:formatDate are called, set the
appropriate TTL on the output. This needs I412febf3 in core to function at
all, and I3f5a80aa in core to function with formatDate.
Change-Id: I59ad364d502fc247500d94c5606516ad9f98a24d
Rather than calling error() when nils get passed to mw.html methods,
either remove whatever it was that the nil would go to (if that makes
sense), or just do nothing. The seemingly inconsistent use of "not x" and
"x ~= nil" is to allow any falsey value where it wouldn't be ambiguous
(such as class names), but not where it could be (such as attribute values).
Bug: 62982
Change-Id: I76773abbb4394aa9bb8c8a08445e019cade3b2bf
When tables are passed from Lua to PHP, their metatables are lost. Because
of this, they need to be kept inside of Lua to allow the __index
metamethod to return a method to be called by #invoke.
Bug: 64141
Change-Id: I0840bc12b25dee72828ec97d2b205812e4929f2b
From wikitext, $parser->callParserFunction() will always get an array of
strings with at least an element [0]. Let's match this from Scribunto:
stringify numbers, and require that [0] (although in Lua it'll be [1]).
Also fix an old broken unit test.
Bug: 63597
Change-Id: Ie7ac34ae4bce70cec455d90c3f02a658644f6866
When displaying a nosuchfunction or nosuchmodule error, include the name
of the nonexistent function or module.
Change-Id: I17fc2c68dc8267302a82eee3cb2c5df9b5a3c46c
It's possible to pass information between multiple #invokes on a page by
having the first call math.randomseed with one of a set of known seeds
and then having the second examine the output from math.random to
determine which of those known seeds was used.
Prevent that by calling math.randomseed( 1 ) when invoking (see the bug
for details on why that seed). But avoid doing so if e.g. a
frame:expandTemplate() call results in a recursive invoke.
Bug: 62291
Change-Id: Id01cb63eca52ced29bf4efebc38beb9f159b7b0e
Various methods are throwing exceptions when passed invalid language
codes. Those need to be caught.
And we should really add unit tests for the mw.language library, too.
Doing so exposed another bug (in lang:gender), which is also fixed here.
Bug: 62242
Change-Id: Ib7d257cbb1ce179c510273526910d6ac5f3cac5d
The LuaStandalone interpreter needs to keep a mapping from integers
returned to PHP to the corresponding function. But if it never releases
these functions when PHP no longer has any reference to them, it can
result in Lua running out of memory if a module with a large number of
functions is invoked many times in one page.
The fix here is to track which function ids are referenced from PHP, and
periodically send the list to Lua so it can remove any that are no
longer used from its cache.
This also takes care of another issue where having multiple interpreter
instances and passing function objects from one into another could call
the wrong function in Lua.
Bug: 51886
Change-Id: I4f15841051f7748d1d6df24080949e5cbd88f217
Message formats other than plain should have never been exposed in this
way, as they allow link tables, etc. to be bypassed and serve no useful
purpose. This removes them, and also removes title, as it serves no
purpose without them.
Bug: 60758
Change-Id: I96284ffbe986a9cd92d2bde1ffdb746029bad989
If we don't do this, then the section edit links point to the wrong page
if we expand a template that contains section headings.
Bug: 55525
Change-Id: I00bda935be3e8b9c0f86fd0f131814207fbb34a7
Include a protectionLevels variable in the output of the mw.title.*
functions, containing the contents of the title's mRestrictions
array (i.e., its protection levels)
Change-Id: I79c9fed64bacfc90aee1d411a3e1b47e44c99755
PHP can't handle having arrays/objects or functions as keys in its
arrays, so make sure we don't try to pass them from Lua. Booleans aren't
really well-handled either, so let's disallow them too.
Also, add tests for proper stringification of floats and infinities when
those are used as keys.
Note this behavior change is needed to match the change in LuaSandbox
for fixing bug 54527, but isn't itself a security issue.
Change-Id: I1e2951bbe8cb78358650ad377bf7119fcac4485d
A module for building complex HTML from Lua using a
fluent interface. The module is originally from enwiki,
but the authors allowed us to reuse it under GPLv2+
(as stated in the file).
The module will be loaded per default and comes with
unit tests.
As discussed on wikitech-l:
http://lists.wikimedia.org/pipermail/wikitech-l/2013-December/073320.html
Change-Id: I7c8d4378091c13d5ace0dd1fcbb4e27163e8c896
This field already exists in PHP with exactly the content requested in
bug 47089, so we may as well expose it on the frame object.
Bug: 47089
Change-Id: I672820589f6ebc7c4daad29b5eb156733a5bc5cc
It's already possible to detect whether the current template is being
substituted via ParserFunctions (see [[en:Template:Ifsubst]]), and a
similar trick works with frame:preprocess. So we may as well provide the
flag directly.
Bug: 47828
Change-Id: Id06d27c6283ee589a8830b78c04e56978e0ac6da
Specifically:
* String conversion in non-URL contexts (e.g. .prefixedText) uses spaces
instead of underscores.
* Setting .fragment now applies the same transformations that are done
(in PHP) by mw.title.new.
Bug: 56217
Change-Id: I12e354636bcde3327864088175fb4de61aecc81a
The PHP call that makes mw.site.namespaces work case-insensitively
doesn't handle non-standard spaces/underscores. So standardize them
before the call.
Bug: 56216
Change-Id: I4758478b126858fb581614f64eb15472f42fef51
A few edge cases were being incorrectly handled:
* mw.ustring.sub( 'abc', 1, 0 ) returned 'a', not ''.
* mw.ustring.codepoint( 'abc', 1, 0 ) returned 97, not no results.
* mw.ustring.codepoint( 'abc', 4, 4 ) returned 99, not no results.
* mw.ustring.gcodepoint had the same issues as mw.ustring.codepoint.
Change-Id: Ib8c0ef5a8073106eb7d90d0aa0513be4525dca08
Negative values for 'i' in mw.ustring.byteoffset are supposed to count
from the end of the string. But in LuaSandbox, it was actually counting
from two bytes before the end of the string due to a typo.
Fix that, and add some tests for it.
Bug: 50176
Change-Id: Iceee1022a55abd7a08df1ea7843e1277eb02798b
The "%f[set]" frontier pattern has been in Lua 5.1 since the beginning,
but was undocumented until Lua 5.2. And the code is even unchanged from
5.1.0 to 5.2.1. So there's no reason not to implement it in ustring too.
Note the changes to UstringLibrary.php are somewhat large, because it
splits the "convert a Lua bracketed charset to PCRE" code into a
separate function and it changes the handling of mw.ustring.find's and
mw.ustring.match's 'init' parameter from "substring, match from 0, then
add back on $init" to "use preg_match's $offset and use \G instead of ^
where this matters". Both of these are necessary to properly support
%f.
This also fixes a bug in the pure-Lua code (not used in Scribunto)
exposed by the unit tests for %f where %z was matching '\1' rather than
'\0' and %Z everything except '\1' instead of everything except '\0'.
Bug: 48331
Change-Id: Ie0b95ef5b734db53d6adc9de5dae4874f8944c08
The following errors are fixed:
* PHP warning and wrong return value with empty pattern and plain
* Incorrect offsets returned when init is larger than the string length
* Incorrect captured offsets returned when init is excessively negative
Bug: 47365
Change-Id: I9741418287dc727747326d6a19678370ce155a2b
Users are reporting disappointment that using pairs on title objects
doesn't let them see the fields available. It's easy enough to add a
__pairs metamethod to allow pairs to work on title objects, so let's do
that.
The same can be done for mw.uri objects.
For mw.message objects, we can easily enough change the implementation
to be like mw.language objects, which doesn't have this problem.
For mw.site.stats, we may as well just remove the load-on-demand feature
since it will be demanded as soon as the environment is cloned for the
first module.
Change-Id: Ie5a3b5684b2bb6c090f9994baa03977687688929
mw.title.new( pageid ) should not throw an error for an nonexisting
pageid, just return nil. Similarly, it should always return nil for 0,
rather than returning the last non-existent title created.
Change-Id: I3cdbb24fc785aef0f8e75fba1feccd26ac5b7370
This exists for some common text-processing functions that aren't
included in string (and therefore also aren't in mw.ustring), as well as
a logical place for the "unstrip" function requested in bug 45085.
Bug: 45085
Change-Id: I47356215fcc8ddeed5f901cd933a30021394bd78
It would be helpful for debugging if a frame object could be gotten in
the console. To that end, add an empty frame when running in the console
and allow it to be returned by mw.getCurrentFrame().
It would also be helpful to be able to create frames with arbitrary
arguments, again for testing. Fortunately support for creating child
frames with arbitrary arguments already exists in core, so we can just
use it.
And for good measure, be sure to restore the $engine->currentFrames
array even if the Lua code throws an exception.
Change-Id: I1dc8602d63af75424f267c42a3743fabbc1827f7
Currently, the time taken to parse the arguments passed to a Lua
function from #invoke will be counted against Lua's 10-second limit.
This is counterintuitive, and can remove incentive for users to convert
templates to Lua since they may have to convert a whole stack at once.
Note this requires change I11881232 to mediawiki/php/luasandbox to
actually have any effect.
Bug: 45684
Change-Id: I773950e4c399b8a1cfa6d1cde781a069d286b3bd
Since the content of a page is being loaded, this should be recorded in
templatelinks so things can be updated properly when that page is
edited.
Change-Id: I3e720fee2705f6c08ac0456c3cab0ed4ede84536
In the unit tests, the engine is given a parser, but the parser is not
being associated back with the engine. So in some cases, that could lead
to *another*, default engine being created for certain operations.
Fix that.
Change-Id: I79995c2635d9e470931b84dc1854dae26772bbe3
Apparently assigning to $wgHooks at the top of the file breaks every
other hook, somehow. So we need to set it in setUp(), but prevent the
interwiki prefix from being queried beforehand when the dataProvider is
run. Fun.
Change-Id: I8535942e3a0e47d9a5b2ea11c1b0b63c5acf2cc5
* mw.ustring.sub( '', 1 ) errors in LuaStandalone
* Default value for ustring.maxStringLength and ustring.maxPatternLength
should be infinity, not nil
* mw.ustring.find() returns one value instead of two in "plain" mode.
Change-Id: I5e65c4ec3a05f0e6930ce7ab7fd4ac72bea95e7f
Clean up the modules in engines/LuaCommon/lualib:
* Fix luabit/bit.lua to return its table instead of trying to set the
global directly.
* Fix luabit/hex.lua to return its table instead of trying to set the
global directly.
* luabit/noki.lua is useless for our purposes
* luabit/utf8.lua is redundant to mw.ustring
* stringtools uses coroutines, which we don't support
Also fix a parser test that has apparently been broken for a long time.
Change-Id: I1284cddb6e9b94327964cb1077d8dbdf7def6d06
Note that fetching any title besides the one for the current page is
considered "expensive". It also records the title fetched in the
ParserOutput so it will be listed in pagelinks, just like #ifexists.
This also moves the ToString test formatter into TestFramework.lua, so
TitleLibraryTests.lua can use it too.
Change-Id: I799f3289a37fe1349b6bca5758829acf82cb718f
In phpunit 3.6.10, assertEquals considered two NaNs to be equivalent.
Somewhere between that and phpunit 3.7.14, this behavior changed so NaNs
are no longer considered equivalent.
Change-Id: I2c664498eb34cf5119a2eaaa96a6be57b821ab94
People sometimes want to load large tables of constant data from a
module. Using require(), this has to be reparsed every time, which can
be slow.
mw.loadData() will load the just data once, and return a table with a
metatable cleverly designed to give read-only access to the loaded data.
Change-Id: Icec192bdbe6cfca7902fd5cb5d0e217bd8399637
Allowing a module to call mw.makeProtectedEnvFuncs() lets it bypass the
allowEnvFuncs setting. It can also be used to manipulate the global
tables that other modules' sandboxes will be copied from.
And for paranoia's sake, let's tighten up what setfenv is allowed to
set. This requires changing a unit test, because it is no longer
sane to do something like
env.setfenv, env.getfenv = mw.makeProtectedEnvFuncs( { [env] = true }, {} )
Nothing real does this, it was only in the unit test.
Change-Id: I8e0d83bb0980ee869af3ac4413afd211717ca92f
