Commit graph

196 commits

Author SHA1 Message Date
Brad Jorsch 29eb0f4ea6 Improve mw.text.nowiki
The following are now correctly escaped:
* Space at the start of a line
* Start-of-line characters after \r
* Magic links such as "RFC 123" with non-space whitespace
* URIs that don't use "://", such as "urn:foo"
* Double-underscore magic words

Bug: 53658
Change-Id: I824417e2937dd27cd1e69bd4e74ab7d21a978c75
2013-09-03 17:36:36 -04:00
jenkins-bot 99b96d8b14 Merge "Add frontier pattern (%f[set]) to ustring" 2013-08-30 06:22:40 +00:00
Brad Jorsch 8e6e07c75f Handle backtrace where Lua can't guess the function name
Current logic is to display the funciton name if Lua provides us with
one, "main" if it's at the main level, or "?" if it's a C function or a
tail call. But we're not handling if it's a Lua call but Lua can't guess
a name for the function; use "?" for that too.

Change-Id: I938b5e5ca55cf4990dbcbb0db8dd8fc93b03bf15
2013-08-28 11:55:55 -04:00
Brad Jorsch d9ba200571 Add mw.logObject
People have requested a method to log a table as something more detailed
than just "table", to be able to inspect values while debugging.

Bug: 48173
Change-Id: Ia58cab834e87842927a2a13d153ee32473f74086
2013-07-26 05:15:20 +00:00
jenkins-bot 3e2972394f Merge "Improve disable_functions handling in LuaStandalone" 2013-07-24 20:16:17 +00:00
Brad Jorsch c7246a1c42 Improve disable_functions handling in LuaStandalone
If the user is on a webhost that has proc_open listed in PHP's
disable_functions directive, we should give a better error message.
Until we no longer support PHP below 5.4, we should do the same for
safe_mode. And since we're doing that, we may as well report any other
warnings if proc_open fails, too.

In addition, this cleans up error handling in
Scribunto_LuaEngine::load() so it doesn't pretend the interpreter is
loaded if getInterpreter() throws an exception. Otherwise other code
winds up with PHP fatal errors trying to access a null value.

Bug: 50706
Change-Id: I2887b722e089fd7a526aa7dcab9c80deb343d8ac
2013-07-24 15:01:36 -04:00
jenkins-bot 635bae4786 Merge "Fix mw.ustring edge cases" 2013-07-11 18:11:51 +00:00
jenkins-bot 0f45f9e5cb Merge "Make lang:parseFormattedNumber more like tonumber" 2013-07-08 21:15:25 +00:00
Brad Jorsch cfdd099ca8 Fix fatal in frame:callParserFunction
If the parser function returns 'isChildObj', we need to create a child
frame to expand the wikitext returned by the parser function. And when
we pass the arguments to the new frame, we need to pass them through the
preprocessor's newPartNodeArray() first.

Bug: 50863
Change-Id: Ieb7cc7007288de1f7d2cd2458f068affe695e8af
2013-07-07 07:43:31 -04:00
Brad Jorsch 25cbfd776d Make lang:parseFormattedNumber more like tonumber
Users seem to expect that mw.language's parseFormattedNumber will act
like tonumber when given nil or other non-string values, returning nil
instead of raising an error. There's no reason not to, so we may as
well.

Change-Id: Ie0ff19efc84ca738e115bbd524bfd92fccf26127
2013-07-03 16:06:32 +00:00
Brad Jorsch d8314539da Fix mw.ustring edge cases
A few edge cases were being incorrectly handled:
* mw.ustring.sub( 'abc', 1, 0 ) returned 'a', not ''.
* mw.ustring.codepoint( 'abc', 1, 0 ) returned 97, not no results.
* mw.ustring.codepoint( 'abc', 4, 4 ) returned 99, not no results.
* mw.ustring.gcodepoint had the same issues as mw.ustring.codepoint.

Change-Id: Ib8c0ef5a8073106eb7d90d0aa0513be4525dca08
2013-07-03 11:49:52 -04:00
Brad Jorsch ad3e5a7c48 Fix LuaSandbox mw.ustring.byteoffset with negative 'i'
Negative values for 'i' in mw.ustring.byteoffset are supposed to count
from the end of the string. But in LuaSandbox, it was actually counting
from two bytes before the end of the string due to a typo.

Fix that, and add some tests for it.

Bug: 50176
Change-Id: Iceee1022a55abd7a08df1ea7843e1277eb02798b
2013-06-25 10:06:01 -04:00
Brad Jorsch 82820aafc8 Add frontier pattern (%f[set]) to ustring
The "%f[set]" frontier pattern has been in Lua 5.1 since the beginning,
but was undocumented until Lua 5.2. And the code is even unchanged from
5.1.0 to 5.2.1. So there's no reason not to implement it in ustring too.

Note the changes to UstringLibrary.php are somewhat large, because it
splits the "convert a Lua bracketed charset to PCRE" code into a
separate function and it changes the handling of mw.ustring.find's and
mw.ustring.match's 'init' parameter from "substring, match from 0, then
add back on $init" to "use preg_match's $offset and use \G instead of ^
where this matters". Both of these are necessary to properly support
%f.

This also fixes a bug in the pure-Lua code (not used in Scribunto)
exposed by the unit tests for %f where %z was matching '\1' rather than
'\0' and %Z everything except '\1' instead of everything except '\0'.

Bug: 48331
Change-Id: Ie0b95ef5b734db53d6adc9de5dae4874f8944c08
2013-05-12 10:27:36 -04:00
Brad Jorsch d6f3633428 (bug 47365) Fix edge cases in mw.ustring.find, mw.ustring.match
The following errors are fixed:
* PHP warning and wrong return value with empty pattern and plain
* Incorrect offsets returned when init is larger than the string length
* Incorrect captured offsets returned when init is excessively negative

Bug: 47365
Change-Id: I9741418287dc727747326d6a19678370ce155a2b
2013-05-10 06:00:02 +00:00
Brad Jorsch 3622f82bd1 (bug 47300) Fix sandboxing with require
Two related issues:
* The package module was inheriting the loaders from the outer sandbox,
  so loaded modules were being loaded into the outer sandbox's
  environment.
* mw.loadData was using the outer sandbox's require(), so again loaded
  modules were being loaded into the outer sandbox's environment.

Bug: 47300
Change-Id: I48d8dd4784c9a890e3abb6389f96f38e1420dbbb
2013-04-24 06:19:44 +00:00
jenkins-bot 24843707df Merge "(bug 47268) lang:parseFormattedNumber should return a number, not a string" 2013-04-22 19:18:13 +00:00
Brad Jorsch 203b3e75dc Don't pass negative years to Language::sprintfDate
Language::sprintfDate cannot handle negative years, so don't pass them.
Return an appropriate error instead.

Change-Id: Ifb633631df98c82aa1c3dc8a555b91f77faf15ed
2013-04-19 17:59:48 -04:00
Brad Jorsch 821ba409d8 (bug 47268) lang:parseFormattedNumber should return a number, not a string
The documentation, and the expectation of users, is that
lang:parseFormattedNumber() should actually return a number, not a
string.

Bug: 47268
Change-Id: Ieabddd0d9192f1fd8ef7e890d5d6268be9636f38
2013-04-15 20:53:47 -04:00
Brad Jorsch 581b2306af Remove mw.getLogBuffer and others from public interface
One of the design goals of Scribunto is that each #invoke should be
independent. Creative use of mw.log and mw.getLogBuffer can get around
this, passing information from one #invoke to the next.

This patch takes the simple solution of removing mw.getLogBuffer from
modules' environments. For good measure, it also removes
mw.clearLogBuffer and mw.executeModule.

Some minor cleanup of the console code is also included.

Change-Id: I30d73928bade4a6bdd7c00ffcd58a3858ff55698
2013-04-14 23:59:07 +00:00
Brad Jorsch ca3b0b55b2 Add timezone support to mw.lang:formatDate
This uses the timezone support added to core Language::sprintfDate in
Iea1f7842 to add support for various timezone-related formatting
characters, and to correct the output for 'c', 'r', and 'U' in local
mode.

This is related to bug 33454, which requests the same for
ParserFunctions' {{#timel:}}.

Bug: 33454
Change-Id: I1b92dc671051a6cc53e35ebd74d383448e16696c
2013-04-02 23:15:16 +00:00
Brad Jorsch adb7b5f939 Make pairs() work with various library objects
Users are reporting disappointment that using pairs on title objects
doesn't let them see the fields available. It's easy enough to add a
__pairs metamethod to allow pairs to work on title objects, so let's do
that.

The same can be done for mw.uri objects.

For mw.message objects, we can easily enough change the implementation
to be like mw.language objects, which doesn't have this problem.

For mw.site.stats, we may as well just remove the load-on-demand feature
since it will be demanded as soon as the environment is cloned for the
first module.

Change-Id: Ie5a3b5684b2bb6c090f9994baa03977687688929
2013-04-02 17:23:42 +00:00
Brad Jorsch 4374903e83 (bug 46635) Recognize Windows path+drive letter
The logic for recognizing absolute versus relative paths needs to take
into account the possibility of a Windows drive letter.

Bug: 46635
Change-Id: I3a43acac2f6e8b481807e1babe5a261b9eb1fe23
2013-03-29 09:26:13 -04:00
jenkins-bot a85275592c Merge "Add text module" 2013-03-27 16:55:07 +00:00
Brad Jorsch 3d4a81653c (bug 46405) Fix errors in mw.title.new( pageid )
mw.title.new( pageid ) should not throw an error for an nonexisting
pageid, just return nil. Similarly, it should always return nil for 0,
rather than returning the last non-existent title created.

Change-Id: I3cdbb24fc785aef0f8e75fba1feccd26ac5b7370
2013-03-21 09:39:25 -04:00
Brad Jorsch 0db3d7c6d2 Add text module
This exists for some common text-processing functions that aren't
included in string (and therefore also aren't in mw.ustring), as well as
a logical place for the "unstrip" function requested in bug 45085.

Bug: 45085
Change-Id: I47356215fcc8ddeed5f901cd933a30021394bd78
2013-03-20 10:10:15 -04:00
Brad Jorsch 541e61a1c4 (bug 41769) Add frame:callParserFunction() and frame::extensionTag()
Requires change I339b882010dedd714e7965e25ad650ed8b8cd48f to
mediawiki/core.

Bug: 41769
Change-Id: I0138836654b0e34c5c23daaedcdf5d4f9d1c7ab2
2013-03-18 13:46:26 -04:00
Brad Jorsch 53c46561ab Don't record self-link from mw.title
Every Lua-using page loads the "self" title for mw.title. But we don't
want to record this self-link in the links table, because that confuses
people.

Change-Id: I1ef57daa615c176b7789f5f09bd15ee5b292ad0c
2013-03-17 14:55:04 -04:00
jenkins-bot b59117fbdd Merge "Portable DIRECTORY_SEPARATOR :)" 2013-03-14 14:35:13 +00:00
Jens Ohlig 8fd450269c Portable DIRECTORY_SEPARATOR :)
Change-Id: I61d2ddc5367c2ffacd68c12bf9b6037cfc39d487
2013-03-14 15:09:45 +01:00
Brad Jorsch b6ea64f132 Make mw.getCurrentFrame() work in console, add frame:newChild()
It would be helpful for debugging if a frame object could be gotten in
the console. To that end, add an empty frame when running in the console
and allow it to be returned by mw.getCurrentFrame().

It would also be helpful to be able to create frames with arbitrary
arguments, again for testing. Fortunately support for creating child
frames with arbitrary arguments already exists in core, so we can just
use it.

And for good measure, be sure to restore the $engine->currentFrames
array even if the Lua code throws an exception.

Change-Id: I1dc8602d63af75424f267c42a3743fabbc1827f7
2013-03-12 14:42:36 +11:00
Brad Jorsch 5faa00150c (bug 45684) Don't count argument parsing time towards Lua limit
Currently, the time taken to parse the arguments passed to a Lua
function from #invoke will be counted against Lua's 10-second limit.
This is counterintuitive, and can remove incentive for users to convert
templates to Lua since they may have to convert a whole stack at once.

Note this requires change I11881232 to mediawiki/php/luasandbox to
actually have any effect.

Bug: 45684
Change-Id: I773950e4c399b8a1cfa6d1cde781a069d286b3bd
2013-03-12 03:18:05 +00:00
Chad Horohoe 3ac6904790 Missing global
Change-Id: I78fc2acd31dd2969971e23239d5c820d867a204c
2013-03-11 14:55:23 -04:00
Brad Jorsch 466ba8668e (bug 45774) Add title.fileExists, improve title.exists
People sometimes want to know if the actual file exists, not just the
file description page. Support that.

Also alias .exists to .fileExists for the Media namespace, since that's
what the Media namespace is for.

And while we're at it, fix .exists for special pages too.

Bug: 45774
Change-Id: I019adc89858a1d32995a38d5e8eef32577fd32d6
2013-03-08 10:33:35 -05:00
Brad Jorsch b926846a7a mw.title:getContent() should record in templatelinks
Since the content of a page is being loaded, this should be recorded in
templatelinks so things can be updated properly when that page is
edited.

Change-Id: I3e720fee2705f6c08ac0456c3cab0ed4ede84536
2013-03-07 13:02:41 -05:00
jenkins-bot efe2def71a Merge "Add language functions" 2013-03-07 00:05:54 +00:00
Brad Jorsch e004b415d6 (bug 45512) Add mw.title:getContent()
Bug: 45512
Change-Id: I630033d367a47f7f80809a0918d3e6feede3fe41
2013-03-05 19:05:00 +00:00
jenkins-bot 61a4dccabd Merge "Clean up argument lists" 2013-03-05 18:49:53 +00:00
jenkins-bot fc766f205a Merge "Fix mw.uri error with empty fragment or query string" 2013-03-05 18:49:06 +00:00
Brad Jorsch fcef54e9d9 Fix ustring errors
* mw.ustring.sub( '', 1 ) errors in LuaStandalone
* Default value for ustring.maxStringLength and ustring.maxPatternLength
  should be infinity, not nil
* mw.ustring.find() returns one value instead of two in "plain" mode.

Change-Id: I5e65c4ec3a05f0e6930ce7ab7fd4ac72bea95e7f
2013-03-05 12:22:15 -05:00
Brad Jorsch 328680ed96 Add language functions
RTL support, duration formatting.

Change-Id: I78db53976abfe04fd6529b36c9434e33a3bc90c7
2013-03-04 21:14:34 -05:00
Brad Jorsch 580cda00d0 Fix mw.uri error with empty fragment or query string
Off-by-one error.

Bug: 45655
Change-Id: Ic6e0dfe8dbf392b1fe9e225f7906b4dd84f4febf
2013-03-05 01:44:10 +00:00
Brad Jorsch 5e548e769a Add requirable bit32 library
Following the interface of the Lua 5.2 bit32 library.

Change-Id: I4bc9edc549dbc2d5fdca15ec3a326397a1212a8a
2013-03-05 01:36:17 +00:00
Brad Jorsch e995922bf4 Clean up argument lists
Remove unused parameters from the require() argument lists of two
modules, left over from an earlier phase of development.

Change-Id: Ib55d19fb9adffed2153a9cb3788533c4b70b5e26
2013-03-04 14:55:46 -05:00
Brad Jorsch 69a717c5ba Fix title library saving
getParser() sometimes returns null (eg: on saving edits). In that case,
don't try injecting link information into the (non-existent)
ParserOutput object.

Change-Id: Ief544922228615fbf674305bff95f874d56b7866
2013-03-01 11:07:45 -08:00
Brad Jorsch 307a5b1be8 Clean up lualibs
Clean up the modules in engines/LuaCommon/lualib:
* Fix luabit/bit.lua to return its table instead of trying to set the
  global directly.
* Fix luabit/hex.lua to return its table instead of trying to set the
  global directly.
* luabit/noki.lua is useless for our purposes
* luabit/utf8.lua is redundant to mw.ustring
* stringtools uses coroutines, which we don't support

Also fix a parser test that has apparently been broken for a long time.

Change-Id: I1284cddb6e9b94327964cb1077d8dbdf7def6d06
2013-02-28 18:39:16 -08:00
Brad Jorsch 04a0a580e3 Add mw.title library
Note that fetching any title besides the one for the current page is
considered "expensive". It also records the title fetched in the
ParserOutput so it will be listed in pagelinks, just like #ifexists.

This also moves the ToString test formatter into TestFramework.lua, so
TitleLibraryTests.lua can use it too.

Change-Id: I799f3289a37fe1349b6bca5758829acf82cb718f
2013-02-28 11:57:11 -08:00
Brad Jorsch ba09ba3fde Add mw.message library
Change-Id: I12ca84f848c34f1227ee8acdc8bc04bdfd0b2d97
2013-02-26 22:20:04 -08:00
Brad Jorsch 00d4b711ed Add mw.loadData, for static data loading
People sometimes want to load large tables of constant data from a
module. Using require(), this has to be reparsed every time, which can
be slow.

mw.loadData() will load the just data once, and return a table with a
metatable cleverly designed to give read-only access to the loaded data.

Change-Id: Icec192bdbe6cfca7902fd5cb5d0e217bd8399637
2013-02-21 23:03:53 +00:00
Tim Starling 96615c97b0 Explicitly destroy the loadedLibraries array
Otherwise there is a circular reference and the engine is not destroyed
until the end of the request.

Change-Id: I9c98fe449c19d22e771e11d485e5516059c45329
2013-02-18 10:41:12 +11:00
Brad Jorsch 2d36fbc492 Scribunto language library
Added a language library. Introduced functions which are easy and safe,
most of them already have parser function interfaces.

Change-Id: I4465150f3e16493a15a056f7ddb7787bdf3b0373
2013-02-18 10:41:12 +11:00
Brad Jorsch 4ff1dbd6a7 Document Scribunto_LuaEngine
Some parts of this that really should be documented aren't. Fix that.

Change-Id: I058f47a08a7529a8d2a70e5b50e4532db7bdb720
2013-02-15 18:42:07 -05:00
jenkins-bot 2980cf4c11 Merge "Fix mw.ustring.gmatch and patterns with '^'" 2013-02-15 08:22:11 +00:00
Brad Jorsch 1efe182e40 Hide mw.makeProtectedEnvFuncs from modules
Allowing a module to call mw.makeProtectedEnvFuncs() lets it bypass the
allowEnvFuncs setting. It can also be used to manipulate the global
tables that other modules' sandboxes will be copied from.

And for paranoia's sake, let's tighten up what setfenv is allowed to
set. This requires changing a unit test, because it is no longer
sane to do something like

 env.setfenv, env.getfenv = mw.makeProtectedEnvFuncs( { [env] = true }, {} )

Nothing real does this, it was only in the unit test.

Change-Id: I8e0d83bb0980ee869af3ac4413afd211717ca92f
2013-02-14 16:49:15 -05:00
Brad Jorsch 4dcac2fcd9 Fix mw.ustring.gmatch and patterns with '^'
The Lua manual says this:

 For this function, a '^' at the start of a pattern does not work as an
 anchor, as this would prevent the iteration.

I had interpreted that to mean that a pattern starting with '^' would
never match in gmatch. But further testing reveals that the '^' is just
treated as a literal character: string.gmatch( "foo ^bar baz", "^%a+" )
will match "^bar".

Change-Id: Id91d6ee2db753ce1d6a4f6ae27764691d9e9fdc4
2013-02-14 14:25:55 -05:00
Chad Horohoe c98cc64545 Allow extensions to add libraries
Wikidata has already requested the ability to add libraries into Lua. We
do this in a simple way: add a $wgScribuntoExtraLibraries global, and
load whatever modules someone puts there.

Change-Id: I460b4e7b968eb02dd86620f1e4b50daf1be9e901
2013-02-14 12:38:03 -05:00
Tim Starling ce062407ab Fix further non-local effects of library registration
Fixed several accidental leaks to the global namespace due to missing
"local" declaration. Removed extension of the string table by mw.uri,
same justification as I5d0ddb70.

Change-Id: Iba1bf8e651d4ce05812e4a9a7a074cb6679297a0
2013-02-13 15:40:18 +11:00
Tim Starling f2f866cbdd Remove global side-effects from mw.ustring library registration
The point of putting the unicode library in mw.ustring instead of
ustring was to avoid conflicts with future upstream work, and with other
libraries. It rather defeats the purpose if you then modify the global
string table during module startup.

Users can always set up local aliases if they feel "mw.ustring" is too
much to type.

Change-Id: I5d0ddb70d999aeb6e36e6ddbcdb19922d0274a39
2013-02-13 15:05:22 +11:00
Tim Starling 2d9e3c74ba Merge "Lua ustring implementation" 2013-02-13 03:32:43 +00:00
Brad Jorsch 0a8757baba Lua ustring implementation
This is a reimplementation of Lua's string library with support for
UTF-8.

The entire ustring library is implemented in pure Lua. PHP callbacks are
also available for overrides: in LuaSandbox these are used for almost
all functions, while in LuaStandalone they are used only for the pattern
matching. Also, ustring.upper and ustring.lower are overridden using
mw.language's .uc and .lc if available.

It also includes a bunch of unit tests.

Note that if you download the normalization tests, they may fail under
LuaSandbox if you have PHP's intl extension installed and libicu on your
system is too old.

Change-Id: Ie76fdf8d3a85d0a3d2a41b0d3b7afe433f247af0
2013-02-12 14:26:29 -05:00
Brad Jorsch d6116fa6ba Make pairs and ipairs work with frame.args
Using the 5.2 compatability added in I37efc59a, we can now make
pairs( frame.args ) work.

Change-Id: Iefdca8805b08ea222251f28514b2c92182a7feb3
2013-02-12 06:41:33 +00:00
Brad Jorsch fddf05c056 Make our loader work like Lua's built-in loaders
Our loader for lua files distributed with the extension doesn't work
like Lua's built-in loaders. Fix that.

Change-Id: I7576f0e173c10a589f7ebf2c121230ed66b538fe
2013-02-08 14:20:32 -05:00
Chad Horohoe 3055ac6aa3 Clean up title handling for defaultUrl
getParser() sometimes returns null (eg: on saving edits), so use
the engine's title instead.

Change-Id: I62259b76d6960a934b5e208fdaacbbfc7c2d057c
2013-02-07 15:45:49 -05:00
Brad Jorsch db9fd2b39b Add mw.uri library
Change-Id: I1d94a8c288537ada038f24f2ec26922d95f14785
2013-02-07 13:31:24 -05:00
Chad Horohoe 04b207829e $wgServerName doesn't exist anymore
This is a followup to I944938f9. I had kind of forgotten that
we removed $wgServerName.

Change-Id: I846cba6853f1a3a09f4e2d362960fea5f60bfcc3
2013-02-06 10:24:59 -05:00
Brad Jorsch bd03237246 Add mw.site library
Contains various constants and functions to access site info.

Change-Id: I944938f9af0203c16d1a3fb2046f332045dec4d9
2013-02-06 09:20:59 -05:00
Brad Jorsch 8b27dc742d Add incrementExpensiveFunctionCount() methods
To allow Lua libraries to mark functions as expensive, add an
incrementExpensiveFunctionCount() method to Scribunto_LuaEngine that
will call the corresponding Parser method and throw an error if the
limit is exceeded.

Also allow libraries to do the same thing from Lua by calling
mw.incrementExpensiveFunctionCount().

Change-Id: I56fded32b1077eff3980371e9abc9b3b7581f7b5
2013-02-01 15:55:16 -05:00
Brad Jorsch 4c69b1350e Lua library support functions
Adds a base class for libraries with some utility functions in PHP, and a
Lua library with utility functions for use from Lua.

Change-Id: I3d67b1de8bc50488fe3a722e4e2de5849285d127
2013-01-31 12:40:39 -05:00
Brad Jorsch 0a8a07de2f Fix package.loaded and other cleanups
The listing of the standard modules in package.loaded seems to have been
removed to avoid leaking information to loaded modules. However, since
the *entire* environment is cloned, *including* package.loaded itself,
this does not seem to actually be a problem. But for good measure, also
add a unit test to verify that the version of the standard module tables
referenced from package.loaded is the same as that in _G.

This change also cleans up some unused local variables and an unused
local function from the package module.

Change-Id: I7ec8227b3273059e8f65ad735c215bfd0c623e64
2013-01-30 17:24:09 +00:00
Brad Jorsch dd5241273b Support __pairs and __ipairs
Lua 5.2 introduces a nice feature where a metatable can override the
standard behavior of the pairs() and ipairs() functions. That would be
very useful in allowing a more standard syntax for our frame.args, and
it's very easy to do both in C and in Lua.

Change-Id: I37efc59a0c8876ee16184807e15fafbc07e2d288
2013-01-24 15:46:33 -05:00
Brad Jorsch bdacaabf21 Fix string extending
Built-in modules (e.g. the upcoming ustring) should be able to extend
the string metatable, but user modules (and the debug console) should
not be able to.

But currently built-in modules cannot extend the string metatable in
LuaStandalone, while in LuaSandbox the string metatable can be extended
in the console. Fix this and add unit tests.

Change-Id: I15f5598fed318f2fe26b08ec47e16053dddc13c4
2013-01-09 15:54:44 +00:00
Brad Jorsch af0f1926f4 Fix mw.allToString( nil )
This should return "nil", not the empty string.

Change-Id: I6363e5ca8c1966e45eec9d05fb583fb4f98a92b5
2013-01-09 05:56:55 +00:00
Brad Jorsch 114f5c13ec Add Scribunto_LuaInterpreter::isLuaFunction method
Add a method to LuaInterpreter to determine whether an object is a
wrapped Lua function.

Change-Id: I20bf16948db025372d68cc89bf5ddcbf617db864
2013-01-09 05:55:12 +00:00
Brad Jorsch 374972c924 Add Scribunto_LuaInterpreter::wrapPhpFunction
Currently, the only way for PHP to pass a function to Lua is to pass
back a function that it received from Lua. This means, for example, that
PHP cannot implement a Lua iterator function except by registering a
library holding the function or by using loadString.

This changeset adds Scribunto_LuaInterpreter::wrapPhpFunction (and
implements it for both LuaSandbox and LuaStandalone), which takes a PHP
callable and returns a Lua function wrapping it.

Note that fallback code is included so this does not depend on
I2e552799.

Change-Id: Ic0a98eec7cc17ef4b1acee032c0f42d617b998d2
2013-01-09 05:54:36 +00:00
Brad Jorsch 8c2300e320 Make loader available to console, interface modules
The package module is loaded into the "base" environment but not
correctly initialized, so interface modules and the console cannot
actually load anything.

Change-Id: I92a47d318ccadd7361edb1ac3b0e4bb304ff8a9c
2013-01-09 05:47:06 +00:00
Brad Jorsch 9a5dc931f5 Allow mw.log and console to handle multiple values
In the debug console, "=unpack( { 1, 2, 3 } )" prints only "1". And
similarly, "mw.log( 1, 2, 3 )" logs only "1". Since Lua uses multiple
return values extensively, this is not particularly helpful.

Following the lead of the lua command-line client, change these to
output multiple values by converting each one using tostring() and then
concatenating them with tab as a separator.

Change-Id: I791d4c92415fc722bbd7c62d0f5f88752d31fe07
2012-12-21 10:26:38 -05:00
Siebrand Mazeland e7dd3e5275 Replace remaining wfMsgExt calls.
All of these were in getScriptTraceHtml, which used some magic ($msgOptions) which is no
longer that easy when using the Message class. As only inContentLanguage ('content') was
used, I documented that, and added the relevant code paths.

I tried to keep the code as readable and brief as possible and also removed an unused
local variable ($linedefined).

Change-Id: I38e1ce0bf90d9aa462e88ca7c795fcc848c4f118
2012-10-09 06:08:48 +00:00
Siebrand Mazeland 42c0c55225 Replace deprecated methods.
Change-Id: Ifab19bbc6f75148c8dcee758e09149280e2b229b
2012-10-07 02:59:30 +02:00
Tim Starling aacca08815 Add mw.getCurrentFrame()
Provide a convenient means to access the current frame so that the
parameter passed to module functions won't need to be conventionally
stored in a global variable.

Change-Id: I0254d86a1094866a3ce4899e4021d0b33367bb35
2012-09-06 14:54:26 +10:00
Tim Starling 996ea62765 LuaSandbox profiler support
Add a profiler report to the limit report for pages with more than 1s
of Lua time. Uses the profiler introduced in I0b83a914 of LuaSandbox.

Also, fixed some circular references which were preventing the
LuaSandbox object from being destroyed when Scribunto::resetEngine()
was called. Otherwise a large number of interval timers could be started
due to a LuaSandbox object resource leak.

Change-Id: I5487fe2623974939d07f09f7197e86a5f297a8f1
2012-09-06 01:00:20 +00:00
Tim Starling ea1347a620 Don't run Preprocessor::preprocessToObj() on cache hits
Don't run Preprocessor::preprocessToObj() for frame:preprocess() on
cache hits, only do it on cache misses. The rationale for passing a
PPNode from expandTemplate() to doCachedExpansion() remains: Parser
implements its own cache, as well as extra logic, inside
getTemplateDom().

Change-Id: I29e85e3e98d9590bbd1ae49fb2bb4deaa895edc9
2012-08-27 11:14:25 +10:00
Tim Starling b5c36bad59 Debug console module
* Added a debug console to the edit page, allowing unsaved modules to be
  tested.
* Removed the "preview" button from the edit page.
* Only show the "ignore code errors" checkbox on module edit pages, not
  all edit pages.
* Added Lua function mw.log() for sending messages to the debug log.

Change-Id: Ia51f439e573a1deb5b83f94ddd1a86792d5569c1
2012-07-14 14:35:55 +10:00
Tim Starling a6a3678bf3 expandTemplate() nonexistent title error
(bug 37324) Raise an error instead of returning nil when the specified
template does not exist.

Change-Id: Idd52c0cb9431cfeb7db73ca2692365aa121c49ab
2012-07-09 13:23:31 +10:00
Tim Starling c45034b250 Give a helpful error message if frame methods are called incorrectly
Change-Id: Iae1ed064eb540122c89f0e93f20e76569ec3d1fe
2012-07-03 13:02:03 +10:00
Reedy 5afbf6a7c7 Some bits of documentation
Left a FIXME

Remove some unused variables

Change-Id: If733608416e68de6afe1e8f6edd4ed78a119979e
2012-06-20 23:54:21 +01:00
Tim Starling 441943bd9b Do not allow access to setfenv() and getfenv() by default
Optionally remove setfenv and getfenv from the global environment in
which user code runs. This will improve the forwards-compatibility of
user code with Lua 5.2.

Porting to Lua 5.2 would still be a daunting project, of questionable
value, but at least only the internal code would need updating, and not
thousands of on-wiki modules. Compared to the environment changes, the
rest of the Lua 5.2 changes are relatively easy to simulate for
backwards compatibility.

Removed module() from the package module, since it depends on setfenv().
The native version of it is deprecated in Lua 5.2 for that reason.

Change-Id: I978903ca98943ac941833da13fe5027949f6b429
2012-05-31 15:02:04 +02:00
Tim Starling 6bc11ff615 New parser interface
* Implemented the new parser interface based on a frame object, as
  described in the design document and wikitech-l.
* Added parser tests for the new interface.
* Removed {{script:}} parser function
* Allow named parameters to {{#invoke:}}
* Don't trim the return value
* If a function invoked by #invoke returns multiple values, concatenate
  them into a single string.
* If there is an error during parse, show the error message as an HTML
  comment as well as via JavaScript. This makes parser test construction
  easier, and probably makes debugging easier also.
* Rename mw_internal to mw_php to clarify its role. It is now strictly a
  private Lua -> PHP interface function table.
* Protect mw.setup() against multiple invocation.
* Fixed a bug in Scribunto_LuaStandaloneInterpreter::receiveMessage():
  large packets caused fread() to return with less than the requested
  amount of data, which previously caused an exception. It's necessary
  to check for EOF and to repeat the read to get all data. The receive
  function on the Lua side does not suffer from this problem.
* In the standalone engine, fixed a bug in the interpretation of null
  return values from PHP callbacks. This should return no values to Lua.
* Updated the Lua unit tests to account for the fact that functions are
  now forced to return strings.
* Updated the getfenv and setfenv tests to account for the extra stack
  level introduced by mw.executeFunction().

Change-Id: If8fdecdfc91ebe7bd4b1dae8489ccbdeb6bbf5ce
2012-05-22 14:18:49 +10:00
Victor Vasiliev f311f8bc96 Minor code cleanup.
* Remove an unused message.
* Remove an unused skin reference which caused notices.

Change-Id: I5d051f3a04da089e1789fd785a80c91b5be9a340
2012-04-29 18:26:04 +04:00
Tim Starling fa6dd44957 LuaSandbox backtraces
* Updates for the LuaSandbox backtrace feature introduced in r115020.
* Fixed a notice in the case where the backtrace is not available
* Implemented error message parsing for file/line as in LuaStandalone

Change-Id: Ie1e053784e174aa53ebf5a1fa782d3cf6243c3f0
2012-04-24 20:40:40 +10:00
Tim Starling 41b93dd7e1 Fixed setfenv() across a tail call
Fixed the issue noticed during testing of da06273e, and which resulted
in satest.setfenv1() being disabled. It's not possible to protect
environments by iterating through every stack level, calling getfenv()
at each one, because if any of the stack levels is a tail call, an error
is raised.

Such a tail call was introduced in da06273e, which is why the test broke.

Instead, just protect the actual specified environments, not their
callers. The callers will have to protect themselves.

Change-Id: If39104010ff2663c1bae5105cc8d37e276532100
2012-04-24 12:33:06 +10:00
Tim Starling da06273ede Nicer errors with backtraces etc.
* Added error backtrace collection to MWServer:handleCall()
* When there is an error on parse, show a short and simple inline error
  message to the user, which when clicked, expands to a full error with
  HTML-formatted backtrace.
* When an error is encountered during module validation, have the code
  editor jump directly to the line. Requires r115011.
* Expose the code location of most errors to Scribunto, by parsing the
  standard error message format.
* During module validation, abbreviate the error location if the error
  is in the same module.
* Do not execute the module during validation, just parse it. Execution
  does not really work without an active parse operation in progress.
  It already caused a fatal error if you called require() from the main
  chunk, and problems would have become more visible as more
  parser-related APIs were added.
* LuaSandbox does not yet provide backtraces, but this is planned.

Change-Id: Id9f6564a41b310792b3fe3ebb527cbf8f8771bd1
2012-04-23 21:58:30 +10:00
tstarling b68cae904a More tests and some related bug fixes
* Added tests for the engine classes.
* Added some tests that run under Lua.
* In the chunk names, fixed truncation of module names at 60 bytes
  by using an "=" prefix instead of @.
* Fixed a bug in mw.clone() which was causing the metatable to be set on
  the source table instead of the destination.
* Put restricted setfenv/getfenv in the cloned environment rather than
  the base environment, they work better that way.
* In setfenv(), check for getfenv() == nil, since that's what our own
  restricted getfenv returns.
* Fixed getfenv() handling of numeric arguments: add one where
  appropriate.

Change-Id: I2b356fd65a3fcb348c4e99a3a4267408fb995739
2012-04-19 17:48:20 +10:00
tstarling cebe775ee8 Added more Lua environment features
Package library:

* Added a simulation of the Lua 5.1 package library.
* Removed mw.import(), replaced it with a package loader. Packages can be
  retrieved from the wiki, using require('Module:Foo'), or from files
  distributed with Scribunto, using require('foo'). The "Module:" prefix allows
  for source compatibility with existing Lua code.
* Added a couple of libraries from LuaForge: luabit and stringtools.
* Made fetchModuleFromParser() return null on error instead of throwing an
  exception, to more easily support the desired behaviour of the package loader,
  which needs to return null on error.
* Renamed mw.setupEnvironment() to mw.setup() since it is setting up things
  other than the environment now.
* In MWServer:handleRegisterLibrary(), remove the feature which interprets dots
  in library names, since LuaSandbox doesn't support this.

Improved module isolation and related refactoring:

* Expose restricted versions of getfenv() and setfenv() to user Lua code.
  Requires luasandbox r114952.
* Don't cache the export list returned by module execution for later function
  calls. This breaks isolation of #invoke calls, since the local variables are
  persistent.
* Removed ScribuntoFunctionBase and its children, since it doesn't really have
  a purpose if it can't cache anything. Instead, invoke functions using a module
  method called invoke().
* Removed Module::initialize(), replaced it with a validate() function. This is
  a more elegant interface and works better with the new module caching scheme.
* Use a Status object for the return value of Engine::validate() instead of an
  array. Use the formatting facilities of the Status class.

Other:

* Removed "too many returns" error, doesn't fit in with Lua conventions.
* Use the standalone engine by default, so that the extension will work without
  configuration for more people.
* Added an accessor for $engine->interpreter
* Fix mw.clone() to correctly clone metatables
* If the standalone interpreter exits due to an error, there are some contexts
  where the initial error will be caught and ignored, and the user will see the
  error from checkValid() instead. In this case, rethrow the original error for
  a more informative message.
* Load mw.lua into the initial standalone environment, to reduce code
  duplication between mw.lua and MWServer.lua.
* Fixed a bug in Scribunto_LuaStandaloneInterpreter::handleCall() for functions
  that return no results.
* Fixed a bug in encodeLuaVar() for strings with "\r". Added test case.
* In MWServer.lua, don't call error() for internal errors, instead just print
  the error and exit. This avoids a protocol violation when an error is
  encountered from within handleCall().
* Added lots of documentation. Lua doc comments are in LuaDoc format.

Change-Id: Ie2fd572c362bedf02f45d3fa5352a5280e034740
2012-04-18 13:46:18 +10:00
tstarling 889f4e42a5 Allow PHP functions to return multiple values to Lua
Corresponding luasandbox commit is r114920.

Change-Id: I10fe726795623e2976c9ddadaa4d9b6d44b30bed
2012-04-16 15:43:34 +10:00
tstarling 16d132312f Remove old check for luasandbox extension
Change-Id: I08594015ce044d7fab129b8bbbef2a92382f6870
2012-04-16 15:06:06 +10:00
tstarling b0f00103e2 Added tests and fixed bugs
* Added unit tests for the two Lua interpreter classes
* Fixed a bug in checkType()
* Have Scribunto_LuaSandboxInterpreter throw an exception on construct
  when the extension doesn't exist, to match the standalone behaviour.
* In Scribunto_LuaSandboxInterpreter, removed debugging statements
  accidentally left in.
* Convert LuaSandboxTimeoutError to the appropriate common error
  message.
* Moved the option munging from the sandbox engine to the interpreter,
  so that the interpreter can be unit tested separately.
* Use /bin/sh instead of bash for lua_ulimit.sh, since dash is smaller
  and still supports ulimit.
* Use exec to run the lua binary, so that the vsize of the shell doesn't
  add to the memory limit.
* Added a quit function to the standalone interpreter. Unused at present.
* Don't add a comma after the last element of a table in a Lua
  expression.
* Make the SIGXCPU detection work: proc_open() runs the command via a
  shell, which reports signals in the child via the exit status, so
  proc_get_status() will never return a valid termsig element.
* In MWServer:call(), fixed a bug causing the return values to be
  wrapped in an array.
* Fixed a misunderstanding of what select() does.
* In MWServer:getStatus(), fixed indexes so that vsize will be correct.
  Removed RSS, since it wasn't used anyway and turns out to be measured
  in multiples of the page size, and I couldn't be bothered trying to
  fetch that from getconf. Return the PID and vsize as numbers rather
  than strings.
* Added a simple table dump feature to MWServer:debug().
* Fixed brackets in MWServer:tostring().
* Added missing Linux 32-bit binary.

Change-Id: Ibf5f4656b1c0a9f81287d363184c3fe9d2abdafd
2012-04-16 14:41:08 +10:00
tstarling 54cedd69b8 Introduced standalone interpreter, implemented module isolation
* Introduced a Lua implementation based on shelling out to a standard Lua binary.
* Bundled several Lua binaries for common platforms. I haven't added a 32-bit Linux binary yet, but that will come.
* Refactored the existing Lua class, bringing out functionality common to all Lua implementations into a set of common base classes.
* Moved the bulk of the implementation-specific functionality into a set of "interpreter" classes.
* Renamed LuaSandboxEngine to Scribunto_LuaSandboxEngine
* Don't create an engine object unconditionally when the ParserLimitReport hook is called.
* Implemented isolation of module global variable namespaces. This means that separate {{#invoke}} calls can't pass data to each other -- this was a desired feature in planning since it allows more flexibility in wikitext parser design. Isolation for mw.import() means that modules cannot accidentally create global variables which affect other modules -- exports are solely via the return value.

Change-Id: I3fa35651fe5b1fbfd85adeadc220b1ea31cd6f0b
2012-04-13 20:45:26 +10:00