Commit graph

27 commits

Author SHA1 Message Date
Mark A. Hershberger 6875792d03 Scribunto should error out sooner if lua isn't executable
Improve error reporting by causing Scribunto to fail earlier if lua
can't be run.

Change-Id: I43fb38c6c3facbd7b65b7ae16f1d23ee72f7a5ff
Bug: 52002
2013-08-30 11:34:19 -04:00
Brad Jorsch 5a9b7cc5a6 More-compatible Linux standalone binaries
The binaries currently provided were compiled against glibc 2.11+, so
people using CentOS 5 (which has glibc 2.5) are not able to use them.

The binaries in this patch were compiled in VMs installed with CentOS
5.9, and so should work for more people; at a glance, it looks like
glibc 2.3 or later will probably work now.

Bug: 51333
Change-Id: Iac1f2373bbc0bbca8783c82c09eff51ffd5e3761
2013-08-13 01:32:50 +00:00
Brad Jorsch c7246a1c42 Improve disable_functions handling in LuaStandalone
If the user is on a webhost that has proc_open listed in PHP's
disable_functions directive, we should give a better error message.
Until we no longer support PHP below 5.4, we should do the same for
safe_mode. And since we're doing that, we may as well report any other
warnings if proc_open fails, too.

In addition, this cleans up error handling in
Scribunto_LuaEngine::load() so it doesn't pretend the interpreter is
loaded if getInterpreter() throws an exception. Otherwise other code
winds up with PHP fatal errors trying to access a null value.

Bug: 50706
Change-Id: I2887b722e089fd7a526aa7dcab9c80deb343d8ac
2013-07-24 15:01:36 -04:00
Brad Jorsch 1ef4cd15ec Fix uncaught exception in LuaStandalone
If the interpreter exits before the end of the page, then the call to
Scribunto_LuaStandaloneEngine::getLimitReport() throws an uncaught
exception when it tries to access the interpreter. Catch it.

Change-Id: I7ce4f09b1b2206f13ab0f422de35e0b69a3b24d5
2013-05-13 16:27:37 -04:00
Brad Jorsch 71d9f600dc (bug 46294) Fix for Windows text-mode file handles
On Windows for LuaStandalone, the lua executable's standard output is a
text-mode file handle, even if the pipe is opened from PHP with the
binary flag. Which means that when Lua returns a "\n", it gets silently
rewritten to "\r\n" and the unserialization fails.

So, change the protocol for Lua→PHP messages to encode \r and \n (and \
itself, as the escape character) to avoid this issue.

Bug: 46294
Change-Id: I73b5f44e8aa0334f5fd03013dc027d1a57318349
2013-04-02 23:05:38 +00:00
Brad Jorsch 5989d28678 (bug 39655) Add Lua version info to Special:Version
Use the SoftwareInfo hook to add the versions of LuaSandbox and Lua to
Special:Version.

Bug: 39655
Change-Id: I912197efee0211066677c4d46e638fb546a410c6
2013-03-25 04:07:30 +00:00
Brad Jorsch 5faa00150c (bug 45684) Don't count argument parsing time towards Lua limit
Currently, the time taken to parse the arguments passed to a Lua
function from #invoke will be counted against Lua's 10-second limit.
This is counterintuitive, and can remove incentive for users to convert
templates to Lua since they may have to convert a whole stack at once.

Note this requires change I11881232 to mediawiki/php/luasandbox to
actually have any effect.

Bug: 45684
Change-Id: I773950e4c399b8a1cfa6d1cde781a069d286b3bd
2013-03-12 03:18:05 +00:00
Brad Jorsch 4c69b1350e Lua library support functions
Adds a base class for libraries with some utility functions in PHP, and a
Lua library with utility functions for use from Lua.

Change-Id: I3d67b1de8bc50488fe3a722e4e2de5849285d127
2013-01-31 12:40:39 -05:00
Brad Jorsch 6b4cfd5b94 Fix LuaStandalone nil handling
In Lua, a table entry with a nil value is the same as a table entry that
doesn't exist. So when serializing for transfer to PHP, these keys will
be skipped. For a table as an associative array this isn't much of a
problem, but for a table as a list it means we have missing indexes.
Some of Lua's functions for handling "lists" (i.e. tables with numeric
keys) also have a problem when the list contains nils.

To work around these issues when passing argument lists and return value
lists, pass the number of elements along with the sparse list. On the
PHP end we can use this to fill in the missing nulls, and on the Lua
end we can pass this count to unpack() to avoid the problems on the Lua
side.

Change-Id: I858e3905a06e377693301da2b8bc534808f00e3e
2013-01-30 10:12:23 -05:00
Brad Jorsch bdacaabf21 Fix string extending
Built-in modules (e.g. the upcoming ustring) should be able to extend
the string metatable, but user modules (and the debug console) should
not be able to.

But currently built-in modules cannot extend the string metatable in
LuaStandalone, while in LuaSandbox the string metatable can be extended
in the console. Fix this and add unit tests.

Change-Id: I15f5598fed318f2fe26b08ec47e16053dddc13c4
2013-01-09 15:54:44 +00:00
Brad Jorsch 114f5c13ec Add Scribunto_LuaInterpreter::isLuaFunction method
Add a method to LuaInterpreter to determine whether an object is a
wrapped Lua function.

Change-Id: I20bf16948db025372d68cc89bf5ddcbf617db864
2013-01-09 05:55:12 +00:00
Brad Jorsch 374972c924 Add Scribunto_LuaInterpreter::wrapPhpFunction
Currently, the only way for PHP to pass a function to Lua is to pass
back a function that it received from Lua. This means, for example, that
PHP cannot implement a Lua iterator function except by registering a
library holding the function or by using loadString.

This changeset adds Scribunto_LuaInterpreter::wrapPhpFunction (and
implements it for both LuaSandbox and LuaStandalone), which takes a PHP
callable and returns a Lua function wrapping it.

Note that fallback code is included so this does not depend on
I2e552799.

Change-Id: Ic0a98eec7cc17ef4b1acee032c0f42d617b998d2
2013-01-09 05:54:36 +00:00
Brad Jorsch eabb081c4f Handle non-finite numbers
There's no reason LuaStandalone cannot handle NaN and Inf.

Also, add some unit tests to check this.

Change-Id: I8570242a792e212489ad24dfc04fb1f2940190a5
2012-12-13 13:40:18 -05:00
Tim Starling d5bd1102d0 Fixed transmission of \0 from Lua standalone
* string.format() truncates the string at a null character, causing a
  deadlock when Lua attempts to send null characters to PHP. Use
  concatenation instead.
* Added test.
* Fixed an error reporting issue in the console, which I happened to
  notice at the same time as the above bug.

Change-Id: I2e6061a04512557492bffbd04bc09ca3bc1d80d6
2012-12-03 17:46:11 +11:00
Tim Starling 9859591271 Allow time functions in LuaStandalone
As in LuaSandbox I7e116c76

Change-Id: I783e2ae802999701ebcd4d1a1b3d113a21b4ca9e
2012-09-06 11:01:53 +10:00
Tim Starling 30dee55eeb Merge "Add a Mac OS X (Lion) lua binary. Compiled for 32 and 64bit. Used automatically on Darwin systems" 2012-06-29 07:19:39 +00:00
Reedy 5afbf6a7c7 Some bits of documentation
Left a FIXME

Remove some unused variables

Change-Id: If733608416e68de6afe1e8f6edd4ed78a119979e
2012-06-20 23:54:21 +01:00
Derk-Jan Hartman 32831ec56e Add a Mac OS X (Lion) lua binary. Compiled for 32 and 64bit.
Used automatically on Darwin systems

Change-Id: Ie4f4d2a2ad3b7d62ae3daea66318ce3947c7fc0e
2012-06-06 23:01:48 +02:00
Tim Starling 6bc11ff615 New parser interface
* Implemented the new parser interface based on a frame object, as
  described in the design document and wikitech-l.
* Added parser tests for the new interface.
* Removed {{script:}} parser function
* Allow named parameters to {{#invoke:}}
* Don't trim the return value
* If a function invoked by #invoke returns multiple values, concatenate
  them into a single string.
* If there is an error during parse, show the error message as an HTML
  comment as well as via JavaScript. This makes parser test construction
  easier, and probably makes debugging easier also.
* Rename mw_internal to mw_php to clarify its role. It is now strictly a
  private Lua -> PHP interface function table.
* Protect mw.setup() against multiple invocation.
* Fixed a bug in Scribunto_LuaStandaloneInterpreter::receiveMessage():
  large packets caused fread() to return with less than the requested
  amount of data, which previously caused an exception. It's necessary
  to check for EOF and to repeat the read to get all data. The receive
  function on the Lua side does not suffer from this problem.
* In the standalone engine, fixed a bug in the interpretation of null
  return values from PHP callbacks. This should return no values to Lua.
* Updated the Lua unit tests to account for the fact that functions are
  now forced to return strings.
* Updated the getfenv and setfenv tests to account for the extra stack
  level introduced by mw.executeFunction().

Change-Id: If8fdecdfc91ebe7bd4b1dae8489ccbdeb6bbf5ce
2012-05-22 14:18:49 +10:00
Tim Starling c9c0aa6fce Peak memory usage, debug.traceback
* Use LuaSandbox::getPeakMemoryUsage() from r115086
* Fixed the debug.traceback function from da06273e, was nil

Change-Id: Iae4b195ffe25a522d4c37f9c8341e1d6ea3ae106
2012-04-30 16:37:41 +10:00
Victor Vasiliev 606aaf30ac Fix Windows support:
* Improve OS detection
* Provide workaround for a proc_open bug.

Change-Id: I02b56cf2af1613406f7175a4ca053adb9d794bdd
2012-04-24 04:47:53 +04:00
Tim Starling da06273ede Nicer errors with backtraces etc.
* Added error backtrace collection to MWServer:handleCall()
* When there is an error on parse, show a short and simple inline error
  message to the user, which when clicked, expands to a full error with
  HTML-formatted backtrace.
* When an error is encountered during module validation, have the code
  editor jump directly to the line. Requires r115011.
* Expose the code location of most errors to Scribunto, by parsing the
  standard error message format.
* During module validation, abbreviate the error location if the error
  is in the same module.
* Do not execute the module during validation, just parse it. Execution
  does not really work without an active parse operation in progress.
  It already caused a fatal error if you called require() from the main
  chunk, and problems would have become more visible as more
  parser-related APIs were added.
* LuaSandbox does not yet provide backtraces, but this is planned.

Change-Id: Id9f6564a41b310792b3fe3ebb527cbf8f8771bd1
2012-04-23 21:58:30 +10:00
tstarling b68cae904a More tests and some related bug fixes
* Added tests for the engine classes.
* Added some tests that run under Lua.
* In the chunk names, fixed truncation of module names at 60 bytes
  by using an "=" prefix instead of @.
* Fixed a bug in mw.clone() which was causing the metatable to be set on
  the source table instead of the destination.
* Put restricted setfenv/getfenv in the cloned environment rather than
  the base environment, they work better that way.
* In setfenv(), check for getfenv() == nil, since that's what our own
  restricted getfenv returns.
* Fixed getfenv() handling of numeric arguments: add one where
  appropriate.

Change-Id: I2b356fd65a3fcb348c4e99a3a4267408fb995739
2012-04-19 17:48:20 +10:00
tstarling cebe775ee8 Added more Lua environment features
Package library:

* Added a simulation of the Lua 5.1 package library.
* Removed mw.import(), replaced it with a package loader. Packages can be
  retrieved from the wiki, using require('Module:Foo'), or from files
  distributed with Scribunto, using require('foo'). The "Module:" prefix allows
  for source compatibility with existing Lua code.
* Added a couple of libraries from LuaForge: luabit and stringtools.
* Made fetchModuleFromParser() return null on error instead of throwing an
  exception, to more easily support the desired behaviour of the package loader,
  which needs to return null on error.
* Renamed mw.setupEnvironment() to mw.setup() since it is setting up things
  other than the environment now.
* In MWServer:handleRegisterLibrary(), remove the feature which interprets dots
  in library names, since LuaSandbox doesn't support this.

Improved module isolation and related refactoring:

* Expose restricted versions of getfenv() and setfenv() to user Lua code.
  Requires luasandbox r114952.
* Don't cache the export list returned by module execution for later function
  calls. This breaks isolation of #invoke calls, since the local variables are
  persistent.
* Removed ScribuntoFunctionBase and its children, since it doesn't really have
  a purpose if it can't cache anything. Instead, invoke functions using a module
  method called invoke().
* Removed Module::initialize(), replaced it with a validate() function. This is
  a more elegant interface and works better with the new module caching scheme.
* Use a Status object for the return value of Engine::validate() instead of an
  array. Use the formatting facilities of the Status class.

Other:

* Removed "too many returns" error, doesn't fit in with Lua conventions.
* Use the standalone engine by default, so that the extension will work without
  configuration for more people.
* Added an accessor for $engine->interpreter
* Fix mw.clone() to correctly clone metatables
* If the standalone interpreter exits due to an error, there are some contexts
  where the initial error will be caught and ignored, and the user will see the
  error from checkValid() instead. In this case, rethrow the original error for
  a more informative message.
* Load mw.lua into the initial standalone environment, to reduce code
  duplication between mw.lua and MWServer.lua.
* Fixed a bug in Scribunto_LuaStandaloneInterpreter::handleCall() for functions
  that return no results.
* Fixed a bug in encodeLuaVar() for strings with "\r". Added test case.
* In MWServer.lua, don't call error() for internal errors, instead just print
  the error and exit. This avoids a protocol violation when an error is
  encountered from within handleCall().
* Added lots of documentation. Lua doc comments are in LuaDoc format.

Change-Id: Ie2fd572c362bedf02f45d3fa5352a5280e034740
2012-04-18 13:46:18 +10:00
tstarling 889f4e42a5 Allow PHP functions to return multiple values to Lua
Corresponding luasandbox commit is r114920.

Change-Id: I10fe726795623e2976c9ddadaa4d9b6d44b30bed
2012-04-16 15:43:34 +10:00
tstarling b0f00103e2 Added tests and fixed bugs
* Added unit tests for the two Lua interpreter classes
* Fixed a bug in checkType()
* Have Scribunto_LuaSandboxInterpreter throw an exception on construct
  when the extension doesn't exist, to match the standalone behaviour.
* In Scribunto_LuaSandboxInterpreter, removed debugging statements
  accidentally left in.
* Convert LuaSandboxTimeoutError to the appropriate common error
  message.
* Moved the option munging from the sandbox engine to the interpreter,
  so that the interpreter can be unit tested separately.
* Use /bin/sh instead of bash for lua_ulimit.sh, since dash is smaller
  and still supports ulimit.
* Use exec to run the lua binary, so that the vsize of the shell doesn't
  add to the memory limit.
* Added a quit function to the standalone interpreter. Unused at present.
* Don't add a comma after the last element of a table in a Lua
  expression.
* Make the SIGXCPU detection work: proc_open() runs the command via a
  shell, which reports signals in the child via the exit status, so
  proc_get_status() will never return a valid termsig element.
* In MWServer:call(), fixed a bug causing the return values to be
  wrapped in an array.
* Fixed a misunderstanding of what select() does.
* In MWServer:getStatus(), fixed indexes so that vsize will be correct.
  Removed RSS, since it wasn't used anyway and turns out to be measured
  in multiples of the page size, and I couldn't be bothered trying to
  fetch that from getconf. Return the PID and vsize as numbers rather
  than strings.
* Added a simple table dump feature to MWServer:debug().
* Fixed brackets in MWServer:tostring().
* Added missing Linux 32-bit binary.

Change-Id: Ibf5f4656b1c0a9f81287d363184c3fe9d2abdafd
2012-04-16 14:41:08 +10:00
tstarling 54cedd69b8 Introduced standalone interpreter, implemented module isolation
* Introduced a Lua implementation based on shelling out to a standard Lua binary.
* Bundled several Lua binaries for common platforms. I haven't added a 32-bit Linux binary yet, but that will come.
* Refactored the existing Lua class, bringing out functionality common to all Lua implementations into a set of common base classes.
* Moved the bulk of the implementation-specific functionality into a set of "interpreter" classes.
* Renamed LuaSandboxEngine to Scribunto_LuaSandboxEngine
* Don't create an engine object unconditionally when the ParserLimitReport hook is called.
* Implemented isolation of module global variable namespaces. This means that separate {{#invoke}} calls can't pass data to each other -- this was a desired feature in planning since it allows more flexibility in wikitext parser design. Isolation for mw.import() means that modules cannot accidentally create global variables which affect other modules -- exports are solely via the return value.

Change-Id: I3fa35651fe5b1fbfd85adeadc220b1ea31cd6f0b
2012-04-13 20:45:26 +10:00