A few edge cases were being incorrectly handled:
* mw.ustring.sub( 'abc', 1, 0 ) returned 'a', not ''.
* mw.ustring.codepoint( 'abc', 1, 0 ) returned 97, not no results.
* mw.ustring.codepoint( 'abc', 4, 4 ) returned 99, not no results.
* mw.ustring.gcodepoint had the same issues as mw.ustring.codepoint.
Change-Id: Ib8c0ef5a8073106eb7d90d0aa0513be4525dca08
The following errors are fixed:
* PHP warning and wrong return value with empty pattern and plain
* Incorrect offsets returned when init is larger than the string length
* Incorrect captured offsets returned when init is excessively negative
Bug: 47365
Change-Id: I9741418287dc727747326d6a19678370ce155a2b
Two related issues:
* The package module was inheriting the loaders from the outer sandbox,
so loaded modules were being loaded into the outer sandbox's
environment.
* mw.loadData was using the outer sandbox's require(), so again loaded
modules were being loaded into the outer sandbox's environment.
Bug: 47300
Change-Id: I48d8dd4784c9a890e3abb6389f96f38e1420dbbb
The documentation, and the expectation of users, is that
lang:parseFormattedNumber() should actually return a number, not a
string.
Bug: 47268
Change-Id: Ieabddd0d9192f1fd8ef7e890d5d6268be9636f38
One of the design goals of Scribunto is that each #invoke should be
independent. Creative use of mw.log and mw.getLogBuffer can get around
this, passing information from one #invoke to the next.
This patch takes the simple solution of removing mw.getLogBuffer from
modules' environments. For good measure, it also removes
mw.clearLogBuffer and mw.executeModule.
Some minor cleanup of the console code is also included.
Change-Id: I30d73928bade4a6bdd7c00ffcd58a3858ff55698
Users are reporting disappointment that using pairs on title objects
doesn't let them see the fields available. It's easy enough to add a
__pairs metamethod to allow pairs to work on title objects, so let's do
that.
The same can be done for mw.uri objects.
For mw.message objects, we can easily enough change the implementation
to be like mw.language objects, which doesn't have this problem.
For mw.site.stats, we may as well just remove the load-on-demand feature
since it will be demanded as soon as the environment is cloned for the
first module.
Change-Id: Ie5a3b5684b2bb6c090f9994baa03977687688929
This exists for some common text-processing functions that aren't
included in string (and therefore also aren't in mw.ustring), as well as
a logical place for the "unstrip" function requested in bug 45085.
Bug: 45085
Change-Id: I47356215fcc8ddeed5f901cd933a30021394bd78
It would be helpful for debugging if a frame object could be gotten in
the console. To that end, add an empty frame when running in the console
and allow it to be returned by mw.getCurrentFrame().
It would also be helpful to be able to create frames with arbitrary
arguments, again for testing. Fortunately support for creating child
frames with arbitrary arguments already exists in core, so we can just
use it.
And for good measure, be sure to restore the $engine->currentFrames
array even if the Lua code throws an exception.
Change-Id: I1dc8602d63af75424f267c42a3743fabbc1827f7
People sometimes want to know if the actual file exists, not just the
file description page. Support that.
Also alias .exists to .fileExists for the Media namespace, since that's
what the Media namespace is for.
And while we're at it, fix .exists for special pages too.
Bug: 45774
Change-Id: I019adc89858a1d32995a38d5e8eef32577fd32d6
* mw.ustring.sub( '', 1 ) errors in LuaStandalone
* Default value for ustring.maxStringLength and ustring.maxPatternLength
should be infinity, not nil
* mw.ustring.find() returns one value instead of two in "plain" mode.
Change-Id: I5e65c4ec3a05f0e6930ce7ab7fd4ac72bea95e7f
Clean up the modules in engines/LuaCommon/lualib:
* Fix luabit/bit.lua to return its table instead of trying to set the
global directly.
* Fix luabit/hex.lua to return its table instead of trying to set the
global directly.
* luabit/noki.lua is useless for our purposes
* luabit/utf8.lua is redundant to mw.ustring
* stringtools uses coroutines, which we don't support
Also fix a parser test that has apparently been broken for a long time.
Change-Id: I1284cddb6e9b94327964cb1077d8dbdf7def6d06
Note that fetching any title besides the one for the current page is
considered "expensive". It also records the title fetched in the
ParserOutput so it will be listed in pagelinks, just like #ifexists.
This also moves the ToString test formatter into TestFramework.lua, so
TitleLibraryTests.lua can use it too.
Change-Id: I799f3289a37fe1349b6bca5758829acf82cb718f
People sometimes want to load large tables of constant data from a
module. Using require(), this has to be reparsed every time, which can
be slow.
mw.loadData() will load the just data once, and return a table with a
metatable cleverly designed to give read-only access to the loaded data.
Change-Id: Icec192bdbe6cfca7902fd5cb5d0e217bd8399637
Added a language library. Introduced functions which are easy and safe,
most of them already have parser function interfaces.
Change-Id: I4465150f3e16493a15a056f7ddb7787bdf3b0373
Allowing a module to call mw.makeProtectedEnvFuncs() lets it bypass the
allowEnvFuncs setting. It can also be used to manipulate the global
tables that other modules' sandboxes will be copied from.
And for paranoia's sake, let's tighten up what setfenv is allowed to
set. This requires changing a unit test, because it is no longer
sane to do something like
env.setfenv, env.getfenv = mw.makeProtectedEnvFuncs( { [env] = true }, {} )
Nothing real does this, it was only in the unit test.
Change-Id: I8e0d83bb0980ee869af3ac4413afd211717ca92f
The Lua manual says this:
For this function, a '^' at the start of a pattern does not work as an
anchor, as this would prevent the iteration.
I had interpreted that to mean that a pattern starting with '^' would
never match in gmatch. But further testing reveals that the '^' is just
treated as a literal character: string.gmatch( "foo ^bar baz", "^%a+" )
will match "^bar".
Change-Id: Id91d6ee2db753ce1d6a4f6ae27764691d9e9fdc4
Fixed several accidental leaks to the global namespace due to missing
"local" declaration. Removed extension of the string table by mw.uri,
same justification as I5d0ddb70.
Change-Id: Iba1bf8e651d4ce05812e4a9a7a074cb6679297a0
The point of putting the unicode library in mw.ustring instead of
ustring was to avoid conflicts with future upstream work, and with other
libraries. It rather defeats the purpose if you then modify the global
string table during module startup.
Users can always set up local aliases if they feel "mw.ustring" is too
much to type.
Change-Id: I5d0ddb70d999aeb6e36e6ddbcdb19922d0274a39
This is a reimplementation of Lua's string library with support for
UTF-8.
The entire ustring library is implemented in pure Lua. PHP callbacks are
also available for overrides: in LuaSandbox these are used for almost
all functions, while in LuaStandalone they are used only for the pattern
matching. Also, ustring.upper and ustring.lower are overridden using
mw.language's .uc and .lc if available.
It also includes a bunch of unit tests.
Note that if you download the normalization tests, they may fail under
LuaSandbox if you have PHP's intl extension installed and libicu on your
system is too old.
Change-Id: Ie76fdf8d3a85d0a3d2a41b0d3b7afe433f247af0
To allow Lua libraries to mark functions as expensive, add an
incrementExpensiveFunctionCount() method to Scribunto_LuaEngine that
will call the corresponding Parser method and throw an error if the
limit is exceeded.
Also allow libraries to do the same thing from Lua by calling
mw.incrementExpensiveFunctionCount().
Change-Id: I56fded32b1077eff3980371e9abc9b3b7581f7b5
Adds a base class for libraries with some utility functions in PHP, and a
Lua library with utility functions for use from Lua.
Change-Id: I3d67b1de8bc50488fe3a722e4e2de5849285d127
The listing of the standard modules in package.loaded seems to have been
removed to avoid leaking information to loaded modules. However, since
the *entire* environment is cloned, *including* package.loaded itself,
this does not seem to actually be a problem. But for good measure, also
add a unit test to verify that the version of the standard module tables
referenced from package.loaded is the same as that in _G.
This change also cleans up some unused local variables and an unused
local function from the package module.
Change-Id: I7ec8227b3273059e8f65ad735c215bfd0c623e64
Lua 5.2 introduces a nice feature where a metatable can override the
standard behavior of the pairs() and ipairs() functions. That would be
very useful in allowing a more standard syntax for our frame.args, and
it's very easy to do both in C and in Lua.
Change-Id: I37efc59a0c8876ee16184807e15fafbc07e2d288
The package module is loaded into the "base" environment but not
correctly initialized, so interface modules and the console cannot
actually load anything.
Change-Id: I92a47d318ccadd7361edb1ac3b0e4bb304ff8a9c
In the debug console, "=unpack( { 1, 2, 3 } )" prints only "1". And
similarly, "mw.log( 1, 2, 3 )" logs only "1". Since Lua uses multiple
return values extensively, this is not particularly helpful.
Following the lead of the lua command-line client, change these to
output multiple values by converting each one using tostring() and then
concatenating them with tab as a separator.
Change-Id: I791d4c92415fc722bbd7c62d0f5f88752d31fe07
Provide a convenient means to access the current frame so that the
parameter passed to module functions won't need to be conventionally
stored in a global variable.
Change-Id: I0254d86a1094866a3ce4899e4021d0b33367bb35
* Added a debug console to the edit page, allowing unsaved modules to be
tested.
* Removed the "preview" button from the edit page.
* Only show the "ignore code errors" checkbox on module edit pages, not
all edit pages.
* Added Lua function mw.log() for sending messages to the debug log.
Change-Id: Ia51f439e573a1deb5b83f94ddd1a86792d5569c1
Optionally remove setfenv and getfenv from the global environment in
which user code runs. This will improve the forwards-compatibility of
user code with Lua 5.2.
Porting to Lua 5.2 would still be a daunting project, of questionable
value, but at least only the internal code would need updating, and not
thousands of on-wiki modules. Compared to the environment changes, the
rest of the Lua 5.2 changes are relatively easy to simulate for
backwards compatibility.
Removed module() from the package module, since it depends on setfenv().
The native version of it is deprecated in Lua 5.2 for that reason.
Change-Id: I978903ca98943ac941833da13fe5027949f6b429
* Implemented the new parser interface based on a frame object, as
described in the design document and wikitech-l.
* Added parser tests for the new interface.
* Removed {{script:}} parser function
* Allow named parameters to {{#invoke:}}
* Don't trim the return value
* If a function invoked by #invoke returns multiple values, concatenate
them into a single string.
* If there is an error during parse, show the error message as an HTML
comment as well as via JavaScript. This makes parser test construction
easier, and probably makes debugging easier also.
* Rename mw_internal to mw_php to clarify its role. It is now strictly a
private Lua -> PHP interface function table.
* Protect mw.setup() against multiple invocation.
* Fixed a bug in Scribunto_LuaStandaloneInterpreter::receiveMessage():
large packets caused fread() to return with less than the requested
amount of data, which previously caused an exception. It's necessary
to check for EOF and to repeat the read to get all data. The receive
function on the Lua side does not suffer from this problem.
* In the standalone engine, fixed a bug in the interpretation of null
return values from PHP callbacks. This should return no values to Lua.
* Updated the Lua unit tests to account for the fact that functions are
now forced to return strings.
* Updated the getfenv and setfenv tests to account for the extra stack
level introduced by mw.executeFunction().
Change-Id: If8fdecdfc91ebe7bd4b1dae8489ccbdeb6bbf5ce
Fixed the issue noticed during testing of da06273e, and which resulted
in satest.setfenv1() being disabled. It's not possible to protect
environments by iterating through every stack level, calling getfenv()
at each one, because if any of the stack levels is a tail call, an error
is raised.
Such a tail call was introduced in da06273e, which is why the test broke.
Instead, just protect the actual specified environments, not their
callers. The callers will have to protect themselves.
Change-Id: If39104010ff2663c1bae5105cc8d37e276532100
* Added tests for the engine classes.
* Added some tests that run under Lua.
* In the chunk names, fixed truncation of module names at 60 bytes
by using an "=" prefix instead of @.
* Fixed a bug in mw.clone() which was causing the metatable to be set on
the source table instead of the destination.
* Put restricted setfenv/getfenv in the cloned environment rather than
the base environment, they work better that way.
* In setfenv(), check for getfenv() == nil, since that's what our own
restricted getfenv returns.
* Fixed getfenv() handling of numeric arguments: add one where
appropriate.
Change-Id: I2b356fd65a3fcb348c4e99a3a4267408fb995739
Package library:
* Added a simulation of the Lua 5.1 package library.
* Removed mw.import(), replaced it with a package loader. Packages can be
retrieved from the wiki, using require('Module:Foo'), or from files
distributed with Scribunto, using require('foo'). The "Module:" prefix allows
for source compatibility with existing Lua code.
* Added a couple of libraries from LuaForge: luabit and stringtools.
* Made fetchModuleFromParser() return null on error instead of throwing an
exception, to more easily support the desired behaviour of the package loader,
which needs to return null on error.
* Renamed mw.setupEnvironment() to mw.setup() since it is setting up things
other than the environment now.
* In MWServer:handleRegisterLibrary(), remove the feature which interprets dots
in library names, since LuaSandbox doesn't support this.
Improved module isolation and related refactoring:
* Expose restricted versions of getfenv() and setfenv() to user Lua code.
Requires luasandbox r114952.
* Don't cache the export list returned by module execution for later function
calls. This breaks isolation of #invoke calls, since the local variables are
persistent.
* Removed ScribuntoFunctionBase and its children, since it doesn't really have
a purpose if it can't cache anything. Instead, invoke functions using a module
method called invoke().
* Removed Module::initialize(), replaced it with a validate() function. This is
a more elegant interface and works better with the new module caching scheme.
* Use a Status object for the return value of Engine::validate() instead of an
array. Use the formatting facilities of the Status class.
Other:
* Removed "too many returns" error, doesn't fit in with Lua conventions.
* Use the standalone engine by default, so that the extension will work without
configuration for more people.
* Added an accessor for $engine->interpreter
* Fix mw.clone() to correctly clone metatables
* If the standalone interpreter exits due to an error, there are some contexts
where the initial error will be caught and ignored, and the user will see the
error from checkValid() instead. In this case, rethrow the original error for
a more informative message.
* Load mw.lua into the initial standalone environment, to reduce code
duplication between mw.lua and MWServer.lua.
* Fixed a bug in Scribunto_LuaStandaloneInterpreter::handleCall() for functions
that return no results.
* Fixed a bug in encodeLuaVar() for strings with "\r". Added test case.
* In MWServer.lua, don't call error() for internal errors, instead just print
the error and exit. This avoids a protocol violation when an error is
encountered from within handleCall().
* Added lots of documentation. Lua doc comments are in LuaDoc format.
Change-Id: Ie2fd572c362bedf02f45d3fa5352a5280e034740
* Introduced a Lua implementation based on shelling out to a standard Lua binary.
* Bundled several Lua binaries for common platforms. I haven't added a 32-bit Linux binary yet, but that will come.
* Refactored the existing Lua class, bringing out functionality common to all Lua implementations into a set of common base classes.
* Moved the bulk of the implementation-specific functionality into a set of "interpreter" classes.
* Renamed LuaSandboxEngine to Scribunto_LuaSandboxEngine
* Don't create an engine object unconditionally when the ParserLimitReport hook is called.
* Implemented isolation of module global variable namespaces. This means that separate {{#invoke}} calls can't pass data to each other -- this was a desired feature in planning since it allows more flexibility in wikitext parser design. Isolation for mw.import() means that modules cannot accidentally create global variables which affect other modules -- exports are solely via the return value.
Change-Id: I3fa35651fe5b1fbfd85adeadc220b1ea31cd6f0b
