Commit graph

145 commits

Author SHA1 Message Date
Jackmcbarn 50a1f556b7 Remove leftover code for titles in messages
Since the code related to titles in messages was removed from
mw.message.lua, remove it from here as well. Titles have no effect since
only the plain format is available.

Change-Id: I0c96a4e831abe61100b48cb6a898ad8dbffd8a72
2014-03-18 13:52:51 -04:00
Jackmcbarn 18f177a3a9 Improve invalid title error message
Include the text of the title being complained about when returning an
invalid title error from expandTemplate.

Change-Id: I2261f9574557c3ae514c39cea71f9777f8f9f431
2014-03-11 17:06:23 +00:00
Brad Jorsch 29452c9bef mw.language: Fix exceptions and add tests
Various methods are throwing exceptions when passed invalid language
codes. Those need to be caught.

And we should really add unit tests for the mw.language library, too.
Doing so exposed another bug (in lang:gender), which is also fixed here.

Bug: 62242
Change-Id: Ib7d257cbb1ce179c510273526910d6ac5f3cac5d
2014-03-09 02:56:06 +00:00
Brad Jorsch 84d96e843c LuaStandalone: Release functions when no longer referenced
The LuaStandalone interpreter needs to keep a mapping from integers
returned to PHP to the corresponding function. But if it never releases
these functions when PHP no longer has any reference to them, it can
result in Lua running out of memory if a module with a large number of
functions is invoked many times in one page.

The fix here is to track which function ids are referenced from PHP, and
periodically send the list to Lua so it can remove any that are no
longer used from its cache.

This also takes care of another issue where having multiple interpreter
instances and passing function objects from one into another could call
the wrong function in Lua.

Bug: 51886
Change-Id: I4f15841051f7748d1d6df24080949e5cbd88f217
2014-02-25 22:59:05 +00:00
Brad Jorsch b58ee1da94 Use the new limit report hooks
Change Ie065c7b5 added an option to show profiling data at the bottom of
preview pages, and with it new hooks to gather and format this data in a
more structured way than is possible with ParserLimitReport. This change
adds support for the new hooks.

Depends-On: I7799616a602d90e1b8d3f0ece35811ca387bade7
Change-Id: Idffd2d78f9a0217c99c07cbbfc844d6daf0172f7
2014-02-10 04:11:35 +00:00
Brad Jorsch ed54cd7032 Fix PHP implementation of mw.ustring %W
Incorrect regex is making it match basically everything.

Bug: 60908
Change-Id: Ic1c5a740c6b825bb7dfacf684d0c7fa905b135ee
2014-02-10 00:45:46 +00:00
Jackmcbarn 2b06e83816 Remove message formats other than plain
Message formats other than plain should have never been exposed in this
way, as they allow link tables, etc. to be bypassed and serve no useful
purpose. This removes them, and also removes title, as it serves no
purpose without them.

Bug: 60758
Change-Id: I96284ffbe986a9cd92d2bde1ffdb746029bad989
2014-02-03 15:29:39 -05:00
Marius Hoch aa943d5795 Validate the type of the tagName given to mw.html.create
Change-Id: I35f70701ba6156325dfd268903452e6279aca528
2014-01-31 02:08:11 +01:00
Brad Jorsch 5cf28c5a15 Use a child frame in frame:expandTemplate
If we don't do this, then the section edit links point to the wrong page
if we expand a template that contains section headings.

Bug: 55525
Change-Id: I00bda935be3e8b9c0f86fd0f131814207fbb34a7
2014-01-30 22:47:31 +00:00
jenkins-bot b1d7bac7b0 Merge "Add protectionLevels variable to mw.title output" 2014-01-15 20:21:48 +00:00
Jackmcbarn 9902ec1284 Add protectionLevels variable to mw.title output
Include a protectionLevels variable in the output of the mw.title.*
functions, containing the contents of the title's mRestrictions
array (i.e., its protection levels)

Change-Id: I79c9fed64bacfc90aee1d411a3e1b47e44c99755
2014-01-15 14:53:18 -05:00
Brad Jorsch f52136eada Handle invalid keys in Lua-to-PHP calls for LuaStandalone
PHP can't handle having arrays/objects or functions as keys in its
arrays, so make sure we don't try to pass them from Lua. Booleans aren't
really well-handled either, so let's disallow them too.

Also, add tests for proper stringification of floats and infinities when
those are used as keys.

Note this behavior change is needed to match the change in LuaSandbox
for fixing bug 54527, but isn't itself a security issue.

Change-Id: I1e2951bbe8cb78358650ad377bf7119fcac4485d
2014-01-14 10:03:42 -05:00
jenkins-bot b34e466b18 Merge "Add mw.html to Scribunto" 2014-01-07 14:09:46 +00:00
Marius Hoch ac62e34952 Add mw.html to Scribunto
A module for building complex HTML from Lua using a
fluent interface. The module is originally from enwiki,
but the authors allowed us to reuse it under GPLv2+
(as stated in the file).
The module will be loaded per default and comes with
unit tests.

As discussed on wikitech-l:
http://lists.wikimedia.org/pipermail/wikitech-l/2013-December/073320.html

Change-Id: I7c8d4378091c13d5ace0dd1fcbb4e27163e8c896
2014-01-03 00:04:34 +01:00
Brad Jorsch 3ced70adb8 Provide access to Language::getFallbacksFor()
The other fallback-related functions in PHP all seem to be based on
this.

Bug: 48328
Change-Id: I4b54ab0c1e1535529f47443f6d0e72a59d3d120c
2013-12-16 14:12:47 -05:00
Brad Jorsch fcb6d9ff92 Add access to Language::fetchLanguageNames
Apparently this is useful on Commons, where they would like to iterate
over all language names in some of their templates.

Bug: 47833
Change-Id: I6e3291bedc72da6630c485ea9bf381d8d2f5453a
2013-12-10 12:22:09 -05:00
Brad Jorsch 0763e22292 Expose PPFrame::getTitle to Lua
This field already exists in PHP with exactly the content requested in
bug 47089, so we may as well expose it on the frame object.

Bug: 47089
Change-Id: I672820589f6ebc7c4daad29b5eb156733a5bc5cc
2013-12-10 04:26:03 +00:00
Marius Hoch 99e69490d8 Fix a PHP Notice in Scribunto_LuaEngine
Change-Id: Ie09ec0a415c8ac72ede1ae77ba2257bbe6062505
2013-12-09 01:40:01 +01:00
jenkins-bot a18a762fd7 Merge "Add mw.isSubsting()" 2013-11-06 18:26:56 +00:00
Brad Jorsch d4331f0c96 Add mw.isSubsting()
It's already possible to detect whether the current template is being
substituted via ParserFunctions (see [[en:Template:Ifsubst]]), and a
similar trick works with frame:preprocess. So we may as well provide the
flag directly.

Bug: 47828
Change-Id: Id06d27c6283ee589a8830b78c04e56978e0ac6da
2013-10-31 22:05:00 -04:00
jenkins-bot 47fc1462c4 Merge "Properly handle spaces/underscores in mw.site.namespaces" 2013-10-31 19:36:03 +00:00
Brad Jorsch ec2f67c7ea Regularize whitespace handling in mw.title
Specifically:
* String conversion in non-URL contexts (e.g. .prefixedText) uses spaces
  instead of underscores.
* Setting .fragment now applies the same transformations that are done
  (in PHP) by mw.title.new.

Bug: 56217
Change-Id: I12e354636bcde3327864088175fb4de61aecc81a
2013-10-31 11:54:26 -04:00
Brad Jorsch f31aa37adc Properly handle spaces/underscores in mw.site.namespaces
The PHP call that makes mw.site.namespaces work case-insensitively
doesn't handle non-standard spaces/underscores. So standardize them
before the call.

Bug: 56216
Change-Id: I4758478b126858fb581614f64eb15472f42fef51
2013-10-31 11:09:16 -04:00
Brad Jorsch 18f858314a Fix frame:extensionTag when not passed args
Bug: 55436
Change-Id: I82b638e89be1d57c941cc4534dc44944a7389740
2013-10-07 14:43:05 -04:00
Brad Jorsch a0a1b05b74 Improve mw.text.nowiki, part 2
The following are now correctly escaped:
* Blank lines (including those with only tabs)
* ---- at the start of a line

Bug: 53658
Change-Id: Ib000ff4f21f76c310741de89de0e0b66f6450344
2013-09-20 13:17:38 -04:00
Brad Jorsch 29eb0f4ea6 Improve mw.text.nowiki
The following are now correctly escaped:
* Space at the start of a line
* Start-of-line characters after \r
* Magic links such as "RFC 123" with non-space whitespace
* URIs that don't use "://", such as "urn:foo"
* Double-underscore magic words

Bug: 53658
Change-Id: I824417e2937dd27cd1e69bd4e74ab7d21a978c75
2013-09-03 17:36:36 -04:00
Mark A. Hershberger 6875792d03 Scribunto should error out sooner if lua isn't executable
Improve error reporting by causing Scribunto to fail earlier if lua
can't be run.

Change-Id: I43fb38c6c3facbd7b65b7ae16f1d23ee72f7a5ff
Bug: 52002
2013-08-30 11:34:19 -04:00
jenkins-bot 99b96d8b14 Merge "Add frontier pattern (%f[set]) to ustring" 2013-08-30 06:22:40 +00:00
Brad Jorsch 8e6e07c75f Handle backtrace where Lua can't guess the function name
Current logic is to display the funciton name if Lua provides us with
one, "main" if it's at the main level, or "?" if it's a C function or a
tail call. But we're not handling if it's a Lua call but Lua can't guess
a name for the function; use "?" for that too.

Change-Id: I938b5e5ca55cf4990dbcbb0db8dd8fc93b03bf15
2013-08-28 11:55:55 -04:00
Brad Jorsch 5a9b7cc5a6 More-compatible Linux standalone binaries
The binaries currently provided were compiled against glibc 2.11+, so
people using CentOS 5 (which has glibc 2.5) are not able to use them.

The binaries in this patch were compiled in VMs installed with CentOS
5.9, and so should work for more people; at a glance, it looks like
glibc 2.3 or later will probably work now.

Bug: 51333
Change-Id: Iac1f2373bbc0bbca8783c82c09eff51ffd5e3761
2013-08-13 01:32:50 +00:00
Brad Jorsch d9ba200571 Add mw.logObject
People have requested a method to log a table as something more detailed
than just "table", to be able to inspect values while debugging.

Bug: 48173
Change-Id: Ia58cab834e87842927a2a13d153ee32473f74086
2013-07-26 05:15:20 +00:00
jenkins-bot 3e2972394f Merge "Improve disable_functions handling in LuaStandalone" 2013-07-24 20:16:17 +00:00
Brad Jorsch c7246a1c42 Improve disable_functions handling in LuaStandalone
If the user is on a webhost that has proc_open listed in PHP's
disable_functions directive, we should give a better error message.
Until we no longer support PHP below 5.4, we should do the same for
safe_mode. And since we're doing that, we may as well report any other
warnings if proc_open fails, too.

In addition, this cleans up error handling in
Scribunto_LuaEngine::load() so it doesn't pretend the interpreter is
loaded if getInterpreter() throws an exception. Otherwise other code
winds up with PHP fatal errors trying to access a null value.

Bug: 50706
Change-Id: I2887b722e089fd7a526aa7dcab9c80deb343d8ac
2013-07-24 15:01:36 -04:00
jenkins-bot 635bae4786 Merge "Fix mw.ustring edge cases" 2013-07-11 18:11:51 +00:00
jenkins-bot 0f45f9e5cb Merge "Make lang:parseFormattedNumber more like tonumber" 2013-07-08 21:15:25 +00:00
Brad Jorsch cfdd099ca8 Fix fatal in frame:callParserFunction
If the parser function returns 'isChildObj', we need to create a child
frame to expand the wikitext returned by the parser function. And when
we pass the arguments to the new frame, we need to pass them through the
preprocessor's newPartNodeArray() first.

Bug: 50863
Change-Id: Ieb7cc7007288de1f7d2cd2458f068affe695e8af
2013-07-07 07:43:31 -04:00
Brad Jorsch 25cbfd776d Make lang:parseFormattedNumber more like tonumber
Users seem to expect that mw.language's parseFormattedNumber will act
like tonumber when given nil or other non-string values, returning nil
instead of raising an error. There's no reason not to, so we may as
well.

Change-Id: Ie0ff19efc84ca738e115bbd524bfd92fccf26127
2013-07-03 16:06:32 +00:00
Brad Jorsch d8314539da Fix mw.ustring edge cases
A few edge cases were being incorrectly handled:
* mw.ustring.sub( 'abc', 1, 0 ) returned 'a', not ''.
* mw.ustring.codepoint( 'abc', 1, 0 ) returned 97, not no results.
* mw.ustring.codepoint( 'abc', 4, 4 ) returned 99, not no results.
* mw.ustring.gcodepoint had the same issues as mw.ustring.codepoint.

Change-Id: Ib8c0ef5a8073106eb7d90d0aa0513be4525dca08
2013-07-03 11:49:52 -04:00
Brad Jorsch ad3e5a7c48 Fix LuaSandbox mw.ustring.byteoffset with negative 'i'
Negative values for 'i' in mw.ustring.byteoffset are supposed to count
from the end of the string. But in LuaSandbox, it was actually counting
from two bytes before the end of the string due to a typo.

Fix that, and add some tests for it.

Bug: 50176
Change-Id: Iceee1022a55abd7a08df1ea7843e1277eb02798b
2013-06-25 10:06:01 -04:00
Brad Jorsch 1ef4cd15ec Fix uncaught exception in LuaStandalone
If the interpreter exits before the end of the page, then the call to
Scribunto_LuaStandaloneEngine::getLimitReport() throws an uncaught
exception when it tries to access the interpreter. Catch it.

Change-Id: I7ce4f09b1b2206f13ab0f422de35e0b69a3b24d5
2013-05-13 16:27:37 -04:00
Brad Jorsch 82820aafc8 Add frontier pattern (%f[set]) to ustring
The "%f[set]" frontier pattern has been in Lua 5.1 since the beginning,
but was undocumented until Lua 5.2. And the code is even unchanged from
5.1.0 to 5.2.1. So there's no reason not to implement it in ustring too.

Note the changes to UstringLibrary.php are somewhat large, because it
splits the "convert a Lua bracketed charset to PCRE" code into a
separate function and it changes the handling of mw.ustring.find's and
mw.ustring.match's 'init' parameter from "substring, match from 0, then
add back on $init" to "use preg_match's $offset and use \G instead of ^
where this matters". Both of these are necessary to properly support
%f.

This also fixes a bug in the pure-Lua code (not used in Scribunto)
exposed by the unit tests for %f where %z was matching '\1' rather than
'\0' and %Z everything except '\1' instead of everything except '\0'.

Bug: 48331
Change-Id: Ie0b95ef5b734db53d6adc9de5dae4874f8944c08
2013-05-12 10:27:36 -04:00
Brad Jorsch d6f3633428 (bug 47365) Fix edge cases in mw.ustring.find, mw.ustring.match
The following errors are fixed:
* PHP warning and wrong return value with empty pattern and plain
* Incorrect offsets returned when init is larger than the string length
* Incorrect captured offsets returned when init is excessively negative

Bug: 47365
Change-Id: I9741418287dc727747326d6a19678370ce155a2b
2013-05-10 06:00:02 +00:00
Brad Jorsch 3622f82bd1 (bug 47300) Fix sandboxing with require
Two related issues:
* The package module was inheriting the loaders from the outer sandbox,
  so loaded modules were being loaded into the outer sandbox's
  environment.
* mw.loadData was using the outer sandbox's require(), so again loaded
  modules were being loaded into the outer sandbox's environment.

Bug: 47300
Change-Id: I48d8dd4784c9a890e3abb6389f96f38e1420dbbb
2013-04-24 06:19:44 +00:00
jenkins-bot 24843707df Merge "(bug 47268) lang:parseFormattedNumber should return a number, not a string" 2013-04-22 19:18:13 +00:00
Brad Jorsch 203b3e75dc Don't pass negative years to Language::sprintfDate
Language::sprintfDate cannot handle negative years, so don't pass them.
Return an appropriate error instead.

Change-Id: Ifb633631df98c82aa1c3dc8a555b91f77faf15ed
2013-04-19 17:59:48 -04:00
Brad Jorsch 821ba409d8 (bug 47268) lang:parseFormattedNumber should return a number, not a string
The documentation, and the expectation of users, is that
lang:parseFormattedNumber() should actually return a number, not a
string.

Bug: 47268
Change-Id: Ieabddd0d9192f1fd8ef7e890d5d6268be9636f38
2013-04-15 20:53:47 -04:00
Brad Jorsch 581b2306af Remove mw.getLogBuffer and others from public interface
One of the design goals of Scribunto is that each #invoke should be
independent. Creative use of mw.log and mw.getLogBuffer can get around
this, passing information from one #invoke to the next.

This patch takes the simple solution of removing mw.getLogBuffer from
modules' environments. For good measure, it also removes
mw.clearLogBuffer and mw.executeModule.

Some minor cleanup of the console code is also included.

Change-Id: I30d73928bade4a6bdd7c00ffcd58a3858ff55698
2013-04-14 23:59:07 +00:00
Brad Jorsch ca3b0b55b2 Add timezone support to mw.lang:formatDate
This uses the timezone support added to core Language::sprintfDate in
Iea1f7842 to add support for various timezone-related formatting
characters, and to correct the output for 'c', 'r', and 'U' in local
mode.

This is related to bug 33454, which requests the same for
ParserFunctions' {{#timel:}}.

Bug: 33454
Change-Id: I1b92dc671051a6cc53e35ebd74d383448e16696c
2013-04-02 23:15:16 +00:00
Brad Jorsch 71d9f600dc (bug 46294) Fix for Windows text-mode file handles
On Windows for LuaStandalone, the lua executable's standard output is a
text-mode file handle, even if the pipe is opened from PHP with the
binary flag. Which means that when Lua returns a "\n", it gets silently
rewritten to "\r\n" and the unserialization fails.

So, change the protocol for Lua→PHP messages to encode \r and \n (and \
itself, as the escape character) to avoid this issue.

Bug: 46294
Change-Id: I73b5f44e8aa0334f5fd03013dc027d1a57318349
2013-04-02 23:05:38 +00:00
Brad Jorsch adb7b5f939 Make pairs() work with various library objects
Users are reporting disappointment that using pairs on title objects
doesn't let them see the fields available. It's easy enough to add a
__pairs metamethod to allow pairs to work on title objects, so let's do
that.

The same can be done for mw.uri objects.

For mw.message objects, we can easily enough change the implementation
to be like mw.language objects, which doesn't have this problem.

For mw.site.stats, we may as well just remove the load-on-demand feature
since it will be demanded as soon as the environment is cloned for the
first module.

Change-Id: Ie5a3b5684b2bb6c090f9994baa03977687688929
2013-04-02 17:23:42 +00:00