Commit graph

261 commits

Author SHA1 Message Date
Jackmcbarn f4501ccd22 Only use mw.ustring when necessary
mw.ustring is really really slow. I've discovered that in a lot of modules
on enwiki, upwards of 2/3 of the total runtime gets used when mw.html
calls mw.ustring.gsub. This change checks whether any Unicode characters
are present, and if not, calls string.gsub instead.

Change-Id: Ia50061584be3901ae7428354c449236225c318db
2016-05-30 18:38:32 +00:00
Brad Jorsch c9de00aeff SECURITY: Don't escape strip markers when escaping attributes in mw.html
Core strip markers were changed in T110143 to include characters that
are normally encoded in attributes, however we want to pass them through
here so they can be unstripped correctly in the output wikitext.

This fix makes "Strip markers in CSS" parser test pass again.

Bug: T110143
Bug: T135961
Change-Id: I1353931a53c668d8a453dfa2300a99f59fdb01c5
2016-05-22 21:40:32 -04:00
Brad Jorsch aa4d72e3ff Fix uncontroversial phpcs errors
The following continue to be ignored:
* Generic.Arrays.DisallowLongArraySyntax.Found, because I'm not sure
  Scribunto is ready to abandon old version support in master.
* MediaWiki.ControlStructures.AssignmentInControlStructures.AssignmentInControlStructures,
  because it's overly strict for its purpose.

Squiz.Classes.ValidClassName.NotCamelCaps isn't ignored globally, we
just ignore it explicitly every place it's needed.

Change-Id: I307668da6ef7b3e23da19b1fd1e08914239b99b3
2016-05-18 16:31:28 -04:00
jenkins-bot c753698eaa Merge "Provide a standard way to get the target of a redirect page" 2016-05-12 19:32:17 +00:00
Brad Jorsch 507827aaf5 Avoid fataling Special:Version if LuaSandbox is enabled without the PHP extension
Such a configuration is completely broken, but it's easy enough to
detect and avoid here.

Bug: T131910
Change-Id: I0bf108ec191a59f5506c0cdab00f3e5e68158ed5
2016-04-06 11:20:20 -04:00
Brad Jorsch b3da8a698d Add toNFKC and toNFKD to mw.ustring
This also makes some updates to make-normalization-table.php to handle
the move of UtfNormal to a separate library.

Bug: T126427
Change-Id: Id4985c3ca441cf92f08ba1f1af85c762ba43d7d2
2016-04-02 15:22:42 +00:00
Jackmcbarn b82ed4aa7d Restrict cached results to their original frame
When caching results from frame:preprocess and frame:expandTemplate,
restrict the scope of the cache to the frame object that was used. This
allows the integrity of the empty-frame expansion cache to be maintained
while also allowing parent frame access. This change is the equivalent of
I621e9075 in core.

Change-Id: Iae4c00e7e19ba12cfdaac135be16c991d9d0cea1
2016-03-09 11:27:23 -05:00
Ricordisamoa 1573bee81a Provide a standard way to get the target of a redirect page
The new Scribunto_LuaTitleLibrary::redirectTarget() method is
used by mw.title objects as read-only attribute 'redirectTarget'.

If the page does not exist or it is not a redirect, the value
of the attribute is `false`; otherwise, it is the target of the
redirect page, as mw.title object.

This is a proper alternative to parsing wikitext as it is done in:
https://en.wikipedia.org/wiki/Module:Redirect

Bug: T68974
Change-Id: Id4d9b0f8c1cd09ebc42c031d4d3fc0c33eea44aa
2016-03-01 14:30:22 +01:00
Brad Jorsch 31dd4d535f Pass language to SpecialVersion::getVersion()
The language used should be $parser->getTargetLanguage(), not the user
language.

Soft-depends on Id14733aaef3e52a2e315bffe74baeb926d46e238.

Bug: T127233
Change-Id: I712e048367d9d65fd223cb085dbf9e5fceca286c
2016-02-24 00:11:17 +00:00
Darian Anthony Patrick 00ed2a567b Update lua binaries to patch CVE-2014-5461
These binaries were compiled from a manually lua-5.1.5 source tree.

Linux binaries were built by Anomie. Mac OS X and Windows by dpatrick.

Bug: T72541
Change-Id: I6af0f042c491785cce26afc186a148c83c4f3414
2016-02-22 09:38:22 -08:00
Jackmcbarn dc9446b84d Remove loadedLibraries
Nothing actually uses this, so I'm not sure why we ever kept track of it.

Change-Id: I60480b96a83731c7b25aed55099886a86efc08b1
2016-01-19 02:25:25 +00:00
Brad Jorsch 29266a9a0f Use correct variable in ustring.lua
Change-Id: Ic576b8c31c487c106593050538f9f2cc5b722b62
2016-01-02 10:49:48 -05:00
jenkins-bot b8830a3e57 Merge "ustring: Handle "empty" charset like Lua does (part 2)" 2015-10-30 16:34:54 +00:00
Ori Livneh b5df651e1e Scribunto_LuaSandboxEngine::getResourceUsage(): call load()
This is required for ensuring $this->interpreter is available. See
::getLimitReportData(), which does the same thing.

Change-Id: I275b093dd7d5f4873ec4b912823322e6e533cae1
2015-10-29 16:52:21 -07:00
Ori Livneh 7e63874c5c Move getResourceUsage to Scribunto_LuaSandboxEngine
Fix-up for I6a4ed03c126.

Change-Id: I69e9218c6a3da6ca2a6f13e5911fee1c78a8f4a0
2015-10-29 16:29:00 -07:00
Ori Livneh 930421d242 Add ScribuntoEngineBase::getResourceUsage()
Introduce a method, ScribuntoEngineBase::getResourceUsage(), which may be
overridden by script engine implementations to provide CPU and memory usage
data.

Change-Id: I6a4ed03c1261f43a7ce7de6f274c32c450e66abb
2015-10-29 03:59:07 +00:00
Brad Jorsch cd618c7a92 ustring: Handle "empty" charset like Lua does (part 2)
Lua actually treats a close-bracket at the start of a bracketed
character class as a literal, rather than using it to close the
character class. Probably unintended behavior, but it happens.

Also, have the pure-lua version throw our more informative errors on
error even when falling back to string.find and the like, and fix some
other weird edge cases that came up in testing.

Bug: T95958
Bug: T115686
Change-Id: Iab783d4a3e58b1514cc09729d4a71c2cb1242ee8
2015-10-16 09:26:55 -04:00
Jan Berkel fb20934b16 Fix a problem with simple pattern detection
A string with a dot pattern is only "simple" if
followed by +, - or *. The end of string condition was not checked
properly.

Change-Id: Ia10b9164caeabe464c76441cc82eef37a7013048
2015-10-07 10:27:45 -04:00
Jan Berkel 7c5454b36c Fix off-by one error in gsub
Change-Id: I49c0386970e007271d23087fd112580af7b21c9c
2015-09-23 17:41:15 +01:00
Ori Livneh eec31286bc Fix-up for I32bad5fd9
Don't return nonexistent variable $content, and don't bypass loadString / callFunction.

Change-Id: Iae493606d0167853c3c79536e35eeb23a54bb6d1
2015-08-25 17:36:26 -07:00
Ori Livneh 7bd4959b55 Cache Lua code files in APC
Cache Lua libraries in APC (if available) for up to 5 minutes. Always check the
file's mtime to avoid serving a stale copy.

This code path is hot enough that using APC makes a difference.

Change-Id: I32bad5fd9443c1759fe6dc91f8df2ac2f120d75b
2015-08-25 16:28:36 -07:00
Jackmcbarn 828c6cf513 Prevent leaking title fragments across invokes
Bug: T106951
Change-Id: Iace5d75deac3d8ffde6f3dec6a4f910dcb77d1e2
2015-07-27 10:46:23 -04:00
Jackmcbarn bd5e46b941 Check content model instead of title
Make Scribunto compatible with storing content model in the database, by
checking for it directly instead of guessing it based on the title.

Change-Id: I94ae07bc47273fbf65d64b2909e5895c1c3fd7e9
2015-07-19 22:16:21 -04:00
Mr. Stradivarius d59d852290 Fix accidental global in mw.uri.parseQueryString
The result of the type function should be compared against the
string "table", not the global variable. This bug probably went
undetected until now, as "table" is also the global variable for the Lua
table library.

Change-Id: Ia28fa10388bfc587d95b522bfa8f3524b4a3ee5f
2015-07-15 23:07:37 +09:00
jenkins-bot 7cf15f43e5 Merge "Display backtraces in the Scribunto console" 2015-07-01 17:01:36 +00:00
Jackmcbarn 52d4915201 Display backtraces in the Scribunto console
When the Scribunto console produces an error, display a full backtrace
instead of just the error message.

Bug: T74462
Change-Id: I305438284eae8e19a51a70b1e83d54e4831de396
2015-07-01 12:21:24 -04:00
jenkins-bot c582834a09 Merge "Mark metatables from mw.loadData" 2015-06-30 20:31:53 +00:00
Jackmcbarn ca7a84b5b2 Fix some PHPCS issues
Change-Id: I5a44d07553d45bc01db070c99856b35a3d275bd1
2015-06-30 13:14:58 -04:00
Jackmcbarn a4cb7efd0d Mark metatables from mw.loadData
Add mw_loadData=true to metatables set by mw.loadData, so that modules can
distinguish them from other tables.

Change-Id: I0795d738891c85600af2621908376474ae21b3fe
2015-06-27 22:38:23 -04:00
Ori Livneh d426627c9b lint: 'if(' => 'if ('
Change-Id: I056ff6bbc5f992bddfd7e3bd82803de107651b80
2015-06-20 21:38:56 -07:00
Brad Jorsch 58d722bcdf Allow nil in mw.text.jsonEncode
If it somehow gets in there (e.g. via a crafty __pairs), let it through.

Change-Id: I9f79dbb1a09cd62b2a8f4b6beb84a3e2f1c85560
2015-06-16 16:36:30 +00:00
Tim Starling e7f5aae520 Fix race condition in SIGXCPU handling
Marius found a race condition in the handling of SIGXCPU: the pipes
would close, causing the read/write to complete, before the status of
the process changed, so the status would randomly be "running" for a few
milliseconds after proc_get_status() was called.

So: terminate the process unconditionally after an I/O error. Get the
exit status from proc_close(), since that's the only way to get the
status of a terminated process while simultaneously waiting for it to
exit. Also fix signal identification as in unmerged patch I57cdf8aa.

Change-Id: I252ec046e82063a868c1094e81705cb5e847db92
2015-05-25 16:40:31 +00:00
Brad Jorsch 4669e43135 ustring: Handle empty charset like Lua does
Both '[]' and '[^]' give a rather odd error, but it's probably best to
follow suit.

Bug: T95958
Change-Id: I3310da55f655537c9082fc9039003f6b2d31eff4
2015-04-13 18:20:33 -04:00
Jackmcbarn 6ffde66c77 SECURITY: Sanitize the content of Lua backtraces
Bug: T85113
Change-Id: Iede661a34f4ec2f384bd0407e2fb8f271ff54a77
2015-04-01 10:02:19 -07:00
Kunal Mehta 3f5f3e247f Use full <?php instead of short <? in ustring generation scripts
Change-Id: Ida6bc4ee1803763b284fdaa7c63769a146fec6ad
2015-03-17 18:16:20 -07:00
Brad Jorsch 3d51662881 Rewrite error handling to avoid OutputPage::addInlineScript
This is apparently unofficially deprecated, and we can do things a bit
more straightforwardly by using ParserOutput::addJsConfigVars() to
communicate the error messages to the JS.

This also takes the opportunity to move "ext.scribunto", which is mostly
about errors, to "ext.scribunto.errors".

Bug: T75618
Change-Id: I1577dab2dab1bd79cb127879de141fdbb8963aeb
2015-03-16 16:08:44 -04:00
Marius Hoch 53ff0bf246 Revert "Revert "Make tests work when $wgLanguageCode != 'en'""
Should be good to apply now after Wikibase has been adopted to
this.

This reverts commit 06f03fe09b.

Bug: T69483
Change-Id: I28073a891b8847fc6aa5ad026bf702e4015f7d4a
2015-02-19 22:30:47 +00:00
Legoktm 06f03fe09b Revert "Make tests work when $wgLanguageCode != 'en'"
Broke Wikibase tests and hoo needs to do backports today.

This reverts commit 499665c36f.

Change-Id: I5151ce963bcd17ed6d3b98a0f332f95c99596bff
2015-02-19 17:41:19 +00:00
Brad Jorsch 499665c36f Make tests work when $wgLanguageCode != 'en'
* For most tests, subclassing MediaWikiLangTestCase instead of
  MediaWikiTestCase works.
* The Site library needs to not reuse its namespace cache if the
  language changed.
* ScribuntoExceptions are still being localized for some reason; test
  the message and args of the exception instead of using getMessage().

Bug: T69483
Change-Id: I1c0ca3b11b6228654b47ceb92c1b1be7bda6709a
2015-02-19 16:05:17 +00:00
rillke fba298c26f Expose file size and MIME to Lua
Use cases:
- size: Showing links (like to SVG validator) conditionally (e.g.
        only if the filesize is below the threshold it can work on)
- mime: No longer have to guess the MIME type based on the file
        extension. We use the MIME type for displaying correct and
        precise information about maximum thumbnailable sizes in
        case an uploaded work exceeds this limit.

This change will not have negative impact on performance; as soon
as width or height is loaded, the two newly added information are
gratis.

Change-Id: I25aad85c2a558a684aca83374aeb628f355fad1c
2015-02-12 01:18:57 +01:00
jenkins-bot f62b6b4379 Merge "Adds support for JSON encoding and decoding" 2015-02-05 02:58:44 +00:00
Kunal Mehta f5a8a3b0ae Update make-normalization-table for core file moves
Depends upon Ib530ad9dbe1d3a33dc53ef8b9620f61d4e1a2d62 in core.

Change-Id: Ib530ad9dbe1d3a33dc53ef8b9620f61d4e1a2d62
2015-02-04 20:04:41 +00:00
Marius Hoch c0480eef77 Fix weird quotes in package.lua
Change-Id: I6d11813ed00489a69c88ab26aeeec4c4dd42d5dd
2015-02-03 00:40:21 +01:00
Jackmcbarn 35e3ea3ce2 Simplify code in mw.html
Replace numeric loops with iteration, don't unnecessarily check for nil
before table.insert (since it's a no-op in that case anyway), and similar
restructuring.

Change-Id: I155839a648f242a1b1de35f4081d8bcfa34f6933
2015-01-31 13:26:40 +00:00
Brad Jorsch 10bc0f7316 Adds support for JSON encoding and decoding
Provides methods to encode and decode JSON in the mw.text module.

Bug: T47470
Change-Id: I274f2ff13adb616e50600ee30e29b35327f3251e
2015-01-26 15:13:22 -08:00
Jackmcbarn f7fe4881a0 Pass the title's fragment to getExpensiveData
Without this, an error occurs when calling mw.title.new('#foo').exists
(or anything similar).

Change-Id: Id2b60fe3f121af95b4b54da3a7042b490ecbc3fe
2015-01-08 13:19:54 -05:00
Mr. Stradivarius ad34c2bd94 Use protocol-relative URLs in mw.site.interwikiMap
At the moment, in mw.site.interwikiMap, URLs stored in the database
with relative protocols are converted to http or https depending
on the protocol of the user who last requested that the page be parsed.
However, this URL isn't updated for users pulling pages from the page
cache, so they may see links with an incorrect protocol. Module authors
can work round this by using mw.uri objects and the isProtocolRelative
property, but it would be simpler just to use relative URLs from the
outset.

Change-Id: If0a169cfd1597e09706fc0e6d5cdcc821ba8d06a
2014-12-26 20:28:49 +09:00
Mr. Stradivarius c58c528d28 Add mw.site.interwikiMap
This makes the interwiki map available to Lua modules. The code is
based on the API interwiki map code in core (the appendInterwikiMap
method of includes/api/ApiQuerySiteInfo.php.) Everything that the
API includes is added, apart from iw_api and iw_wikiid, which I
couldn't think of a use for from Lua modules.

Accessing the interwiki map would be useful for modules like
enwiki's Module:InterwikiTable,[1] as it would stop module writers
having to duplicate the data.

[1] https://en.wikipedia.org/wiki/Module:InterwikiTable

Change-Id: Ie8ad2582aaf5e422824f7da51714a347bb4041d1
2014-12-24 01:17:48 +09:00
Brad Jorsch ce034c2f92 Normalize newlines before calling preprocessor
The behavior of the different preprocessors differs when given \r or
\r\n newlines.

Bug: T78488
Change-Id: I74faee1d3deb57a70d0c8fb690e5abb388d1f61a
2014-12-22 04:15:52 +00:00
Thiemo Mättig 93579c7363 Add more comments and type hints
Change-Id: I8c4b91d3a6695cc33c9c3d3fb427af5412323ff7
2014-12-19 19:25:53 +00:00