Commit graph

178 commits

Author SHA1 Message Date
Lucas Werkmeister (WMDE) a55cfc2078 Revert "Adding a unit test for CVE-2014-5461 in Scribunto."
This test causes spurious CI failures in other projects; to unblock them
for now, remove the test again until we can figure out how to make it
work better.

This reverts commit 7a7f522676.

Bug: T209232
Change-Id: Id2eeeb781b7a8a6298ba06d78bab238b37dac9ca
2018-11-30 14:14:40 +00:00
Mogmog123 7a7f522676 Adding a unit test for CVE-2014-5461 in Scribunto.
Bug: T209232
Change-Id: I84a4ec014875764bcba4d603b0e27d210d4a9308
2018-11-26 20:36:45 +00:00
Umherirrender 55bd9d22bb Add method scope visibility
Change-Id: I2efe0f71266d70e9a41e044406d82ef7daa31296
2018-11-19 21:18:12 +00:00
Brad Jorsch 18c08c23fc ustring: Match undocumented string.gsub behavior
As documented, string.gub( 'foo', '%a', '%1' ) should raise an invalid
capture index error because there is no capture with index 1 in the
pattern. But in fact it treats %1 as %0 in this situation. The ustring
library should match this behavior.

This patch also adds some tests for the behavior of gsub with table and
function replacements when the pattern does have captures.

Bug: T207623
Change-Id: Ie3e6c2eafa4a05989815c62c7037167642581751
2018-11-01 03:59:35 +00:00
libraryupgrader 8b489ca160 build: Updating mediawiki/mediawiki-codesniffer to 22.0.0
And updating CoC link to use Special:MyLanguage (T202047).

Change-Id: I091003f69b82c7cacc4cda320a38b1b07f3cdb6b
2018-09-03 21:33:35 +00:00
Max Semenik eb8ccf03db Get rid of call_user_func_array()
Yay PHP7!

Change-Id: I777ed78d22efbddacaab22c4614a0defa6ad3f94
2018-07-03 19:40:19 -07:00
Kunal Mehta d91a8cf8a0 Add @covers tags to all tests
Bug: T195160
Change-Id: I77ce544e9c166fef6c6fb02f67d1de6ddf0c2465
2018-05-29 11:37:27 -07:00
Kunal Mehta f76ba3c465 Disable Squiz.Classes.ValidClassName.NotCamelCaps globally
Instead of per-file. This happens to also fix a false positive with the
PhpunitAnnotations sniff.

Change-Id: I22621c37217ed2db9d8b3591df1a1421c25fa7f6
2018-05-24 22:26:11 -07:00
Brad Jorsch 32718af677 ustring: Handle invalid types in gsub
If the replacement table or function results in a value that isn't a
string or number (or nil), string.gsub raises an error. Have ustring
raise the same error.

Bug: T195326
Change-Id: Ic36f9f5d7adc0c14e7a4a94d3747335107acd8b6
2018-05-22 18:55:49 -04:00
Kunal Mehta d245edbb94 Add phan configuration
Manually import LuaSandbox's git repository as a composer dependency to
provide the PHP stubs for phan.

Change-Id: I6226b9211f31d829da5a2775c6f5cf3599dd8ebc
2018-05-14 18:41:59 -07:00
Jayprakash12345 7da38c8e10 Update extensions to take advantage of parser test autodiscovery
Bug: T170037
Change-Id: Ifeffbf7a27dfd4915998159617d37d4d9b9a1c17
2018-04-13 15:07:17 +00:00
Umherirrender 44fe3df9bf Rename test classes to end with Test
Change-Id: I510e5516d56b28d26510423da840b4b496a10833
2018-03-10 14:25:38 +01:00
Kunal Mehta 27a7eb0f58 Use namespaced PHPUnit\Framework\TestCase
For future compatibility with PHPUnit 6.

Bug: T188166
Change-Id: Id1e951d7e9a2092500408ab865427db45c025bde
2018-02-24 00:23:15 -08:00
libraryupgrader 6d1a6ffb01 build: Updating mediawiki/mediawiki-codesniffer to 16.0.0
The following sniffs are failing and were disabled:
* MediaWiki.Commenting.MissingCovers.MissingCovers

Change-Id: I07b2cf945f44fd5532812a712f7dd40d2f208be2
2018-02-15 13:57:18 +00:00
Brad Jorsch 790311faa3 Sanify handling of array keys
When passing an array from PHP to Lua, stringify integer array keys
beyond the range a lua_Number can represent.

When passing a table from Lua to PHP,
* Avoid exponential encoding for integer keys beyond 1e14, so Zend PHP
  will interpret them as integers.
* Always encode integer keys as integers, so HHVM will interpret them as
  integers.
* Detect collisions, e.g. { [0] = 'foo', ["0"] = 'bar' }

Bug: T186240
Change-Id: I078068ed57df078248a307608381614bdfc70801
2018-02-06 17:13:20 -05:00
Kunal Mehta ec7b1b05cf Fix Scribunto_LuaStandaloneInterpreterTest::testGetStatus on 7.1+
The extraneous whitespace in the return value from wfShellExec() causes
multiplying $size to trigger the newly introduced "A non well formed
numeric value encountered" warning in PHP 7.1+.

Work around that by using trim() to get rid of the whitespace.

Bug: T186299
Change-Id: I3d47ef6cc7fb99b4d4840dc847d150c3939ee535
2018-02-01 21:08:32 -08:00
jenkins-bot 252d117fdd Merge "Improve some parameter docs" 2018-01-16 16:21:51 +00:00
Umherirrender cfbd0a1a1b Improve some parameter docs
Change-Id: Ic85f74fc8dcefe86a3620e2d12f0b2ad2386ee23
2018-01-11 21:27:53 +01:00
Kunal Mehta 76dbe5d804 Treat phpdbg as being run from the command-line
The two lualib/ustring generation scripts run independently of MediaWiki, so
the new wfIsCLI() isn't usable there.

Bug: T184043
Change-Id: I217657d12e16a7b76dc814be5fed03540c461e7c
2018-01-10 19:47:19 +05:30
Marius Hoch 12acfa95ef Make Scribunto_LuaSandboxTests::testArgumentParsingTime more robust
This should make sure the tests still work, even if the running
node is heavily overloaded.

Bug: T143389
Change-Id: Ic40c8d76c8799c2e9d11f53945276747c199fd02
2017-10-10 10:38:23 +02:00
Brad Jorsch 9fefb11d56 Replace uses of each()
It's deprecated in PHP 7.2, may as well replace it now.

Bug: T174354
Change-Id: I2c37229d69a9646edaa61e25e28b13e8190a8359
2017-09-19 16:46:51 -04:00
Brad Jorsch 7418a571ac Fix tests
Iaa880531 added extra frames in the call stack, so the frame being
tested by the "setfenv invalid level" and "getfenv invalid level" tests
was no longer invalid.

Bug: T175065
Change-Id: Id1028e7c8bbb92fb9d7d01ebeabd94e8ba284b1c
2017-09-05 15:44:55 -04:00
Brad Jorsch ca85f20099 Make mw.uri.encode 'WIKI' mode match core {{urlencode:}}
The core {{urlencode:}} parser function doesn't encode various
characters in WIKI mode that it does in other modes. mw.uri.encode
should match that.

Bug: T174239
Change-Id: I2be0811cf39c02c5c0ad3433e4b0ef9030350e24
2017-08-28 10:34:16 -04:00
Kunal Mehta b8ff734aa4 Use namespaced ScopedCallback
The non-namespaced version is deprecated since 1.28

Change-Id: Icb3fed78882913a26aad4bdb1a84cb5a3e8ca6bb
2017-08-21 14:06:34 -07:00
WMDE-Fisch e1763ff69a Updating mediawiki-codesniffer to 0.10.1 and fix issues
Change-Id: Iac61e49ab0f1318fcb7b23b4c90b6a427fe8957f
2017-07-25 17:16:47 +00:00
Umherirrender 18b22b3a3d build: Updating mediawiki/mediawiki-codesniffer to 0.10.0
Change-Id: I92b9fb936cb8fa8411850b97804e2aacf3984322
2017-07-08 15:42:23 +02:00
Kunal Mehta 31820c673a Move tests into tests/phpunit and remove UnitTestsList hook
This takes advantage of extension.json's unit tests autodiscovery
mechanism.

Bug: T142120
Change-Id: Id526f3368fc73ba7e6ef1d793ea70ab05fbd9517
2017-07-07 13:50:39 -07:00
Umherirrender 4abed1d7c7 Use short array syntax
Done by phpcbf over composer fix

Change-Id: I9b7419e025ef499ff68be79789d76ad4b886d256
2017-06-16 13:26:30 +00:00
Tim Starling 0cf603ca9d Make the maximum language cache size configurable
Make the language cache size configurable, and increase the default from
20 to 30. It needs to be fairly small on default installations, but can
be essentially unlimited if $wgLocalisationCacheConf['manualRecache'] is
true.

Bug: T85461
Change-Id: Idb17691b30b0d2565a1624e5159df7d9b795764d
2017-03-23 15:24:00 +11:00
Brad Jorsch 7f94d88733 LuaStandalone: Fix signal handling
I252ec046 noticeably broke things by adding a dependency on the pcntl
functions, which tend not to be present under Apache.

It also subtly broke exit handling by using proc_close()'s return value,
which PHP mangles in such a way that we can't tell the difference
between an actual XCPU kill and exit( SIGXCPU ). This one wasn't noticed
because the pcntl functions interpret everything proc_close() is going
to return as a signal kill and we didn't test the 'exited' code path.

I'm not sure what was going on in I57cdf8aa since it provides no details
about what it was trying to fix, but that would have broken signal
handling in the other way: Ibf5f4656 worked because proc_open() on Linux
executes the command by passing it to /bin/sh -c, and that shell is
going to turn any signal that kills Lua (e.g. the SIGXCPU) into an exit
status of 128+signum.

To avoid proc_close()'s broken return value while also avoiding the
race, we can loop on proc_get_status() until $status['running'] is
false.

To have signals that kill Lua actually be interpreted as signals, we
have two options: add an "exec" in front of the command so proc_open()'s
/bin/sh -c is execed away, or detect shell-style signal reporting and
convert it. We may as well do both.

Bug: T128048
Change-Id: I8a62e1660fe1694e9ba5de77d01960c1ab4580aa
2017-03-09 23:16:28 +00:00
Antoine Musso 1cf6339a1b test: change interwiki to a meaningful name
When investigating a test failure on T155600 I found the title
'scribuntotitletest:Module:TestFramework' to be rather obscure.  Had to
browse the code to findout 'scribuntotitletest' is an interwiki prefix.

Update TitleLibraryTest php/lua and change the interwiki prefix to the
more meaningful 'interwikiprefix'.

Bug: T155600
Change-Id: Iec63734c82b0835a7f2444ea9af35617876831d5
2017-01-19 09:48:55 +01:00
jenkins-bot ae677fbc0d Merge "Ustring: Let gcodepoint work with moderately long strings" 2016-12-16 00:42:02 +00:00
Brad Jorsch db07787390 Cleanup backwards-compatibility code
https://www.mediawiki.org/wiki/Extension:Scribunto says that master
requires 1.25+, so let's remove checks for stuff that was added before
that.

* PPFrame::getTTL() was in 1.24.
* PPFrame::setTTL() was in 1.24.
* PPFrame::isVolatile() was in 1.24.
* Parser::fetchCurrentRevisionOfTitle() was in 1.24.
* ObjectCache::getLocalServerInstance() was added in 1.27, so restore the call to ObjectCache::newAccelerator() as BC.

This also removes BC with the php-luasandbox extension older than 1.6, which
was released before MediaWiki 1.22.

Bug: T148012
Change-Id: I36e37f3b65d0f167e1d28b00e0842d9721feee31
2016-10-13 11:07:44 -04:00
Brad Jorsch 629f11d0dd Fix pure-Lua ustring and empty patterns
An empty pattern isn't "safe" since it could match in between the
bytes of a UTF-8 character.

Also, it turns out there's a bug in PHP <5.6.9 preg_replace() that we
need to work around too.

Change-Id: I282e5909e4663461d60c5386693db182de2fd44c
2016-10-05 14:32:27 -04:00
jenkins-bot c48bda0698 Merge "Add handling for PCRE errors in ustringGsub" 2016-10-05 18:15:10 +00:00
Kunal Mehta 48748a6046 Improve validation of ScribuntoContent
Implement Content::prepareSave() to ensure that any content
directly passed to WikiPage::doEditContent() that doesn't run edit
filters will still be validated. We have to use prepareSave() instead of
Content::isValid() because validation depends upon the current Title.

Create a ScribuntoContent::validate() convenience function to hold the
logic for that and add a todo to use it in the EditFilterMerged hook.

Also, remove a parser test that depended upon being able to save invalid
modules directly, as what it is testing is no longer possible (unless it
pre-dates making valid syntax a requirement).

Bug: T145548
Change-Id: Ie57eff36100963f02899d669df7375577f7375e1
2016-09-14 11:50:07 -07:00
Marius Hoch 0f4db74148 Add mw.hash to Scribunto
Provides a simple wrapper for PHP's hash() and
hash_algos() functions.

I will add docs to the Lua reference manual once
this is merged.

Bug: T142585
Change-Id: I6697463974a175e99f9b77428a1085247165ebc9
2016-08-18 04:39:04 +02:00
Brad Jorsch ba19a82c06 Add handling for PCRE errors in ustringGsub
Bug: T130823
Change-Id: I6fab71c82ddab92daf6b369cb9857d9892f2d246
2016-07-15 15:43:58 -04:00
Brad Jorsch d643f40de9 Ustring: Let gcodepoint work with moderately long strings
For the PHP implementation, return the codepoints as a table instead of
multiple return values that get table-ified in Lua, to avoid hitting
too-many-values stack limits.

For the pure-Lua version, inline most of ustring.codepoint instead of
calling it to avoid what's effectively "{ unpack( stuff ) }".

Bug: T118687
Change-Id: I105f388cc23ab55d4124739700ef89d5354b7dbc
2016-07-15 19:35:58 +00:00
Brad Jorsch c9de00aeff SECURITY: Don't escape strip markers when escaping attributes in mw.html
Core strip markers were changed in T110143 to include characters that
are normally encoded in attributes, however we want to pass them through
here so they can be unstripped correctly in the output wikitext.

This fix makes "Strip markers in CSS" parser test pass again.

Bug: T110143
Bug: T135961
Change-Id: I1353931a53c668d8a453dfa2300a99f59fdb01c5
2016-05-22 21:40:32 -04:00
Brad Jorsch aa4d72e3ff Fix uncontroversial phpcs errors
The following continue to be ignored:
* Generic.Arrays.DisallowLongArraySyntax.Found, because I'm not sure
  Scribunto is ready to abandon old version support in master.
* MediaWiki.ControlStructures.AssignmentInControlStructures.AssignmentInControlStructures,
  because it's overly strict for its purpose.

Squiz.Classes.ValidClassName.NotCamelCaps isn't ignored globally, we
just ignore it explicitly every place it's needed.

Change-Id: I307668da6ef7b3e23da19b1fd1e08914239b99b3
2016-05-18 16:31:28 -04:00
jenkins-bot c753698eaa Merge "Provide a standard way to get the target of a redirect page" 2016-05-12 19:32:17 +00:00
Brad Jorsch b3da8a698d Add toNFKC and toNFKD to mw.ustring
This also makes some updates to make-normalization-table.php to handle
the move of UtfNormal to a separate library.

Bug: T126427
Change-Id: Id4985c3ca441cf92f08ba1f1af85c762ba43d7d2
2016-04-02 15:22:42 +00:00
aude 399b94fa24 Update LanguageLibraryTests per changes in core
appears the test depends on kk-cyrl translation of
limitreport-cputime-value, which has changed:

https://gerrit.wikimedia.org/r/#/c/278597/1/languages/i18n/kk-cyrl.json

Change-Id: Iedf847b7dd0dde094ce0c1c2af4f44b71b9c477f
2016-03-20 18:02:22 -04:00
Jackmcbarn b82ed4aa7d Restrict cached results to their original frame
When caching results from frame:preprocess and frame:expandTemplate,
restrict the scope of the cache to the frame object that was used. This
allows the integrity of the empty-frame expansion cache to be maintained
while also allowing parent frame access. This change is the equivalent of
I621e9075 in core.

Change-Id: Iae4c00e7e19ba12cfdaac135be16c991d9d0cea1
2016-03-09 11:27:23 -05:00
Ricordisamoa 1573bee81a Provide a standard way to get the target of a redirect page
The new Scribunto_LuaTitleLibrary::redirectTarget() method is
used by mw.title objects as read-only attribute 'redirectTarget'.

If the page does not exist or it is not a redirect, the value
of the attribute is `false`; otherwise, it is the target of the
redirect page, as mw.title object.

This is a proper alternative to parsing wikitext as it is done in:
https://en.wikipedia.org/wiki/Module:Redirect

Bug: T68974
Change-Id: Id4d9b0f8c1cd09ebc42c031d4d3fc0c33eea44aa
2016-03-01 14:30:22 +01:00
Brad Jorsch 1365e35d89 Waste CPU cycles, not wall clock time
Apparently microtime is a bit unreliable for this test for some reason,
so let's use getrusage() to measure actual CPU time instead.

Bug: T125045
Change-Id: Ia1ab6b043b99abb4fc6c2989ad09a24d97dd09c3
2016-02-11 14:00:03 -05:00
Jan Zerebecki 1517a1ba2e Fix typo in comment and missing spaces
Change-Id: Ifda05edd9b488768814d3251a93d71e9b55ea5a6
2015-11-11 23:08:15 +00:00
Jan Zerebecki fa0e32ce99 Temporarily skip a test that fails on HHVM.
HHVM has a bug in json decode https://github.com/facebook/hhvm/issues/5813 .
Skip the test until there is a fix upstream.

Bug: T103346
Change-Id: I7e44d98c29ba9b9f5443665fc046382f696193c9
2015-11-11 21:49:12 +01:00
Brad Jorsch cd618c7a92 ustring: Handle "empty" charset like Lua does (part 2)
Lua actually treats a close-bracket at the start of a bracketed
character class as a literal, rather than using it to close the
character class. Probably unintended behavior, but it happens.

Also, have the pure-lua version throw our more informative errors on
error even when falling back to string.find and the like, and fix some
other weird edge cases that came up in testing.

Bug: T95958
Bug: T115686
Change-Id: Iab783d4a3e58b1514cc09729d4a71c2cb1242ee8
2015-10-16 09:26:55 -04:00