Commit graph

97 commits

Author SHA1 Message Date
jenkins-bot 968bd9b817 Merge "Add tests for tokenizer caching" 2019-04-17 23:27:19 +00:00
Daimona Eaytoy 4b10a544ab Add tests for tokenizer caching
Caching the result of the tokenization is pretty important
performance-wise, so this test ensures that caching works as expected.
I have also extracted the method used to generate the cache key for
easier testing, and moved the cache instance to a class member because
otherwise that piece of code can't be tested...

Bug: T201193
Change-Id: I9b3bc36b552901bc6ca7609ee51e80be2979a9c4
2019-04-15 16:59:55 +02:00
Daimona Eaytoy 23fe973544 Remove pointless number cast
If the number is int there's not need to intval it, and if it's float
there's no need to floatval... Just use it to determine the internal
data type, like it happens for sum and sub.

Change-Id: Ie00c4bb4e67ad1eface0cea3eb42d04aa5fb39cc
2019-04-14 10:49:09 +02:00
Daimona Eaytoy 909eec6716 Tweak coverage part 2
Follow-up of Ic30883f7d261d974a2be46308d023e2714119e95, with two files
that I forgot to git-add and a repositioning of comments to avoid the
last bracket to be reported as uncovered.

Bug: T201193
Change-Id: I6bf7e5892a0f49f6a138792f0aedf230a70c18a8
2019-04-13 19:26:01 +02:00
Daimona Eaytoy 4bcb64b01a Increase code coverage a bit
This patch mostly adds coverageIgnore comments for intendedly
unreachable code etc. Some of them could be made testable by adding a new
filter function (e.g. array cast), but this patch is meant to be
comment-only (aside from the parser test).
Ignoring coverage for these lines makes some methods reach 100%
coverage, which in turn makes it easier to look at the coverage chart
and identify at a glance which parts of the code *really* need to be
covered.

Bug: T201193
Change-Id: Ic30883f7d261d974a2be46308d023e2714119e95
2019-04-13 18:30:14 +02:00
jenkins-bot efe32b7c93 Merge "Add doc for every class member" 2019-04-06 14:37:19 +00:00
jenkins-bot d53c84da36 Merge "Restore check for dividebyzero" 2019-04-06 12:35:23 +00:00
Daimona Eaytoy 27545422b4 Add a function to the parser to retrieve the next token
It provides some sort of look ahead capability, avoiding to move and
then roll back.

Change-Id: I6293cbd355572c9de3a8591dd8286b14a239ffb2
2019-03-23 11:55:08 +01:00
Daimona Eaytoy 9144f20245 Restore check for dividebyzero
Follow-up of I1721a3ba532d481e3ecf35f51099c1438b6b73b2. This is the only
wrong replacement: strict checking will let 5 / 0.0 pass, with
unexpected results. Adding a regression test for it, too.

Change-Id: I25dbe9fafa92fd9a11bd8bc6ab8e66f305b8d48e
2019-03-23 11:38:39 +01:00
Daimona Eaytoy f2c1beec44 Replace double-equals with triple-equals
Since double-equals are evil. I left some of them in place where I
wasn't sure, but I may be changed some which were intended to be
doubles. It could be a good idea to delay merging this patch until we'll
have more code coverage.

Change-Id: I1721a3ba532d481e3ecf35f51099c1438b6b73b2
2019-03-22 16:12:13 +01:00
Daimona Eaytoy bedbe36744 Add doc for every class member
Adding PHPdocs to every class members, in every file. This patch only
touches comments, and moved properties on their own lines. Note that
some of these properties would need to be moved, somehow changed, or
just removed (either because they're old, unused leftovers, or just
because we can move them to local scope), but I wanted to keep this
patch doc-only.

Change-Id: I9fe701445bea8f09d82783789ff1ec537ac6704b
2019-03-17 11:40:24 +01:00
jenkins-bot 3f3e98fbc5 Merge "Fix shortcircuit for consecutive operations" 2019-03-17 10:04:14 +00:00
jenkins-bot e2f1880922 Merge "Don't use wgLang and wgContLang" 2019-03-17 09:53:16 +00:00
Daimona Eaytoy 53ab2b5067 Fix documentation errors reported by Phan
Change-Id: I5788147ba1998235ded9eedbf64ebad37fce236f
2019-03-16 09:27:05 +00:00
Daimona Eaytoy 6f4bfc9597 Fix shortcircuit for consecutive operations
Using break could halt parsing between operations, instead use continue
to parse all operations.

Bug: T214642
Change-Id: If67ddaffef280c2448c55ae536013758617bba68
2019-02-08 17:55:59 +00:00
Daimona Eaytoy 51120e51c5 Don't use wgLang and wgContLang
For wgLang, there's a Language object available in the proximity, so just pass it.
For wgContLang, use MediaWikiServices.

Change-Id: Ic492007f2d5eeb8048d0919a4b9b7dd98c15c350
2019-02-06 12:00:44 +01:00
libraryupgrader b744d18526 build: Updating mediawiki/mediawiki-codesniffer to 24.0.0
The following sniffs are failing and were disabled:
* MediaWiki.Usage.DeprecatedGlobalVariables.Deprecated$wgContLang

Change-Id: Ic167fc5e836c5437edc6b330e5d73f9913bc2859
2019-02-06 09:28:26 +00:00
jenkins-bot b44984c50a Merge "Remove unused stuff" 2019-01-19 12:18:22 +00:00
jenkins-bot 575646393b Merge "Improve code readability" 2019-01-19 12:11:06 +00:00
jenkins-bot a2bee3bcf3 Merge "Simplify parser methods" 2019-01-19 12:11:04 +00:00
Daimona Eaytoy 6217ffb928 Remove unused stuff
Variables declared but never used, redundant code, and old leftovers.

Change-Id: Ic51044a45a1b49ad6c7af06c646b11893411a7cd
2019-01-18 17:04:19 +01:00
Daimona Eaytoy 93e8cb5ac5 Tune logging channel
As follow-up of I10b1fd2d9bdfe518089c053d77fef568170ecb65, use
'AbuseFilter' instead of 'AbuseFilterDeprecatedVars' as channel name.
Raise level for null-title filtering. Since with a null title
several things are likely to break, a warning is more appropriate here.
Tweaked the message as well, to include the bug number and to avoid
pointlessly including the title (which is null).
Lower the level for stashedit hit/miss (as it's really spammy and not
that useful right now).
Use 'abusefilter' instead of 'AbuseFilter' for statsd so that everything
has the same prefix.
Also raise the level for parser exceptions and unrecognized
consequences.

Change-Id: I1f9988155e924232b201281795cd322636da8082
2019-01-16 08:56:22 +00:00
jenkins-bot 23a7aa69a5 Merge "Fix regex group counting for get_matches" 2018-12-04 13:58:06 +00:00
Daimona Eaytoy 7427333ed5 Improve code readability
Simplify some logic constructs, reduce the amount of return statements
inside methods, explicitly declare variables before using them, reduce
code duplication, add names to JS anonymous function to produce clearer
stack traces.

Change-Id: Ife4546a91c30d4c519d09a712ba56a2f33abe579
2018-11-19 16:01:37 +01:00
Daimona Eaytoy badde6ba75 Revert "Revert "Add typehinting for every object-only parameter""
This reverts commit 1ed75b4ae0.
Fixed the one which caused errors, by making articleFromTitle
only use WikiPage, instead of silently mixing WikiPage and Article.

Note for reviewers: this patch is identical to the one which was
previously +2ed, which was mostly correct. To see the actual change,
diff AFComputedVariable with 1..current.

Change-Id: I6747eaed861af6c40a3b1610aebcc1174296e9ed
2018-11-15 10:09:16 +01:00
Daimona Eaytoy 16475c0266 Fix regex group counting for get_matches
Adding the * as character to match after parentheses, since it may be
used with backtrack verbs (e.g. (*FAIL), (*SKIP)). I guess this is a
very, very rare use case, but since the fix is easy, let's include it.
Also, added a ToDo since we should probably find a better way to count
capturing groups, although I cannot figure out any.

Change-Id: Idcb303b4740530af9d3f009414d35d68f59effd0
2018-11-01 11:52:33 +01:00
jenkins-bot 7e151f5edc Merge "Unbreak short circuit for arrays" 2018-10-18 04:04:31 +00:00
jenkins-bot eb1303c8cd Merge "Revert "Add typehinting for every object-only parameter"" 2018-10-17 03:09:55 +00:00
Jforrester 1ed75b4ae0 Revert "Add typehinting for every object-only parameter"
This reverts commit 69d7669069.

Reason for revert: Causing UBN train blocker

Bug: T207220
Change-Id: I3445d9b3065149e2beb149e10fbbf5502b480f57
2018-10-17 01:22:23 +00:00
jenkins-bot 7a8a2fa3e1 Merge "Add typehinting for every object-only parameter" 2018-10-16 02:48:39 +00:00
Daimona Eaytoy 69d7669069 Add typehinting for every object-only parameter
This patch covers every object-only parameter, adding a typehint for it
to avoid errors.

Change-Id: Iebf700621b9dbff78c3bd8f3c136ed15ef4b8d4b
2018-10-15 09:56:09 +02:00
Daimona Eaytoy eafb4f56c7 Avoid useless error message for regexfailure exception
Users writing filters probably don't care about preg_match or whatever
happens in PHP. Also, it's not that useful to see "unspecified error".

Change-Id: I014742fa6f678126f55ac5ccff38e44b2c5a7d15
2018-10-08 19:19:01 +02:00
Daimona Eaytoy 6d54b83f2c Simplify parser methods
Use a single function to check parameters amount, avoid duplication
between keywordIn and keywordContains, use if...elseif instead of
if-else when statements have a return inside, simplify some other logic,
add typehinting, and change method visibility according to use of such
methods.

Change-Id: I22225a5cbbb93679a0e78bf6e15866829167fbf4
2018-10-03 17:19:40 +02:00
Daimona Eaytoy e60dacbbea Fix code comments
Fixed some comments adding explanations, fixing syntax, and parameter types
for docblocks. Also fixed some whitespace mess, and added a missing use
statement.

Change-Id: I3547c90bdaa2cab5443e8bf0c63b217fe6ba663f
2018-10-03 16:45:03 +02:00
Daimona Eaytoy d9d5af3890 Unbreak short circuit for arrays
This problem have been making filters potentially fail silently since
2009. Also add tests for arrays to make sure that no problems arise
when short circuit is used.

Bug: T204841
Change-Id: Ie4e2e06498c1202ba73afcc5d164a72427abbca5
2018-10-03 16:44:10 +02:00
jenkins-bot a0a4755c59 Merge "Remove unused method from parser" 2018-09-09 12:32:56 +00:00
jenkins-bot 121df619da Merge "Improve coverage for AbuseFilterTokenizer" 2018-09-09 12:30:49 +00:00
Daimona Eaytoy bffba28713 Add full tests for deprecated variables
This test checks every deprecated variable to be identical to the
newly-named one, and to emit a debug notice. It also changes such debug
to be emitted via logger instead of wfDebug.

Bug: T201193
Bug: T173889
Change-Id: Ie55746bb7731062ae2d46d84857af2a05d78cf4c
2018-08-29 11:00:28 +02:00
Daimona Eaytoy 775c736512 Improve coverage for AbuseFilterTokenizer
This will make tokenizer almost fully covered. The only uncovered parts
are the one with cache and an else condition which I think won't ever be
executed, and thus added a comment for that. Also, remove an obsolete
xxx comment from ComputedVariable (fixed in
I8e420f0259ef6c9e579f7a00beb58f28af9da37d)

Bug: T201193
Change-Id: I6e9a73aa9e437f096f6a1e20d53a7cb50e5ed85d
2018-08-25 10:25:16 +02:00
Daimona Eaytoy a8b62dc828 Remove unused method from parser
AbuseFilterParser::setVarsimply calls the setVar method in
VariableHolder and is currently unused. Its only call was removed in
I80cbc4033ff96f2fe8c1da263b1877bfb4c7c0c4. After this patch we'll only
have an uncovered line in the parser, which is likely due to a bug in
the coverage check.

Change-Id: Ic860b03b2d23fec073a9294e356e074ae1b14ae5
2018-08-24 12:30:47 +02:00
Daimona Eaytoy 03b52c2b37 Remove unused function and improve unit test
AbuseFilterParser::setVars is only used in a parser test. In the past it
was also used in the actual code (see for instance
https://phabricator.wikimedia.org/diffusion/EABF/browse/master/;5cc8dac63ca585c288ca4c8605db810774e39666?grep=setVars), but at the moment it's pretty unuseful.
This patch removes such function and makes the unit test use literals
instead of variables to avoid calling it.

Change-Id: I80cbc4033ff96f2fe8c1da263b1877bfb4c7c0c4
2018-08-23 11:00:16 +02:00
jenkins-bot a762c82fe7 Merge "Add aliases for "_text" and "article_" variables" 2018-08-22 12:44:20 +00:00
Daimona Eaytoy 6bc630cfef Add aliases for "_text" and "article_" variables
Variables regarding title (full list in task description) are quite
deceiving, since they use "text" instead of "title". As proposed in the
task, this is the first patch to add aliases for those variables and
slightly deprecate the old ones. In the future we may be able to replace
every occurrence (either with a search function or directly on the
database), but even a coexistence would be enough to avoid
confusion. A wfDebug log is generated whenever a deprecated variable is
parsed. The "article_" prefix is also changed to "title_", in the same
way as above.
Also, added a hook which other extension may use to specify their
deprecated variables, which will be handled the same as core ones.

Bug: T173889
Change-Id: I5c370b54e6516889624088e27928ad3a1f48a821
2018-08-21 16:59:56 +02:00
Daimona Eaytoy 4f3b020f5d Improve code coverage for AbuseFilterParser
Add some tests and improve others to raise coverage percentage. This
should lead to almost 100% for the AbuseFilterParser class. Aside from
this, a couple of changes:
* Remove an unused function
* Let equals_to_any return a genuine result with empty strings
* Remove an if which will never be true in skipOverBraces, since the
function is called after checking the same conditions.

Bug: T201193
Change-Id: I7020b2ed996236c38c5784d161ad98ec44163406
2018-08-20 14:38:40 +02:00
jenkins-bot f587230fea Merge "Use noparams exception and correctly count function parameters" 2018-08-02 08:16:12 +00:00
Kunal Mehta 404e098c3b Fix MediaWiki.Usage.InArrayUsage.Found issues
Change-Id: I1898d95d92cda279c1b9c8a452fb7d054ff263bf
2018-07-29 15:19:09 -07:00
Daimona Eaytoy cea1b0aec3 Compact some other comments
Remove unnecessary ones, make clear what the cryptic ones mean, and
inline them when possible.

Change-Id: I384859871a66ced8cb0d81260c06c5a5b278866f
2018-07-17 17:17:44 +02:00
Daimona Eaytoy 32718888c0 Use noparams exception and correctly count function parameters
For the counting part I used this a relatively simple approach. It might
not be the best one, but should work without changing too much code. As
for the exception, I added it to every function which takes a single
parameter. Plus a couple of minor fixes: removed an unused function and
replaced "__METHOD__" with function names.

Bug: T198300
Change-Id: I484fe2994292970276150d2e417801453339e540
2018-07-15 15:32:26 +00:00
Daimona Eaytoy 7a64280893 Add phpunit tests for all exception thrown in the parser
All uses of "throw" inside AbuseFilterParser are now covered.
Bonus: added a standard suppresswarning when checking regex validity.

Change-Id: Iacb8f7a361079e3e117dc6845597c7bd8473e54a
2018-07-01 18:31:11 +02:00
Daimona Eaytoy 7104c40518 Copy levels documentation on AbuseFilterParser
doLevel- functions are currently documented in AFPToken. This patch
copies such comments on docblocks in AbuseFilterParser, the place where
this docs can really be helpful.

Change-Id: I4e47e760a56800faa9b0a1146e0d79f8955dca9a
2018-06-30 20:35:49 +02:00