Commit graph

162 commits

Author SHA1 Message Date
sbailey bef1042e69 Replace use of select or selectRow with QueryBuilder
* All database methods using old access methods converted to use
   queryBuilder except row counters

Bug: T312434
Change-Id: I8796adf63179c4505fbb76ab6fd71cf3775c9b89
2023-03-25 15:13:59 +00:00
sbailey 6aa4cdeba9 Linter Tag and Template search feature, UI and report code
* Tag and Template search is enabled using config variable
   'LinterUserInterfaceTagAndTemplateStage' and also checks for
   the linter table column 'linter_tag' to exist to protect the
   report code from error if the column is absent. As the linter
   table alter maintenance added both the linter_tag and
   linter_template at the same time, there is no reason to check
   both. The user interface code does not check for the field
   presence only the config variable.

 * This code depends on the recordLintJob code writing the tag
   and template data which is enabled by the config variable
   'LinterWriteTagAndTemplateColumnsStage' and also assumes the
   data migration maintenance script migrateTagTemplate.php has
   been run to populate linter error records created prior to
   the table alter and the write code being enabled.

Bug: T175177
Change-Id: I2f951dfcd34e3dc6ca17e8754cfaeba8baa3e835
2023-02-27 06:55:06 -08:00
sbailey 499a1c3796 Use new linter_namespace column and index in the Linter table
* This performance improvement patch uses the namespace from the
   new field 'linter_namespace' in the linter table instead of
   the 'page_namespace' in the page table. It checks for and
   requires the presence of the linter_namespace field in the
   linter table, as well as the config variable
   'LinterUseNamaspaceColumnStage' being set true.

 * If the linter_namespace field is present and aforementioned
   config variable is true, the code assumes that the config
   variable 'LinterWriteNamespaceColumnStage' is set true and
   recording the linter_namespace for new lint errors is
   active and the migrateNamespace.php migrate code has been
   run to migrate the page_namespace data into existing linter
   records that were created before the linter_namespace column
   existed and were left NULL during the table alter maintenance
   operation.

 * A follow on patch should remove the configuration variables and
   conditional code producing the final, refactored code dependent
   on the new namespace column index.

Bug: T299612
Change-Id: I4a1497d9e4dcd6a9a7befdaccf3e34c61694365d
2023-02-24 06:54:08 -08:00
Kunal Mehta eaf2d26db8 Remove API action=record-lint
The action=record-lint was a hack that allowed Parsoid/JS to send data
to MediaWiki to be stored in the linter database. Thankfully we no
longer need it in the glorious Parsoid/PHP world because it can write
directly to the database.

The API module, i18n messages and $wgLinterSubmitterWhitelist are all
removed.

Bug: T329992
Change-Id: Iba70e05a2e28f4ecd02101cff51993ebe65f19d0
2023-02-20 20:44:00 -05:00
sbailey d12bf639f6 Change linter maintenance scripts to use existing config varaibles
* Having separate config variables to enable the maintenance
   migrateNamespace and migrateTagTemplate scripts is duplicitous
   and should be shared with the write enable config variables.

Bug: T329342
Change-Id: I4cb453fc0678b065cb42a2ca59863da1ab9cdbe4
2023-02-14 09:43:54 -08:00
jenkins-bot f82904c185 Merge "Link to detailed Special:LintErrors from action=info" 2023-02-10 02:10:20 +00:00
jenkins-bot 107b89a901 Merge "Move LoadExtensionSchemaUpdatesHook handler to separate class" 2023-02-10 02:10:19 +00:00
sbailey 2768a70218 Fix migrate data error when params has excessively long strings
* The linter migrate code for linter_tag field and linter_template
   field are constrained by the database schema to 32 characters
   for the tag field and 255 characters for the template field.
   In some anomalous circumstances parsoid can report tag and or
   template fields in the linter_params object that exceed those
   character limits. This code truncates these excessively long
   strings to protect the database migrate update code from a
   length exceeded error.

Bug: T329113
Change-Id: I8af7c44759f172eae77d3519a6eac47110e9b1e7
2023-02-09 18:20:46 +00:00
sbailey 07046457f0 Fix write error when linter_params has excessively long strings
* The linter write code for linter_tag field and linter_template
   field are constrained by the database schema to 32 characters
   for the tag field and 255 characters for the template field.
   In some anomalous circumstances parsoid can report tag and or
   template fields in the linter_params object that exceed those
   character limits. This code truncates these anomalous strings
   to protect the database update code from a length exceeded
   error.

Bug: T328979
Change-Id: I057ae2e32a9e1a7735b5300409e5693e8db5c764
2023-02-08 10:40:12 -08:00
Kunal Mehta cb84f3a872 Link to detailed Special:LintErrors from action=info
action=info has a summary table of number of lint errors by category,
but we have richer information available via Special:LintErrors. If
there is a "Lint errors" section, provide a link below the table to
Special:LintErrors for the errors on this page.

Update ApiRecordLint for the new Hooks constructor and leave a FIXME
to eliminate the coupling.

Bug: T301374
Change-Id: Ic1fcf42b50d1392ac53201ceb256691133cf62ff
2023-02-06 20:41:45 -05:00
Kunal Mehta b5a7aaf7e0 Move LoadExtensionSchemaUpdatesHook handler to separate class
This hook is not allowed to have a service, so before we can add
services to the main Hooks class, it needs to be split out.

Change-Id: Ia7b4b8bf7c91ebb851c5de9f0f54f56b0993bf83
2023-02-06 20:36:45 -05:00
sbailey abbcc17511 Fix linter table linter_params definition to not be nullable
* update all schema files to correct linter_params being set nullable by accident in
   2020 during abstract definition update

 * include hook to provide 3rd party support, use $updater->modifyExtensionField(

Bug: T327806
Change-Id: I7fea08280cf2fa7956e5b300dc3365dc3561ca52
2023-01-26 12:47:36 -08:00
Umherirrender eb31c6c4dc Replace deprecated HTMLForm::addHeaderText
Bug: T325474
Change-Id: If1a4b14b6cec480f600065c913cb1f27260fbc5f
2022-12-27 12:48:42 +01:00
sbailey 350d677c5b Phase 3 of T175177: Migrate linter_params into new fields
* The migrate code is designed to perform a one-time update of
   linter_params JSON encoded template and tag information into
   the new discrete template and tag text fields for use as
   additional search criteria. The function can be restarted if
   it is interrupted.
 * It now uses configurable batching and sleep times between
   batches to allow the database to do other work and replication
   to occur without stressing infrastructure.
 * The migrate code is only called by test code and needs to be
   called one-time from a maintenance script.

Bug: T175177
Change-Id: Idc4ca88d4762bc7a3bcbc4e66c0f275562083867
2022-12-09 12:01:06 -08:00
sbailey 702ce215d0 Phase 3 migrate code for namespace column add to Linter table
* Migrates namespace info from the page tables page_namespace field
   to the new linter table field linter_namespace. This duplication
   of the namespace value was requested to greatly reduce the amount
   of database activity required by the linter search and reporting
   code.

 * This patch has been prepared as a dark launch patch enabled with
   config value LinterMigrateNamespaceStage and assumes that the
   Linter table has had the linter_namespace column added to it,
   and recording of the namespace field is already enabled and is
   populating the namespace column.

 * The migrate code now runnable from Linter/maintenance directory,
   using migrateNamespace.php, which will be deployed in a separate
   patch. The maintenance code creates an appropriate environment
   to call migrateNamespace( in Database.php.

Bug: T299612
Change-Id: I73cb80729d6a5a8716fe93164ad1e42e6958d672
2022-11-28 08:07:54 -08:00
James D. Forrester 1f119cd5b1 ApiQueryLinterStats: Change the 'ls' prefix to 'lntrst' to avoid conflict
In practice this isn't a breaking change as no-one has any params to
re-name in callers.

Bug: T319202
Change-Id: Ic925e258ad3c2e194fd9f04f8afd255a9aa99e45
2022-10-03 12:06:55 -04:00
Reedy 1c526b7443 Convert to HookHandler
Change-Id: Ib8ad9a58e781f7dad4b047d170a8f689802c4bad
2022-09-07 02:58:04 +01:00
Reedy 89d3f6152b Minor cleanup
Change-Id: I0b8abdbeaece73fe8759ee220b9a3aefce240e68
2022-09-07 02:48:18 +01:00
sbailey b358b20dca Second phase of T175177: Adds template and tag to RecordLintJob
Bug: T175177
Change-Id: I59be7cabb80ace98da3c7f6f36a0d3d4f6b17d23
2022-08-22 12:47:01 -07:00
Arlo Breault 0f46ed3dbc Get config from services, not globals
If908c4dc99c966cde2981f9a03be38a577406a4e introduced the
wgLinterWriteNamespaceColumnStage global but didn't add it to the
extension.json

Change-Id: I8ee5849f67ddfd894a25425582b59404eb52aef2
2022-08-03 13:01:42 -04:00
Arlo Breault f6607e8818 Stop using wfGetDB
This is pulled out of I59be7cabb80ace98da3c7f6f36a0d3d4f6b17d23

Change-Id: Iab6a47320995e9adb1666cd0bb728f516a2fde69
2022-08-02 14:40:30 -04:00
Kunal Mehta faf474e139 Only call getRequest()/getOutput() once in same function
Change-Id: I17b90a7d9f65dda563daa9cb09b40b965f3c49d3
2022-07-02 15:00:46 -07:00
Kunal Mehta be6c6a4eb0 Treat empty titlecategorysearch parameter same as not being present
The form will automatically add an empty &titlecategorysearch= parameter,
so treat the parameter being absent the same as empty string.

Bug: T311202
Change-Id: I377c55a59ce60f81532ae7effba60fbe29da609b
2022-07-02 14:57:43 -07:00
sbailey e306f5f681 Add namespace column and new index to Linter table - part 2
* Writes namespaceID to new "linter_namespace" field if global
   "wgLinterWriteNamespaceColumnStage" is present

Bug: T299612
Change-Id: If908c4dc99c966cde2981f9a03be38a577406a4e
2022-07-01 06:51:29 -07:00
DannyS712 c97d9e329d Start catch blocks on the same line as the preceding }
Change-Id: Idafe486275588e20846f7b3fd03adc97744d83b8
2022-06-23 00:27:00 +00:00
sbailey 79e825a466 Provide search by title prefix for any category of lint error
Bug: T185685
Change-Id: Ib667fcf5b2b1e752fde297b32b8bbe37dceabc5a
2022-06-16 13:27:14 -07:00
gerritbot 044ba77182 Fix usage of ApiBase::PARAM_* deprecated constants
The ones that are replaced with ParamValidator

Bug: T275455
Change-Id: I773e3ec3072bb25c455c080b8cd5870b3b27d23e
2022-04-04 00:37:17 +00:00
gerritbot e7dccae886 Replace deprecated ApiBase::PARAM_ with IntegerDef ones
The rest of ApiBase::PARAM_  will be done in separate patches

Bug: T275455
Change-Id: I491d4306611f53ed19aa92e119de7b386ef6bda4
2022-04-03 22:15:33 +02:00
sbailey ffcbb3af84 Add title interactive lookup to category based linter error search
* Just one of two or more patches enhancing linter error searching

Bug: T185685
Change-Id: Ic103d1ece079f5689f24425b8b7e78e0fb00449e
2022-03-28 10:07:27 -07:00
jenkins-bot d772f211dd Merge "Use HTMLTitleTextField on SpecialLintErrors" 2022-03-16 20:52:40 +00:00
sbailey 7309cc139c Linter allows specification of a page prefix to refine selection
* UI and functionality working, SQL injection risk is handled using
   the database wrapper function buildLike()

Bug: T185685
Change-Id: I1f630a83f01a1c93f29643e4fc8baf55391b758d
2022-03-10 08:48:19 -08:00
Subramanya Sastry b08fd17186 Hooks.php: Check for non-array $tags
Bug: T302918
Change-Id: I96fdbbc443025d9461dc168e58164004a0f8d7ed
2022-03-02 15:40:05 -06:00
sbailey 6925519cb5 Delete lint errors when blank page saved while changing content type
* Add "mw-blank" as another tag value that erases all lint errors
   for a page as a blank page cannot have any lint errors.

Bug: T280193
Change-Id: Iaad8ce75950588b2676de5dfb5f5221d64231f0e
2022-02-28 15:03:16 -08:00
Arlo Breault 285838961b Account for all lintable contentmodels after contentmodel change
Change-Id: I54495ec01d4fc3b29f1c924f86be3bcd2ed94977
2022-02-25 17:46:56 -05:00
sbailey 0e56c22277 Delete lint error records when content model changes from wikitext
* Determines if new content type is not wikitext and if so
   deletes all existing lint error records for that pageID.

Bug: T298343
Change-Id: I20fac9a0c901f3e7a5cc898566a4487fbe70798f
2022-02-25 13:22:10 -08:00
Umherirrender ba6b0db292 Use HTMLTitleTextField on SpecialLintErrors
This gives autocomplete on the special page

Change-Id: Ic29cf4c16886609c6689c66aff6ebc28a20879ea
2022-02-16 18:19:12 +00:00
Ammarpad daa6275a20 SpecialLintErrors: Make code more resilient to bad input
Use WebRequest::getText, which discards array values since
the `pagename` param requires string value.

Bug: T301360
Change-Id: Ifc23a7020e410603ac86b71d7aee2d28f06b9ab2
2022-02-15 18:49:40 -05:00
Alexander Vorwerk 8c7fa34662 Fix typo in patch file
Bug: T175177
Follow-Up: Iac22d99109bb1253f450a64254f50677e3cdefeb
Change-Id: I0c1ad29b67a732faa06add8faff4f70988b8613b
2022-02-07 20:47:53 +00:00
sbailey 56567da56f First phase of T175177 database alter table adds two fields
* Adds template and tag name fields to enable better search
   capability to the Linter extension

Bug: T175177
Change-Id: Iac22d99109bb1253f450a64254f50677e3cdefeb
2022-02-03 14:57:17 -08:00
sbailey 4544a9c7cc Add namespace column and new index to Linter table - part 1
* This patch contains the code needed to update the database with
   the addition Linter table column linter_namespace <int/null>
   and add an index with category and namespace for faster
   selection of linter records.

Bug: T299612
Change-Id: I05da381b9a74294b44d4aef968614277d601a176
2022-01-28 20:19:38 +00:00
jenkins-bot bd45eeda91 Merge "Replace deprecated JobQueueGroup::singleton()" 2022-01-23 21:39:36 +00:00
Umherirrender 354012f419 Replace deprecated JobQueueGroup::singleton()
Change-Id: Ib19314b8a8a4265daf67e31d50f1c61aec2edb8e
2022-01-21 17:22:20 +00:00
Subramanya Sastry 70ffca650e Drop 'inline-media-caption' lint requests
Bug: T297443
Bug: T299302
Change-Id: Id158f1fef8be06ddac733c71b7c1e26a58270955
2022-01-17 12:55:51 -06:00
Matěj Suchánek 1335b79a0b Pass HTML element attributes as an array
Html::openElement checks for this and logs a warning.
Html::closeElement does not, and this makes it return
</span class="error">.

Change-Id: I31c2809d2fd5421606fa877021d1636ac0eb4d26
2022-01-15 14:18:56 +01:00
Arlo Breault 8a797fd625 s/linter-pager-title/linter-pager-title-header/
Change-Id: I51e4ffa89954b0b6bf798562668dc4a95c00d9da
2022-01-11 23:24:52 +00:00
Arlo Breault ace2d5c17d s/linter-pager-template/linter-pager-template-header/
Change-Id: I658b258fe46bdeefbfc0b1d348e4ac4d9c423205
2022-01-11 12:41:56 -05:00
Arlo Breault 850ec4fa70 Add i18n for pager category
Follow up to I70fde2676ded6a112f7f2b07f94f6f4b616f0e39

Change-Id: I38647833938fb27793827649bc5c785dc3fd63ea
2022-01-11 12:21:19 -05:00
Arlo Breault fc8c39baa5 Fix lint error updating
The article id of the title is set to 0 when the page is deleted so,
although the lint job from the hook runs, it doesn't remove anything.

Reverts most of I06b821b65f65609ddac8ed4e7c662336082d8266

Bug: T298782
Bug: T170313
Change-Id: I2610b9b16d4032b0e18b3537cc9ed51bfdaff299
2022-01-10 16:24:00 -05:00
Arlo Breault 2218e87c05 Move outputting headers back to the top
Change-Id: I175d69c990de032686033ef909eec0b1865e795f
2022-01-10 15:21:20 -05:00
Arlo Breault f821fa7f47 Add a title for single page view
Follow up to I70fde2676ded6a112f7f2b07f94f6f4b616f0e39

Bug: T151362
Change-Id: I21b8b9085f13480075e0b572f38ec9a05a398f01
2022-01-10 15:16:04 -05:00