Issue #3397: always delete config_log records in cleanup task - #3398
Open
Guillermo Gomez Arias (guillogo) wants to merge 1 commit into
Open
Issue #3397: always delete config_log records in cleanup task#3398Guillermo Gomez Arias (guillogo) wants to merge 1 commit into
Guillermo Gomez Arias (guillogo) wants to merge 1 commit into
Conversation
The config_log deletion was nested inside the "if (!empty($logidstodelete))" block, so the records were only removed when a matching record still existed in logstore_standard_log. That match is not guaranteed: old log records are removed every night by \logstore_standard\task\cleanup_task according to the loglifetime setting, while config_log is never purged by Moodle. When no match is found nothing is deleted, the task then counts the records again, sees they are still there and queues another adhoc task, which reads the same records and repeats. Close the block after the logstore deletion so the config_log records are always deleted. The execution time check and the pause between chunks move out of the block as well, so they apply to every chunk.
Author
|
@microsoft-github-policy-service agree company="Catalyst IT Canada" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3397
Problem
The adhoc task
\local_o365\task\deleteinvalidconfiglogdoes not delete theconfig_logrecords it is supposed to clean up, but it still queues a new copy of itself after every run, so the number of adhoc tasks keeps growing.The deletion of the
config_logrecords is nested inside theif (!empty($logidstodelete))block, so those records are removed only when a matching record still exists inlogstore_standard_log.That match is not guaranteed. Old log records are removed every night by
\logstore_standard\task\cleanup_task, according to theloglifetimesetting, whileconfig_logis never purged by Moodle. So the oldconfig_logrecords stay while their log entries are already gone.When there is no match, nothing is deleted. The task then counts the records again, sees they are still there, sets
$hasmore = trueand queues another task, which reads the same records (ORDER BY id ASC) and does the same thing.On our production site this created around 6400 adhoc tasks, each running heavy
COUNTqueries onmdl_config_log. See #3397 for the full report and the cron output.Change
Close the
ifblock right after the logstore deletion, so theconfig_logrecords are always deleted for every chunk. The execution time check and the pause between chunks move out of the block as well, so they also apply to every chunk.This is the smallest possible change: no new variables, no new conditions, and the logstore deletion logic is untouched.
Introduced in
Commit
807457d(PR #3067). Before that commit theconfig_logdeletion was in its own loop and was unconditional, so the task always made progress.Testing
config_logrecords forlocal_o365whose matching entries inlogstore_standard_logno longer exist (for example after the log cleanup task has run).\local_o365\task\checkinvalidconfiglog, then run the queued adhoc task.... N records remainingwith the same number as... Found N config_log records, and a new adhoc task is queued on every run.... Deleted 500 config_log records for chunkand the remaining count goes down on each run, untilAll invalid config log records have been deleted.is reported and no new task is queued.Note
The issue also mentions two other points that are not addressed here, to keep this PR minimal:
manager::queue_adhoc_task()is called without the$checkforexistingparameter, andcheckinvalidconfiglog.phponly recognises existing tasks withfaildelay == 0. Happy to send a follow-up PR for those if you want.