Optimise create remixgroups#2125
Open
alastair wants to merge 2 commits into
Open
Conversation
If a sound's sources or remixes change then we need to update both this sound and the ones on the other end of the relation so that search works as expected. If a sound is updated and its sources change perform the mark dirty operation at the same time that we expire caches and send emails. This means that relationships will be updated in solr the next time the update script runs.
This command deletes all remix groups, recreates them, and then marks every sound that is part of a remix group as dirty. This causes a spike of 10,000 sounds to be reindexed every hour, when these items almost never change. Instead, take advantage of the previous commit to just mark sounds as dirty directly when their sources/remixes are edited. Clean up the script a bit to reduce queries
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue(s)
Description
create_remix_groups deletes all remix groups, recreates them, and then marks
every sound that is part of a remix group as dirty. This causes a spike
of 10,000 sounds to be reindexed every hour, when these items almost
never change.
Instead, just mark a sound's sources/remixes as dirty if they are edited/deleted and update them the next time the reindex script runs.