Skip to content

fix(ta): batch testrun updates in ProcessFlakesTask#951

Draft
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/ta-process-flakes-n-plus-1
Draft

fix(ta): batch testrun updates in ProcessFlakesTask#951
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/ta-process-flakes-n-plus-1

Conversation

@sentry
Copy link
Copy Markdown
Contributor

@sentry sentry Bot commented May 30, 2026

This PR addresses an N+1 query performance issue identified in app.tasks.flakes.ProcessFlakesTask, specifically within the services/test_analytics/ta_process_flakes.py module.

Problem:
Previously, the process_flakes_for_commit function iterated over each upload, calling process_single_upload for every item. Each call to process_single_upload then executed a separate SELECT query via get_testruns and a bulk_update on the ta_timeseries_testrun table. This resulted in an N+1 query pattern, where N is the number of uploads, leading to inefficient database operations.

Solution:
To resolve this, the following changes were implemented:

  1. get_testruns function modified: It now accepts a list of upload_ids and filters Testrun objects using upload_id__in, allowing all relevant testruns to be fetched in a single database query.
  2. process_single_upload replaced by process_all_uploads: The logic for processing testruns (handling passes and failures) was consolidated into a new function, process_all_uploads. This function operates on all testruns fetched in bulk and performs a single Testrun.objects.bulk_update at the end.
  3. process_flakes_for_commit updated: The loop iterating over individual uploads and calling process_single_upload has been removed. Instead, process_flakes_for_commit now collects all upload IDs and passes them to the new process_all_uploads function, effectively reducing N SELECTs + N UPDATEs to a single SELECT + a single UPDATE operation.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

Fixes WORKER-YNC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants