Skip to content

fix(worker): N+1 query in ProcessFlakesTask for testruns and expired flakes#967

Draft
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/worker-n1-flakes-task-hV5i4t
Draft

fix(worker): N+1 query in ProcessFlakesTask for testruns and expired flakes#967
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/worker-n1-flakes-task-hV5i4t

Conversation

@sentry
Copy link
Copy Markdown
Contributor

@sentry sentry Bot commented Jun 2, 2026

This PR addresses an N+1 query issue in app.tasks.flakes.ProcessFlakesTask (apps/worker/services/test_analytics/ta_process_flakes.py).

Problem:
Previously, when processing flakes for a commit:

  1. process_flakes_for_commit would fetch all relevant uploads for a commit.
  2. For each upload, process_single_upload would then execute a separate database query via get_testruns(upload) to fetch its associated testruns, leading to an N+1 query pattern.
  3. Additionally, handle_pass would call flake.save() individually when a flake reached 30 recent passes, potentially causing many single UPDATE statements.

Solution:

  1. Bulk Fetching Testruns: The get_testruns function has been refactored into get_testruns_for_uploads to fetch all testruns for all uploads related to a commit in a single database query using upload_id__in.
  2. In-Memory Grouping: These testruns are then grouped by upload_id in memory before being passed to process_single_upload.
  3. Bulk Update for Expired Flakes: handle_pass no longer calls flake.save() directly. Instead, expired flakes are collected into a list and a single Flake.objects.bulk_update is performed after all uploads have been processed.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

Fixes WORKER-YNX

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants