Skip to content

fix(search): wait for merge threads before dropping the bulk writer#6731

Open
Azur1337 wants to merge 1 commit into
matrix-org:mainfrom
Azur1337:fix/search-bulk-execute-wait-merges
Open

fix(search): wait for merge threads before dropping the bulk writer#6731
Azur1337 wants to merge 1 commit into
matrix-org:mainfrom
Azur1337:fix/search-bulk-execute-wait-merges

Conversation

@Azur1337

@Azur1337 Azur1337 commented Jul 7, 2026

Copy link
Copy Markdown

RoomIndex::bulk_execute oepns a tantivy IndexWriter, commits, and then lets it drop without calling wait_merging_threads(). The writers background merge threads can outlive it and still be running when the next bulk_execute opens a new writer on the same index, so the two race over the same segments

During bulk indexing (e.g. backfilling a room history) this surfaces as

      tantivy::indexer::segment_manager: couldn't find segment in SegmentManager

warnings and leaves the index more fragmented than necessary

To fix this, we wait for the writers merge threads to finish before it is dropped at the end of bulk_execute. No public api change


  • I've documented the public API changes in the appropriate changelog files (see [Writing changelog entries][pull-request-template-changelog]).
  • This PR was made with the help of AI.

@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 89.91%. Comparing base (7bf142a) to head (fa5f4d5).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
crates/matrix-sdk-search/src/index/mod.rs 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6731      +/-   ##
==========================================
- Coverage   89.92%   89.91%   -0.02%     
==========================================
  Files         398      398              
  Lines      111157   111161       +4     
  Branches   111157   111161       +4     
==========================================
- Hits        99954    99946       -8     
- Misses       7400     7412      +12     
  Partials     3803     3803              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@Azur1337 Azur1337 force-pushed the fix/search-bulk-execute-wait-merges branch from 5bb7102 to 6cb8ec9 Compare July 7, 2026 16:17
@Azur1337 Azur1337 force-pushed the fix/search-bulk-execute-wait-merges branch from 6cb8ec9 to fa5f4d5 Compare July 7, 2026 16:23
@codspeed-hq

codspeed-hq Bot commented Jul 7, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 98.38%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 49 untouched benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation Restore session [memory store] 224.5 ms 113.1 ms +98.38%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing Azur1337:fix/search-bulk-execute-wait-merges (fa5f4d5) with main (7bf142a)

Open in CodSpeed

@Azur1337 Azur1337 marked this pull request as ready for review July 7, 2026 17:19
@Azur1337 Azur1337 requested a review from a team as a code owner July 7, 2026 17:19
@Azur1337 Azur1337 requested review from Hywan and removed request for a team July 7, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant