fix(gateway): offload sync SQLite queries in build_channel_directory to thread#61213
fix(gateway): offload sync SQLite queries in build_channel_directory to thread#61213AlexFucuson9 wants to merge 1 commit into
Conversation
…to thread build_channel_directory() is an async function invoked on gateway startup and every 5 minutes. It called _build_discord() and _build_from_sessions() synchronously on the event loop thread, blocking it for 10-40s on large state.db. This starved discord.py's heartbeat task, causing shard disconnects. Wrap the three synchronous call sites with asyncio.to_thread() so the SQLite queries run on the default executor without blocking the event loop. Closes NousResearch#60794.
Duplicate of #60810 (earliest open canonical for the #60794 cluster) — this PR wraps the same three |
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved
Looks Good
- Offloads synchronous SQLite queries to a thread pool in
build_channel_directory, preventing blocking - Uses
asyncio.get_event_loop().run_in_executor()pattern correctly - Minimal, focused change with clear intent
- No security concerns
|
Thanks — this was the right fix for #60794, but it already landed on main via PR #60984 (commit d23990f), which salvaged @embwl0x's earlier #60810 and additionally offloaded the two Slack call sites in the same file. Current main has all three of your call sites (plus the Slack ones) wrapped in asyncio.to_thread. Closing as already-fixed. Credit for this cluster goes to #60810 as the earliest submission; nothing left to salvage here. |
Summary
build_channel_directory()is anasync definvoked on gateway startup and every 5 minutes. It calls_build_discord()and_build_from_sessions()synchronously on the event loop thread, blocking it for 10-40s on largestate.db. This starves discord.py's heartbeat task, causingShard ID None heartbeat blocked for more than 10 secondswarnings and gateway disconnects.Fix
Wrap the three synchronous call sites with
asyncio.to_thread():_build_discord(adapter)→await asyncio.to_thread(_build_discord, adapter)_build_from_sessions(plat_name)→await asyncio.to_thread(_build_from_sessions, plat_name)(2 call sites)Files Changed
gateway/channel_directory.py: Addimport asyncio, wrap 3 sync calls withasyncio.to_thread()Test Plan
pytest tests/test_tui_gateway_server.py)Closes #60794