Skip to content

fix(gateway): offload sync SQLite queries in build_channel_directory to thread#61213

Closed
AlexFucuson9 wants to merge 1 commit into
NousResearch:mainfrom
AlexFucuson9:fix/channel-directory-async
Closed

fix(gateway): offload sync SQLite queries in build_channel_directory to thread#61213
AlexFucuson9 wants to merge 1 commit into
NousResearch:mainfrom
AlexFucuson9:fix/channel-directory-async

Conversation

@AlexFucuson9

Copy link
Copy Markdown
Contributor

Summary

build_channel_directory() is an async def invoked on gateway startup and every 5 minutes. It calls _build_discord() and _build_from_sessions() synchronously on the event loop thread, blocking it for 10-40s on large state.db. This starves discord.py's heartbeat task, causing Shard ID None heartbeat blocked for more than 10 seconds warnings and gateway disconnects.

Fix

Wrap the three synchronous call sites with asyncio.to_thread():

  1. _build_discord(adapter)await asyncio.to_thread(_build_discord, adapter)
  2. _build_from_sessions(plat_name)await asyncio.to_thread(_build_from_sessions, plat_name) (2 call sites)

Files Changed

  • gateway/channel_directory.py: Add import asyncio, wrap 3 sync calls with asyncio.to_thread()

Test Plan

  • All 315 gateway tests pass (pytest tests/test_tui_gateway_server.py)
  • Module imports successfully

Closes #60794

…to thread

build_channel_directory() is an async function invoked on gateway startup
and every 5 minutes. It called _build_discord() and _build_from_sessions()
synchronously on the event loop thread, blocking it for 10-40s on large
state.db. This starved discord.py's heartbeat task, causing shard
disconnects.

Wrap the three synchronous call sites with asyncio.to_thread() so the
SQLite queries run on the default executor without blocking the event loop.

Closes NousResearch#60794.
@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery platform/discord Discord bot adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages P1 High — major feature broken, no workaround duplicate This issue or pull request already exists labels Jul 9, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #60810 (earliest open canonical for the #60794 cluster) — this PR wraps the same three build_channel_directory call sites (_build_discord + 2× _build_from_sessions) in asyncio.to_thread, byte-equivalent to #60810's mechanism but without its regression tests. The maintainer salvage #60984 (covers the two extra Slack sites) is the canonical merge lane.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Looks Good

  • Offloads synchronous SQLite queries to a thread pool in build_channel_directory, preventing blocking
  • Uses asyncio.get_event_loop().run_in_executor() pattern correctly
  • Minimal, focused change with clear intent
  • No security concerns

@teknium1

teknium1 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Thanks — this was the right fix for #60794, but it already landed on main via PR #60984 (commit d23990f), which salvaged @embwl0x's earlier #60810 and additionally offloaded the two Slack call sites in the same file. Current main has all three of your call sites (plus the Slack ones) wrapped in asyncio.to_thread.

Closing as already-fixed. Credit for this cluster goes to #60810 as the earliest submission; nothing left to salvage here.

@teknium1 teknium1 closed this Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery duplicate This issue or pull request already exists P1 High — major feature broken, no workaround platform/discord Discord bot adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

build_channel_directory blocks the event loop with synchronous SQLite queries (Discord heartbeat stalls)

4 participants