Skip to content

Background auto-sync uses a synchronous file walk that can block the event loop #1

Description

@ashishmishra26

The MCP server auto-syncs the index in the background as queries arrive (maybeAutoSync in src/mcp/server.ts). The sync runs syncRepo -> walkRepo, and walkRepo (src/core/walker.ts) uses synchronous fs (readdirSync, statSync, readFileSync). So while a background sync's stat-walk runs, it blocks the Node event loop for the duration of the walk.

Impact

The triggering query is usually unaffected: the sync is deferred with setImmediate, debounced (default CX_SYNC_INTERVAL_SECS=30), and single-flight, so a fast query typically resolves before the walk starts. The residual case is a concurrent or immediately-following query that lands while the walk is running, which can be delayed by up to the walk time. That is a few tens of ms on typical repos but can reach ~1-2s on very large trees.

This does not affect token counts or tool-call counts (the walk is server-side and invisible to the model). It is purely a tail-latency concern on large repos.

Proposed fix

Move the walk and the change-detection reads off the event loop:

  • async fs (fs/promises / opendir) so the walk yields between entries, or
  • run the sync in a worker thread.

Either one removes the event-loop blocking entirely, so a background sync can never delay a concurrent query.

Notes

  • Debounce + deferral + single-flight already bound how often and how much this happens. This is about removing the residual blocking, so it is a v0.2 candidate rather than urgent.
  • For latency benchmarking in the meantime: on a static repo the walk finds no changes (near no-op), and CX_AUTO_SYNC=0 removes even the stat-walk from the query path.

Refs: src/core/walker.ts (walkRepo, readdirSync/statSync), src/mcp/server.ts (maybeAutoSync).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions