Skip to content

fix(lsp): abort drain when a live server never replies#145

Merged
jrollin merged 3 commits into
mainfrom
fix/lsp-mute-server-abort
Jul 6, 2026
Merged

fix(lsp): abort drain when a live server never replies#145
jrollin merged 3 commits into
mainfrom
fix/lsp-mute-server-abort

Conversation

@jrollin

@jrollin jrollin commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Problem

During cartog index's LSP edge-resolution phase, a language server that starts but never answers definition requests (e.g. Volar 3 pointed at a Vue 2 project it can't parse) burned the full 10s batch deadline on every file. On a large repo that is hours of pure timeout — only genuine process death aborted the drain, and a mute-but-alive server stays is_alive() == true.

Observed on a 2,119-file React+Vue monorepo: the Vue drain (805 .vue files) would hang for ~2.2h and had to be killed by hand.

Fix

  • Window-based mute detection. Count consecutive all-timeout request windows (each window shares one 10s batch deadline) across files, and abandon the language after UNRESPONSIVE_WINDOW_LIMIT = 3 (~30s). Counting windows, not files, keeps the bound at ~30s regardless of how unresolved edges distribute (one huge file is many windows; three tiny files are three windows). Logs a WARN and falls back to heuristics for that language, keeping edges already resolved.
  • stopped_early vs server_died. A mute stop is not a death: the answers drained from responsive files are trustworthy, so apply_lang_outcomes still commits their external/unresolvable markings. A process death continues to distrust everything. Reusing server_died for the mute case would have silently discarded a responsive prefix's markings and re-queried them on every future index.
  • Precise timeout classification (client::is_request_timeout, prefix-matched like is_cancelled) so a disconnect or an LSP-error reply is not miscounted as "responsive."
  • Deterministic abortby_file is sorted, so the decision is identical run-to-run (it was a HashMap).

Verification

  • New tests: mute_server_stops_early_after_consecutive_timeout_windows, replying_server_is_never_flagged_unresponsive, stopped_early_still_commits_drained_answers. Test fakes moved to a shared manager::test_support module; scripted_fake_server hardened against shell-quoting.
  • Live on the monorepo: Vue drain aborts in ~30–36s instead of ~2.2h; resolution 29% → 58% (identical js/ts/tsx yields to a manually-rescued run), full index in ~36s.
  • Gates: cartog-lsp 73 green, cartog-mcp 119 green (the any_server_started consumer), clippy -D warnings clean on default and --no-default-features, fmt clean.

Docs

Behavior documented in the three surfaces that own LSP failure modes: docs/troubleshooting.md ("LSP server looks stuck"), docs/explanation/concurrency.md, and docs/explanation/architecture.md. No new command/flag/config-key/MCP-tool/language, so site/cli.md/config.md/README/SKILL unchanged.

Deferred (separate PR)

A fully-wedged server that stops reading stdin can block didOpen's write_all forever (no write deadline). Pre-existing, in the same feature area, but the correct fix is a client write-path redesign (writer thread + bounded channel) that shouldn't ride on this diff.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • LSP-backed resolution now stops retrying after repeated request timeouts and falls back to heuristics, helping runs complete when a language server is unresponsive but still alive.
    • Resolution results are now handled more consistently, with earlier progress preserved even when processing stops partway through.
  • Bug Fixes

    • Improved detection of stalled language servers so they’re no longer treated the same as fully failed ones.
  • Documentation

    • Updated architecture, concurrency, and troubleshooting docs to describe the new timeout behavior and fallback flow.

jrollin added 2 commits July 5, 2026 11:25
A server that starts but never answers definition requests (e.g. Volar 3
on a Vue 2 project) burned the full 10s batch deadline per file — hours on
a large repo — because only process death aborted the drain. Track
consecutive all-timeout request windows (each = one 10s deadline) across
files and abandon the language after three (~30s): warn, fall back to
heuristics, and keep the edges already resolved.

Introduce `stopped_early` distinct from `server_died`: a mute stop keeps
the answers drained from responsive files (external/unresolvable get
committed), whereas a death distrusts them all. Classify a request
timeout precisely (`is_request_timeout`) so a disconnect or LSP-error
reply is not miscounted as responsive, and sort files so the abort
decision is deterministic run-to-run.

Measured on a 2,119-file monorepo with a mute vue server: the vue drain
aborts after ~30s instead of ~2.2h, with identical js/ts/tsx yields.
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@jrollin, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 43 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2ff3fd4e-3fff-453e-bbcd-82fa9a74c657

📥 Commits

Reviewing files that changed from the base of the PR and between e54b2af and a1c5644.

📒 Files selected for processing (3)
  • .cargo/audit.toml
  • AGENTS.md
  • deny.toml
📝 Walkthrough

Walkthrough

Adds a shared timeout-error prefix and is_request_timeout helper in the LSP client, tracks consecutive timeout windows in resolve.rs to detect mute-but-alive servers, adds a stopped_early flag to commit partial results, introduces manager test-support helpers, and updates documentation.

Changes

LSP unresponsive server handling

Layer / File(s) Summary
Timeout error detection in LSP client
crates/cartog-lsp/src/client.rs
Adds REQUEST_TIMEOUT_PREFIX constant and is_request_timeout helper to identify batch-deadline timeouts from error root causes.
Mute-server early-stop logic
crates/cartog-lsp/src/resolve.rs
Adds stopped_early flag to LangOutcomes, tracks consecutive all-timeout windows in drain_language, stops draining early past a threshold while committing already-drained outcomes, and makes file iteration deterministic.
Manager test-support module
crates/cartog-lsp/src/manager.rs
Makes DEFINITION_BATCH_WINDOW crate-visible, and centralizes fake-server and test accessor helpers into a new test_support module.
Resolve tests for stopped_early
crates/cartog-lsp/src/resolve.rs
Updates existing tests and adds new tests validating early-stop behavior, drained-answer commits, and correct handling of replying vs. mute servers.
Documentation updates
docs/explanation/architecture.md, docs/explanation/concurrency.md, docs/troubleshooting.md
Documents abandonment of unresponsive LSP servers after consecutive timeout windows and fallback to heuristics.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Resolver as drain_language
  participant LspClient
  participant Outcomes as LangOutcomes

  Resolver->>LspClient: send definition batch window
  LspClient-->>Resolver: is_request_timeout = true (repeated)
  Resolver->>Resolver: increment consecutive_timeout_windows
  alt threshold reached
    Resolver->>Outcomes: set stopped_early = true
    Resolver->>Outcomes: commit drained outcomes so far
  else replies received
    Resolver->>Outcomes: reset consecutive_timeout_windows
  end
Loading

Related issues: None specified.

Related PRs: None specified.

Suggested labels: lsp, documentation, tests

Suggested reviewers: jrollin

🐰 A server gone mute, no reply in sight,
after three silent windows, we call off the fight,
we keep what we've drained, and switch to our guess,
with docs updated so no one's in distress.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly captures the main fix: aborting LSP drain when a live server stops replying.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/lsp-mute-server-abort

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

RUSTSEC-2026-0194/0195 landed against quick-xml, a transitive dep via
self_update (0.37) and rust-s3/aws-creds (0.38). The fix is quick-xml
>=0.41, but both dependents pin <0.41, so no upgrade or [patch] applies —
CI's audit + deny gates go red on every PR and on main until then.

Ignore the two IDs in deny.toml and .cargo/audit.toml (kept in sync) with
a dated rationale. Exposure is narrow: XML parsed is GitHub release
metadata and S3 responses, not attacker-controlled input. Remove once
self_update/rust-s3 pull quick-xml >=0.41.
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.47368% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.31%. Comparing base (f102d45) to head (a1c5644).

Files with missing lines Patch % Lines
crates/cartog-lsp/src/resolve.rs 86.79% 14 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #145      +/-   ##
==========================================
+ Coverage   88.03%   88.31%   +0.28%     
==========================================
  Files          96       96              
  Lines       32200    32307     +107     
==========================================
+ Hits        28347    28533     +186     
+ Misses       3853     3774      -79     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (3)
crates/cartog-lsp/src/client.rs (1)

257-268: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add #[must_use] and a co-located unit test for is_request_timeout.
is_request_timeout is a pure predicate, so its return value shouldn’t be dropped. Add a direct #[cfg(test)] case in crates/cartog-lsp/src/client.rs; the module’s current tests cover other helpers, but not this one.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/cartog-lsp/src/client.rs` around lines 257 - 268, Add #[must_use] to
the is_request_timeout predicate so callers don’t accidentally ignore its
result, and add a co-located #[cfg(test)] unit test in client.rs that exercises
is_request_timeout directly. Use the existing REQUEST_TIMEOUT_PREFIX and
is_request_timeout symbols to create one case that matches the timeout prefix
and one that does not, keeping the test in the same module as the helper.

Source: Coding guidelines

crates/cartog-lsp/src/manager.rs (1)

640-645: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

insert_client_for_test hardcodes language_id to "python" regardless of the language argument.

Currently every caller happens to use "python", so it's latent, but a future test calling insert_client_for_test("ruby", ...) (or any non-python language) would silently mislabel the stored language_id, potentially causing confusing failures if didOpen/languageId content ever matters for that test.

♻️ Proposed fix
-        pub(crate) fn insert_client_for_test(&mut self, language: &str, client: LspClient) {
-            self.clients
-                .insert(language.to_string(), (client, "python"));
+        pub(crate) fn insert_client_for_test(
+            &mut self,
+            language: &str,
+            language_id: &'static str,
+            client: LspClient,
+        ) {
+            self.clients
+                .insert(language.to_string(), (client, language_id));
         }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/cartog-lsp/src/manager.rs` around lines 640 - 645,
`insert_client_for_test` in `LspManager` hardcodes the stored `language_id` to
`"python"` instead of using the `language` argument, so update that test helper
to persist the provided language value alongside the client. Keep the change
localized to `insert_client_for_test` and ensure the inserted tuple uses the
same language string passed by the caller so future tests with non-Python
languages are stored correctly.
crates/cartog-lsp/src/resolve.rs (1)

296-362: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Core windowing/mute-detection logic is correct.

Traced through all-timeout windows, mixed windows, mid-window server_died, and cross-file counter persistence — the consecutive_timeout_windows bookkeeping, break ordering (server_died checked before the window-limit check), and close_file cleanup on every exit path all look sound. Nice touch keeping drained in_root/pending_external/pending_unresolvable intact on early stop.

One process point: this is squarely the hot path referenced by the existing ~33% faster edge resolution benchmark note in concurrency.md. Per coding guidelines, changes to an indexing/query hot path should add or re-run the matching benchmark — worth capturing updated numbers (or at least a mute-server-specific benchmark) now that the worst case is bounded to UNRESPONSIVE_WINDOW_LIMIT windows instead of the full edge set.

As per coding guidelines, "Performance-critical code must avoid per-row or per-edge queries inside loops, avoid quadratic index/resolve behavior, prefer borrowed data over cloning in hot paths, and add or re-run the matching benchmark when changing an indexing or query hot path."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/cartog-lsp/src/resolve.rs` around lines 296 - 362, This change touches
a hot indexing/query path in resolve.rs, so you should add or re-run the
relevant performance benchmark and record the updated numbers. Use the
definition resolution loop around the windowed timeout handling in resolve.rs as
the target, and verify the existing benchmark tied to the edge-resolution
performance note (or add a mute-server-specific benchmark if needed) after the
UNRESPONSIVE_WINDOW_LIMIT behavior change.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/cartog-lsp/src/client.rs`:
- Around line 257-268: Add #[must_use] to the is_request_timeout predicate so
callers don’t accidentally ignore its result, and add a co-located #[cfg(test)]
unit test in client.rs that exercises is_request_timeout directly. Use the
existing REQUEST_TIMEOUT_PREFIX and is_request_timeout symbols to create one
case that matches the timeout prefix and one that does not, keeping the test in
the same module as the helper.

In `@crates/cartog-lsp/src/manager.rs`:
- Around line 640-645: `insert_client_for_test` in `LspManager` hardcodes the
stored `language_id` to `"python"` instead of using the `language` argument, so
update that test helper to persist the provided language value alongside the
client. Keep the change localized to `insert_client_for_test` and ensure the
inserted tuple uses the same language string passed by the caller so future
tests with non-Python languages are stored correctly.

In `@crates/cartog-lsp/src/resolve.rs`:
- Around line 296-362: This change touches a hot indexing/query path in
resolve.rs, so you should add or re-run the relevant performance benchmark and
record the updated numbers. Use the definition resolution loop around the
windowed timeout handling in resolve.rs as the target, and verify the existing
benchmark tied to the edge-resolution performance note (or add a
mute-server-specific benchmark if needed) after the UNRESPONSIVE_WINDOW_LIMIT
behavior change.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d0376ee2-9baf-491a-99ce-2e52dad8b933

📥 Commits

Reviewing files that changed from the base of the PR and between f102d45 and e54b2af.

📒 Files selected for processing (7)
  • .gitignore
  • crates/cartog-lsp/src/client.rs
  • crates/cartog-lsp/src/manager.rs
  • crates/cartog-lsp/src/resolve.rs
  • docs/explanation/architecture.md
  • docs/explanation/concurrency.md
  • docs/troubleshooting.md

@jrollin jrollin merged commit ce3ff3f into main Jul 6, 2026
14 checks passed
@jrollin jrollin deleted the fix/lsp-mute-server-abort branch July 6, 2026 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant