feat(mcp): MCP doctor — verify every listed MCP server is actually usable#3715
Merged
Conversation
… (cave-0kux)
The well-known servers grid listed 50+ registry entries with no way to tell
which would actually work. Add an MCP doctor:
- src/lib/mcp-doctor.ts: per-entry diagnosis — remote (http/sse) entries get
a real JSON-RPC initialize probe, stdio entries a launcher-on-PATH check,
and ${PLACEHOLDER} requirements surface by name (names only, never values;
never spawns server processes)
- GET /api/mcp/health: runs the doctor over marketplace/exports/mcp/mcp.json
- familiar-tab MCP card: on-demand Check servers action pins honest verdict
pills (ready / needs config / unavailable) on each grid card, detail +
unmet requirement names in the tooltip
- docs/mcp-doctor.md: what each verdict means and how to fix it
Verified live: 53 servers diagnosed in ~1.3s through the running app —
31 ready, 21 needs-config (exact env names reported), 1 unavailable (dnx).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an “MCP doctor” workflow to the Familiar tab so the Well-known servers catalog can be validated against real runtime conditions (remote endpoint reachability + stdio launcher availability) and surface actionable “ready / needs config / unavailable” verdicts without leaking secret values.
Changes:
- Introduces
src/lib/mcp-doctor.ts(+ tests) to diagnose registry entries via injectable endpoint probing and PATH checks, reporting placeholder names only. - Adds
GET /api/mcp/health(wired into api-contracts) to run the doctor over the marketplace MCP registry. - Updates the Familiar tab MCP catalog UI to run an on-demand health check and display verdict pills + tooltips, with supporting CSS + tests and documentation.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/styles/familiar-tab-mcp.css | Adds layout + semantic styling for the “Check servers” row and verdict pills. |
| src/lib/mcp-doctor.ts | Implements the MCP doctor (placeholder extraction, per-entry diagnosis, PATH lookup). |
| src/lib/mcp-doctor.test.ts | Adds behavioral tests ensuring honest verdicts and no secret-value leakage. |
| src/components/familiar-tab-mcp.tsx | Adds on-demand “Check servers” action and renders verdict pills per server. |
| src/components/familiar-tab-mcp.test.ts | Extends static source assertions to cover the on-demand health check UI and pill semantics. |
| src/app/api/mcp/health/route.ts | Adds the /api/mcp/health route to run the doctor over the registry. |
| src/app/api/api-contracts.test.ts | Registers the new /mcp/health route in the contract test. |
| scripts/run-tests.mjs | Wires src/lib/mcp-doctor.test.ts into the test suite list. |
| docs/mcp-doctor.md | Documents verdict meanings and a terminal debugging workflow for the new route. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+124
to
+128
| /** Default deps: real MCP initialize probe + real PATH lookup. */ | ||
| export const systemDoctorDeps: DoctorDeps = { | ||
| probe: checkMcpEndpoint, | ||
| commandExists: systemCommandExists, | ||
| }; |
Comment on lines
+200
to
+218
| const checkServers = useCallback(async () => { | ||
| setChecking(true); | ||
| setHealthError(null); | ||
| try { | ||
| const res = await fetch("/api/mcp/health", { cache: "no-store" }); | ||
| const body = (await res.json()) as McpHealthResponse; | ||
| const next: Record<string, McpServerHealth> = {}; | ||
| for (const server of Array.isArray(body?.servers) ? body.servers : []) next[server.id] = server; | ||
| if (Object.keys(next).length === 0) { | ||
| setHealthError("Health check returned no servers — is the marketplace registry present?"); | ||
| } else { | ||
| setHealth(next); | ||
| } | ||
| } catch { | ||
| setHealthError("Health check failed — the cave server did not respond."); | ||
| } finally { | ||
| setChecking(false); | ||
| } | ||
| }, []); |
…race CI measured 5,655 (Ubuntu) / 5,657 (Windows) with the new health route's mcp-doctor + endpoint-validators chunks; pin 5,667 per the documented ten-file-headroom convention. Byte ceiling unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…t extractor) The budget is deliberately cross-pinned: sidecar-bundle-deps.test.mjs (source regexes), sidecar-runtime-closure.test.mjs, sidecar-runtime-smoke.mjs, and the Windows archive extractor's MAX_FILE_COUNT. All now match the closure budget raised for the /api/mcp/health trace. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The familiar tab's Well-known servers grid lists every entry in the marketplace MCP registry (
marketplace/exports/mcp/mcp.json, 53 servers) with no signal of whether any of them would actually work — remote endpoints were never probed from this surface, stdio launchers never checked, and${PLACEHOLDER}requirements were invisible. Debugging a broken MCP meant guessing.What
src/lib/mcp-doctor.ts— per-entry diagnosis with three honest verdicts:ready— remote (http/sse) endpoint answered a real JSON-RPCinitializeprobe (401/403 counts: remote MCPs auth in-client via OAuth), or the stdio launcher (npx/uvx/docker/…) is installed with nothing left to configureneeds-config— entry references${PLACEHOLDER}values the user must supply; reported by name only, never valuesunavailable— endpoint unreachable / launcher not installedNever spawns server processes; probes are injectable so tests touch neither network nor PATH.
GET /api/mcp/health— runs the doctor over the registry (registered in api-contracts).Familiar tab · MCP card — on-demand Check servers action (never on mount) pins verdict pills on each grid card; tooltip carries detail + unmet requirement names; failures surface as alerts, not silence.
docs/mcp-doctor.md— verdict semantics + terminal debugging recipe.Verification
node --test src/lib/mcp-doctor.test.ts— 11/11 (verdict honesty, placeholder-URL never probed, secret-value leak guard)familiar-tab-mcp.test.ts— 10/10 incl. new pins (GET-only, on-demand, token-toned pills)api-contracts,check-tests-wired,pnpm typecheck,pnpm lint,design-token-drift,pnpm build(bundle budgets ok) — all greencurl /api/mcp/health→ 53 servers in ~1.3s: 31 ready, 21 needs-config (exact env names, e.g.GITHUB_PAT,COVEN_MCP_FILESYSTEM_ROOT), 1 unavailable (dnxnot installed); secret-leak spot check cleanBead: cave-0kux