New conversation creation blocks/fails while an agent in the same workspace is responding

[inercia]

Summary

Creating a NEW conversation in a workspace frequently fails (or the "New
Conversation" button appears dead) while an existing agent in that same
workspace is busy responding. Confirmed root cause: agent-side serial request
handling on the shared ACP process, combined with a 25s creation-RPC timeout and
silent frontend backoff.

Only affects new conversations in the same workspace (same shared ACP
process) as a busy agent. A new conversation in a different workspace uses a
different process and is unaffected.

Root cause chain

1. Sessions in the same workspace share ONE ACP agent process, keyed by
   (workingDir, acpServer) — getSharedProcess() in
   internal/web/session_manager.go:1104.
2. Creating a conversation issues a synchronous session/new RPC over that
   shared stdio connection — startSharedACPSession() at
   internal/web/background_session.go:2599
   (sharedProcess.NewSession(rpcCtx, ...)).
3. The vendored github.com/coder/acp-go-sdk is concurrent-capable (unique
   JSON-RPC IDs, pending map, writeMu held only for the wire write), so the
   SDK is NOT the bottleneck.
4. The agent subprocess (Claude Code / Auggie) services requests serially: it
   does not answer session/new until the in-flight session/prompt turn
   finishes.
5. The creation RPC is capped at sessionCreationRPCTimeout = 25s
   (internal/web/background_session.go:1306, applied in creationRPCCtx() at
   :2567), kept under the 30s HTTP request-timeout middleware on purpose.
6. On timeout the handler returns 503 session_creation_timeout /
   "Agent is busy — please try again in a moment" —
   internal/web/session_api.go:136-142.
7. The frontend then enters silent exponential backoff: createNewSession() in
   web/static/hooks/useWebSocket.js gates on _sessionCreationNextAllowedMs
   (:4485) and bumps _sessionCreationFailureCount /
   calculateSessionCreationDelay() on failure (:4515-4519). Subsequent clicks
   return instantly with no network request, so the button looks broken. The
   button itself (web/static/app.js:3139) is never disabled and shows no
   busy/spinner state.

Verification performed

• Read backend creation path end-to-end (handler → SessionManager → shared
  process → ACP RPC) and confirmed locks (sm.mu, SharedACPProcess.mu,
  MultiplexClient.mu) are short-lived / read-only — NOT a global-lock issue.
• Inspected the vendored acp-go-sdk Connection: confirmed concurrent-capable
  (per-ID pending channels, brief writeMu). Verdict: blocking is agent-side.

Fix options (not yet implemented)

• A — Lazy ACP session (best UX): create the Mitto conversation + metadata
  immediately and defer the session/new RPC until the shared process is idle
  or until the new conversation's first prompt. Conversation appears instantly.
  Largest change; touches creation flow + lazy ACP handshake on first prompt.
• B — Dedicated-process fallback when busy: if the workspace's shared
  process has in-flight RPCs (see SharedACPProcess.ActiveRPCs() /
  IsPrompting), create the new conversation on its own dedicated ACP process
  instead of blocking. Simpler/localized; costs extra memory for that session
  and forgoes shared-process benefits for it.
• C — UX feedback only: keep current blocking behavior but make the button
  reflect "agent busy" (disable/spinner), show a clearer toast, and auto-retry
  when the agent frees up instead of silently backing off. No architectural
  change; does not remove the underlying wait.

Key code references

• internal/web/session_manager.go:1104 — getSharedProcess (process keyed by workspace)
• internal/web/session_manager.go:1439 — CreateSessionWithWorkspace
• internal/web/background_session.go:2599 — blocking sharedProcess.NewSession RPC
• internal/web/background_session.go:1306 / :2567 — sessionCreationRPCTimeout (25s) and creationRPCCtx()
• internal/web/shared_acp_process.go:622 — SharedACPProcess.NewSession
• internal/web/session_api.go:136-142 — 503 "Agent is busy" mapping
• web/static/hooks/useWebSocket.js:4485-4519 — frontend silent backoff
• web/static/app.js:3139 — New Conversation button (never disabled)