Skip to content

[Claude Code - Stage 4] Normalize Messages session identity for cache-aware routing #159

Description

@franciscojavierarceo

Part of #113 (Claude Code integration). Related to #18 and vllm-project/vllm#48049.

Problem

The Messages path currently forwards Claude Code's x-claude-code-session-id, but agentic-api does not normalize or act on it. Anthropic Messages remains stateless with respect to transcript persistence, but clients can still provide a stable session coordinate for routing, retention, and cache lifecycle policy.

Without one internal contract, agentic-api, llm-d, and vLLM must each understand client-specific headers or lose the session signal.

Proposed contract

  • Claude Code: x-claude-code-session-idsession_id
  • Generic Messages clients: X-Session-IDsession_id
  • Neither header present: leave session_id unset
  • Both headers present with different values: return a clear 4xx error
  • Scope the normalized value to the authenticated tenant/workspace
  • Preserve the original client headers for protocol compatibility and observability
  • Do not derive session identity from cache_control, message content, prompt hashes, or request IDs

Ownership

  • agentic-api normalizes protocol-specific identity.
  • llm-d can use the normalized coordinate for soft affinity, routing, retention, and lifecycle policy.
  • vLLM can receive it as typed request metadata once the upstream contract lands.
  • Exact KV reuse is still determined by rendered token IDs, canonical block keys, and event-confirmed residency. session_id is not KV block identity and does not prove residency.

Messages remains stateless in the transcript sense: this issue does not add message persistence, rehydration, or a Responses-style stored-response object.

Acceptance criteria

  • /v1/messages recognizes x-claude-code-session-id.
  • /v1/messages recognizes provider-neutral X-Session-ID.
  • Matching values normalize to one internal session coordinate.
  • Conflicting values return a deterministic 4xx error.
  • Missing values leave the internal session coordinate unset.
  • The normalized value is tenant/workspace scoped.
  • Original Messages headers and body remain protocol-faithful.
  • The canonical session coordinate can be forwarded to llm-d and bridged to vLLM's typed session_id.
  • Tests cover Claude Code, generic Messages, missing, matching, conflicting, and cross-tenant cases.

Out of scope

  • Persisting Anthropic Messages transcripts
  • Hard replica pinning
  • Treating session identity as exact KV/cache-block identity
  • Implementing llm-d or vLLM lifecycle policy in this repository

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions