[Feature]: 算力亲和设计文档

## 1. 背景描述

### 1.1 问题

多轮 Agent 对话中，上下文引擎（Context Engine）会持续对历史消息做两类收缩操作：

- **压缩（Compress）**：`DialogueCompressor`、`RoundLevelCompressor` 调用 LLM 将多轮历史总结为 memory block，替换原消息。
- **卸载（Offload / Compact）**：`MicroCompactProcessor`、`MessageOffloader` 将冗余 `ToolMessage` 内容替换为 `[Old tool result content cleared]` 占位，保留元数据与 `tool_call_id`。

消息序列变化后，推理引擎（vLLM）已缓存的 KV cache 中，对应 token 范围失效。vLLM 默认 LRU 策略：被释放的 cache block 仍停在 LRU 队列尾部，需等满驱逐才能复用，导致：

- 缓存命中率下降（新轮次命中不到旧 prefix）
- TTFT（Time To First Token）上升（失效部分重新计算）
- 显存碎片化（尾部 block 占位不释放）

### 1.2 目标

上下文变化时，由 Agent 侧主动通知推理引擎："从 `messages_released_index` 起的后缀消息对应 cache 可释放"，让 vLLM 立即将对应 block 移入"可复用优先级段"，跳过 LRU 等待。

### 1.3 关键约束

- **不强依赖推理引擎**：`/release_kv_cache` 是 vLLM + openJiuwen-vllm-affinity 插件提供的可选端点。普通 OpenAI/DashScope 端点不实现，调通后返回 4xx，需容错。
- **不破坏现有上下文处理器**：release 是 `getContextWindow` 末尾的一个旁路调用，不改处理器触发/执行逻辑。

---

## 2. 设计思路

### 2.1 三段链路

```
[Agent 侧]                [Context 侧]              [Inference 侧]

ReActAgent.callModel
  └ buildContextWindowKwargs()
        ↓ model=llm 注入 kwargs
  └ context.getContextWindow(..., kwargs)
        ↓ processors 执行压缩/卸载 → window 变化
SessionModelContext.getContextWindow 末尾
  └ kvCacheManager.release(window, model)
        ↓ diff prev vs curr window → 首个变化 index
KVCacheManager.release
  ├ Path 1: model instanceof InferenceAffinityModel
  └ Path 2: model instanceof Model && supportsKvCacheRelease
        ↓ 委托
Model.release / InferenceAffinityModel.release
  └ client.release(...)
        ↓ HTTP POST
InferenceAffinityModelClient.release
  └ POST {apiBase}/release_kv_cache
        ↓
[server] vllm + jiuwen_vllm_affinity.kv_cache_plugin
  └ api_server.release_kv_cache handler
  └ serving_chat.release_kv_cache → token-level diff
  └ EngineCore.release_kv_cache → block_hashes[release_block_index:]
  └ SingleTypeKVCacheManager.aging_block → 解绑 session
  └ TwoPhaseBlockQueue.aging_block → 移入 release 段
```

### 2.2 关键设计决策

| 决策点 | 选择 | 理由 |
|---|---|---|
| model 注入位置 | `ReActAgent.buildContextWindowKwargs()` | kwargs 是已有通道，无需新签名 |
| model 类型检查 | 双路径 `instanceof InferenceAffinityModel` OR `instanceof Model` | 用 `instanceof` + `supportsKvCacheRelease()` 等价 duck typing |
| release 触发位置 | `SessionModelContext.getContextWindow` 末尾 | 处理器跑完再 release，diff 才准确 |
| 不可逆性 | release 失败容错 catch + warning | HTTP 404/500 不应中断 Agent 循环 |
| 配置开关 | `ContextEngineConfig.enableKvCacheRelease` (默认 false) | opt-in，不影响默认用户 |
| 能力探测 | `Model.supportsKvCacheRelease()` → `BaseModelClient.supportsKvCacheRelease()` | 类型安全版本 duck typing |

### 2.3 与 vllm-affinity 插件的契约

- **请求**：`POST /release_kv_cache`，body 是 `ChatCompletionRequest` 超集：
  ```json
  {
    "model": "GLM-5.2",
    "cache_salt": "session-abc",      // = sessionId
    "cache_sharing": true,
    "messages": [...],                 // 当前完整对话
    "messages_released_index": 3,      // 首个变化消息 index
    "tools": [...],                    // 可选
    "tools_released_index": 1          // 可选
  }
  ```
- **响应（成功）**：`200 {"cache_salt": "...", "block_released": <int>}`
- **响应（错误）**：400/404/500 + `ErrorResponse`
- **服务端语义**：`block_released` = session 绑定被解的 block 数；block 不立即释放，而是从 `normal` 段移入 `release` 段，下次分配优先复用（`free > release > normal`）。

### 2.4 为什么两条路径

- `Model`（factory 路径，ReActAgent 常用）持 `BaseModelClient client`，client 可能是 `InferenceAffinityModelClient`
- `InferenceAffinityModel`（独立类，demo / 直接构造场景）持 `InferenceAffinityModelClient` 直接

二者不继承。用 `instanceof` 双分支等价 duck typing：

```java
if (model instanceof InferenceAffinityModel iam) { ... }       // Path 1: 独立类
else if (model instanceof Model m && m.supportsKvCacheRelease()) { ... }  // Path 2: Model 包装
```

---

## 3. 架构关系图（子模块 + 上下游）

### 3.1 子模块依赖

```
com.openjiuwen.core.singleagent.agents
  ├ ReActAgent                 ← KV 注入入口 (callModel / callModelStream)
  └ ReActAgentEvolve           ← 同步注入 (prepareModelCall)

com.openjiuwen.core.context
  ├ ModelContext (abstract)    ← getContextWindow(..., Map<String,Object> kwargs) 抽象签名
  └ context.SessionModelContext ← release 实际触发点 (line 440-442)
       └ context.KVCacheManager ← diff + 双路径分发

com.openjiuwen.core.context.schema
  └ ContextEngineConfig        ← enableKvCacheRelease 开关

com.openjiuwen.core.foundation.llm
  ├ Model                      ← release 委托 client (Path 2)
  ├ InferenceAffinityModel     ← 独立 release (Path 1, demo 场景)
  └ model_clients
       ├ BaseModelClient       ← supportsKvCacheRelease() / release() 默认 no-op
       └ InferenceAffinityModelClient ← @Override release() POST /release_kv_cache
```

### 3.2 上下游

```
[上游：触发源]
  ReActAgent.invoke → callModel → buildContextWindowKwargs → getContextWindow(kwargs)
  ReActAgentEvolve.invoke → prepareModelCall → buildContextWindowKwargs → getContextWindow(kwargs)

[中游：上下文引擎]
  SessionModelContext.getContextWindow
    ├ processors: MicroCompactProcessor / DialogueCompressor / RoundLevelCompressor
    │   └ 修改 messages (清 ToolMessage content / 替换 memory block)
    └ KVCacheManager.release(window, model)
        ├ checkReleaseNeeded → 首个变化 index
        └ Path 1 / Path 2 分发

[下游：LLM 客户端]
  Model.release / InferenceAffinityModel.release
    └ InferenceAffinityModelClient.release
        └ OkHttp POST → {apiBase}/release_kv_cache

[远端：推理引擎]
  vllm + openJiuwen-vllm-affinity v0.21.0
    └ jiuwen_vllm_affinity.kv_cache_plugin.entrypoints.openai.api_server.release_kv_cache
        └ serving_chat → engine_core → scheduler → kv_cache_manager → aging_block
```

### 3.3 数据流

```
Agent 轮次
   │
   ▼
prevContextWindow (上一轮 snapshot)
   │
   ▼ processors 执行
currContextWindow (本轮 snapshot, messages 已变)
   │
   ▼ KVCacheManager diff
ReleaseCheckResult{shouldRelease, msgIdx, toolIdx}
   │
   ▼ HTTP POST
/release_kv_cache body{cache_salt=sessionId, messages_released_index=msgIdx, ...}
   │
   ▼ 服务端 token-level diff
block_hashes[release_block_index:] → aging_block → 移入 release 段
```

---

## 4. 开发核心代码实现

### 4.1 `BaseModelClient` — 默认 no-op（子类选择性 @Override）

```java
public boolean supportsKvCacheRelease() {
    return false;
}

public boolean release(String sessionId, Object messages, int messagesReleasedIndex,
        Object tools, Integer toolsReleasedIndex, String model) throws Exception {
    return false;
}
```

文件: `src/main/java/com/openjiuwen/core/foundation/llm/model_clients/BaseModelClient.java:513-536`

### 4.2 `Model` — 委托 client

```java
public boolean supportsKvCacheRelease() {
    return client.supportsKvCacheRelease();
}

public boolean release(String sessionId, List<?> messages, int messagesReleasedIndex,
        List<?> tools, Integer toolsReleasedIndex, String model) throws Exception {
    if (!supportsKvCacheRelease()) {
        return false;
    }
    return client.release(sessionId, (Object) messages, messagesReleasedIndex,
            (Object) tools, toolsReleasedIndex, model);
}
```

文件: `src/main/java/com/openjiuwen/core/foundation/llm/Model.java:373-401`

### 4.3 `InferenceAffinityModelClient` — HTTP POST

```java
@Override
public boolean supportsKvCacheRelease() {
    return true;
}

@Override
public boolean release(String sessionId, Object messages, int messagesReleasedIndex,
        Object tools, Integer toolsReleasedIndex, String model) throws Exception {
    Map<String, Object> body = new LinkedHashMap<>();
    body.put("model", resolveModelName(model, null));
    body.put("cache_salt", sessionId);
    body.put("cache_sharing", true);
    body.put("messages", convertMessagesToDict(messages));
    body.put("messages_released_index", messagesReleasedIndex);
    if (tools != null) {
        body.put("tools", convertToolsToDict(tools));
    }
    if (toolsReleasedIndex != null) {
        body.put("tools_released_index", toolsReleasedIndex);
    }
    HttpResponse<String> response = httpClient.send(buildJsonRequest("/release_kv_cache", body, null),
            HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));
    return response.statusCode() >= 200 && response.statusCode() < 300;
}
```

文件: `src/main/java/com/openjiuwen/core/foundation/llm/model_clients/InferenceAffinityModelClient.java:239-258`

### 4.4 `KVCacheManager` — diff + 双路径

```java
public void release(ContextWindow contextWindow, Object model) {
    if (lastContextWindow == null) {
        lastContextWindow = contextWindow;
        return;  // 首轮不 release，仅记录基线
    }

    ReleaseCheckResult result = checkReleaseNeeded(contextWindow);

    if (result.shouldRelease
            && (result.messagesReleasedIndex != null || result.toolsReleasedIndex != null)) {
        Loggers.CONTEXT_ENGINE.info("KV cache release triggered for session " + sessionId
                + " (msg_idx=" + result.messagesReleasedIndex
                + ", tool_idx=" + result.toolsReleasedIndex + ")");

        // Path 1: 独立 InferenceAffinityModel (demo / 直接构造场景)
        if (model instanceof InferenceAffinityModel iam) {
            try {
                iam.release(sessionId, lastContextWindow.getMessages(),
                        result.messagesReleasedIndex != null ? result.messagesReleasedIndex : 0,
                        lastContextWindow.getToolList(), result.toolsReleasedIndex, null);
            } catch (Exception e) {
                Loggers.CONTEXT_ENGINE.warning(
                        "Failed to release inference-affinity KV cache: " + e.getMessage());
            }
        }
        // Path 2: Model 包装 InferenceAffinityModelClient (ReActAgent 常用路径)
        else if (model instanceof Model m && m.supportsKvCacheRelease()) {
            try {
                m.release(sessionId, lastContextWindow.getMessages(),
                        result.messagesReleasedIndex != null ? result.messagesReleasedIndex : 0,
                        lastContextWindow.getToolList(), result.toolsReleasedIndex, null);
            } catch (Exception e) {
                Loggers.CONTEXT_ENGINE.warning(
                        "Failed to release KV cache via Model: " + e.getMessage());
            }
        }
    }

    lastContextWindow = contextWindow;
}
```

文件: `src/main/java/com/openjiuwen/core/context/context/KVCacheManager.java:64-98`

### 4.5 `SessionModelContext` — 自动触发点

```java
// 字段 (line 73):
private final KVCacheManager kvCacheManager;

// 实例化 (line 108, 仅当开关打开):
this.kvCacheManager = config.isEnableKvCacheRelease() ? new KVCacheManager(sessionId) : null;

// getContextWindow 末尾 (line 440-442):
validateAndFixContextWindow(window);
if (kvCacheManager != null) {
    Object model = effectiveKwargs.get("model");
    kvCacheManager.release(window, model);
}
window.setStatistic(statContextWindow(window));
return window;
```

文件: `src/main/java/com/openjiuwen/core/context/context/SessionModelContext.java`

### 4.6 `ReActAgent` — kwargs 注入

```java
// 字段:
private boolean kvReleaseWarningLogged;

// 公共 helper:
private Map<String, Object> buildContextWindowKwargs() {
    Map<String, Object> kwargs = new HashMap<>();
    if (llm == null) {
        return kwargs;
    }
    ContextEngineConfig ceConfig = config.getContextEngineConfig();
    boolean enableKvRelease = ceConfig != null && ceConfig.isEnableKvCacheRelease();
    boolean supportsKvRelease = llm.supportsKvCacheRelease();

    if (enableKvRelease && !supportsKvRelease && !kvReleaseWarningLogged) {
        Loggers.AGENT.warning("ContextEngineConfig.enable_kv_cache_release is True, "
                + "but the current LLM does not support KV cache release; "
                + "KV cache release will not take effect.");
        kvReleaseWarningLogged = true;
    }

    if (enableKvRelease && supportsKvRelease) {
        kwargs.put("model", llm);
    }
    return kwargs;
}

// callModel / callModelStream 改成传 kwargs 第 5 参:
private AssistantMessage callModel(AgentCallbackContext ctx, ModelContext context,
        List<BaseMessage> systemMessages, List<ToolInfo> tools) {
    var contextWindow = context.getContextWindow(systemMessages,
            tools != null ? tools : null, (Integer) null, (Integer) null,
            buildContextWindowKwargs());
    ctx.setInputs(ModelCallInputs.builder()
            .messages(new ArrayList<>(contextWindow.getMessages()))
            .tools(contextWindow.getToolList()).build());
    return railedModelCall(ctx).orElse(null);
}

// config 变更时重置 warning flag:
if (!safeEquals(oldConfig.getContextEngineConfig(), newConfig.getContextEngineConfig())) {
    this.contextEngine = new ContextEngine(newConfig.getContextEngineConfig());
    this.kvReleaseWarningLogged = false;
}
```

文件: `src/main/java/com/openjiuwen/core/singleagent/agents/ReActAgent.java:91, 157, 317-326, 341-361, 1469-1478`

### 4.7 `ReActAgentEvolve` — 独立类同步

`ReActAgentEvolve extends BaseAgent`（不继承 `ReActAgent`），需独立复制 `buildContextWindowKwargs()` + 字段 + 重置 + `prepareModelCall` 改造。代码与 `ReActAgent` 等价。

文件: `src/main/java/com/openjiuwen/core/singleagent/agents/ReActAgentEvolve.java:64, 139, 293-307, 322-342`

---

## 5. 主要接口类关系图

```
                    ┌─────────────────────────────┐
                    │  ReActAgent / ReActAgentEvolve  │
                    │  - llm: Model                  │
                    │  - kvReleaseWarningLogged: bool │
                    │  + buildContextWindowKwargs()  │
                    └──────────────┬───────────────┘
                                   │ kwargs{model=llm}
                                   ▼
                    ┌──────────────────────────────┐
                    │  ModelContext (abstract)      │
                    │  + getContextWindow(          │
                    │      sys, tools, winSize,     │
                    │      dialogueRound, kwargs)   │
                    └──────────────┬───────────────┘
                                   │
                                   ▼
              ┌────────────────────────────────────────┐
              │  SessionModelContext                    │
              │  - kvCacheManager: KVCacheManager       │
              │  + getContextWindow(...)                │
              │    末尾: kvCacheManager.release(win, m) │
              └────────────────────┬───────────────────┘
                                   │
                                   ▼
              ┌──────────────────────────────────────────┐
              │  KVCacheManager                           │
              │  - sessionId: String                      │
              │  - lastContextWindow: ContextWindow       │
              │  + release(window, model)                 │
              │    ├ Path 1: InferenceAffinityModel        │
              │    └ Path 2: Model + supportsKvCacheRelease│
              │  - checkReleaseNeeded → ReleaseCheckResult│
              └──────┬──────────────────────┬────────────┘
                     │ Path 1               │ Path 2
                     ▼                      ▼
    ┌──────────────────────────┐  ┌──────────────────────────┐
    │  InferenceAffinityModel   │  │  Model                    │
    │  - client: IAMClient      │  │  - client: BaseModelClient │
    │  + release(...)          │  │  + release(...)             │
    │  + supportsKvCacheRelease()│  │  + supportsKvCacheRelease() │
    └──────────┬───────────────┘  └──────────┬────────────────┘
               │                              │
               └──────────┬───────────────────┘
                          ▼
            ┌─────────────────────────────────────┐
            │  BaseModelClient (abstract)          │
            │  + supportsKvCacheRelease() default  │
            │  + release(...) default no-op       │
            └──────────────┬──────────────────────┘
                           │ @Override
                           ▼
            ┌─────────────────────────────────────┐
            │  InferenceAffinityModelClient        │
            │  + release(...) → POST /release_kv_cache│
            │  + supportsKvCacheRelease() → true  │
            └─────────────────────────────────────┘
                           │
                           ▼ HTTP POST /release_kv_cache
            ┌──────────────────────────────────────────────┐
            │  vllm + jiuwen_vllm_affinity plugin           │
            │                                              │
            │  ┌────────────────────────────────────────┐  │
            │  │ api_server (FastAPI router)             │  │
            │  │ + release_kv_cache(request, raw_request)│  │
            │  └──────────────┬─────────────────────────┘  │
            │                 │                              │
            │                 ▼                              │
            │  ┌────────────────────────────────────────┐  │
            │  │ OpenAIServingChatEx                    │  │
            │  │ + release_kv_cache(request, raw_request)│  │
            │  │   ├ render full prompt → before_tokens  │  │
            │  │   ├ render partial (messages[:idx])     │  │
            │  │   │     → after_tokens                  │  │
            │  │   ├ token-level diff:                   │  │
            │  │   │   released_token_index              │  │
            │  │   ├ tool_changed? scan for divergence   │  │
            │  │   └ engine_client.release_kv_cache(     │  │
            │  │       cache_salt, request_params)       │  │
            │  └──────────────┬─────────────────────────┘  │
            │                 │                              │
            │                 ▼                              │
            │  ┌────────────────────────────────────────┐  │
            │  │ EngineCoreEx                           │  │
            │  │ + release_kv_cache(cache_salt, params)  │  │
            │  │   ├ unpack sharing_cache_salt from      │  │
            │  │   │   request_id                        │  │
            │  │   ├ resolve session binding id          │  │
            │  │   └ split block_hashes at               │  │
            │  │     release_block_index                 │  │
            │  └──────────────┬─────────────────────────┘  │
            │                 │                              │
            │                 ▼                              │
            │  ┌────────────────────────────────────────┐  │
            │  │ KVCacheManagerEx (v1/core)              │  │
            │  │ + release_kv_cache(session_id, ...)     │  │
            │  │   └ coordinator.release_block(...)     │  │
            │  └──────────────┬─────────────────────────┘  │
            │                 │ fan-out                      │
            │                 ▼                              │
            │  ┌────────────────────────────────────────┐  │
            │  │ KVCacheCoordinator                      │  │
            │  │ + aging_block(block_ids, session_id)    │  │
            │  │   └ per SingleTypeKVCacheManager         │  │
            │  └──────────────┬─────────────────────────┘  │
            │                 │                              │
            │                 ▼                              │
            │  ┌────────────────────────────────────────┐  │
            │  │ SingleTypeKVCacheManager               │  │
            │  │ + aging_block(block_ids, session_id)   │  │
            │  │   ├ KvCacheSessionManager               │  │
            │  │   │   .release_blocks(blocks, session)  │  │
            │  │   │   → aging_blocks list               │  │
            │  │   └ TwoPhaseBlockQueue                  │  │
            │  │       .aging_block(aging_blocks)        │  │
            │  │       → move normal → release 段        │  │
            │  └────────────────────────────────────────┘  │
            │                                              │
            │  block reuse priority: free > release > normal
            └──────────────────────────────────────────────┘
```

### 5.1 类关系总结（agent-core-java 侧）

| 类 | 角色 | release 能力 |
|---|---|---|
| `ReActAgent` | 入口, kwargs 注入 | 不直接 release |
| `ReActAgentEvolve` | 同 ReActAgent, 独立类 | 不直接 release |
| `ModelContext` | 抽象签名 | 不实现 |
| `SessionModelContext` | 触发点 | 委托 KVCacheManager |
| `KVCacheManager` | diff + 分发 | 不发 HTTP, 分发到 model |
| `Model` | Path 2 包装 | 委托 client |
| `InferenceAffinityModel` | Path 1 独立 | 委托 client |
| `BaseModelClient` | 抽象基, 默认 no-op | 默认 false |
| `InferenceAffinityModelClient` | HTTP 叶 | POST `/release_kv_cache` |

### 5.2 类关系总结（vllm affinity 插件侧）

| 插件类 | 角色 | 职责 |
|---|---|---|
| `api_server` (FastAPI router) | HTTP 入口 | `@router.post("/release_kv_cache")` 接请求 |
| `ReleaseKvCacheRequest` / `ReleaseKvCacheResponse` | 协议 schema | 定义 body 字段与响应结构 |
| `OpenAIServingChatEx` | token-level diff | render full/partial prompt, 定位 `released_token_index`, 调 engine client |
| `EngineCoreEx` | block hash 切分 | 解包 `sharing_cache_salt`, 解析 session binding id, 在 `release_block_index` 处切分 `block_hashes` |
| `KVCacheManagerEx` | 协调器入口 | 转发到 `KVCacheCoordinator` |
| `KVCacheCoordinator` | fan-out | `aging_block` 分发到各 `SingleTypeKVCacheManager` |
| `SingleTypeKVCacheManager` | 实际解绑 | 调 `KvCacheSessionManager.release_blocks` + `TwoPhaseBlockQueue.aging_block` |
| `KvCacheSessionManager` | session 绑定解除 | 按 `session_id` 解绑 block, 返回 `aging_blocks` |
| `TwoPhaseBlockQueue` | 段迁移 | `aging_block` 把 block 从 `normal` 段移入 `release` 段 |

---

## 6. 涉及到的对外 API

### 6.1 Java 框架对外 API

#### `ContextEngineConfig`（用户配置入口）

```java
ContextEngineConfig.builder()
    .maxContextMessageNum(200)
    .defaultWindowRoundNum(10)
    .enableKvCacheRelease(true)   // ← 开关
    .build();
```

- 字段: `boolean enableKvCacheRelease`，默认 `false`。
- 文件: `com.openjiuwen.core.context.schema.ContextEngineConfig`

#### `ReActAgentConfig` / `ReActAgentEvolve` 配置

```java
ReActAgentConfig config = ReActAgentConfig.builder()
    .contextEngineConfig(ContextEngineConfig.builder()
        .enableKvCacheRelease(true)
        .build())
    .build();
```

#### `ModelClientConfig`（选 InferenceAffinity provider）

```java
ModelClientConfig.builder()
    .clientProvider("InferenceAffinity")  // 或 "inference_affinity"
    .apiBase("http://vllm-host:8000")     // vllm + affinity 插件端点
    .apiKey("...")
    .build();
```

### 6.2 HTTP API（对外契约）

#### 请求

```
POST {apiBase}/release_kv_cache
Content-Type: application/json

{
  "model": "<model_name>",
  "cache_salt": "<sessionId>",
  "cache_sharing": true,
  "messages": [...],                   // 完整当前对话
  "messages_released_index": <int>,   // 首个变化消息 index
  "tools": [...],                     // 可选, 当前 tools
  "tools_released_index": <int|null>  // 可选, 首个变化 tool index
}
```

#### 响应

| 状态 | Body | 说明 |
|---|---|---|
| 200 | `{"cache_salt": "...", "block_released": <int>}` | 成功, `block_released` = 解绑 block 数 |
| 400 | `ErrorResponse` | 请求格式错误 |
| 404 | `ErrorResponse` 或 HTML | 端点未实现（普通 OpenAI 兼容端点） |
| 500 | `ErrorResponse` | 服务端异常 |

#### Java 调用方容错

`InferenceAffinityModelClient.release` 仅在 HTTP 2xx 返 `true`，否则返 `false`，不抛异常。`KVCacheManager.release` 内 try/catch，失败仅打 `Loggers.CONTEXT_ENGINE.warning`，不中断 Agent 循环。

### 6.3 日志 API（可观测）

| Logger | 级别 | 信息 |
|---|---|---|
| `Loggers.AGENT` | warning | "ContextEngineConfig.enable_kv_cache_release is True, but the current LLM does not support KV cache release; KV cache release will not take effect." |
| `Loggers.CONTEXT_ENGINE` | info | "KV cache release triggered for session {sessionId} (msg_idx={msgIdx}, tool_idx={toolIdx})" |
| `Loggers.CONTEXT_ENGINE` | info | "[RELEASE REASON] Message modified at index {idx}" |
| `Loggers.CONTEXT_ENGINE` | info | "[RELEASE REASON] Tool modified at index {idx}" |
| `Loggers.CONTEXT_ENGINE` | warning | "Failed to release inference-affinity KV cache: {msg}" |
| `Loggers.CONTEXT_ENGINE` | warning | "Failed to release KV cache via Model: {msg}" |

---

## 7. 测试设计与测试计划

### 7.1 测试分层

```
Unit (单测)              Integration (集成)             E2E (端到端)
─────────────            ───────────────────            ─────────────
KVCacheManagerTest       SessionModelContextKVTest       MicroCompactProcessorKvCacheExample
BaseModelClientTest      ReActAgentKVReleaseTest
ModelReleaseTest         ReActAgentEvolveKVReleaseTest
InferenceAffinityModelClientTest (HTTP mock)
```

### 7.2 单元测试

#### 7.2.1 `KVCacheManagerTest`（已存在）

文件: `src/test/java/com/openjiuwen/core/context/context/KVCacheManagerTest.java`

**已有覆盖**:
- 首轮不 release（仅记基线）
- 相同 window 不 release
- 消息变化触发, msgIdx 正确
- tools 变化触发, toolIdx 正确
- 连续 5 次 release
- empty → non-empty 转换

**待补**:
- Path 1: 传 `InferenceAffinityModel` mock → 验证 `iam.release(...)` 被调
- Path 2: 传 `Model` mock (supportsKvCacheRelease=true) → 验证 `m.release(...)` 被调
- model=null → 不调 release
- model 非 InferenceAffinityModel 非 Model (e.g. String) → 不调
- release 抛 Exception → 仅 warning, 不中断

#### 7.2.2 `BaseModelClientTest`（新）

- 默认 `supportsKvCacheRelease()` 返 false
- 默认 `release(...)` 返 false, 不抛
- 子类不 override → 用默认

#### 7.2.3 `ModelReleaseTest`（新）

- `Model(client=mock InferenceAffinityModelClient)` → `supportsKvCacheRelease()` 返 true
- `Model(client=mock OpenAIModelClient)` → 返 false
- `release(...)` 委托 client, 参数透传
- `release(...)` 不支持时返 false, 不调 client

#### 7.2.4 `InferenceAffinityModelClientTest`（新, HTTP mock）

- 用 MockWebServer mock vllm 端点
- 200 + `{"block_released": 3}` → 返 true
- 404 → 返 false, 不抛
- 500 → 返 false, 不抛
- 验证 body 字段: model, cache_salt, cache_sharing, messages, messages_released_index, tools (when not null), tools_released_index (when not null)
- model=null → fallback 到 modelConfig.modelName

### 7.3 集成测试

#### 7.3.1 `SessionModelContextKVTest`（新）

- `enableKvCacheRelease=false` → `kvCacheManager` 字段为 null, 不调
- `enableKvCacheRelease=true` + kwargs 无 model → release 触发 log 但不发 HTTP
- `enableKvCacheRelease=true` + kwargs 有 model (Model mock) → release 链路走 Path 2
- 处理器触发压缩后 → diff 检测到变化 → release 被调

#### 7.3.2 `ReActAgentKVReleaseTest`（新）

- `enableKvCacheRelease=true` + `InferenceAffinity` provider → kwargs 有 `model=llm`
- `enableKvCacheRelease=true` + 普通 OpenAI provider → kwargs 空, 打一次 warning
- `enableKvCacheRelease=false` → kwargs 空
- 第二次同配置 → warning 不再打（`kvReleaseWarningLogged` 守护）
- 配置变更 → flag 重置, warning 再打一次

#### 7.3.3 `ReActAgentEvolveKVReleaseTest`（新）

同 7.3.2, 跑 `ReActAgentEvolve`, 验证独立类逻辑等价。

### 7.4 端到端测试

#### 7.4.1 `MicroCompactProcessorKvCacheExample`（已存在 demo）

文件: `examples/context_evolver/MicroCompactProcessorKvCacheExampleSupport.java`

**当前已验证**:
- 3 轮天气查询触发 `MicroCompactProcessor`
- 第 3 轮清掉北京+上海 2 个旧 ToolMessage, 保留广州
- 手动调 `kvCacheManager.release(window, affinityModel)` → Path 1

**修复后预期（自动链路打通）**:
- Round 2 后: `context_engine` log `KV cache release triggered (msg_idx=0, tool_idx=1)` → `affinityModel.release` (Path 2, Model 委托) → POST `/release_kv_cache` → vllm 端点返 200/404
- Round 3 命中 MicroCompactProcessor: `[RELEASE REASON] Message modified at index 3` → `KV cache release triggered (msg_idx=3, tool_idx=1)` → POST `/release_kv_cache`
- 无 `[kv_cache] release failed`（自动链路容错 catch, demo 手动调仍 catch）

#### 7.4.2 真 vllm 联调测试（system-test 标签）

前置: 部署 vllm 0.21+ + openJiuwen-vllm-affinity v0.21.0 插件。

| 场景 | 操作 | 预期 |
|---|---|---|
| 基线命中率 | 不开 enableKvCacheRelease, 跑 3 轮 | TTFT 上升（cache 失效但 LRU 不清） |
| 开启 release | 开 enableKvCacheRelease, 跑 3 轮 | Round 2/3 触发 POST, TTFT 优于基线 |
| 多 session 隔离 | 两 session 并发跑 | `cache_salt` 区分, 不串释放 |
| 不支持端点 | 指向普通 OpenAI 端点 | 4xx, warning, Agent 不中断 |
| 服务端错误 | vllm 临时 kill | 5xx/连接错, warning, 不中断 |

### 7.5 兼容性测试

跨实现一致性校验, 通过 `*CompatibilityTest.java` 自动跑同等输入, 校验 Java 输出/HTTP body 一致。

| 已有测试场景 | Java 对应 |
|---|---|
| `KVCacheManager` 行为 | `KVCacheManagerTest` (已存在, 需补 Path 1/2) |
| ReActAgent KV 注入 | `ReActAgentKVReleaseTest` (新) |
| 处理器 + release 联动 | `SessionModelContextKVTest` (新) |

### 7.6 测试计划执行命令

```bash
# 单测
mvn test -Dtest=KVCacheManagerTest,BaseModelClientTest,ModelReleaseTest,InferenceAffinityModelClientTest

# 集成
mvn test -Dtest=SessionModelContextKVTest,ReActAgentKVReleaseTest,ReActAgentEvolveKVReleaseTest

# 端到端 demo
java -Dfile.encoding=UTF-8 \
  -cp "target/classes;$(cat target/dialogue_compressor.classpath)" \
  examples.context_evolver.MicroCompactProcessorKvCacheExample

# system-test (真 vllm 联调)
mvn test -Dsurefire.groups=system-test

# 全量回归
mvn test

# 兼容性校验
mvn test -Dtest='*CompatibilityTest'
```

### 7.7 验收标准

- [ ] 所有单测通过, 覆盖 Path 1 / Path 2 / model=null / 异常容错
- [ ] 集成测试验证 kwargs 注入正确, flag 守护生效
- [ ] demo 跑通, log 显示 `KV cache release triggered` + HTTP POST 发出
- [ ] 真 vllm 联调: `block_released > 0`, TTFT 优于基线
- [ ] 兼容性测试通过
- [ ] `mvn compile` + `mvn test` 全绿

---

## 附录 A: 文件清单

| 文件 | 角色 | 行数变化 |
|---|---|---|
| `BaseModelClient.java` | 默认 no-op 方法 | +36 |
| `Model.java` | 委托 client | +39 |

| `InferenceAffinityModelClient.java` | @Override + HTTP POST | +36/-12 |
| `InferenceAffinityModel.java` | 独立 Path 1 | 0 |
| `KVCacheManager.java` | 双路径分发 | +23/-3 |
| `SessionModelContext.java` | 触发点 | 0 |
| `ReActAgent.java` | kwargs 注入 | +48/-3 |
| `ReActAgentEvolve.java` | 同步注入 | +42/-1 |
| `ContextEngineConfig.java` | 开关字段 | 0 |

## 附录 B: vllm affinity 插件端实现参考

| 插件文件 | 职责 |
|---|---|
| `entrypoints/openai/api_server` | `@router.post("/release_kv_cache")` HTTP handler |
| `entrypoints/openai/protocol` | `ReleaseKvCacheRequest` / `ReleaseKvCacheResponse` schema |
| `entrypoints/openai/serving_chat` | `OpenAIServingChatEx.release_kv_cache` token-level diff |
| `v1/engine/core` | `EngineCoreEx.release_kv_cache` block_hashes 切分 |
| `v1/core/kv_cache_manager` | `KVCacheManagerEx.release_kv_cache` → coordinator |
| `v1/core/kv_cache_coordinator` | `aging_block` fan-out |
| `v1/core/single_type_kv_cache_manager` | 实际解绑 + 回收 |
| `v1/core/two_phase_block_queue` | `aging_block` 移入 release 段 |
| `v1/core/kv_cache_session_manager` | `release_blocks` session 绑定解除 |

插件版本: `jiuwen_vllm_affinity 0.1.0`, 依赖 `vllm>=0.21.0`。

插件类	角色	职责
`api_server` (FastAPI router)	HTTP 入口	`@router.post("/release_kv_cache")` 接请求
`ReleaseKvCacheRequest` / `ReleaseKvCacheResponse`	协议 schema	定义 body 字段与响应结构
`OpenAIServingChatEx`	token-level diff	render full/partial prompt, 定位 `released_token_index`, 调 engine client
`EngineCoreEx`	block hash 切分	解包 `sharing_cache_salt`, 解析 session binding id, 在 `release_block_index` 处切分 `block_hashes`
`KVCacheManagerEx`	协调器入口	转发到 `KVCacheCoordinator`
`KVCacheCoordinator`	fan-out	`aging_block` 分发到各 `SingleTypeKVCacheManager`
`SingleTypeKVCacheManager`	实际解绑	调 `KvCacheSessionManager.release_blocks` + `TwoPhaseBlockQueue.aging_block`
`KvCacheSessionManager`	session 绑定解除	按 `session_id` 解绑 block, 返回 `aging_blocks`
`TwoPhaseBlockQueue`	段迁移	`aging_block` 把 block 从 `normal` 段移入 `release` 段

插件文件	职责
`entrypoints/openai/api_server`	`@router.post("/release_kv_cache")` HTTP handler
`entrypoints/openai/protocol`	`ReleaseKvCacheRequest` / `ReleaseKvCacheResponse` schema
`entrypoints/openai/serving_chat`	`OpenAIServingChatEx.release_kv_cache` token-level diff
`v1/engine/core`	`EngineCoreEx.release_kv_cache` block_hashes 切分
`v1/core/kv_cache_manager`	`KVCacheManagerEx.release_kv_cache` → coordinator
`v1/core/kv_cache_coordinator`	`aging_block` fan-out
`v1/core/single_type_kv_cache_manager`	实际解绑 + 回收
`v1/core/two_phase_block_queue`	`aging_block` 移入 release 段
`v1/core/kv_cache_session_manager`	`release_blocks` session 绑定解除

决策点	选择	理由
model 注入位置	`ReActAgent.buildContextWindowKwargs()`	kwargs 是已有通道，无需新签名
model 类型检查	双路径 `instanceof InferenceAffinityModel` OR `instanceof Model`	用 `instanceof` + `supportsKvCacheRelease()` 等价 duck typing
release 触发位置	`SessionModelContext.getContextWindow` 末尾	处理器跑完再 release，diff 才准确
不可逆性	release 失败容错 catch + warning	HTTP 404/500 不应中断 Agent 循环
配置开关	`ContextEngineConfig.enableKvCacheRelease` (默认 false)	opt-in，不影响默认用户
能力探测	`Model.supportsKvCacheRelease()` → `BaseModelClient.supportsKvCacheRelease()`	类型安全版本 duck typing

类	角色	release 能力
`ReActAgent`	入口, kwargs 注入	不直接 release
`ReActAgentEvolve`	同 ReActAgent, 独立类	不直接 release
`ModelContext`	抽象签名	不实现
`SessionModelContext`	触发点	委托 KVCacheManager
`KVCacheManager`	diff + 分发	不发 HTTP, 分发到 model
`Model`	Path 2 包装	委托 client
`InferenceAffinityModel`	Path 1 独立	委托 client
`BaseModelClient`	抽象基, 默认 no-op	默认 false
`InferenceAffinityModelClient`	HTTP 叶	POST `/release_kv_cache`

状态	Body	说明
200	`{"cache_salt": "...", "block_released": <int>}`	成功, `block_released` = 解绑 block 数
400	`ErrorResponse`	请求格式错误
404	`ErrorResponse` 或 HTML	端点未实现（普通 OpenAI 兼容端点）
500	`ErrorResponse`	服务端异常

Logger	级别	信息
`Loggers.AGENT`	warning	"ContextEngineConfig.enable_kv_cache_release is True, but the current LLM does not support KV cache release; KV cache release will not take effect."
`Loggers.CONTEXT_ENGINE`	info	"KV cache release triggered for session {sessionId} (msg_idx={msgIdx}, tool_idx={toolIdx})"
`Loggers.CONTEXT_ENGINE`	info	"[RELEASE REASON] Message modified at index {idx}"
`Loggers.CONTEXT_ENGINE`	info	"[RELEASE REASON] Tool modified at index {idx}"
`Loggers.CONTEXT_ENGINE`	warning	"Failed to release inference-affinity KV cache: {msg}"
`Loggers.CONTEXT_ENGINE`	warning	"Failed to release KV cache via Model: {msg}"

场景	操作	预期
基线命中率	不开 enableKvCacheRelease, 跑 3 轮	TTFT 上升（cache 失效但 LRU 不清）
开启 release	开 enableKvCacheRelease, 跑 3 轮	Round 2/3 触发 POST, TTFT 优于基线
多 session 隔离	两 session 并发跑	`cache_salt` 区分, 不串释放
不支持端点	指向普通 OpenAI 端点	4xx, warning, Agent 不中断
服务端错误	vllm 临时 kill	5xx/连接错, warning, 不中断

已有测试场景	Java 对应
`KVCacheManager` 行为	`KVCacheManagerTest` (已存在, 需补 Path 1/2)
ReActAgent KV 注入	`ReActAgentKVReleaseTest` (新)
处理器 + release 联动	`SessionModelContextKVTest` (新)

文件	角色	行数变化
`BaseModelClient.java`	默认 no-op 方法	+36
`Model.java`	委托 client	+39

Uh oh!

[Feature]: 算力亲和设计文档 #112

Description

1. 背景描述

1.1 问题

1.2 目标

1.3 关键约束

2. 设计思路

2.1 三段链路

2.2 关键设计决策

2.3 与 vllm-affinity 插件的契约

2.4 为什么两条路径

3. 架构关系图（子模块 + 上下游）

3.1 子模块依赖

3.2 上下游

3.3 数据流

4. 开发核心代码实现

4.1 BaseModelClient — 默认 no-op（子类选择性 @OverRide）

4.2 Model — 委托 client

4.3 InferenceAffinityModelClient — HTTP POST

4.4 KVCacheManager — diff + 双路径

4.5 SessionModelContext — 自动触发点

4.6 ReActAgent — kwargs 注入

4.7 ReActAgentEvolve — 独立类同步

5. 主要接口类关系图

5.1 类关系总结（agent-core-java 侧）

5.2 类关系总结（vllm affinity 插件侧）

6. 涉及到的对外 API

6.1 Java 框架对外 API

ContextEngineConfig（用户配置入口）

ReActAgentConfig / ReActAgentEvolve 配置

ModelClientConfig（选 InferenceAffinity provider）

6.2 HTTP API（对外契约）

请求

响应

Java 调用方容错

6.3 日志 API（可观测）

7. 测试设计与测试计划

7.1 测试分层

7.2 单元测试

7.2.1 KVCacheManagerTest（已存在）

7.2.2 BaseModelClientTest（新）

7.2.3 ModelReleaseTest（新）

7.2.4 InferenceAffinityModelClientTest（新, HTTP mock）

7.3 集成测试

7.3.1 SessionModelContextKVTest（新）

7.3.2 ReActAgentKVReleaseTest（新）

7.3.3 ReActAgentEvolveKVReleaseTest（新）

7.4 端到端测试

7.4.1 MicroCompactProcessorKvCacheExample（已存在 demo）

7.4.2 真 vllm 联调测试（system-test 标签）

7.5 兼容性测试

7.6 测试计划执行命令

7.7 验收标准

附录 A: 文件清单

附录 B: vllm affinity 插件端实现参考

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

4.1 `BaseModelClient` — 默认 no-op（子类选择性 @OverRide）

4.2 `Model` — 委托 client

4.3 `InferenceAffinityModelClient` — HTTP POST

4.4 `KVCacheManager` — diff + 双路径

4.5 `SessionModelContext` — 自动触发点

4.6 `ReActAgent` — kwargs 注入

4.7 `ReActAgentEvolve` — 独立类同步

`ContextEngineConfig`（用户配置入口）

`ReActAgentConfig` / `ReActAgentEvolve` 配置

`ModelClientConfig`（选 InferenceAffinity provider）

7.2.1 `KVCacheManagerTest`（已存在）

7.2.2 `BaseModelClientTest`（新）

7.2.3 `ModelReleaseTest`（新）

7.2.4 `InferenceAffinityModelClientTest`（新, HTTP mock）

7.3.1 `SessionModelContextKVTest`（新）

7.3.2 `ReActAgentKVReleaseTest`（新）

7.3.3 `ReActAgentEvolveKVReleaseTest`（新）

7.4.1 `MicroCompactProcessorKvCacheExample`（已存在 demo）