Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ config.local.yaml
# Local data
data/
.cache/
.neocode/projects/**/.transcripts/

# Editor/IDE
.idea/
Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,13 @@ go run ./cmd/neocode

- `/provider` — 切换内建模型提供商
- `/model` — 切换模型
- `/compact` — 手动压缩当前会话上下文

### 上下文压缩(Compact)

- 每次向模型发起请求前,runtime 会先执行一次 `micro compact`,对旧且较长的 `tool result` 做轻量占位替换,控制上下文膨胀。
- 你可以随时输入 `/compact` 触发 `manual compact`,把较早历史折叠为结构化 summary,并保留最近若干对话跨度。
- 每次 compact 前都会先写入完整 transcript(`~/.neocode/projects/<project-hash>/.transcripts/`),便于追溯和恢复历史。

## 架构概览

Expand Down
29 changes: 29 additions & 0 deletions docs/guides/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,14 @@ current_model: gpt-4.1
workdir: /Users/username/projects/myproject
shell: bash
max_loops: 8
context:
compact:
micro_enabled: true
tool_result_keep_recent: 3
tool_result_placeholder_min_chars: 100
manual_strategy: keep_recent
manual_keep_recent_spans: 6
max_summary_chars: 1200

tools:
webfetch:
Expand Down Expand Up @@ -88,6 +96,17 @@ tools:
| `tools.webfetch.max_response_bytes` | int | `1048576` (1MB) | WebFetch 工具最大响应字节数 |
| `tools.webfetch.supported_content_types` | []string | `[text/html, text/plain, application/json]` | 支持的内容类型 |

#### 上下文压缩配置(Compact)

| 字段 | 类型 | 默认值 | 说明 |
|------|------|--------|------|
| `context.compact.micro_enabled` | bool | `true` | 是否在每轮 provider 请求前启用 micro compact |
| `context.compact.tool_result_keep_recent` | int | `3` | micro compact 中保留最近 N 条 tool result 原文 |
| `context.compact.tool_result_placeholder_min_chars` | int | `100` | micro compact 触发占位替换的最小字符数阈值 |
| `context.compact.manual_strategy` | string | `keep_recent` | 手动 `/compact` 策略,可选 `keep_recent` / `full_replace` |
| `context.compact.manual_keep_recent_spans` | int | `6` | `keep_recent` 模式下保留最近 N 个 span |
| `context.compact.max_summary_chars` | int | `1200` | 手动 compact 生成 summary 的最大字符数 |

### 配置文件特点

**自动管理**:
Expand Down Expand Up @@ -182,6 +201,16 @@ NeoCode 提供以下 slash 命令用于快速切换配置:
gpt-5.3-codex
```

### /compact - 手动压缩当前会话

```
/compact
```

- 显式触发一次 manual compact。
- 执行顺序为:写入 transcript -> 生成/校验 summary -> 重写会话消息 -> 发送 compact 事件。
- 若压缩前 transcript 写盘失败,compact 会直接报错并终止,不会写坏会话。

## 配置管理

配置管理由 `internal/config` 模块负责:
Expand Down
41 changes: 35 additions & 6 deletions docs/runtime-provider-event-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,51 @@

当前 runtime 对外暴露一组小而稳定的事件:

- `user_message`
- `agent_chunk`
- `agent_done`
- `tool_start`
- `tool_chunk`
- `tool_result`
- `tool_call_thinking`
- `provider_retry`
- `compact_start`
- `compact_done`
- `compact_error`
- `micro_compact_applied`
- `run_canceled`
- `error`

## ReAct 主循环

1. 加载目标会话或创建新会话。
2. 追加最新的用户消息。
3. 读取最新配置快照。
4. 解析当前 provider 配置并构建 provider 实例。
5. 调用 `context.Builder` 生成本轮请求使用的 `system prompt` 和消息上下文。
6. 调用 `Provider.Chat`,并把流式事件桥接给 TUI。
7. 保存 assistant 完整回复。
8. 执行返回的工具调用,并保存每一个工具结果。
9. 如果仍需继续推理,则进入下一轮;否则结束。
4. 每轮请求前执行一次 `micro compact`(轻量压缩旧 tool result)。
5. 解析当前 provider 配置并构建 provider 实例。
6. 调用 `context.Builder` 生成本轮请求使用的 `system prompt` 和消息上下文。
7. 调用 `Provider.Chat`,并把流式事件桥接给 TUI。
8. 保存 assistant 完整回复。
9. 执行返回的工具调用,并保存每一个工具结果。
10. 如果仍需继续推理,则进入下一轮;否则结束。

## 手动 `/compact` 命令链路

1. TUI 识别 `/compact` 并调用 `runtime.Compact(...)`。
2. runtime 发出 `compact_start` 事件。
3. compact 服务先写 transcript(完整原始消息,JSONL)。
4. 按策略(`keep_recent` / `full_replace`)执行手动压缩并生成指标。
5. runtime 回写 session(仅在 `applied=true` 时写回)。
6. 成功时发出 `compact_done`;失败时发出 `compact_error`。

## Compact 结果与可观测字段

- `before_chars`
- `after_chars`
- `saved_ratio`
- `trigger_mode`(`micro` / `manual`)
- `transcript_id`
- `transcript_path`

### Context Builder 输入与职责

Expand Down Expand Up @@ -64,4 +92,5 @@
- 用户消息提交后保存
- assistant 完整回复后保存
- 每个工具结果完成后保存
- 每次 compact 执行前先保存 transcript(完整留痕)
- 避免在高频 UI 刷新路径中做磁盘 I/O
178 changes: 178 additions & 0 deletions internal/config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -829,6 +829,184 @@ func TestConstructorsRejectMissingDependencies(t *testing.T) {
})
}

func TestCompactConfigDefaultsAndRoundTrip(t *testing.T) {
tempDir := t.TempDir()
loader := NewLoader(tempDir, testDefaultConfig())

cfg, err := loader.Load(context.Background())
if err != nil {
t.Fatalf("Load() error = %v", err)
}

compactCfg := cfg.Context.Compact
if !compactCfg.MicroEnabled {
t.Fatalf("expected micro_enabled default true")
}
if compactCfg.ToolResultKeepRecent != DefaultCompactToolResultKeepRecent {
t.Fatalf("expected tool_result_keep_recent=%d, got %d", DefaultCompactToolResultKeepRecent, compactCfg.ToolResultKeepRecent)
}
if compactCfg.ToolResultPlaceholderMinChars != DefaultCompactPlaceholderMinChars {
t.Fatalf("expected tool_result_placeholder_min_chars=%d, got %d", DefaultCompactPlaceholderMinChars, compactCfg.ToolResultPlaceholderMinChars)
}
if compactCfg.ManualStrategy != CompactManualStrategyKeepRecent {
t.Fatalf("expected manual strategy %q, got %q", CompactManualStrategyKeepRecent, compactCfg.ManualStrategy)
}
if compactCfg.ManualKeepRecentSpans != DefaultCompactManualKeepRecentSpans {
t.Fatalf("expected manual_keep_recent_spans=%d, got %d", DefaultCompactManualKeepRecentSpans, compactCfg.ManualKeepRecentSpans)
}
if compactCfg.MaxSummaryChars != DefaultCompactMaxSummaryChars {
t.Fatalf("expected max_summary_chars=%d, got %d", DefaultCompactMaxSummaryChars, compactCfg.MaxSummaryChars)
}

if err := loader.Save(context.Background(), cfg); err != nil {
t.Fatalf("Save() error = %v", err)
}
reloaded, err := loader.Load(context.Background())
if err != nil {
t.Fatalf("Reload() error = %v", err)
}

if reloaded.Context.Compact.ManualStrategy != CompactManualStrategyKeepRecent {
t.Fatalf("expected manual strategy to persist, got %q", reloaded.Context.Compact.ManualStrategy)
}
}

func TestCompactConfigMicroEnabledFalsePersistsAcrossReload(t *testing.T) {
tempDir := t.TempDir()
loader := NewLoader(tempDir, testDefaultConfig())

cfg, err := loader.Load(context.Background())
if err != nil {
t.Fatalf("Load() error = %v", err)
}
cfg.Context.Compact.MicroEnabled = false

if err := loader.Save(context.Background(), cfg); err != nil {
t.Fatalf("Save() error = %v", err)
}
reloaded, err := loader.Load(context.Background())
if err != nil {
t.Fatalf("Reload() error = %v", err)
}
if reloaded.Context.Compact.MicroEnabled {
t.Fatalf("expected micro_enabled to persist as false after reload")
}
}

func TestCompactConfigMissingMicroEnabledFallsBackToDefaultTrue(t *testing.T) {
tempDir := t.TempDir()
loader := NewLoader(tempDir, testDefaultConfig())
if err := os.MkdirAll(loader.BaseDir(), 0o755); err != nil {
t.Fatalf("create base dir: %v", err)
}

legacyConfig := []byte(strings.Join([]string{
"selected_provider: openai",
"current_model: gpt-4.1",
"workdir: " + filepath.ToSlash(tempDir),
"shell: powershell",
"",
}, "\n"))
if err := os.WriteFile(loader.ConfigPath(), legacyConfig, 0o644); err != nil {
t.Fatalf("write legacy config: %v", err)
}

cfg, err := loader.Load(context.Background())
if err != nil {
t.Fatalf("Load() error = %v", err)
}
if !cfg.Context.Compact.MicroEnabled {
t.Fatalf("expected missing micro_enabled to fallback to default true")
}
}

func TestCompactConfigValidateFailures(t *testing.T) {
tests := []struct {
name string
compact CompactConfig
expectErr string
}{
{
name: "invalid keep recent",
compact: CompactConfig{
ToolResultKeepRecent: 0,
ToolResultPlaceholderMinChars: 100,
ManualStrategy: CompactManualStrategyKeepRecent,
ManualKeepRecentSpans: 6,
MaxSummaryChars: 1200,
},
expectErr: "tool_result_keep_recent",
},
{
name: "invalid placeholder min chars",
compact: CompactConfig{
ToolResultKeepRecent: 3,
ToolResultPlaceholderMinChars: 0,
ManualStrategy: CompactManualStrategyKeepRecent,
ManualKeepRecentSpans: 6,
MaxSummaryChars: 1200,
},
expectErr: "tool_result_placeholder_min_chars",
},
{
name: "invalid manual strategy",
compact: CompactConfig{
ToolResultKeepRecent: 3,
ToolResultPlaceholderMinChars: 100,
ManualStrategy: "unsupported",
ManualKeepRecentSpans: 6,
MaxSummaryChars: 1200,
},
expectErr: "manual_strategy",
},
{
name: "invalid manual keep spans",
compact: CompactConfig{
ToolResultKeepRecent: 3,
ToolResultPlaceholderMinChars: 100,
ManualStrategy: CompactManualStrategyKeepRecent,
ManualKeepRecentSpans: 0,
MaxSummaryChars: 1200,
},
expectErr: "manual_keep_recent_spans",
},
{
name: "invalid summary chars",
compact: CompactConfig{
ToolResultKeepRecent: 3,
ToolResultPlaceholderMinChars: 100,
ManualStrategy: CompactManualStrategyKeepRecent,
ManualKeepRecentSpans: 6,
MaxSummaryChars: 0,
},
expectErr: "max_summary_chars",
},
}

for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
err := tt.compact.Validate()
if err == nil || !strings.Contains(err.Error(), tt.expectErr) {
t.Fatalf("expected error containing %q, got %v", tt.expectErr, err)
}
})
}
}

func TestCompactConfigValidateSupportsFullReplace(t *testing.T) {
err := (CompactConfig{
ToolResultKeepRecent: 3,
ToolResultPlaceholderMinChars: 100,
ManualStrategy: CompactManualStrategyFullReplace,
ManualKeepRecentSpans: 6,
MaxSummaryChars: 1200,
}).Validate()
if err != nil {
t.Fatalf("expected full_replace strategy to be valid, got %v", err)
}
}

func restoreEnv(t *testing.T, key string) {
t.Helper()
value, ok := os.LookupEnv(key)
Expand Down
Loading
Loading