Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
bf88561
docs: draft quality gate design — dual-gate auto-send routing
Jun 10, 2026
dc8f419
feat: add quality gate metrics to evaluation pipeline
Jun 10, 2026
1d0a8d9
feat: add draft quality scoring module
Jun 10, 2026
a7e473d
feat: dual-gate routing — confidence + draft quality
Jun 10, 2026
96115c1
fix: add encoding safety at data entry points (retrieve_evidence, key…
Jun 11, 2026
f101dc8
fix: replace datetime.now() with eval_ticket.submitted_at for reprodu…
Jun 11, 2026
fbcaccb
feat: add exclusion_rule support to optimizer + classifier (fixes fir…
Jun 11, 2026
6b6cdba
test: add encoding safety and exclusion rule tests
Jun 11, 2026
3faff5e
optimizer: 增量评测 — run_partial_evaluation() + _verify_fix() 增量支持
Jun 11, 2026
11df78f
optimizer: 诊断增强 — _analyze_causal_features() 因果分析 + exclusion_rule 关键词优化
Jun 11, 2026
98c1a83
optimizer: 最佳状态追踪 + 提前终止(3轮无改进自动停止)
Jun 11, 2026
e49667a
test: 补充_analyze_causal_features/提前终止/verify_fix增量路径测试
Jun 11, 2026
1b6b07b
feat(guard): add check_safe_escalation_language() and check_manual_re…
Jun 11, 2026
14947a2
feat(guard): wire safe escalation + manual review into check_claim_gu…
Jun 11, 2026
2701b5a
feat(eval): add guard_failure_types to DraftEvaluationRow + per-failu…
Jun 11, 2026
0ec8c67
phase14: complete Guard Architecture — taxonomy, safe language, integ…
Jun 11, 2026
9f067aa
feat: replace n-gram Chinese keyword extraction with jieba
Jun 11, 2026
6bcbbfc
[verified] feat: scoring-based intent classifier
Jun 11, 2026
c3c5f37
[verified] Phase 16: keyword trade-off analysis + LLM approval + eval…
Jun 11, 2026
ec5ac10
docs: 建立项目文档体系 — CHANGELOG + INDEX + MAINTENANCE + AGENTS 维护约定
Jun 11, 2026
fa32304
feat(retrieval): add retrieval sufficiency assessment
Jun 11, 2026
e996ef7
feat(retrieval): rule-based query rewriting (v1 synonym map)
Jun 11, 2026
85d626f
fix: CORS env var, embedding type, OpenAPI tags, pre-commit
Jun 12, 2026
3a7f84c
fix: batch bug fixes, lint cleanup, OpenAPI tags, pre-commit
Jun 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# TicketPilot Environment Configuration
# Copy this file to .env and fill in your values
# Copy this file to .env.local and fill in your values

# =====================
# Database Configuration
Expand All @@ -15,10 +15,15 @@ DB_PASSWORD=your_password_here
# =====================
EMBEDDING_PROVIDER=openai_compatible
EMBEDDING_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
EMBEDDING_API_KEY=your_dashscope_api_key_here
EMBEDDING_API_KEY=your_api_key_here
EMBEDDING_MODEL=text-embedding-v3
EMBEDDING_DIM=1024

# =====================
# CORS Configuration (comma-separated origins)
# =====================
CORS_ORIGINS=http://localhost:3000,http://localhost:5173

# =====================
# LLM Provider (DeepSeek)
# =====================
Expand Down
20 changes: 20 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.11.13
hooks:
Comment on lines +4 to +6
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
args: ['--maxkb=500']
- id: check-merge-conflict
- id: detect-private-key
32 changes: 32 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -308,3 +308,35 @@ Each phase (logical unit from tasks.md) follows a 7-step loop. See `docs/harness
- Loop back: Review/Doc fails → back to Implementation (max 3 retries, then escalate)
- Phase done: All steps pass → commit + push → next phase
- Controller never implements code directly (always delegate to subagent)

## 17. 文档维护约定

> 见 `docs/MAINTENANCE.md` 完整版。以下是本文件的核心摘录。

### 17.1 不更新文档 = 没做完

任何阶段完成后,**必须**更新:
1. `CHANGELOG.md` — `[Unreleased]` 填写变更内容。**未更新 CHANGELOG 的 commit 不能标记 `[verified]`**
2. `docs/INDEX.md` — 如果新增/修改了文件
3. 受影响的技术文档 — 至少加过期标记

### 17.2 更新时机速查

| 时机 | 必须更新 |
|------|---------|
| 阶段 merge 时 | CHANGELOG.md + 受影响的 technical docs |
| 新增文件 | docs/INDEX.md |
| 评测数据/指标变化 | CHANGELOG.md |
| 淘汰旧功能 | 标记 deprecated + 更新 INDEX |

### 17.3 文档清理

| 类型 | 处理 |
|------|------|
| `docs/plans/` | 阶段归档后移入 `archive/` 或删除 |
| `docs/CHECKPOINT_*.md` | 归档到 CHANGELOG 后删除 |
| `docs/technical/` | 永久保留,修改时同步更新 |

### 17.4 一句话

> 每完成一件事,停下来更新一次文档。**不更新等于没做完**。
221 changes: 221 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
# Changelog

> **格式**: 基于 [Keep a Changelog](https://keepachangelog.com/),按阶段组织。
> **维护规则**: 见 `docs/MAINTENANCE.md#changelog`。每个阶段合并前必须更新本文件。
> **版本**: 语义化版本,阶段作为次版本。

---

## [Unreleased]

### Added
- tradeoff.py: 关键词候选混淆簇模拟(TP/FP/Net Gain 量化)
- llm_reviewer.py: OpenAI-compatible LLM 审批替代硬性回滚
- 54 个新测试(tradeoff + llm_reviewer + keyword_search + engine)

### Changed
- engine.py: 集成 tradeoff 分析 + llm_reviewer 到优化器循环
- diagnostics.py: jieba 因果特征分析增强
- evaluator.py: 支持扩展评测数据集 + 混淆分析
- 评测数据扩展: golden_expectations +503/-503

### Docs
- CHANGELOG.md: 正式 changelog,覆盖所有阶段迭代记录
- docs/INDEX.md: 文档索引导航(给 Agent 和人共用)
- docs/MAINTENANCE.md: 文档维护公约("不更新 = 没做完")
- AGENTS.md §17: 文档维护约定嵌入 Agent 工作准则

---

## [0.16.0] — 2026-06-11 — Scoring Classifier + Keyword Trade-off Engine

> **Git:** `6bcbbfc` | **Tests:** 1,574 ✅ / 14 known | **Ruff:** clean

### Added
- `ScoringIntentClassifier`: 替代硬 first-match-wins,per-intent 关键词评分+阈值门控
- jieba FTS 分词 + 42 停用词过滤 + `%` SQL 转义
- 排除规则 (`IntentRule.exclusions`) — 解决 first-match-wins 误分类
- 编码安全防线(3 层 UTF-8 防御)
- 增量评测 (`run_partial_evaluation()` → 6min → 30s)
- 最佳状态追踪 + 3 轮无改进提前终止

### Fixed
- `UnicodeDecodeError` in keyword_search (psycopg3 LIKE)
- `submitted_at` 使用 `datetime.fromisoformat()` 确保可复现

### Metrics
| 指标 | 基线 | 备注 |
|------|------|------|
| Intent accuracy | 69.3% | v1 classifier |
| Severity accuracy | 57.4% | |
| Risk flag F1 | 34.7% | 瓶颈 (22 keywords, 6 flags) |
| Composite | 0.6125 | 加权综合分 |

---

## [0.15.0] — 2026-06-10 — Chat UI + Controller Harness + 跨境电商

> **Git:** `a7e473d` | **Tests:** ~1,700 | **Coverage:** 83%

### Added
- Streamlit Chat 演示 UI(multi-turn 上下文)
- Pipeline-to-chat 适配器(证据映射+上下文助手)
- Controller Harness master skill + OpenSpec 插件
- 风险升级显示、证据面板、复核队列链接
- 跨境电商 DraftAgent + Chunking + 知识库 144→340 chunks
- BM25 tsvector + ts_rank_cd 优化
- Re-ranking 框架(embedding tiebreaker,保留 RRF)
- Self-reflection loop(幻觉检测和修正)
- 置信度路由 — 分级审核
- DashScope text-embedding-v3 接入
- Agent Harness 三阶段:追踪+评估+护栏
- Docker 部署、Multi-Agent 架构
- 草稿质量门禁:`DraftQualityScorer` + 双重路由

### Changed
- 意图分类 80%→100%(强指示词+支付关键词优化)
- README metrics 更新:1,239 unit tests, 83% coverage
- 知识库 1,505 chunks / ~2,360 原始文档

---

## [0.14.0] — 2026-06-09 — Guard Architecture

> **Git:** `0ec8c67`

- `GuardFailureType` taxonomy(3 大类: hallucination, risk, escalation)
- 安全升级检测 + 人工复核确认 + per-failure-type pass rates

---

## [0.13.0] — 2026-06-08 — Extended Eval Metrics

> **Git:** `0e050d6`

- Extended draft evaluation metrics via comparison runner
- Real provider extended comparison
- Guard-aware provider prompting experiment

---

## [0.12.0] — 2026-06-07 — LLM Provider Comparison

> **Git:** `bb88d9a`

- OpenAI-compatible LLM provider for offline comparison
- Fake vs Real provider evaluation
- Agent error memory system

---

## [0.11.0] — 2026-06-06 — Evidence-Grounded LLM Draft

> **Git:** `ac1b01a`

- Draft schema + LLM provider interface + Fake provider
- Evidence-grounded prompt builder
- Citation validation + Unsupported-claim guard
- Offline draft generation metrics

---

## [0.10.0] — 2026-06-05 — Hybrid Retrieval Diagnosis

> **Git:** `199fbf2` | **Eval tickets:** 86 doc-level golden labels

- Retrieval trace readiness audit + P0 ranking diagnosis
- Doc-level golden metrics + real pipeline eval
- AI Development Harness + ChatGPT controller harness

---

## [0.9.0] — 2026-06-04 — Knowledge Coverage Expansion

> **Git:** `b9c2ed8`

- Wrong-case taxonomy + knowledge gap map
- 11 P0 records, evaluation rerun (knowledge coverage impact)
- Real embedding provider identity audit
- 54 修复 skipped integration tests (WSL DLL + dimension)

---

## [0.8.0] — 2026-06-03 — Real Retrieval Upgrade

> **Git:** `facbc1d`

- Embedding provider config + factory
- DashScope text-embedding-v3 (1024-dim)
- Retrieval comparison metrics (real vs fake)

---

## [0.7.0] — 2026-06-02 — Evidence Pack Scale-Up

> **Git:** `ef6a3b0`

- Evaluation dataset + knowledge base expansion
- 7 demo scenarios + limitations doc

---

## [0.6.0] — 2026-06-01 — Agent Kernel Runtime

> **Git:** `b99e5ec`

- Agent schemas + trace events + tool registry
- Deterministic agent planner + loop
- Runtime skill loader + full integration tests

---

## [0.5.0] — 2026-05-30 — Public GitHub Package

> **Git:** `c58c769`

- Public README + demo guide + release checklist + MIT License

---

## [0.4.0] — 2026-05-28 — Evaluation Pipeline

> **Git:** `165a279`

- 101 synthetic eval tickets + metric computation + offline CLI

---

## [0.3.0] — 2026-05-25 — Human Review Console

> **Git:** `def4afa`

- Review schema + JSONL store + Streamlit MVP

---

## [0.2.0] — 2026-05-20 — Evidence Drafting

> **Git:** `afa8885`

- Evidence-grounded drafting + pipeline entrypoint

---

## [0.1.0] — 2026-04-29 — Project Init + Audit

> **Git:** `9738f37`

- Project scaffold + OpenSpec workflow + audit fixes

---

## 关键指标演变

| 日期 | Tests | 知识库 Chunks | 评测工单 | 综合分 |
|------|-------|--------------|---------|--------|
| 2026-04 | ~50 | 0 | 0 | — |
| 2026-05 | ~400 | 0 | 101 | — |
| 2026-06-02 | ~700 | 340 | 101 | — |
| 2026-06-03 | ~800 | 1,505 | 101 | — |
| 2026-06-10 | 1,574 | 1,505 | 101 | 0.6125 |
| 2026-06-11 | 1,628 | 1,505 | ~400 | 0.6255 |
Loading
Loading