From 5b1c3a1f67a2b3a7e6dc617064c50f2f9818d7a5 Mon Sep 17 00:00:00 2001 From: JunkaiWang-TheoPhy <1181100960@qq.com> Date: Thu, 2 Apr 2026 20:55:38 +0800 Subject: [PATCH 1/3] feat: add diversity skills and improve onboarding/status guidance --- README.en.md | 5 + README.md | 14 ++ openclaw.plugin.json | 3 + skills/algorithm-selection/SKILL.md | 103 ++++++++++ .../references/candidate-template.md | 13 ++ .../references/selection-template.md | 39 ++++ skills/baseline-runner/SKILL.md | 103 ++++++++++ .../references/baseline-matrix-template.md | 9 + .../references/baseline-report-template.md | 25 +++ skills/dataset-validate/SKILL.md | 104 ++++++++++ .../references/data-validation-template.md | 38 ++++ src/commands.ts | 180 +++++++++++++++++- src/templates/bootstrap.ts | 34 +++- 13 files changed, 668 insertions(+), 2 deletions(-) create mode 100644 skills/algorithm-selection/SKILL.md create mode 100644 skills/algorithm-selection/references/candidate-template.md create mode 100644 skills/algorithm-selection/references/selection-template.md create mode 100644 skills/baseline-runner/SKILL.md create mode 100644 skills/baseline-runner/references/baseline-matrix-template.md create mode 100644 skills/baseline-runner/references/baseline-report-template.md create mode 100644 skills/dataset-validate/SKILL.md create mode 100644 skills/dataset-validate/references/data-validation-template.md diff --git a/README.en.md b/README.en.md index cf163fd..e75c804 100644 --- a/README.en.md +++ b/README.en.md @@ -226,6 +226,8 @@ Or invoke a specific skill directly with a slash command: /research-pipeline /research-collect /idea-generation +/algorithm-selection +/dataset-validate ``` ### 3. Monitor sub-agent progress @@ -262,7 +264,10 @@ Check status anytime: | **research-pipeline** | `/research-pipeline` | Orchestrator. Spawns sub-agents for each phase, verifies outputs between steps. | | **research-collect** | `/research-collect` | Search arXiv → filter → download .tex sources → cluster → generate survey report. | | **research-survey** | `/research-survey` | Deep analysis of papers: extract formulas, map to code, produce method comparison table. | +| **algorithm-selection** | `/algorithm-selection` | Choose between 2-3 plausible ML routes, record rejected routes, and keep a fallback. | | **research-plan** | `/research-plan` | Create 4-part implementation plan (Dataset/Model/Training/Testing) from survey results. | +| **dataset-validate** | `/dataset-validate` | Audit data reality, splits, labels, and leakage risk before model review. | +| **baseline-runner** | `/baseline-runner` | Run or audit strong baselines under the same protocol before headline comparisons. | | **research-implement** | `/research-implement` | Implement ML code from plan, run 2-epoch validation with `uv` venv isolation. | | **research-review** | `/research-review` | Review implementation. Iterates fix → rerun → review up to 3 times. | | **research-experiment** | `/research-experiment` | Full training + ablation experiments. Requires review PASS. | diff --git a/README.md b/README.md index 0c848fe..957bf2d 100644 --- a/README.md +++ b/README.md @@ -226,8 +226,22 @@ openclaw gateway /research-pipeline /research-collect /idea-generation +/algorithm-selection +/dataset-validate ``` +## 机器学习中段任务的新增技能 + +- `/algorithm-selection` + - 用在 `/research-survey` 之后、`/research-plan` 之前 + - 作用:把 2-3 条候选路线写清楚,明确 `Chosen Route / Rejected Routes / Fallback Route` +- `/dataset-validate` + - 用在 `plan_res.md` 已经存在、准备实现或审查模型之前 + - 作用:单独审数据真实性、split、label、leakage 和 mock 风险,把数据质量和模型质量分开 +- `/baseline-runner` + - 用在 `plan_res.md` 已经存在、需要真实 baseline 对比时 + - 作用:统一 baseline、协议、指标和结果记录,产出 `baseline_res.md` + ### 3. 监控子 agent 进度 编排器 spawn 子 agent 后,你会看到: diff --git a/openclaw.plugin.json b/openclaw.plugin.json index b6555ce..38ff131 100644 --- a/openclaw.plugin.json +++ b/openclaw.plugin.json @@ -8,6 +8,9 @@ "properties": {} }, "skills": [ + "skills/algorithm-selection", + "skills/baseline-runner", + "skills/dataset-validate", "skills/idea-generation", "skills/research-pipeline", "skills/research-survey", diff --git a/skills/algorithm-selection/SKILL.md b/skills/algorithm-selection/SKILL.md new file mode 100644 index 0000000..7cd9531 --- /dev/null +++ b/skills/algorithm-selection/SKILL.md @@ -0,0 +1,103 @@ +--- +name: algorithm-selection +description: "Use this when the user needs to choose between multiple ML routes after survey but before committing to implementation. Compares candidate approaches, selects one, records rejected routes, and keeps a fallback." +metadata: + { + "openclaw": + { + "emoji": "🧭", + }, + } +--- + +# Algorithm Selection + +**Don't ask permission. Just do it.** + +Use this skill after `/research-survey` when there are several plausible ML approaches and the project needs a deliberate route choice instead of jumping straight into implementation. + +Outputs go to the workspace root. + +## Use This When + +- `survey_res.md` already exists +- there are at least 2 plausible methods or model families +- the user wants a chosen route plus backups + +## Do Not Use This When + +- the project has no survey yet +- the team already decided the model route and only needs implementation details + +## Required Inputs + +- `SOUL.md` +- `survey_res.md` +- `knowledge/paper_*.md` when available + +If `survey_res.md` is missing, stop and say: `需要先运行 /research-survey 完成深度分析`. + +## Required Output + +- `selection_res.md` + +## Workflow + +### Step 1: Read the Current Project Direction + +Read: + +- `SOUL.md` +- `survey_res.md` +- relevant `knowledge/paper_*.md` + +Extract: + +- the task and evaluation target +- method families mentioned in survey +- constraints such as compute, data, latency, interpretability, or deployment needs + +### Step 2: Build 2-3 Candidate Routes + +Create 2-3 realistic candidate routes only. For each route, record: + +- route name +- core idea +- supporting papers +- expected strengths +- expected risks +- implementation cost +- baseline compatibility + +Use `references/candidate-template.md`. + +### Step 3: Select One Route and Keep Backups + +Choose: + +- one `Chosen Route` +- one or more `Rejected Routes` +- one `Fallback Route` + +The fallback should be the route most likely to work if the chosen route underperforms or proves too expensive to implement. + +### Step 4: Write `selection_res.md` + +Use `references/selection-template.md`. + +The final output must include: + +- project goal +- decision criteria +- candidate options table +- chosen route +- rejected routes +- fallback route +- next recommended command + +## Rules + +1. Do not present only one route unless the survey truly leaves no meaningful alternative. +2. Every route must cite at least one paper or survey-derived basis. +3. The chosen route must match the project constraints in `SOUL.md`. +4. The fallback route must be different from the chosen route. diff --git a/skills/algorithm-selection/references/candidate-template.md b/skills/algorithm-selection/references/candidate-template.md new file mode 100644 index 0000000..05ab1bd --- /dev/null +++ b/skills/algorithm-selection/references/candidate-template.md @@ -0,0 +1,13 @@ +# Candidate Route Template + +Use one block per route. + +```markdown +## Route A: {name} +- Core idea: +- Based on: +- Expected strengths: +- Expected risks: +- Implementation cost: low / medium / high +- Baseline compatibility: +``` diff --git a/skills/algorithm-selection/references/selection-template.md b/skills/algorithm-selection/references/selection-template.md new file mode 100644 index 0000000..c08e4a7 --- /dev/null +++ b/skills/algorithm-selection/references/selection-template.md @@ -0,0 +1,39 @@ +# Selection Result Template + +```markdown +# Algorithm Selection + +## Project Goal +- task: +- main metric: +- key constraints: + +## Decision Criteria +- criterion 1: +- criterion 2: +- criterion 3: + +## Candidate Options + +| Route | Core idea | Strengths | Risks | Cost | Basis | +|-------|-----------|-----------|-------|------|-------| +| A | ... | ... | ... | ... | ... | +| B | ... | ... | ... | ... | ... | + +## Chosen Route +- route: +- why this route: +- what to implement first: + +## Rejected Routes +- route: + - why not now: + +## Fallback Route +- route: +- when to switch: + +## Next Step +- recommended command: +- expected output: +``` diff --git a/skills/baseline-runner/SKILL.md b/skills/baseline-runner/SKILL.md new file mode 100644 index 0000000..e18af44 --- /dev/null +++ b/skills/baseline-runner/SKILL.md @@ -0,0 +1,103 @@ +--- +name: baseline-runner +description: "Use this when the project needs real baseline results before or alongside the main model. Runs classical or literature-aligned baselines under the same protocol and writes a reproducible baseline summary." +metadata: + { + "openclaw": + { + "emoji": "📏", + "requires": { "bins": ["python3", "uv"] }, + }, + } +--- + +# Baseline Runner + +**Don't ask permission. Just do it.** + +Use this skill when the project needs trustworthy baseline numbers instead of only evaluating the proposed model in isolation. + +Outputs go to the workspace root. + +## Use This When + +- `plan_res.md` already names baselines +- `project/` already exists or a baseline implementation path is known +- the experiment stage needs matched comparison numbers + +## Do Not Use This When + +- the project has not finished survey or planning +- no baseline method has been identified yet + +## Required Inputs + +- `plan_res.md` +- `survey_res.md` +- `project/` when the current project already has runnable code + +If `plan_res.md` is missing, stop and say: `需要先运行 /research-plan 完成实现计划`. + +## Required Outputs + +- `baseline_res.md` +- `experiments/baselines/` when runnable artifacts are created + +## Workflow + +### Step 1: Read the Evaluation Contract + +Read: + +- `plan_res.md` +- `survey_res.md` +- current `experiment_res.md` if it exists + +Extract: + +- baseline names +- evaluation metric +- protocol or guardrail +- dataset or workload assumptions + +### Step 2: Define the Baseline Matrix + +Create a small comparison matrix with: + +- baseline name +- source or basis +- expected setup +- metric +- status: `ready`, `needs adaptation`, or `missing` + +Use `references/baseline-matrix-template.md`. + +### Step 3: Run or Approximate Baselines Conservatively + +For each baseline: + +- if code is runnable under the current workspace, run it +- if only a lightweight adaptation is needed, implement the minimal adapter +- if a baseline cannot be run honestly, mark it as unavailable instead of inventing numbers + +All numeric results must come from actual execution logs or explicit imported evidence. + +### Step 4: Write `baseline_res.md` + +Use `references/baseline-report-template.md`. + +The report must include: + +- which baselines were attempted +- which ones ran successfully +- the exact metric values +- the evaluation protocol +- missing or partial baselines +- the most comparable baseline for the current project + +## Rules + +1. Never fabricate baseline numbers. +2. Keep the protocol aligned with the main experiment whenever possible. +3. If a baseline is only partly comparable, say so explicitly. +4. Prefer 2-3 strong baselines over a long weak list. diff --git a/skills/baseline-runner/references/baseline-matrix-template.md b/skills/baseline-runner/references/baseline-matrix-template.md new file mode 100644 index 0000000..736ecc3 --- /dev/null +++ b/skills/baseline-runner/references/baseline-matrix-template.md @@ -0,0 +1,9 @@ +# Baseline Matrix Template + +```markdown +# Baseline Matrix + +| Baseline | Source | Metric | Protocol | Status | Notes | +|----------|--------|--------|----------|--------|-------| +| {name} | {paper/repo} | {metric} | {protocol} | ready / needs adaptation / missing | {note} | +``` diff --git a/skills/baseline-runner/references/baseline-report-template.md b/skills/baseline-runner/references/baseline-report-template.md new file mode 100644 index 0000000..8b5dd84 --- /dev/null +++ b/skills/baseline-runner/references/baseline-report-template.md @@ -0,0 +1,25 @@ +# Baseline Report Template + +```markdown +# Baseline Results + +## Evaluation Contract +- dataset or workload: +- metric: +- guardrail or protocol: + +## Baselines Attempted + +| Baseline | Status | Result | Evidence Source | Notes | +|----------|--------|--------|-----------------|-------| +| {name} | ran / partial / missing | {value or N/A} | {log or file} | {notes} | + +## Most Comparable Baseline +- baseline: +- why this is the main comparison: + +## Gaps +- baseline not run: +- reason: +- how to close the gap: +``` diff --git a/skills/dataset-validate/SKILL.md b/skills/dataset-validate/SKILL.md new file mode 100644 index 0000000..716b737 --- /dev/null +++ b/skills/dataset-validate/SKILL.md @@ -0,0 +1,104 @@ +--- +name: dataset-validate +description: "Use this when the project needs a dedicated data-quality review before model review. Checks data reality, split correctness, label health, leakage risk, shape consistency, and mock-data disclosure." +metadata: + { + "openclaw": + { + "emoji": "🗂️", + "requires": { "bins": ["python3", "uv"] }, + }, + } +--- + +# Dataset Validate + +**Don't ask permission. Just do it.** + +Use this skill before or alongside model implementation review when data quality needs to be checked separately from model quality. + +Outputs go to the workspace root. + +## Use This When + +- `plan_res.md` already exists +- the project is about to implement or has just implemented a model +- data quality, split quality, or label integrity is still uncertain + +## Do Not Use This When + +- the project has no concrete plan yet +- there is no dataset or data-loading path to inspect + +## Required Inputs + +- `plan_res.md` +- `project/` if a data pipeline already exists +- `survey_res.md` when it defines dataset or protocol expectations + +If `plan_res.md` is missing, stop and say: `需要先运行 /research-plan 完成实现计划`. + +## Required Output + +- `data_validation.md` + +## Workflow + +### Step 1: Read the Data Contract + +Read: + +- `plan_res.md` +- `survey_res.md` if present +- current data-loading code under `project/data/` if present + +Extract: + +- expected dataset name +- source +- split structure +- label or target format +- expected shapes + +### Step 2: Audit Data Reality + +Check: + +- whether dataset files actually exist +- whether the data is real or mock +- whether mock usage is clearly declared +- whether row count / sample count is plausible + +### Step 3: Audit Data Integrity + +Check: + +- train / val / test split existence and separation +- label distribution or target sanity +- shape / dtype consistency +- obvious leakage risks +- preprocessing consistency with `plan_res.md` + +If code exists, run lightweight inspection commands under the project environment to verify counts and sample structure. + +### Step 4: Write `data_validation.md` + +Use `references/data-validation-template.md`. + +The report must include: + +- dataset identity +- data reality check +- split integrity +- label / target health +- leakage risk +- mock-data disclosure +- verdict: `PASS`, `NEEDS_REVISION`, or `BLOCKED` +- exact next step + +## Rules + +1. Keep data quality separate from model quality. +2. Never infer that data is real if the files or loading path are missing. +3. If mock data is used, call it out explicitly. +4. If data leakage is plausible, treat it as blocking until clarified. diff --git a/skills/dataset-validate/references/data-validation-template.md b/skills/dataset-validate/references/data-validation-template.md new file mode 100644 index 0000000..ae64151 --- /dev/null +++ b/skills/dataset-validate/references/data-validation-template.md @@ -0,0 +1,38 @@ +# Data Validation Template + +```markdown +# Data Validation + +## Dataset Identity +- dataset: +- source: +- expected split: + +## Reality Check +- files present: +- real or mock: +- evidence: + +## Split Integrity +- train split: +- val split: +- test split: +- leakage risk: + +## Label / Target Health +- label format: +- distribution or range: +- obvious anomalies: + +## Preprocessing Check +- expected preprocessing: +- observed preprocessing: +- mismatch: + +## Verdict +- PASS / NEEDS_REVISION / BLOCKED + +## Next Step +- recommended command: +- reason: +``` diff --git a/src/commands.ts b/src/commands.ts index 8c34ef8..77085d7 100644 --- a/src/commands.ts +++ b/src/commands.ts @@ -10,6 +10,25 @@ interface ResearchAgent { workspace: string; } +interface ProjectSnapshot { + hasConfig: boolean; + hasSurvey: boolean; + hasSelection: boolean; + hasPlan: boolean; + hasDataValidation: boolean; + hasBaseline: boolean; + hasImplementationReport: boolean; + latestReviewVerdict: "PASS" | "NEEDS_REVISION" | "NEEDS_ALGORITHM_REVIEW" | "BLOCKED" | "MISSING" | "UNKNOWN"; + hasExperiment: boolean; +} + +interface NextActionState { + stage: string; + command: string; + expectedOutputs: string[]; + reason: string; +} + /** * List all research agents from openclaw.json. */ @@ -38,6 +57,158 @@ function countFiles(dirPath: string, filter?: (name: string) => boolean): number } } +function fileExists(filePath: string): boolean { + try { + return fs.existsSync(filePath); + } catch { + return false; + } +} + +function readLatestReviewVerdict(workspace: string): ProjectSnapshot["latestReviewVerdict"] { + const iterationsDir = path.join(workspace, "iterations"); + if (!fileExists(iterationsDir)) return "MISSING"; + + try { + const files = fs.readdirSync(iterationsDir) + .filter((f) => /^judge_v\d+\.md$/.test(f)) + .sort((a, b) => { + const na = Number(a.match(/\d+/)?.[0] ?? "0"); + const nb = Number(b.match(/\d+/)?.[0] ?? "0"); + return nb - na; + }); + + const latest = files[0]; + if (!latest) return "MISSING"; + + const content = fs.readFileSync(path.join(iterationsDir, latest), "utf-8"); + const verdictMatch = content.match(/##\s+Verdict:\s+([A-Z_]+)/); + const verdict = verdictMatch?.[1] as ProjectSnapshot["latestReviewVerdict"] | undefined; + return verdict ?? "UNKNOWN"; + } catch { + return "UNKNOWN"; + } +} + +function buildProjectSnapshot(workspace: string): ProjectSnapshot { + return { + hasConfig: fileExists(path.join(workspace, "config.json")), + hasSurvey: fileExists(path.join(workspace, "survey_res.md")), + hasSelection: fileExists(path.join(workspace, "selection_res.md")), + hasPlan: fileExists(path.join(workspace, "plan_res.md")), + hasDataValidation: fileExists(path.join(workspace, "data_validation.md")), + hasBaseline: fileExists(path.join(workspace, "baseline_res.md")), + hasImplementationReport: fileExists(path.join(workspace, "ml_res.md")), + latestReviewVerdict: readLatestReviewVerdict(workspace), + hasExperiment: fileExists(path.join(workspace, "experiment_res.md")), + }; +} + +function inferNextAction(snapshot: ProjectSnapshot): NextActionState { + if (!snapshot.hasConfig) { + return { + stage: "Bootstrap pending", + command: "完成 BOOTSTRAP 配置", + expectedOutputs: ["config.json", "SOUL.md"], + reason: "项目还没有基础配置,后续 survey、selection 和 experiment 没有统一方向依据。", + }; + } + + if (!snapshot.hasSurvey) { + return { + stage: "Survey needed", + command: "/research-survey", + expectedOutputs: ["knowledge/", "survey_res.md"], + reason: "当前还没有深度调研结果,无法可靠进入路线选择和实现。", + }; + } + + if (!snapshot.hasSelection && !snapshot.hasPlan) { + return { + stage: "Route selection", + command: "/algorithm-selection", + expectedOutputs: ["selection_res.md"], + reason: "已经有 survey,但还没有把候选路线收敛成 Chosen / Rejected / Fallback。", + }; + } + + if (!snapshot.hasPlan) { + return { + stage: "Planning", + command: "/research-plan", + expectedOutputs: ["plan_res.md"], + reason: "还缺 Dataset / Model / Training / Testing 四部分计划。", + }; + } + + if (!snapshot.hasDataValidation) { + return { + stage: "Dataset validation", + command: "/dataset-validate", + expectedOutputs: ["data_validation.md"], + reason: "先把数据真实性、split、label 和 leakage 风险单独审清楚,再做模型质量判断。", + }; + } + + if (!snapshot.hasBaseline) { + return { + stage: "Baseline setup", + command: "/baseline-runner", + expectedOutputs: ["baseline_res.md", "experiments/baselines/"], + reason: "当前还缺统一协议下的 baseline 结果,不适合直接写 headline comparison。", + }; + } + + if (!snapshot.hasImplementationReport) { + return { + stage: "Implementation", + command: "/research-implement", + expectedOutputs: ["project/", "ml_res.md"], + reason: "路线、计划、数据检查和 baseline 契约都已经具备,下一步应进入实现和 2 epoch 验证。", + }; + } + + if (snapshot.latestReviewVerdict !== "PASS") { + return { + stage: "Review", + command: "/research-review", + expectedOutputs: ["iterations/judge_v{N}.md"], + reason: "实现已经存在,但还没有拿到 review PASS,模型质量还需要单独审。", + }; + } + + if (!snapshot.hasExperiment) { + return { + stage: "Full experiment", + command: "/research-experiment", + expectedOutputs: ["experiment_res.md", "experiment_analysis/"], + reason: "实现和 review 已通过,下一步应补 full training、ablation 和补充实验。", + }; + } + + return { + stage: "Experiment complete", + command: "/write-review-paper", + expectedOutputs: ["review/"], + reason: "核心机器学习主链已经跑完,可以进入总结、综述或对外整理阶段。", + }; +} + +function formatArtifactPresence(snapshot: ProjectSnapshot): string { + const items = [ + ["survey", snapshot.hasSurvey], + ["selection", snapshot.hasSelection], + ["plan", snapshot.hasPlan], + ["data_validation", snapshot.hasDataValidation], + ["baseline", snapshot.hasBaseline], + ["implement", snapshot.hasImplementationReport], + ["review", snapshot.latestReviewVerdict === "PASS"], + ["experiment", snapshot.hasExperiment], + ]; + + return items.map(([label, ok]) => `${ok ? "yes" : "no"} ${label}`).join(" | "); +} + /** * /research-status - Show workspace status for all research agents */ @@ -58,6 +229,8 @@ export function handleResearchStatus(_ctx: PluginCommandContext): PluginCommandR const ideasCount = countFiles(path.join(w, "ideas"), (f) => f.endsWith(".md")); const topicCount = countFiles(path.join(w, "knowledge"), (f) => f.startsWith("topic-")); const hypothesisCount = countFiles(path.join(w, "ideas"), (f) => f.startsWith("hyp-")); + const snapshot = buildProjectSnapshot(w); + const next = inferNextAction(snapshot); let currentDay = 0; try { @@ -67,7 +240,12 @@ export function handleResearchStatus(_ctx: PluginCommandContext): PluginCommandR output += `**${projectId}** (Day ${currentDay})\n`; output += ` Workspace: \`${w}\`\n`; - output += ` Topics: ${topicCount} | Hypotheses: ${hypothesisCount} | Papers: ${papersCount} | Ideas: ${ideasCount}\n\n`; + output += ` Topics: ${topicCount} | Hypotheses: ${hypothesisCount} | Papers: ${papersCount} | Ideas: ${ideasCount}\n`; + output += ` Stage: ${next.stage}\n`; + output += ` Artifacts: ${formatArtifactPresence(snapshot)}\n`; + output += ` Next: \`${next.command}\`\n`; + output += ` Why: ${next.reason}\n`; + output += ` Expected: ${next.expectedOutputs.map((p) => `\`${p}\``).join(", ")}\n\n`; } return { text: output }; diff --git a/src/templates/bootstrap.ts b/src/templates/bootstrap.ts index b8c9497..0f2a1ac 100644 --- a/src/templates/bootstrap.ts +++ b/src/templates/bootstrap.ts @@ -21,7 +21,24 @@ export function renderBootstrapMd(projectName: string): string { - 生成 config.json(参考下方模板) 5. 询问用户是否立即执行 Day 0(构建初始知识状态) - 如果是,执行 /metabolism 完成首轮文献检索和知识库构建(Day 0 模式) -6. 删除本文件(BOOTSTRAP.md) +6. 如果用户已经有部分产物,优先走最短路径,不要默认从 Day 0 重头开始: + - 已有 papers/ -> /research-survey + - 已有 survey_res.md,但路线还没定 -> /algorithm-selection + - 已有 survey_res.md -> /research-plan + - 已有 plan_res.md,需要先审数据质量 -> /dataset-validate + - 已有 plan_res.md,且需要真实对比方法 -> /baseline-runner + - 已有实现代码 -> /research-review + - 已有 review PASS -> /research-experiment +7. 在完成首次配置后,告诉用户下一步推荐命令和预期输出文件,再删除本文件(BOOTSTRAP.md) + +## 常见入口 + +- 我只有一个研究方向,没有材料 -> 先完成配置,再运行 /metabolism +- 我已经有一批论文 -> 直接运行 /research-survey +- 我已经做完 survey,但还没决定模型路线 -> 运行 /algorithm-selection +- 我已经有 plan,想先确认数据没问题 -> 运行 /dataset-validate +- 我已经有 plan,想先把 baseline 跑出来 -> 运行 /baseline-runner +- 我已经有实现代码 -> 运行 /research-review ## config.json 模板 @@ -131,11 +148,26 @@ $W/ | /metabolism | Day 0: config.json, knowledge/ / Day 1+: papers/, knowledge/, ideas/hyp-*.md, log/ | | /research-collect | papers/ | | /research-survey | knowledge/, survey_res.md | +| /algorithm-selection | selection_res.md | | /research-plan | plan_res.md | +| /dataset-validate | data_validation.md | +| /baseline-runner | baseline_res.md, experiments/baselines/ | | /research-implement | experiments/ | | /research-review | experiments/review/ | | /research-experiment | experiments/results/ | | /idea-generation | ideas/ | | /write-review-paper | review/ | + +## Common ML Midstream Paths + +- Survey exists but route choice is still unclear: + - run \`/algorithm-selection\` + - expected output: \`selection_res.md\` +- Plan exists and the project needs a dedicated data-quality check: + - run \`/dataset-validate\` + - expected output: \`data_validation.md\` +- Plan exists and the project needs honest comparison numbers: + - run \`/baseline-runner\` + - expected outputs: \`baseline_res.md\`, optional baseline artifacts under \`experiments/baselines/\` `; } From 7c192549d4244a808d1b50dc88ad5292dd130025 Mon Sep 17 00:00:00 2001 From: JunkaiWang-TheoPhy <1181100960@qq.com> Date: Thu, 2 Apr 2026 21:00:04 +0800 Subject: [PATCH 2/3] chore --- skills/algorithm-selection/SKILL.md | 2 +- skills/baseline-runner/SKILL.md | 2 +- skills/dataset-validate/SKILL.md | 2 +- src/cli/research.ts | 2 +- src/commands.ts | 22 ++-- src/templates/bootstrap.ts | 156 ++++++++++++++-------------- 6 files changed, 93 insertions(+), 93 deletions(-) diff --git a/skills/algorithm-selection/SKILL.md b/skills/algorithm-selection/SKILL.md index 7cd9531..c6014c1 100644 --- a/skills/algorithm-selection/SKILL.md +++ b/skills/algorithm-selection/SKILL.md @@ -35,7 +35,7 @@ Outputs go to the workspace root. - `survey_res.md` - `knowledge/paper_*.md` when available -If `survey_res.md` is missing, stop and say: `需要先运行 /research-survey 完成深度分析`. +If `survey_res.md` is missing, stop and say: `Run /research-survey first to complete the deep analysis.` ## Required Output diff --git a/skills/baseline-runner/SKILL.md b/skills/baseline-runner/SKILL.md index e18af44..190e0fd 100644 --- a/skills/baseline-runner/SKILL.md +++ b/skills/baseline-runner/SKILL.md @@ -36,7 +36,7 @@ Outputs go to the workspace root. - `survey_res.md` - `project/` when the current project already has runnable code -If `plan_res.md` is missing, stop and say: `需要先运行 /research-plan 完成实现计划`. +If `plan_res.md` is missing, stop and say: `Run /research-plan first to complete the implementation plan.` ## Required Outputs diff --git a/skills/dataset-validate/SKILL.md b/skills/dataset-validate/SKILL.md index 716b737..a344c1c 100644 --- a/skills/dataset-validate/SKILL.md +++ b/skills/dataset-validate/SKILL.md @@ -36,7 +36,7 @@ Outputs go to the workspace root. - `project/` if a data pipeline already exists - `survey_res.md` when it defines dataset or protocol expectations -If `plan_res.md` is missing, stop and say: `需要先运行 /research-plan 完成实现计划`. +If `plan_res.md` is missing, stop and say: `Run /research-plan first to complete the implementation plan.` ## Required Output diff --git a/src/cli/research.ts b/src/cli/research.ts index be3cb90..feb61bc 100644 --- a/src/cli/research.ts +++ b/src/cli/research.ts @@ -71,7 +71,7 @@ function addCronJob(agentId: string): void { payload: { kind: "agentTurn", agentId, - message: "执行每日知识新陈代谢。阅读 AGENTS.md 了解工作流,然后使用 /metabolism 技能完成今日代谢。", + message: "Run daily knowledge metabolism. Read AGENTS.md for the workflow, then use the /metabolism skill to complete today's cycle.", }, delivery: { mode: "announce" }, enabled: true, diff --git a/src/commands.ts b/src/commands.ts index 77085d7..13ce80b 100644 --- a/src/commands.ts +++ b/src/commands.ts @@ -108,9 +108,9 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { if (!snapshot.hasConfig) { return { stage: "Bootstrap pending", - command: "完成 BOOTSTRAP 配置", + command: "complete BOOTSTRAP configuration", expectedOutputs: ["config.json", "SOUL.md"], - reason: "项目还没有基础配置,后续 survey、selection 和 experiment 没有统一方向依据。", + reason: "The project is missing its base configuration, so later survey, selection, and experiment steps do not yet share a stable direction.", }; } @@ -119,7 +119,7 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { stage: "Survey needed", command: "/research-survey", expectedOutputs: ["knowledge/", "survey_res.md"], - reason: "当前还没有深度调研结果,无法可靠进入路线选择和实现。", + reason: "There is no deep survey result yet, so route selection and implementation would be premature.", }; } @@ -128,7 +128,7 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { stage: "Route selection", command: "/algorithm-selection", expectedOutputs: ["selection_res.md"], - reason: "已经有 survey,但还没有把候选路线收敛成 Chosen / Rejected / Fallback。", + reason: "A survey exists, but the project has not yet narrowed candidate approaches into Chosen / Rejected / Fallback routes.", }; } @@ -137,7 +137,7 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { stage: "Planning", command: "/research-plan", expectedOutputs: ["plan_res.md"], - reason: "还缺 Dataset / Model / Training / Testing 四部分计划。", + reason: "The project still needs a concrete Dataset / Model / Training / Testing plan before implementation.", }; } @@ -146,7 +146,7 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { stage: "Dataset validation", command: "/dataset-validate", expectedOutputs: ["data_validation.md"], - reason: "先把数据真实性、split、label 和 leakage 风险单独审清楚,再做模型质量判断。", + reason: "Data reality, splits, labels, and leakage risk should be reviewed separately before judging model quality.", }; } @@ -155,7 +155,7 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { stage: "Baseline setup", command: "/baseline-runner", expectedOutputs: ["baseline_res.md", "experiments/baselines/"], - reason: "当前还缺统一协议下的 baseline 结果,不适合直接写 headline comparison。", + reason: "The project still lacks baseline results under a matched protocol, so headline comparisons would be too early.", }; } @@ -164,7 +164,7 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { stage: "Implementation", command: "/research-implement", expectedOutputs: ["project/", "ml_res.md"], - reason: "路线、计划、数据检查和 baseline 契约都已经具备,下一步应进入实现和 2 epoch 验证。", + reason: "The route, plan, data check, and baseline contract are already in place, so the next step is implementation plus 2-epoch validation.", }; } @@ -173,7 +173,7 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { stage: "Review", command: "/research-review", expectedOutputs: ["iterations/judge_v{N}.md"], - reason: "实现已经存在,但还没有拿到 review PASS,模型质量还需要单独审。", + reason: "Implementation exists, but review has not yet reached PASS, so model quality still needs a dedicated review pass.", }; } @@ -182,7 +182,7 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { stage: "Full experiment", command: "/research-experiment", expectedOutputs: ["experiment_res.md", "experiment_analysis/"], - reason: "实现和 review 已通过,下一步应补 full training、ablation 和补充实验。", + reason: "Implementation and review are ready, so the next step is full training, ablations, and supplementary experiments.", }; } @@ -190,7 +190,7 @@ function inferNextAction(snapshot: ProjectSnapshot): NextActionState { stage: "Experiment complete", command: "/write-review-paper", expectedOutputs: ["review/"], - reason: "核心机器学习主链已经跑完,可以进入总结、综述或对外整理阶段。", + reason: "The core ML execution chain is complete, so the project can move into synthesis, survey writing, or outward-facing summaries.", }; } diff --git a/src/templates/bootstrap.ts b/src/templates/bootstrap.ts index 0f2a1ac..696725f 100644 --- a/src/templates/bootstrap.ts +++ b/src/templates/bootstrap.ts @@ -6,46 +6,46 @@ export function renderBootstrapMd(projectName: string): string { return `# Research Agent Bootstrap -你刚刚被创建为课题「${projectName}」的研究 agent。你需要完成首次配置。 - -## 引导流程 - -1. 向用户问好,说明你是课题「${projectName}」的研究 agent,请用户描述研究方向 -2. 根据用户回答,提取: - - 核心域关键词(3-5 个) - - 建议的 arXiv 分类(如 cs.LG, cs.AI),如果研究方向不属于 arXiv 覆盖范围则留空 - - 建议的文献来源偏好(arXiv、OpenAlex、或两者兼用) -3. 与用户讨论研究方向的边界和重点,确认以上配置,接受调整 -4. 确认后执行以下写入操作: - - 更新 SOUL.md:填写研究方向、核心域各字段 - - 生成 config.json(参考下方模板) -5. 询问用户是否立即执行 Day 0(构建初始知识状态) - - 如果是,执行 /metabolism 完成首轮文献检索和知识库构建(Day 0 模式) -6. 如果用户已经有部分产物,优先走最短路径,不要默认从 Day 0 重头开始: - - 已有 papers/ -> /research-survey - - 已有 survey_res.md,但路线还没定 -> /algorithm-selection - - 已有 survey_res.md -> /research-plan - - 已有 plan_res.md,需要先审数据质量 -> /dataset-validate - - 已有 plan_res.md,且需要真实对比方法 -> /baseline-runner - - 已有实现代码 -> /research-review - - 已有 review PASS -> /research-experiment -7. 在完成首次配置后,告诉用户下一步推荐命令和预期输出文件,再删除本文件(BOOTSTRAP.md) - -## 常见入口 - -- 我只有一个研究方向,没有材料 -> 先完成配置,再运行 /metabolism -- 我已经有一批论文 -> 直接运行 /research-survey -- 我已经做完 survey,但还没决定模型路线 -> 运行 /algorithm-selection -- 我已经有 plan,想先确认数据没问题 -> 运行 /dataset-validate -- 我已经有 plan,想先把 baseline 跑出来 -> 运行 /baseline-runner -- 我已经有实现代码 -> 运行 /research-review - -## config.json 模板 +You were just created as the research agent for project "${projectName}". Complete the initial setup before starting downstream work. + +## Bootstrap Flow + +1. Greet the user, explain that you are the research agent for project "${projectName}", and ask them to describe the research direction. +2. Based on the user's answer, extract: + - core-domain keywords (3-5) + - suggested arXiv categories (for example "cs.LG", "cs.AI"); leave empty if the topic is outside arXiv coverage + - preferred literature sources ("arxiv", "openalex", or both) +3. Discuss scope and priorities with the user, confirm the configuration, and accept adjustments. +4. Once confirmed, write the following files: + - update SOUL.md with the project direction and domain fields + - generate config.json using the template below +5. Ask whether the user wants to run Day 0 immediately to build the initial knowledge state. + - If yes, run /metabolism to perform the first literature retrieval and knowledge-base construction pass. +6. If the project already has partial outputs, use the shortest matching path instead of restarting from Day 0: + - existing papers/ -> /research-survey + - existing survey_res.md, but route still undecided -> /algorithm-selection + - existing survey_res.md -> /research-plan + - existing plan_res.md, but data quality still needs review -> /dataset-validate + - existing plan_res.md, and matched baseline evidence is needed -> /baseline-runner + - existing implementation code -> /research-review + - existing review PASS -> /research-experiment +7. After the initial setup, tell the user the recommended next command and expected output files, then delete this file (BOOTSTRAP.md). + +## Common Entry Paths + +- I only have a research direction and no materials yet -> finish setup, then run /metabolism +- I already have a batch of papers -> run /research-survey +- I finished the survey but have not chosen a model route -> run /algorithm-selection +- I already have a plan and want to validate the data first -> run /dataset-validate +- I already have a plan and want honest baseline numbers first -> run /baseline-runner +- I already have implementation code -> run /research-review + +## config.json Template \`\`\`json { "projectId": "${projectName}", - "keywords": ["关键词1", "关键词2"], + "keywords": ["keyword-1", "keyword-2"], "arxivCategories": ["cs.LG"], "sources": ["arxiv", "openalex"], "currentDay": 0, @@ -54,23 +54,23 @@ export function renderBootstrapMd(projectName: string): string { } \`\`\` -> **注意:** \`arxivCategories\` 和 \`sources\` 根据研究领域灵活配置。 -> 自然科学、社会科学等非 CS 领域可将 \`arxivCategories\` 设为 \`[]\`,主要依赖 OpenAlex。 +> **Note:** Configure \`arxivCategories\` and \`sources\` based on the research domain. +> For natural science, social science, or other non-CS topics, it is fine to set \`arxivCategories\` to \`[]\` and rely primarily on OpenAlex. `; } export function renderSoulMd(projectName: string): string { return `# Project Agent — ${projectName} -你是课题「${projectName}」的研究 agent。 +You are the research agent for project "${projectName}". -## 研究方向 -{由 BOOTSTRAP 流程填写} +## Research Direction +{Filled during the BOOTSTRAP flow} -## 核心域 -关键词: {由 BOOTSTRAP 流程填写} -arXiv 分类: {由 BOOTSTRAP 流程填写,不适用则留空} -文献来源: {arXiv / OpenAlex / 两者兼用} +## Core Domain +Keywords: {Filled during the BOOTSTRAP flow} +arXiv Categories: {Filled during the BOOTSTRAP flow; leave empty if not applicable} +Literature Sources: {arXiv / OpenAlex / both} `; } @@ -79,32 +79,32 @@ export function renderAgentsMd(): string { ## Workspace Layout -本 agent 的工作目录即为项目根目录(\`$W\`)。所有文件相对于 \`$W\` 组织: +The agent workspace root is the project root (\`$W\`). All files are organized relative to \`$W\`: \`\`\` $W/ -├── SOUL.md # 身份 + 研究方向 -├── AGENTS.md # 本文档 -├── config.json # 项目配置(关键词、分类、当前天数) +├── SOUL.md # identity and project direction +├── AGENTS.md # this document +├── config.json # project config (keywords, categories, current day) │ -├── papers/ # 文献区:下载的论文 -│ ├── {arxiv_id}/ # arXiv 论文源文件 -│ └── {doi_slug}.pdf # PDF 文件 +├── papers/ # literature artifacts +│ ├── {arxiv_id}/ # arXiv source files +│ └── {doi_slug}.pdf # PDF files │ -├── knowledge/ # 知识区:持久知识状态 -│ ├── _index.md # 全景索引 -│ └── topic-*.md # 主题文件(上限 50) +├── knowledge/ # persistent knowledge state +│ ├── _index.md # global index +│ └── topic-*.md # topic files (limit 50) │ -├── ideas/ # 想法区:假设与研究想法 -│ ├── hyp-*.md # 生成的假设 -│ └── selected_idea.md # 选中的研究想法 +├── ideas/ # hypotheses and research ideas +│ ├── hyp-*.md # generated hypotheses +│ └── selected_idea.md # selected research idea │ -├── experiments/ # 实验代码区 -│ ├── run.py # 入口脚本 +├── experiments/ # experiment code +│ ├── run.py # entry script │ ├── requirements.txt -│ └── results/ # 实验结果 +│ └── results/ # experiment outputs │ -├── log/ # 运行日志 +├── log/ # run logs │ └── {YYYY-MM-DD}.md │ └── skills/ # workspace skills @@ -112,34 +112,34 @@ $W/ ## Session Context -你可能在不同类型的 session 中被唤醒: -- **Main session**:与人类直接对话,可触发 research-pipeline 等编排 skill -- **Cron session**:定时触发,执行周期性任务(如每日 metabolism) -- **Spawn session**:被 main session 调度(sessions_spawn),执行一次性重任务 +You may be activated in different session types: +- **Main session**: direct interaction with a human; can trigger orchestration skills such as \`research-pipeline\` +- **Cron session**: scheduled execution for recurring work such as daily metabolism +- **Spawn session**: dispatched by the main session (\`sessions_spawn\`) for one-off heavy tasks -任务指令会在 session 启动时注入,按指令执行即可。 +Task instructions are injected when the session starts. Follow the injected instructions for that session. ## Conventions ### File Existence = Step Completion -检查产出文件是否存在再执行。已存在则跳过。支持崩溃恢复和增量推进。 +Check whether the output file already exists before running a step. If it exists, skip it. This enables crash recovery and incremental progress. ### Immutability -产出文件一旦写入不修改,除非用户明确要求。例外:\`project/\` 在 implement-review 迭代中可变。 +Do not modify output files once written unless the user explicitly asks you to. Exception: \`project/\` may change during implement-review iteration. ### Knowledge File Rules -- knowledge/ 下的文件是持久知识状态,修改需谨慎 -- 每次修改必须先读取当前内容再更新 -- _index.md 是全景索引,必须与 topic 文件保持同步 -- topic 文件数上限 50,低活跃主题应合并归档 +- Files under \`knowledge/\` are persistent knowledge state and must be edited carefully. +- Always read the current file before updating it. +- \`_index.md\` is the global index and must stay in sync with the topic files. +- Limit topic files to 50. Merge or archive low-activity topics when needed. ## Research Rigor -- 先读原文,再思考,最后作答 -- 不捏造引用或实验结果,每个断言需有来源 -- 不确定时说「不确定」而非猜测 -- 读论文全文(.tex),不能只读 abstract -- 想法必须扎根于真实论文 +- Read the source material first, think second, answer third. +- Do not fabricate citations or experiment results. Every claim needs a source. +- Say "uncertain" when uncertain instead of guessing. +- Read the full paper source (\`.tex\`) when available; do not rely on the abstract alone. +- Ground ideas in real papers rather than unsupported intuition. ## Skill Outputs Summary From 65b7809d36ab1e2af2d27f1f2ea551296498d471 Mon Sep 17 00:00:00 2001 From: JunkaiWang-TheoPhy <1181100960@qq.com> Date: Thu, 2 Apr 2026 21:10:17 +0800 Subject: [PATCH 3/3] Update README.en.md --- README.en.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/README.en.md b/README.en.md index e75c804..34201af 100644 --- a/README.en.md +++ b/README.en.md @@ -230,6 +230,18 @@ Or invoke a specific skill directly with a slash command: /dataset-validate ``` +## New Skills for Midstream ML Work + +- `/algorithm-selection` + - use this after `/research-survey` and before `/research-plan` + - purpose: make 2-3 candidate routes explicit and record `Chosen Route / Rejected Routes / Fallback Route` +- `/dataset-validate` + - use this when `plan_res.md` already exists and you want to validate data quality before implementation or model review + - purpose: review data reality, splits, labels, leakage risk, and mock-data usage separately from model quality +- `/baseline-runner` + - use this when `plan_res.md` already exists and the project needs real baseline comparisons + - purpose: standardize baselines, protocol, metrics, and result recording, then write `baseline_res.md` + ### 3. Monitor sub-agent progress When the orchestrator spawns sub-agents, you'll see: