diff --git a/README.en.md b/README.en.md index 00b2bd3..ac64416 100644 --- a/README.en.md +++ b/README.en.md @@ -226,8 +226,22 @@ Or invoke a specific skill directly with a slash command: /research-pipeline /research-collect /idea-generation +/algorithm-selection +/dataset-validate ``` +## New Skills for Midstream ML Work + +- `/algorithm-selection` + - use this after `/research-survey` and before `/research-plan` + - purpose: make 2-3 candidate routes explicit and record `Chosen Route / Rejected Routes / Fallback Route` +- `/dataset-validate` + - use this when `plan_res.md` already exists and you want to validate data quality before implementation or model review + - purpose: review data reality, splits, labels, leakage risk, and mock-data usage separately from model quality +- `/baseline-runner` + - use this when `plan_res.md` already exists and the project needs real baseline comparisons + - purpose: standardize baselines, protocol, metrics, and result recording, then write `baseline_res.md` + ### 3. Monitor sub-agent progress When the orchestrator spawns sub-agents, you'll see: @@ -262,7 +276,10 @@ Check status anytime: | **research-pipeline** | `/research-pipeline` | Orchestrator. Spawns sub-agents for each phase, verifies outputs between steps. | | **research-collect** | `/research-collect` | Search arXiv → filter → download .tex sources → cluster → generate survey report. | | **research-survey** | `/research-survey` | Deep analysis of papers: extract formulas, map to code, produce method comparison table. | +| **algorithm-selection** | `/algorithm-selection` | Choose between 2-3 plausible ML routes, record rejected routes, and keep a fallback. | | **research-plan** | `/research-plan` | Create 4-part implementation plan (Dataset/Model/Training/Testing) from survey results. | +| **dataset-validate** | `/dataset-validate` | Audit data reality, splits, labels, and leakage risk before model review. | +| **baseline-runner** | `/baseline-runner` | Run or audit strong baselines under the same protocol before headline comparisons. | | **research-implement** | `/research-implement` | Implement ML code from plan, run 2-epoch validation with `uv` venv isolation. | | **research-review** | `/research-review` | Review implementation. Iterates fix → rerun → review up to 3 times. | | **research-experiment** | `/research-experiment` | Full training + ablation experiments. Requires review PASS. | diff --git a/README.md b/README.md index 369d261..2a31f10 100644 --- a/README.md +++ b/README.md @@ -226,8 +226,22 @@ openclaw gateway /research-pipeline /research-collect /idea-generation +/algorithm-selection +/dataset-validate ``` +## 机器学习中段任务的新增技能 + +- `/algorithm-selection` + - 用在 `/research-survey` 之后、`/research-plan` 之前 + - 作用：把 2-3 条候选路线写清楚，明确 `Chosen Route / Rejected Routes / Fallback Route` +- `/dataset-validate` + - 用在 `plan_res.md` 已经存在、准备实现或审查模型之前 + - 作用：单独审数据真实性、split、label、leakage 和 mock 风险，把数据质量和模型质量分开 +- `/baseline-runner` + - 用在 `plan_res.md` 已经存在、需要真实 baseline 对比时 + - 作用：统一 baseline、协议、指标和结果记录，产出 `baseline_res.md` + ### 3. 监控子 agent 进度 编排器 spawn 子 agent 后，你会看到： diff --git a/openclaw.plugin.json b/openclaw.plugin.json index 4dbd3c1..23ffc2f 100644 --- a/openclaw.plugin.json +++ b/openclaw.plugin.json @@ -8,6 +8,9 @@ "properties": {} }, "skills": [ + "skills/algorithm-selection", + "skills/baseline-runner", + "skills/dataset-validate", "skills/idea-generation", "skills/research-pipeline", "skills/research-survey", diff --git a/skills/algorithm-selection/SKILL.md b/skills/algorithm-selection/SKILL.md new file mode 100644 index 0000000..c6014c1 --- /dev/null +++ b/skills/algorithm-selection/SKILL.md @@ -0,0 +1,103 @@ +--- +name: algorithm-selection +description: "Use this when the user needs to choose between multiple ML routes after survey but before committing to implementation. Compares candidate approaches, selects one, records rejected routes, and keeps a fallback." +metadata: + { + "openclaw": + { + "emoji": "🧭", + }, + } +--- + +# Algorithm Selection + +**Don't ask permission. Just do it.** + +Use this skill after `/research-survey` when there are several plausible ML approaches and the project needs a deliberate route choice instead of jumping straight into implementation. + +Outputs go to the workspace root. + +## Use This When + +- `survey_res.md` already exists +- there are at least 2 plausible methods or model families +- the user wants a chosen route plus backups + +## Do Not Use This When + +- the project has no survey yet +- the team already decided the model route and only needs implementation details + +## Required Inputs + +- `SOUL.md` +- `survey_res.md` +- `knowledge/paper_*.md` when available + +If `survey_res.md` is missing, stop and say: `Run /research-survey first to complete the deep analysis.` + +## Required Output + +- `selection_res.md` + +## Workflow + +### Step 1: Read the Current Project Direction + +Read: + +- `SOUL.md` +- `survey_res.md` +- relevant `knowledge/paper_*.md` + +Extract: + +- the task and evaluation target +- method families mentioned in survey +- constraints such as compute, data, latency, interpretability, or deployment needs + +### Step 2: Build 2-3 Candidate Routes + +Create 2-3 realistic candidate routes only. For each route, record: + +- route name +- core idea +- supporting papers +- expected strengths +- expected risks +- implementation cost +- baseline compatibility + +Use `references/candidate-template.md`. + +### Step 3: Select One Route and Keep Backups + +Choose: + +- one `Chosen Route` +- one or more `Rejected Routes` +- one `Fallback Route` + +The fallback should be the route most likely to work if the chosen route underperforms or proves too expensive to implement. + +### Step 4: Write `selection_res.md` + +Use `references/selection-template.md`. + +The final output must include: + +- project goal +- decision criteria +- candidate options table +- chosen route +- rejected routes +- fallback route +- next recommended command + +## Rules + +1. Do not present only one route unless the survey truly leaves no meaningful alternative. +2. Every route must cite at least one paper or survey-derived basis. +3. The chosen route must match the project constraints in `SOUL.md`. +4. The fallback route must be different from the chosen route. diff --git a/skills/algorithm-selection/references/candidate-template.md b/skills/algorithm-selection/references/candidate-template.md new file mode 100644 index 0000000..05ab1bd --- /dev/null +++ b/skills/algorithm-selection/references/candidate-template.md @@ -0,0 +1,13 @@ +# Candidate Route Template + +Use one block per route. + +```markdown +## Route A: {name} +- Core idea: +- Based on: +- Expected strengths: +- Expected risks: +- Implementation cost: low / medium / high +- Baseline compatibility: +``` diff --git a/skills/algorithm-selection/references/selection-template.md b/skills/algorithm-selection/references/selection-template.md new file mode 100644 index 0000000..c08e4a7 --- /dev/null +++ b/skills/algorithm-selection/references/selection-template.md @@ -0,0 +1,39 @@ +# Selection Result Template + +```markdown +# Algorithm Selection + +## Project Goal +- task: +- main metric: +- key constraints: + +## Decision Criteria +- criterion 1: +- criterion 2: +- criterion 3: + +## Candidate Options + +| Route | Core idea | Strengths | Risks | Cost | Basis | +|-------|-----------|-----------|-------|------|-------| +| A | ... | ... | ... | ... | ... | +| B | ... | ... | ... | ... | ... | + +## Chosen Route +- route: +- why this route: +- what to implement first: + +## Rejected Routes +- route: + - why not now: + +## Fallback Route +- route: +- when to switch: + +## Next Step +- recommended command: +- expected output: +``` diff --git a/skills/baseline-runner/SKILL.md b/skills/baseline-runner/SKILL.md new file mode 100644 index 0000000..190e0fd --- /dev/null +++ b/skills/baseline-runner/SKILL.md @@ -0,0 +1,103 @@ +--- +name: baseline-runner +description: "Use this when the project needs real baseline results before or alongside the main model. Runs classical or literature-aligned baselines under the same protocol and writes a reproducible baseline summary." +metadata: + { + "openclaw": + { + "emoji": "📏", + "requires": { "bins": ["python3", "uv"] }, + }, + } +--- + +# Baseline Runner + +**Don't ask permission. Just do it.** + +Use this skill when the project needs trustworthy baseline numbers instead of only evaluating the proposed model in isolation. + +Outputs go to the workspace root. + +## Use This When + +- `plan_res.md` already names baselines +- `project/` already exists or a baseline implementation path is known +- the experiment stage needs matched comparison numbers + +## Do Not Use This When + +- the project has not finished survey or planning +- no baseline method has been identified yet + +## Required Inputs + +- `plan_res.md` +- `survey_res.md` +- `project/` when the current project already has runnable code + +If `plan_res.md` is missing, stop and say: `Run /research-plan first to complete the implementation plan.` + +## Required Outputs + +- `baseline_res.md` +- `experiments/baselines/` when runnable artifacts are created + +## Workflow + +### Step 1: Read the Evaluation Contract + +Read: + +- `plan_res.md` +- `survey_res.md` +- current `experiment_res.md` if it exists + +Extract: + +- baseline names +- evaluation metric +- protocol or guardrail +- dataset or workload assumptions + +### Step 2: Define the Baseline Matrix + +Create a small comparison matrix with: + +- baseline name +- source or basis +- expected setup +- metric +- status: `ready`, `needs adaptation`, or `missing` + +Use `references/baseline-matrix-template.md`. + +### Step 3: Run or Approximate Baselines Conservatively + +For each baseline: + +- if code is runnable under the current workspace, run it +- if only a lightweight adaptation is needed, implement the minimal adapter +- if a baseline cannot be run honestly, mark it as unavailable instead of inventing numbers + +All numeric results must come from actual execution logs or explicit imported evidence. + +### Step 4: Write `baseline_res.md` + +Use `references/baseline-report-template.md`. + +The report must include: + +- which baselines were attempted +- which ones ran successfully +- the exact metric values +- the evaluation protocol +- missing or partial baselines +- the most comparable baseline for the current project + +## Rules + +1. Never fabricate baseline numbers. +2. Keep the protocol aligned with the main experiment whenever possible. +3. If a baseline is only partly comparable, say so explicitly. +4. Prefer 2-3 strong baselines over a long weak list. diff --git a/skills/baseline-runner/references/baseline-matrix-template.md b/skills/baseline-runner/references/baseline-matrix-template.md new file mode 100644 index 0000000..736ecc3 --- /dev/null +++ b/skills/baseline-runner/references/baseline-matrix-template.md @@ -0,0 +1,9 @@ +# Baseline Matrix Template + +```markdown +# Baseline Matrix + +| Baseline | Source | Metric | Protocol | Status | Notes | +|----------|--------|--------|----------|--------|-------| +| {name} | {paper/repo} | {metric} | {protocol} | ready / needs adaptation / missing | {note} | +``` diff --git a/skills/baseline-runner/references/baseline-report-template.md b/skills/baseline-runner/references/baseline-report-template.md new file mode 100644 index 0000000..8b5dd84 --- /dev/null +++ b/skills/baseline-runner/references/baseline-report-template.md @@ -0,0 +1,25 @@ +# Baseline Report Template + +```markdown +# Baseline Results + +## Evaluation Contract +- dataset or workload: +- metric: +- guardrail or protocol: + +## Baselines Attempted + +| Baseline | Status | Result | Evidence Source | Notes | +|----------|--------|--------|-----------------|-------| +| {name} | ran / partial / missing | {value or N/A} | {log or file} | {notes} | + +## Most Comparable Baseline +- baseline: +- why this is the main comparison: + +## Gaps +- baseline not run: +- reason: +- how to close the gap: +``` diff --git a/skills/dataset-validate/SKILL.md b/skills/dataset-validate/SKILL.md new file mode 100644 index 0000000..a344c1c --- /dev/null +++ b/skills/dataset-validate/SKILL.md @@ -0,0 +1,104 @@ +--- +name: dataset-validate +description: "Use this when the project needs a dedicated data-quality review before model review. Checks data reality, split correctness, label health, leakage risk, shape consistency, and mock-data disclosure." +metadata: + { + "openclaw": + { + "emoji": "🗂️", + "requires": { "bins": ["python3", "uv"] }, + }, + } +--- + +# Dataset Validate + +**Don't ask permission. Just do it.** + +Use this skill before or alongside model implementation review when data quality needs to be checked separately from model quality. + +Outputs go to the workspace root. + +## Use This When + +- `plan_res.md` already exists +- the project is about to implement or has just implemented a model +- data quality, split quality, or label integrity is still uncertain + +## Do Not Use This When + +- the project has no concrete plan yet +- there is no dataset or data-loading path to inspect + +## Required Inputs + +- `plan_res.md` +- `project/` if a data pipeline already exists +- `survey_res.md` when it defines dataset or protocol expectations + +If `plan_res.md` is missing, stop and say: `Run /research-plan first to complete the implementation plan.` + +## Required Output + +- `data_validation.md` + +## Workflow + +### Step 1: Read the Data Contract + +Read: + +- `plan_res.md` +- `survey_res.md` if present +- current data-loading code under `project/data/` if present + +Extract: + +- expected dataset name +- source +- split structure +- label or target format +- expected shapes + +### Step 2: Audit Data Reality + +Check: + +- whether dataset files actually exist +- whether the data is real or mock +- whether mock usage is clearly declared +- whether row count / sample count is plausible + +### Step 3: Audit Data Integrity + +Check: + +- train / val / test split existence and separation +- label distribution or target sanity +- shape / dtype consistency +- obvious leakage risks +- preprocessing consistency with `plan_res.md` + +If code exists, run lightweight inspection commands under the project environment to verify counts and sample structure. + +### Step 4: Write `data_validation.md` + +Use `references/data-validation-template.md`. + +The report must include: + +- dataset identity +- data reality check +- split integrity +- label / target health +- leakage risk +- mock-data disclosure +- verdict: `PASS`, `NEEDS_REVISION`, or `BLOCKED` +- exact next step + +## Rules + +1. Keep data quality separate from model quality. +2. Never infer that data is real if the files or loading path are missing. +3. If mock data is used, call it out explicitly. +4. If data leakage is plausible, treat it as blocking until clarified. diff --git a/skills/dataset-validate/references/data-validation-template.md b/skills/dataset-validate/references/data-validation-template.md new file mode 100644 index 0000000..ae64151 --- /dev/null +++ b/skills/dataset-validate/references/data-validation-template.md @@ -0,0 +1,38 @@ +# Data Validation Template + +```markdown +# Data Validation + +## Dataset Identity +- dataset: +- source: +- expected split: + +## Reality Check +- files present: +- real or mock: +- evidence: + +## Split Integrity +- train split: +- val split: +- test split: +- leakage risk: + +## Label / Target Health +- label format: +- distribution or range: +- obvious anomalies: + +## Preprocessing Check +- expected preprocessing: +- observed preprocessing: +- mismatch: + +## Verdict +- PASS / NEEDS_REVISION / BLOCKED + +## Next Step +- recommended command: +- reason: +``` diff --git a/src/cli/research.ts b/src/cli/research.ts index 210d6fd..879a9f7 100644 --- a/src/cli/research.ts +++ b/src/cli/research.ts @@ -72,7 +72,7 @@ function addCronJob(agentId: string): void { payload: { kind: "agentTurn", agentId, - message: "执行每日知识新陈代谢。阅读 AGENTS.md 了解工作流，然后使用 /metabolism 技能完成今日代谢。", + message: "Run daily knowledge metabolism. Read AGENTS.md for the workflow, then use the /metabolism skill to complete today's cycle.", }, delivery: { mode: "announce" }, enabled: true, diff --git a/src/commands.ts b/src/commands.ts index 8e0e7c3..b2c69ff 100644 --- a/src/commands.ts +++ b/src/commands.ts @@ -11,6 +11,25 @@ interface ResearchAgent { workspace: string; } +interface ProjectSnapshot { + hasConfig: boolean; + hasSurvey: boolean; + hasSelection: boolean; + hasPlan: boolean; + hasDataValidation: boolean; + hasBaseline: boolean; + hasImplementationReport: boolean; + latestReviewVerdict: "PASS" | "NEEDS_REVISION" | "NEEDS_ALGORITHM_REVIEW" | "BLOCKED" | "MISSING" | "UNKNOWN"; + hasExperiment: boolean; +} + +interface NextActionState { + stage: string; + command: string; + expectedOutputs: string[]; + reason: string; +} + /** * List all research agents from openclaw.json. */ @@ -39,6 +58,158 @@ function countFiles(dirPath: string, filter?: (name: string) => boolean): number } } +function fileExists(filePath: string): boolean { + try { + return fs.existsSync(filePath); + } catch { + return false; + } +} + +function readLatestReviewVerdict(workspace: string): ProjectSnapshot["latestReviewVerdict"] { + const iterationsDir = path.join(workspace, "iterations"); + if (!fileExists(iterationsDir)) return "MISSING"; + + try { + const files = fs.readdirSync(iterationsDir) + .filter((f) => /^judge_v\d+\.md$/.test(f)) + .sort((a, b) => { + const na = Number(a.match(/\d+/)?.[0] ?? "0"); + const nb = Number(b.match(/\d+/)?.[0] ?? "0"); + return nb - na; + }); + + const latest = files[0]; + if (!latest) return "MISSING"; + + const content = fs.readFileSync(path.join(iterationsDir, latest), "utf-8"); + const verdictMatch = content.match(/##\s+Verdict:\s+([A-Z_]+)/); + const verdict = verdictMatch?.[1] as ProjectSnapshot["latestReviewVerdict"] | undefined; + return verdict ?? "UNKNOWN"; + } catch { + return "UNKNOWN"; + } +} + +function buildProjectSnapshot(workspace: string): ProjectSnapshot { + return { + hasConfig: fileExists(path.join(workspace, "config.json")), + hasSurvey: fileExists(path.join(workspace, "survey_res.md")), + hasSelection: fileExists(path.join(workspace, "selection_res.md")), + hasPlan: fileExists(path.join(workspace, "plan_res.md")), + hasDataValidation: fileExists(path.join(workspace, "data_validation.md")), + hasBaseline: fileExists(path.join(workspace, "baseline_res.md")), + hasImplementationReport: fileExists(path.join(workspace, "ml_res.md")), + latestReviewVerdict: readLatestReviewVerdict(workspace), + hasExperiment: fileExists(path.join(workspace, "experiment_res.md")), + }; +} + +function inferNextAction(snapshot: ProjectSnapshot): NextActionState { + if (!snapshot.hasConfig) { + return { + stage: "Bootstrap pending", + command: "complete BOOTSTRAP configuration", + expectedOutputs: ["config.json", "SOUL.md"], + reason: "The project is missing its base configuration, so later survey, selection, and experiment steps do not yet share a stable direction.", + }; + } + + if (!snapshot.hasSurvey) { + return { + stage: "Survey needed", + command: "/research-survey", + expectedOutputs: ["knowledge/", "survey_res.md"], + reason: "There is no deep survey result yet, so route selection and implementation would be premature.", + }; + } + + if (!snapshot.hasSelection && !snapshot.hasPlan) { + return { + stage: "Route selection", + command: "/algorithm-selection", + expectedOutputs: ["selection_res.md"], + reason: "A survey exists, but the project has not yet narrowed candidate approaches into Chosen / Rejected / Fallback routes.", + }; + } + + if (!snapshot.hasPlan) { + return { + stage: "Planning", + command: "/research-plan", + expectedOutputs: ["plan_res.md"], + reason: "The project still needs a concrete Dataset / Model / Training / Testing plan before implementation.", + }; + } + + if (!snapshot.hasDataValidation) { + return { + stage: "Dataset validation", + command: "/dataset-validate", + expectedOutputs: ["data_validation.md"], + reason: "Data reality, splits, labels, and leakage risk should be reviewed separately before judging model quality.", + }; + } + + if (!snapshot.hasBaseline) { + return { + stage: "Baseline setup", + command: "/baseline-runner", + expectedOutputs: ["baseline_res.md", "experiments/baselines/"], + reason: "The project still lacks baseline results under a matched protocol, so headline comparisons would be too early.", + }; + } + + if (!snapshot.hasImplementationReport) { + return { + stage: "Implementation", + command: "/research-implement", + expectedOutputs: ["project/", "ml_res.md"], + reason: "The route, plan, data check, and baseline contract are already in place, so the next step is implementation plus 2-epoch validation.", + }; + } + + if (snapshot.latestReviewVerdict !== "PASS") { + return { + stage: "Review", + command: "/research-review", + expectedOutputs: ["iterations/judge_v{N}.md"], + reason: "Implementation exists, but review has not yet reached PASS, so model quality still needs a dedicated review pass.", + }; + } + + if (!snapshot.hasExperiment) { + return { + stage: "Full experiment", + command: "/research-experiment", + expectedOutputs: ["experiment_res.md", "experiment_analysis/"], + reason: "Implementation and review are ready, so the next step is full training, ablations, and supplementary experiments.", + }; + } + + return { + stage: "Experiment complete", + command: "/write-review-paper", + expectedOutputs: ["review/"], + reason: "The core ML execution chain is complete, so the project can move into synthesis, survey writing, or outward-facing summaries.", + }; +} + +function formatArtifactPresence(snapshot: ProjectSnapshot): string { + const items = [ + ["survey", snapshot.hasSurvey], + ["selection", snapshot.hasSelection], + ["plan", snapshot.hasPlan], + ["data_validation", snapshot.hasDataValidation], + ["baseline", snapshot.hasBaseline], + ["implement", snapshot.hasImplementationReport], + ["review", snapshot.latestReviewVerdict === "PASS"], + ["experiment", snapshot.hasExperiment], + ]; + + return items.map(([label, ok]) => `${ok ? "yes" : "no"} ${label}`).join(" | "); +} + /** * /research-status - Show workspace status for all research agents */ @@ -59,6 +230,8 @@ export function handleResearchStatus(_ctx: PluginCommandContext): PluginCommandR const ideasCount = countFiles(path.join(w, "ideas"), (f) => f.endsWith(".md")); const topicCount = countFiles(path.join(w, "knowledge"), (f) => f.startsWith("topic-")); const hypothesisCount = countFiles(path.join(w, "ideas"), (f) => f.startsWith("hyp-")); + const snapshot = buildProjectSnapshot(w); + const next = inferNextAction(snapshot); let currentDay = 0; try { @@ -69,6 +242,11 @@ export function handleResearchStatus(_ctx: PluginCommandContext): PluginCommandR output += `**${projectId}** (Day ${currentDay})\n`; output += ` Workspace: \`${w}\`\n`; output += ` Topics: ${topicCount} | Hypotheses: ${hypothesisCount} | Papers: ${papersCount} | Ideas: ${ideasCount}\n`; + output += ` Stage: ${next.stage}\n`; + output += ` Artifacts: ${formatArtifactPresence(snapshot)}\n`; + output += ` Next: \`${next.command}\`\n`; + output += ` Why: ${next.reason}\n`; + output += ` Expected: ${next.expectedOutputs.map((p) => `\`${p}\``).join(", ")}\n`; const gateStatus = readReleaseGateStatus(w); if (hasReleaseFacingArtifacts(w) || gateStatus.state !== "missing") { output += ` Release Gate: ${formatReleaseGateStatus(gateStatus)}\n`; @@ -77,7 +255,7 @@ export function handleResearchStatus(_ctx: PluginCommandContext): PluginCommandR } const nextStep = getReleaseGateNextStep(w, gateStatus); if (nextStep) { - output += ` Next: ${nextStep}\n`; + output += ` Release Next: ${nextStep}\n`; } } output += `\n`; diff --git a/src/templates/bootstrap.ts b/src/templates/bootstrap.ts index 30b4d94..f8d8f10 100644 --- a/src/templates/bootstrap.ts +++ b/src/templates/bootstrap.ts @@ -13,21 +13,38 @@ You were just created as the research agent for project "${projectName}". Comple 1. Greet the user, explain that you are the research agent for project "${projectName}", and ask them to describe the research direction. 2. Based on the user's answer, extract: - core-domain keywords (3-5) - - suggested arXiv categories (for example \`cs.LG\`, \`cs.AI\`); leave empty if the topic is outside arXiv coverage - - preferred literature sources (\`arxiv\`, \`openalex\`, or both) + - suggested arXiv categories (for example "cs.LG", "cs.AI"); leave empty if the topic is outside arXiv coverage + - preferred literature sources ("arxiv", "openalex", or both) 3. Discuss scope and priorities with the user, confirm the configuration, and accept adjustments. 4. Once confirmed, write the following files: - - update \`SOUL.md\` with the project direction and domain fields - - generate \`config.json\` using the template below - - create \`progress_status.json\` for the current goal, including overall goal, current focus, next step, total tasks, completed tasks, and 3-7 subtasks + - update SOUL.md with the project direction and domain fields + - generate config.json using the template below 5. Ask whether the user wants to run Day 0 immediately to build the initial knowledge state. - - If yes, run \`/metabolism\` to perform the first literature retrieval and knowledge-base construction pass. -6. If the project already has partial outputs, use the shortest matching path instead of restarting the full pipeline: - - existing \`experiment_res.md\` plus figures -> \`/write-paper\` - - existing \`paper/draft.md\` -> \`/artifact-review\` - - existing figures with inconsistent captions or style -> \`/figure-standardize\` - - preparing \`README.md\` or release-facing docs -> \`/release-layout\` after a fresh release gate exists -7. Delete this file (\`BOOTSTRAP.md\`) after the setup is complete. + - If yes, run /metabolism to perform the first literature retrieval and knowledge-base construction pass. +6. If the project already has partial outputs, use the shortest matching path instead of restarting from Day 0: + - existing papers/ -> /research-survey + - existing survey_res.md, but route still undecided -> /algorithm-selection + - existing survey_res.md -> /research-plan + - existing plan_res.md, but data quality still needs review -> /dataset-validate + - existing plan_res.md, and matched baseline evidence is needed -> /baseline-runner + - existing implementation code -> /research-review + - existing review PASS -> /research-experiment + - existing experiment_res.md plus figures -> /write-paper + - existing paper/draft.md -> /artifact-review + - existing figures with inconsistent captions or style -> /figure-standardize + - preparing README.md or release-facing docs -> /release-layout after a fresh release gate exists +7. After the initial setup, tell the user the recommended next command and expected output files, then delete this file (BOOTSTRAP.md). + +## Common Entry Paths + +- I only have a research direction and no materials yet -> finish setup, then run /metabolism +- I already have a batch of papers -> run /research-survey +- I finished the survey but have not chosen a model route -> run /algorithm-selection +- I already have a plan and want to validate the data first -> run /dataset-validate +- I already have a plan and want honest baseline numbers first -> run /baseline-runner +- I already have implementation code -> run /research-review +- I already have experiment results and figures -> run /write-paper +- I already have a draft that needs review -> run /artifact-review ## config.json Template @@ -49,7 +66,7 @@ You were just created as the research agent for project "${projectName}". Comple } export function renderSoulMd(projectName: string): string { - return `# Project Agent - ${projectName} + return `# Project Agent — ${projectName} You are the research agent for project "${projectName}". @@ -154,7 +171,10 @@ Do not modify output files once written unless the user explicitly asks you to. | /metabolism | Day 0: config.json, knowledge/ / Day 1+: papers/, knowledge/, ideas/hyp-*.md, log/ | | /research-collect | papers/ | | /research-survey | knowledge/, survey_res.md | +| /algorithm-selection | selection_res.md | | /research-plan | plan_res.md | +| /dataset-validate | data_validation.md | +| /baseline-runner | baseline_res.md, experiments/baselines/ | | /research-implement | experiments/ | | /research-review | experiments/review/ | | /research-experiment | experiments/results/, experiment_res.md | @@ -165,6 +185,18 @@ Do not modify output files once written unless the user explicitly asks you to. | /figure-standardize | reports/figures/figure_spec.md or project/figures/figure_spec.md | | /release-layout | README.md, docs/index.html, release-facing pages | +## Common ML Midstream Paths + +- Survey exists but route choice is still unclear: + - run \`/algorithm-selection\` + - expected output: \`selection_res.md\` +- Plan exists and the project needs a dedicated data-quality check: + - run \`/dataset-validate\` + - expected output: \`data_validation.md\` +- Plan exists and the project needs honest comparison numbers: + - run \`/baseline-runner\` + - expected outputs: \`baseline_res.md\`, optional baseline artifacts under \`experiments/baselines/\` + ## Writing and Release Entry Points Use these entry points when the project already has partial outputs and does not need to restart the full research pipeline: