Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,8 +226,22 @@ Or invoke a specific skill directly with a slash command:
/research-pipeline
/research-collect
/idea-generation
/algorithm-selection
/dataset-validate
```

## New Skills for Midstream ML Work

- `/algorithm-selection`
- use this after `/research-survey` and before `/research-plan`
- purpose: make 2-3 candidate routes explicit and record `Chosen Route / Rejected Routes / Fallback Route`
- `/dataset-validate`
- use this when `plan_res.md` already exists and you want to validate data quality before implementation or model review
- purpose: review data reality, splits, labels, leakage risk, and mock-data usage separately from model quality
- `/baseline-runner`
- use this when `plan_res.md` already exists and the project needs real baseline comparisons
- purpose: standardize baselines, protocol, metrics, and result recording, then write `baseline_res.md`

### 3. Monitor sub-agent progress

When the orchestrator spawns sub-agents, you'll see:
Expand Down Expand Up @@ -262,7 +276,10 @@ Check status anytime:
| **research-pipeline** | `/research-pipeline` | Orchestrator. Spawns sub-agents for each phase, verifies outputs between steps. |
| **research-collect** | `/research-collect` | Search arXiv → filter → download .tex sources → cluster → generate survey report. |
| **research-survey** | `/research-survey` | Deep analysis of papers: extract formulas, map to code, produce method comparison table. |
| **algorithm-selection** | `/algorithm-selection` | Choose between 2-3 plausible ML routes, record rejected routes, and keep a fallback. |
| **research-plan** | `/research-plan` | Create 4-part implementation plan (Dataset/Model/Training/Testing) from survey results. |
| **dataset-validate** | `/dataset-validate` | Audit data reality, splits, labels, and leakage risk before model review. |
| **baseline-runner** | `/baseline-runner` | Run or audit strong baselines under the same protocol before headline comparisons. |
| **research-implement** | `/research-implement` | Implement ML code from plan, run 2-epoch validation with `uv` venv isolation. |
| **research-review** | `/research-review` | Review implementation. Iterates fix → rerun → review up to 3 times. |
| **research-experiment** | `/research-experiment` | Full training + ablation experiments. Requires review PASS. |
Expand Down
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,8 +226,22 @@ openclaw gateway
/research-pipeline
/research-collect
/idea-generation
/algorithm-selection
/dataset-validate
```

## 机器学习中段任务的新增技能

- `/algorithm-selection`
- 用在 `/research-survey` 之后、`/research-plan` 之前
- 作用:把 2-3 条候选路线写清楚,明确 `Chosen Route / Rejected Routes / Fallback Route`
- `/dataset-validate`
- 用在 `plan_res.md` 已经存在、准备实现或审查模型之前
- 作用:单独审数据真实性、split、label、leakage 和 mock 风险,把数据质量和模型质量分开
- `/baseline-runner`
- 用在 `plan_res.md` 已经存在、需要真实 baseline 对比时
- 作用:统一 baseline、协议、指标和结果记录,产出 `baseline_res.md`

### 3. 监控子 agent 进度

编排器 spawn 子 agent 后,你会看到:
Expand Down
3 changes: 3 additions & 0 deletions openclaw.plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@
"properties": {}
},
"skills": [
"skills/algorithm-selection",
"skills/baseline-runner",
"skills/dataset-validate",
"skills/idea-generation",
"skills/research-pipeline",
"skills/research-survey",
Expand Down
103 changes: 103 additions & 0 deletions skills/algorithm-selection/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
name: algorithm-selection
description: "Use this when the user needs to choose between multiple ML routes after survey but before committing to implementation. Compares candidate approaches, selects one, records rejected routes, and keeps a fallback."
metadata:
{
"openclaw":
{
"emoji": "🧭",
},
}
---

# Algorithm Selection

**Don't ask permission. Just do it.**

Use this skill after `/research-survey` when there are several plausible ML approaches and the project needs a deliberate route choice instead of jumping straight into implementation.

Outputs go to the workspace root.

## Use This When

- `survey_res.md` already exists
- there are at least 2 plausible methods or model families
- the user wants a chosen route plus backups

## Do Not Use This When

- the project has no survey yet
- the team already decided the model route and only needs implementation details

## Required Inputs

- `SOUL.md`
- `survey_res.md`
- `knowledge/paper_*.md` when available

If `survey_res.md` is missing, stop and say: `Run /research-survey first to complete the deep analysis.`

## Required Output

- `selection_res.md`

## Workflow

### Step 1: Read the Current Project Direction

Read:

- `SOUL.md`
- `survey_res.md`
- relevant `knowledge/paper_*.md`

Extract:

- the task and evaluation target
- method families mentioned in survey
- constraints such as compute, data, latency, interpretability, or deployment needs

### Step 2: Build 2-3 Candidate Routes

Create 2-3 realistic candidate routes only. For each route, record:

- route name
- core idea
- supporting papers
- expected strengths
- expected risks
- implementation cost
- baseline compatibility

Use `references/candidate-template.md`.

### Step 3: Select One Route and Keep Backups

Choose:

- one `Chosen Route`
- one or more `Rejected Routes`
- one `Fallback Route`

The fallback should be the route most likely to work if the chosen route underperforms or proves too expensive to implement.

### Step 4: Write `selection_res.md`

Use `references/selection-template.md`.

The final output must include:

- project goal
- decision criteria
- candidate options table
- chosen route
- rejected routes
- fallback route
- next recommended command

## Rules

1. Do not present only one route unless the survey truly leaves no meaningful alternative.
2. Every route must cite at least one paper or survey-derived basis.
3. The chosen route must match the project constraints in `SOUL.md`.
4. The fallback route must be different from the chosen route.
13 changes: 13 additions & 0 deletions skills/algorithm-selection/references/candidate-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Candidate Route Template

Use one block per route.

```markdown
## Route A: {name}
- Core idea:
- Based on:
- Expected strengths:
- Expected risks:
- Implementation cost: low / medium / high
- Baseline compatibility:
```
39 changes: 39 additions & 0 deletions skills/algorithm-selection/references/selection-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Selection Result Template

```markdown
# Algorithm Selection

## Project Goal
- task:
- main metric:
- key constraints:

## Decision Criteria
- criterion 1:
- criterion 2:
- criterion 3:

## Candidate Options

| Route | Core idea | Strengths | Risks | Cost | Basis |
|-------|-----------|-----------|-------|------|-------|
| A | ... | ... | ... | ... | ... |
| B | ... | ... | ... | ... | ... |

## Chosen Route
- route:
- why this route:
- what to implement first:

## Rejected Routes
- route:
- why not now:

## Fallback Route
- route:
- when to switch:

## Next Step
- recommended command:
- expected output:
```
103 changes: 103 additions & 0 deletions skills/baseline-runner/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
name: baseline-runner
description: "Use this when the project needs real baseline results before or alongside the main model. Runs classical or literature-aligned baselines under the same protocol and writes a reproducible baseline summary."
metadata:
{
"openclaw":
{
"emoji": "📏",
"requires": { "bins": ["python3", "uv"] },
},
}
---

# Baseline Runner

**Don't ask permission. Just do it.**

Use this skill when the project needs trustworthy baseline numbers instead of only evaluating the proposed model in isolation.

Outputs go to the workspace root.

## Use This When

- `plan_res.md` already names baselines
- `project/` already exists or a baseline implementation path is known
- the experiment stage needs matched comparison numbers

## Do Not Use This When

- the project has not finished survey or planning
- no baseline method has been identified yet

## Required Inputs

- `plan_res.md`
- `survey_res.md`
- `project/` when the current project already has runnable code

If `plan_res.md` is missing, stop and say: `Run /research-plan first to complete the implementation plan.`

## Required Outputs

- `baseline_res.md`
- `experiments/baselines/` when runnable artifacts are created

## Workflow

### Step 1: Read the Evaluation Contract

Read:

- `plan_res.md`
- `survey_res.md`
- current `experiment_res.md` if it exists

Extract:

- baseline names
- evaluation metric
- protocol or guardrail
- dataset or workload assumptions

### Step 2: Define the Baseline Matrix

Create a small comparison matrix with:

- baseline name
- source or basis
- expected setup
- metric
- status: `ready`, `needs adaptation`, or `missing`

Use `references/baseline-matrix-template.md`.

### Step 3: Run or Approximate Baselines Conservatively

For each baseline:

- if code is runnable under the current workspace, run it
- if only a lightweight adaptation is needed, implement the minimal adapter
- if a baseline cannot be run honestly, mark it as unavailable instead of inventing numbers

All numeric results must come from actual execution logs or explicit imported evidence.

### Step 4: Write `baseline_res.md`

Use `references/baseline-report-template.md`.

The report must include:

- which baselines were attempted
- which ones ran successfully
- the exact metric values
- the evaluation protocol
- missing or partial baselines
- the most comparable baseline for the current project

## Rules

1. Never fabricate baseline numbers.
2. Keep the protocol aligned with the main experiment whenever possible.
3. If a baseline is only partly comparable, say so explicitly.
4. Prefer 2-3 strong baselines over a long weak list.
9 changes: 9 additions & 0 deletions skills/baseline-runner/references/baseline-matrix-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Baseline Matrix Template

```markdown
# Baseline Matrix

| Baseline | Source | Metric | Protocol | Status | Notes |
|----------|--------|--------|----------|--------|-------|
| {name} | {paper/repo} | {metric} | {protocol} | ready / needs adaptation / missing | {note} |
```
25 changes: 25 additions & 0 deletions skills/baseline-runner/references/baseline-report-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Baseline Report Template

```markdown
# Baseline Results

## Evaluation Contract
- dataset or workload:
- metric:
- guardrail or protocol:

## Baselines Attempted

| Baseline | Status | Result | Evidence Source | Notes |
|----------|--------|--------|-----------------|-------|
| {name} | ran / partial / missing | {value or N/A} | {log or file} | {notes} |

## Most Comparable Baseline
- baseline:
- why this is the main comparison:

## Gaps
- baseline not run:
- reason:
- how to close the gap:
```
Loading
Loading