Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1341,7 +1341,7 @@ Save to `vault/06_system/vault-philosophy.md` - this teaches your agent HOW to u

### The Graph Tools

MOCs and wiki-links create a graph, but the agent needs tooling to traverse it. See `scripts/vault-graph/` for the complete tools:
MOCs and wiki-links create a graph, but the agent needs tooling to traverse it. See [`scripts/vault-graph/`](./scripts/vault-graph/) for the complete tools (committed to this repo — see [`scripts/`](./scripts/)):

| Script | Purpose |
|--------|---------|
Expand Down
2 changes: 1 addition & 1 deletion part15-infrastructure-hardening.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,7 +375,7 @@ git branch -D agent/auth-refactor # if you don't want to keep it

### The 20-Line Spawner

Copy-paste harness for spawning N agents across N worktrees. Pairs with the [Ralph Loop (Part 30)](./part30-ralph-loop-in-openclaw.md) when you want each agent to run autonomously:
Copy-paste harness for spawning N agents across N worktrees. Pairs with the [Ralph Loop (Part 30)](./part30-ralph-loop-in-openclaw.md) when you want each agent to run autonomously. The full script is committed to this repo at [`scripts/fan-out.sh`](./scripts/fan-out.sh):

```bash
#!/usr/bin/env bash
Expand Down
2 changes: 1 addition & 1 deletion part30-ralph-loop-in-openclaw.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ The prompt is deliberately boring. Ralph iterations should feel like a factory l

## The Wrapper

A minimal bash wrapper that runs on macOS/Linux. PowerShell port is mechanical.
A minimal bash wrapper that runs on macOS/Linux. PowerShell port is mechanical. The full script is committed to this repo at [`scripts/ralph.sh`](./scripts/ralph.sh).

```bash
#!/usr/bin/env bash
Expand Down
2 changes: 2 additions & 0 deletions part9-vault-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,8 @@ This document is not for you — it's for the agent. You're programming behavior

MOCs and wiki-links create a graph, but the agent can't traverse it without tooling. You need two scripts: one that builds the graph, one that searches it.

> The full source for every script in this part is committed to this repo under [`scripts/vault-graph/`](./scripts/vault-graph/) — clone it and run, or copy the blocks below.

#### graph-indexer.mjs

This script scans every `.md` file across your vault, memory, and workspace root. It parses `[[wiki-links]]`, resolves them to actual files, and builds a JSON adjacency graph.
Expand Down
57 changes: 57 additions & 0 deletions scripts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Scripts

Runnable companions to the guide. The parts describe what each script does and
why; the source lives here so you can `git clone` once and run them instead of
copy-pasting out of Markdown.

All scripts are zero-dependency: the `.mjs` tools need only Node.js (ES
modules), and the `.sh` wrappers need only `bash` plus the CLIs called out in
their comments (`openclaw`, `git`, `jq`, `awk`, `bc`).

## Vault graph tools — [Part 9](../part9-vault-memory.md)

Run these from your workspace root (or set `OPENCLAW_WORKSPACE` to point at it).

| Script | Purpose |
|--------|---------|
| [`vault-graph/graph-indexer.mjs`](./vault-graph/graph-indexer.mjs) | Scans every `.md` file, parses `[[wiki-links]]`, writes a JSON adjacency graph to `vault/06_system/graph-index.json`. |
| [`vault-graph/graph-search.mjs`](./vault-graph/graph-search.mjs) | Traverses the graph — finds a file plus its direct and 2nd-degree connections. |
| [`vault-graph/auto-capture.mjs`](./vault-graph/auto-capture.mjs) | Turns an insight into a claim-named note in `vault/00_inbox/` and links it to related MOCs. |
| [`vault-graph/process-inbox.mjs`](./vault-graph/process-inbox.mjs) | Reviews inbox notes and suggests (or moves them to) the right vault folder. |
| [`vault-graph/update-mocs.mjs`](./vault-graph/update-mocs.mjs) | Health check — finds broken wiki-links, stale items, and orphaned notes. |

```bash
# typical first run
mkdir -p vault/{00_inbox,01_thinking,02_reference,03_creating,04_published,05_archive,06_system}
node scripts/vault-graph/graph-indexer.mjs
node scripts/vault-graph/graph-search.mjs "memory"
```

> Note: `vault-graph/auto-capture.mjs` (Part 9) is **not** the same as the
> `auto-capture` hook in [`hooks/auto-capture/`](../hooks/auto-capture/) (Part
> 11). The Part 9 script files a note you hand it; the Part 11 hook extracts
> notes from a session transcript automatically.

## Autonomy wrappers

| Script | Part | Purpose |
|--------|------|---------|
| [`ralph.sh`](./ralph.sh) | [Part 30](../part30-ralph-loop-in-openclaw.md) | Runs the Ralph loop — a fresh OpenClaw session per iteration until `PRD.json` is `done` or the iteration / wall-clock / USD budget is exhausted. Needs `PRD.json` and `loop-prompt.md` in the project root. |
| [`fan-out.sh`](./fan-out.sh) | [Part 15](../part15-infrastructure-hardening.md) | Spawns one agent per `*.md` task prompt across N git worktrees, then waits on all of them and reports failures. |

```bash
scripts/ralph.sh /path/to/project
scripts/fan-out.sh ./tasks
```

## Referenced elsewhere (not bundled here)

A few scripts mentioned in the guide intentionally live outside this repo —
their source isn't reproduced in the Markdown, so committing a copy here would
just go stale. Here's where each one actually comes from:

| Path in the guide | Part | Where it lives |
|-------------------|------|----------------|
| `scripts/memory-bridge/memory-query.js`, `preflight-context.js` | [Part 13](../part13-memory-bridge.md) | Standalone Memory Bridge tool — copy the two scripts into `scripts/memory-bridge/` in your own workspace as described in Part 13. |
| `lightrag-watcher.py` | [Part 21](../part21-realtime-knowledge-sync.md) | Your real-time sync watcher — Part 21 shows the config block to edit; deploy it under your own `scripts/`. |
| `scripts/qwen_embed_server_v3.py` | [Part 10](../part10-state-of-the-art-embeddings.md) | Ships inside your local OpenClaw install (installed by the one-shot prompt in Part 17), not in this guide repo. |
34 changes: 34 additions & 0 deletions scripts/fan-out.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#!/usr/bin/env bash
# scripts/fan-out.sh <tasks-dir>
set -euo pipefail
tasks_dir="$(realpath "${1:?pass a directory containing one *.md task-prompt per agent}")"
base_repo="$(pwd)"

mkdir -p "$base_repo/.worktrees"
declare -a pids=()

for task in "$tasks_dir"/*.md; do
abs_task="$(realpath "$task")" # pin before we cd into the worktree
name=$(basename "$task" .md)
wt="$base_repo/.worktrees/$name"
branch="agent/$name"

git worktree add "$wt" -b "$branch" >/dev/null
(
cd "$wt"
openclaw run --prompt-file "$abs_task" --ephemeral --output json \
> "$base_repo/.worktrees/$name.log" 2>&1
) &
pids+=($!)
echo "[fan-out] spawned $name (pid ${pids[-1]}) in $wt"
done

# Don't let one failing agent orphan the rest: track failures but keep waiting.
set +e
failures=0
for pid in "${pids[@]}"; do
wait "$pid" || ((failures++))
done
set -e
echo "[fan-out] all agents done ($failures failed of ${#pids[@]}). Branches: agent/*"
[[ $failures -eq 0 ]] || exit 1
48 changes: 48 additions & 0 deletions scripts/ralph.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# scripts/ralph.sh — run the Ralph loop until PRD is done or budget is exhausted
set -euo pipefail

PROJECT_ROOT="${1:-$(pwd)}"
cd "$PROJECT_ROOT"

[ ! -f PRD.json ] && { echo "PRD.json missing"; exit 1; }
[ ! -f loop-prompt.md ] && { echo "loop-prompt.md missing"; exit 1; }

max_iter=$(jq -r '.budget.max_iterations // 40' PRD.json)
max_usd=$(jq -r '.budget.max_usd // 20' PRD.json)
max_hours=$(jq -r '.budget.max_wall_hours // 4' PRD.json)

start=$(date +%s)
total_usd=0.00
i=0

while : ; do
i=$((i+1))
status=$(jq -r '.status' PRD.json)
[ "$status" = "done" ] && { echo "[ralph] PRD.done — exiting at iter $i"; break; }

elapsed=$(( ($(date +%s) - start) / 3600 ))
[ "$elapsed" -ge "$max_hours" ] && { echo "[ralph] wall-clock budget exhausted"; break; }
[ "$i" -gt "$max_iter" ] && { echo "[ralph] iteration budget exhausted"; break; }

echo "[ralph] === iter $i (elapsed ${elapsed}h, spent \$$total_usd) ==="

# Fresh OpenClaw session per iteration. --ephemeral keeps it out of your main history.
out_json=$(openclaw run \
--prompt-file loop-prompt.md \
--ephemeral \
--output json)

iter_usd=$(echo "$out_json" | jq -r '.usage.cost_usd // 0')
total_usd=$(echo "$total_usd $iter_usd" | awk '{print $1 + $2}')

(( $(echo "$total_usd >= $max_usd" | bc -l) )) && { echo "[ralph] USD budget exhausted"; break; }

# Let the dreaming scheduler consolidate anything learned before the next iteration.
sleep 2
done

# Final summary.
echo
echo "[ralph] final status=$(jq -r '.status' PRD.json) iterations=$i spent=\$${total_usd}"
jq '.tasks[] | {id, status}' PRD.json
101 changes: 101 additions & 0 deletions scripts/vault-graph/auto-capture.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
#!/usr/bin/env node
/**
* Auto-Capture: Converts insights into claim-named vault notes.
*
* Usage:
* node auto-capture.mjs --claim "nemotron mamba wont train on windows" --body "details..."
* node auto-capture.mjs --file summary.txt
* echo "insight text" | node auto-capture.mjs
*/

import fs from 'fs';
import path from 'path';

const VAULT = process.env.OPENCLAW_WORKSPACE
? path.join(process.env.OPENCLAW_WORKSPACE, 'vault')
: path.resolve('vault');
const INBOX = path.join(VAULT, '00_inbox');
const THINKING = path.join(VAULT, '01_thinking');

function toClaimName(text) {
let claim = text.split(/[.!?\n]/)[0].trim();
if (claim.length > 80) claim = claim.substring(0, 80);
return claim.toLowerCase()
.replace(/[^a-z0-9\s-]/g, '')
.replace(/\s+/g, '-')
.replace(/-+/g, '-')
.replace(/^-|-$/g, '')
.substring(0, 60) + '.md';
}

function findRelatedMOCs(text) {
if (!fs.existsSync(THINKING)) return [];
const mocs = fs.readdirSync(THINKING).filter(f => f.endsWith('.md') && f !== 'README.md');
const textLower = text.toLowerCase();
return mocs.filter(moc => {
const keywords = moc.replace('.md', '').split('-').filter(w => w.length > 3);
return keywords.filter(kw => textLower.includes(kw)).length >= 2;
}).map(m => m.replace('.md', ''));
}

function buildNote(claim, body, relatedMOCs) {
const date = new Date().toISOString().split('T')[0];
let note = `# ${claim}\n\n## Key Facts\n\n${body}\n\n`;
if (relatedMOCs.length > 0) {
note += `## Connected Topics\n\n`;
for (const moc of relatedMOCs) note += `- [[${moc}]]\n`;
note += `\n`;
}
note += `## Agent Notes\n\n- [ ] Review and verify this capture\n`;
note += `- [ ] Link to additional related notes\n`;
note += `- [ ] Move from inbox to appropriate vault folder\n`;
note += `\n_Captured: ${date}_\n`;
return note;
}

function updateMOCs(filename, relatedMOCs) {
const linkName = filename.replace('.md', '');
for (const moc of relatedMOCs) {
const mocPath = path.join(THINKING, moc + '.md');
if (!fs.existsSync(mocPath)) continue;
let content = fs.readFileSync(mocPath, 'utf8');
if (!content.includes(`[[${linkName}]]`)) {
const idx = content.indexOf('## Agent Notes');
if (idx !== -1) {
const insertPoint = content.indexOf('\n', idx) + 1;
content = content.substring(0, insertPoint)
+ `\n- [ ] New capture linked: [[${linkName}]]\n`
+ content.substring(insertPoint);
Comment on lines +65 to +68

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 updateMOCs corrupts MOC file when ## Agent Notes has no trailing newline

If the ## Agent Notes heading is the last line of a MOC file (no trailing newline), content.indexOf('\n', idx) at auto-capture.mjs:65 returns -1. Adding 1 gives insertPoint = 0, so content.substring(0, 0) is the empty string. The result is the new link line prepended to the beginning of the file, with the entire original content appended after it — corrupting the MOC structure.

Suggested change
const insertPoint = content.indexOf('\n', idx) + 1;
content = content.substring(0, insertPoint)
+ `\n- [ ] New capture linked: [[${linkName}]]\n`
+ content.substring(insertPoint);
const nlIdx = content.indexOf('\n', idx);
const insertPoint = nlIdx !== -1 ? nlIdx + 1 : content.length;
content = content.substring(0, insertPoint)
+ `\n- [ ] New capture linked: [[${linkName}]]\n`
+ content.substring(insertPoint);
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same situation as the other finding — real bug, but pre-existing and copied verbatim from Part 9, so I kept the file identical to the guide block in this "extract scripts to files" PR. The suggested guard (const nlIdx = content.indexOf('\n', idx); const insertPoint = nlIdx !== -1 ? nlIdx + 1 : content.length;) is correct. If you want it fixed, I'll patch both auto-capture.mjs and the Part 9 code block together so they stay in sync.

fs.writeFileSync(mocPath, content, 'utf8');
console.log(` Updated MOC: ${moc}.md`);
}
}
}
}

async function main() {
const args = process.argv.slice(2);
let claim = '', body = '';
if (args.includes('--claim')) claim = args[args.indexOf('--claim') + 1] || '';
if (args.includes('--body')) body = args[args.indexOf('--body') + 1] || '';
if (args.includes('--file')) {
const f = args[args.indexOf('--file') + 1];
if (f && fs.existsSync(f)) { body = fs.readFileSync(f, 'utf8'); if (!claim) claim = body.split('\n')[0]; }
}
if (!claim && !body) {
console.log('Usage: node auto-capture.mjs --claim "insight" --body "details..."');
process.exit(1);
}
const filename = toClaimName(claim);
Comment on lines +85 to +89

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 toClaimName generates hidden .md file when claim is empty

When --body is provided without --claim, the guard at auto-capture.mjs:85 (!claim && !body) passes because body is truthy while claim is ''. Then toClaimName('') at line 89 runs all its replacements on an empty string, which all produce '', and appends .md — creating a Unix hidden file named .md in the inbox. This file won't appear in normal ls listings, won't be found by subsequent process-inbox.mjs (which filters .endsWith('.md') but .md starts with a dot), and silently loses the user's capture.

Suggested change
if (!claim && !body) {
console.log('Usage: node auto-capture.mjs --claim "insight" --body "details..."');
process.exit(1);
}
const filename = toClaimName(claim);
if (!claim && !body) {
console.log('Usage: node auto-capture.mjs --claim "insight" --body "details..."');
process.exit(1);
}
if (!claim) claim = body.split('\n')[0] || 'untitled-capture';
const filename = toClaimName(claim);
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed real bug. Note this is pre-existing — the script was copied verbatim out of Part 9's code block, so the same defect lives in the guide prose too. Since this PR is purely "commit the inline scripts as files," I left the logic untouched to keep the file and the Part 9 block identical. Happy to fix it in a follow-up (or here) — but the fix should land in both scripts/vault-graph/auto-capture.mjs and the Part 9 code block so they don't drift. The suggested if (!claim) claim = body.split('\n')[0] || 'untitled-capture'; guard is the right shape. Deferring to the maintainer on whether to bundle that into this PR.

const filepath = path.join(INBOX, filename);
const relatedMOCs = findRelatedMOCs(claim + ' ' + body);
if (!fs.existsSync(INBOX)) fs.mkdirSync(INBOX, { recursive: true });
fs.writeFileSync(filepath, buildNote(claim, body, relatedMOCs), 'utf8');
console.log(`✅ Captured: ${filename}`);
if (relatedMOCs.length > 0) {
console.log(` Related MOCs: ${relatedMOCs.join(', ')}`);
updateMOCs(filename, relatedMOCs);
}
}

main().catch(console.error);
Loading
Loading