Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions .brain/BRAIN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# `.brain/` — agent workflow rules

This repo has a per-project second brain at `.brain/`. Hooks in
`.claude/settings.json` drive it automatically. Your job during a session:
keep the brain useful for the *next* session.

## At session start (already automatic)

The `SessionStart` hook runs `.brain/scripts/assemble-context.sh`, which
generates `.brain/_CONTEXT.md` and injects it into context. It contains:
- Pointers to CKIS (`_MEMORY.md`, `_overview.md`, architecture spec)
- Last 3 session summaries
- Open decisions and bugs
- Top of `GRAPH_REPORT.md` (if Graphify has run)

You don't need to read `_CONTEXT.md` again — it's already in context.

## During the session — what's auto-captured

The `PostToolUse` hook runs `.brain/scripts/log-tool-event.sh` after every
Bash call and silently appends one line to `.brain/sessions/_active.md`
**only** for these objectively important events:

- `npm run build` / `npm test` / `npm run lint` → success or failure (with last
~8 lines of output on failure, so the error is preserved next session)
- `git commit` → SHA + message + diffstat

The `UserPromptSubmit` hook runs `.brain/scripts/log-compact.sh` on every user
prompt but only acts on `/compact` commands — everything else is ignored. When
detected, it writes a timestamp breadcrumb to `.brain/.compact-triggers`.

Everything else (file edits, reads, other shell commands, regular prompts) is
**not logged** — that would be noise. The Stop hook merges `_active.md` into
the final session log under `## Iterations`. This means: every build, every
test, every commit you make in a session is permanently in the brain —
searchable, visible to the next session, and impossible to lose to a `/clear`.

When `/compact` runs, the Stop hook extracts the full summary from the session
JSONL transcript and writes it to `.brain/sessions/compacts/<timestamp>-compact.md`.
The session file gets a `## Compactions` section with a pointer and a 200-char
excerpt — the full summary is never auto-inlined to keep `_CONTEXT.md` lean.

## During the session

When the work warrants it, write to:

- **`.brain/decisions/YYYY-MM-DD-<slug>.md`** — for any decision Eduardo
makes about Korvex Web (architecture, dependency, deploy, scope). Use the
CKIS decision-log format. See `.brain/decisions/README.md`.
- **`.brain/bugs/YYYY-MM-DD-<slug>.md`** — when a bug is found *and* fixed,
capture the lesson (root cause, why it happened, how to prevent it). The
patch lives in the commit; this file is for the *why*.

Important decisions also get a one-line cross-post to
`~/Documents/Second Brain/00-inbox/_MEMORY.md` Open Decisions.

## Before ending the session

The Stop hook fills in the objective metadata automatically:
- Iterations (every build/test/lint/commit, with timestamps)
- Commits made, files changed, duration, branch state

Your only job is the **narrative `## Summary`** — and only when it adds
value. If the session was a routine commit-and-iterate cycle, the iteration
log already tells the story. Fill in the Summary when:

- A non-obvious decision was made mid-session
- A bug had a *why* worth remembering (root cause, not patch)
- Something is unfinished and the next session needs to know

Format: 2–4 bullets max. What was done · What was decided · What's next.
Skip it if the iteration log + commits speak for themselves.

## CKIS bridge — when to escalate

| Situation | Goes to |
| --- | --- |
| Routine code change, bug fix, refactor | Just commit. No brain entry needed. |
| Decision about *this* project | `.brain/decisions/` |
| Bug worth a postmortem | `.brain/bugs/` |
| Strategic / cross-project / personal | CKIS `_MEMORY.md` + `02-projects/korvex/_overview.md` |
| Pattern reusable across projects | CKIS `03-knowledge/permanent-notes/` |

When in doubt: project decision → `.brain/`. Strategic → CKIS.

## Hard rules

- Never modify `.brain/_CONTEXT.md` by hand — it's regenerated each session.
- Never delete files in `.brain/decisions/` or `.brain/bugs/` — supersede instead.
- `.brain/sessions/` is gitignored (personal); `decisions/` and `bugs/` are committed.
- Graphify rebuilds `.brain/graph/` on every commit (post-commit hook). Don't edit it.
288 changes: 288 additions & 0 deletions .brain/OPUS_BOOTSTRAP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,288 @@
---
type: opus-bootstrap
project: magnus-cli
created: 2026-06-03
status: active
---

# OPUS ORCHESTRATOR BOOTSTRAP — magnus-cli

> Paste this as your first message after `claude --resume <session-id>` in the magnus-cli tab.
> Model: `claude-opus-4-8` (set via `/model claude-opus-4-8` after resume).

---

## YOUR ROLE

You are the **Opus 4.8 orchestrator** for magnus-cli. Your mission: take this project from its scaffolded **v0.1.0** to a working, published **v1.0.0** — the universal media downloader + CKIS social capture engine.

This is a two-track project:
- **Track A (Download MVP):** Single-URL yt-dlp wrapper that works today, no external service dependency
- **Track B (Capture Pipeline):** Bulk social ingestion via Apify — requires external setup (has blockers), implement after Track A ships

You will orchestrate to v1.0.0 = Track A complete + Track B architecturally complete + npm published.

---

## STEP 1 — LOAD CONTEXT FIRST

```bash
cat .brain/_CONTEXT.md
cat graphify-out/GRAPH_REPORT.md
cat ~/Documents/Second\ Brain/02-projects/magnus-cli/_overview.md
cat package.json
find src/ -name "*.ts" | sort
```

Also read these source files before planning (they contain the critical implementation specs):
```bash
cat ~/Documents/Second\ Brain/04-resources/tools/yt-dlp-universal-downloader.md
cat ~/Documents/Second\ Brain/00-systems/workflows/ckis-social-captures-pipeline-master-plan.md
```

Key facts (ground truth):
- **Current version:** v0.1.0
- **Stack:** Node.js 20 ESM + TypeScript 5.5 + Ink v5 + Commander v13 + SQLite (better-sqlite3)
- **Core god nodes:** `runYouTube()` (19), `runInstagram()` (18), `runTikTok()` (18), `MagnusError` (18), `openDb()` (12), `Logger` (11), `resolvePath()` (10), `planDownload()` (9), `renderNote()` (9)
- **What exists:** Full architecture scaffold — platform handlers, SQLite queue, capture pipeline wiring, config/secrets store, Apify client integration points
- **License:** AGPL-3.0 + Dual Commercial (FINAL)
- **Open decisions to resolve in ultraplan:** See CKIS overview Step 2 below

---

## STEP 2 — ULTRAPLAN TARGET (v1.0.0)

### OPEN DECISIONS — RESOLVE THESE FIRST (before any code)

Read the open decisions from the CKIS overview and resolve them:

1. **Language for capture scripts:** Resolve to **Node.js/TypeScript** (single-stack, avoids Python dep, consistent with CLI). `runInstagram()` and `runTikTok()` are already TypeScript god nodes — stay in-stack.

2. **Config surface:** Resolve to **`~/.config/magnus/config.json` + `~/.config/magnus/.env`** — separate concerns (paths/prefs vs secrets). Mirror `loadConfig()` + `loadSecrets()` from `store.ts` (already wired per graph).

3. **TUI framework:** Resolve to **Ink v5 with `ora` spinners for battery operations** — Ink is already in the scaffold. For capture mode (long-running), use `ora` + `cli-progress`. For single URL download, simple Ink progress line.

4. **Package name:** Resolve to **`magnus-cli`** (global) — consistent with the suite naming pattern. Binary: `magnus`.

5. **First milestone scope:** Resolve to **Track A (single URL download) first** — this ships as v0.2.0. Track B (capture) ships as v0.3.0–v1.0.0.

6. **`magnus capture --watch` spec:** Implement as documented in [[recmp3-social-capture-pipeline]] — detect monitor source, record on play, stop at reel end. This is v1.0.0 scope (after basic capture works).

Write resolved decisions as ADR files in `.brain/decisions/` before generating the sprint.

### Track A — Single URL Download (v0.2.0)

The `planDownload()` god node is the core — implement it fully:
- `magnus <url>` — auto-detect platform from URL (CRITICAL: `music.youtube.com` must match before `youtube.com`)
- Platform detection: YouTube, YouTube Music, Instagram, TikTok, Twitter/X, Twitch, Bandcamp, SoundCloud, generic
- Audio mode (music platforms): `yt-dlp -x --audio-format mp3 --embed-metadata --embed-thumbnail`
- Video mode (all else): `yt-dlp -f "bv*+ba/b"`
- Smart output folders: `~/Videos/Social Media/{YouTube,TikTok,...}`, `~/Music/YT Music/{General,Soundtracks}/`
- Ink progress bar during download (stream yt-dlp stdout)
- `magnus audio <url>` — force audio mode on any URL
- `magnus doctor` — verify yt-dlp, ffmpeg, output dir write access
- Full test suite for platform detection logic (this is pure logic, no subprocess needed)
- `magnus stats` — download history by platform/date (reads SQLite history)

### Track B — Capture Pipeline (v0.3.0 → v0.4.0)

Build on the `runInstagram()`, `runTikTok()`, `runCapture()` scaffold already present:
- `magnus capture instagram` — Apify `apify/instagram-reel-scraper` + `apify/instagram-post-scraper`
- `magnus capture tiktok` — Apify `scrape-creators/best-tiktok-transcripts-scraper`
- `magnus capture youtube` — yt-dlp `--write-auto-subs` + webvtt-py, no Apify
- **20-URL canary mandatory** before bulk (hardcode this check — never bypass)
- **Spend cap enforcement:** `$40` soft warn / `$50` hard abort (Apify cost tracking)
- SQLite resume queue (`ig_queue`, `tt_queue`, `yt_queue`) — survives mid-run abort
- `renderNote()` — CKIS Type-B frontmatter: `type: resource`, `subtype: social-capture`, `transcript_source: native|auto-subs|unavailable`
- Atomic vault writes (write complete note or nothing — no partial files)

### Track C — `magnus capture --watch` (v1.0.0)

Integrate with recmp3-cli's system audio capture pipeline:
- Detect PulseAudio monitor source (same logic as recmp3's source detection)
- Start recording when user triggers, stop at content boundary
- Prompt: URL + author (paste-in)
- Write CKIS note via `renderNote()` with transcript from the recorded audio
- This is the automation of `[[recmp3-social-capture-pipeline]]`

### D. Agent-Native Mode
- `--json` flag on all commands (already scaffolded via `Logger` god node)
- `MagnusError` must serialize to JSON cleanly
- Document `magnus <url> --json` output schema in README for Claude Code integration

---

## STEP 3 — SPRINT ATOMIZATION RULES

1. **Resolve all open decisions FIRST** (Task 0) — write ADRs, no code yet
2. **Track A before Track B** — nothing in Track B until `magnus <url>` works E2E
3. **Platform detection = its own task** — pure logic, testable without subprocess
4. **SQLite schema is a foundational task** — must be stable before any capture tasks
5. **Apify integration tasks are marked `[BLOCKED-EXTERNAL]`** — implement the code, but flag that they need Apify account + credentials to test E2E
6. **Every task leaves `npm run build && npm test` passing**

Example:
```
TASK-01 [INFRA]: Resolve all open decisions → write 6 ADR files in .brain/decisions/
Done: 6 ADR files exist, each with decision + rationale + consequences
Depends: nothing

TASK-02 [CORE]: Implement platform URL detection (planDownload() layer)
Files: src/core/detect.ts (new), src/core/detect.test.ts
Done: all platform URLs in test suite route to correct platform enum;
music.youtube.com detected before youtube.com
Depends: TASK-01
```

---

## STEP 4 — WORKER ORCHESTRATION PROTOCOL

**Worker briefing template:**
```
You are a Sonnet execution agent for magnus-cli — a universal media downloader and CKIS social capture tool.

ARCHITECTURE FACTS:
- yt-dlp is called as a subprocess (spawn, not exec) — always stream stdout for progress
- music.youtube.com MUST be checked before youtube.com in platform detection (order is critical)
- SQLite via better-sqlite3 (synchronous API) — openDb() is the entry point, never open directly
- MagnusError is the error base class — all thrown errors must extend it
- renderNote() produces CKIS Type-B frontmatter — never write vault notes manually
- Never download MP4s for capture mode — CDN URLs expire in 3 days
- Spend cap: $40 warn / $50 abort — this is hardcoded, never bypassed, never made configurable

TASK: [exact task spec]
CONTEXT: [relevant god nodes]
CONSTRAINTS:
- Read before edit
- Immediately runnable
- npm run build + npm test must pass
- [BLOCKED-EXTERNAL] tasks: write the code, mock the external calls in tests
- Do NOT commit

Report back: files changed + verification commands.
```

---

## STEP 5 — AUDIT PROTOCOL

```bash
npm run build && npm test
npx tsc --noEmit
npm audit --audit-level=moderate
```

**Audit checklist:**
- [ ] `magnus doctor` passes: yt-dlp found, ffmpeg found, output dirs writable
- [ ] Platform detection: `music.youtube.com` routes to `youtube-music` (not `youtube`)
- [ ] `MagnusError` serializes to `{ code, message, platform? }` — no raw Error objects leaked
- [ ] `openDb()` creates tables on first run (idempotent)
- [ ] Spend cap check is NOT configurable — hardcoded at $40/$50
- [ ] No Apify API key in any test fixture or default config
- [ ] `renderNote()` output validates against CKIS Type-B frontmatter schema
- [ ] SQLite resume queue: aborting mid-run and restarting resumes from checkpoint
- [ ] `--json` flag produces valid JSON on stdout on all commands

---

## STEP 6 — COMPLETION CRITERIA + COMMIT

- [ ] `npm run build` → exit 0
- [ ] `npm test` → all tests pass (platform detection fully covered)
- [ ] `npx tsc --noEmit` → 0 errors
- [ ] `npm audit` → 0 moderate/high
- [ ] `magnus doctor` → passes (yt-dlp + ffmpeg verified)
- [ ] `magnus https://www.youtube.com/watch?v=dQw4w9WgXcQ` → downloads to correct folder
- [ ] `npm publish --dry-run` → exits cleanly

```bash
git add [specific files]
git commit -m "feat: magnus-cli v1.0.0 — universal download, social capture pipeline, CKIS integration"
git tag v1.0.0
git push origin main --tags
```

---

## STEP 7 — PERSONAL GUIDE (after v1.0.0 is pushed)

This step runs **only after Step 6 is fully complete** — v1.0.0 tagged, pushed, and npm published.

Write Eduardo's personal reference guide to:
`~/Documents/Second Brain/02-projects/magnus-cli/personal-guide.md`

**What this IS:** Eduardo's personal manual — how he uses Magnus to download media and ingest social content into his CKIS vault, written in plain language with his real folder structure and real commands.
**What this is NOT:** developer docs or a README for strangers.

### Guide structure:

```
---
type: guide
subtype: personal-tool-reference
project: magnus-cli
created: <today>
modified: <today>
tags: [guide, cli, personal-reference, magnus, descarga, media, social-capture, ckis]
related:
- "[[02-projects/magnus-cli/_overview]]"
- "[[00-systems/workflows/ckis-social-captures-pipeline-master-plan]]"
- "[[00-systems/workflows/recmp3-social-capture-pipeline]]"
---

# magnus — Mi Guía Personal
```

**Section 1 — ¿Qué es esto y para qué lo uso?**
Plain-language: Magnus reemplaza la función `yt()` del `.bashrc` y automatiza todo el pipeline de captura social. Qué hacía Eduardo antes (bash aliases, descarga manual, copiar-pegar). Qué resuelve Magnus ahora. Los dos tracks: descargas simples vs captura masiva de posts guardados.

**Section 2 — Cómo lo tengo instalado y configurado**
`npm i -g magnus-cli` → `magnus` global. Dónde están los archivos de config (`~/.config/magnus/`). Las carpetas de output: `~/Videos/Social Media/`, `~/Music/YT Music/`. El archivo `.env` con las API keys (Apify, etc). `magnus doctor` — output limpio en la máquina de Eduardo.

**Section 3 — Todos los comandos con ejemplos reales**
Every command with Eduardo's exact usage:
- `magnus <url>` — detecta plataforma automáticamente, descarga al folder correcto. Ejemplo real: YouTube video, TikTok, Instagram Reel, YouTube Music
- `magnus audio <url>` — fuerza modo MP3. Cuándo usar esto (música, podcasts)
- `magnus batch <file>` — descarga desde lista de URLs. Cómo Eduardo arma la lista
- `magnus capture instagram` — ingesta masiva de posts guardados via Apify. Qué genera (notas en CKIS)
- `magnus capture tiktok` — igual para TikTok
- `magnus capture youtube` — playlists/historial via yt-dlp (gratis, sin Apify)
- `magnus capture --watch` — el pipeline de system audio con recmp3 (el use case más poderoso)
- `magnus stats` — historial por plataforma y fecha
- `magnus config` — qué tiene Eduardo configurado
- `magnus doctor` — verificación completa

**Section 4 — Mis casos de uso personales**

1. **Descarga rápida:** `magnus https://...` → el URL que acabo de ver, descargado al folder correcto automáticamente. Antes: abrir yt-dlp, recordar los flags, especificar el folder. Ahora: un comando.
2. **Captura de contenido de Instagram:** tengo 5,200 posts guardados — `magnus capture instagram` los convierte en notas CKIS con transcript nativo. Eduardo puede buscar cualquier cosa que haya guardado en Instagram en Obsidian.
3. **Pipeline recmp3 + Magnus:** ver un Reel → `recwatch` graba el audio → `magnus capture --watch` lo convierte en nota CKIS con transcript y metadatos del post. El workflow completo sin abrir ninguna app.
4. **Música:** `magnus audio https://music.youtube.com/...` → MP3 con metadata y thumbnail en `~/Music/YT Music/`

**Section 5 — El pipeline de captura social explicado simple**
Cómo funciona el proceso completo: Apify obtiene el post → Magnus extrae el transcript → `renderNote()` crea la nota en Obsidian → Eduardo puede buscar en su vault todo lo que consumió. Por qué no descarga los MP4s (los CDN URLs expiran en 3 días). Por qué el canary de 20 URLs antes del bulk (para no gastar $50 en Apify si algo falla).

**Section 6 — Lo que aprendí**
- La trampa del orden en detección de plataformas: `music.youtube.com` DEBE detectarse antes que `youtube.com`
- Por qué el resume queue en SQLite salva la vida en capturas de miles de posts
- El spend cap de $40/$50 de Apify: por qué está hardcodeado y por qué eso es bueno
- bgutil-ytdlp-pot-provider: qué es y por qué YouTube lo necesita ahora

**Section 7 — Referencia rápida**
Cheat sheet de todos los comandos con una línea de descripción.

### After writing the guide:
1. Write the file to the exact CKIS path above
2. Add `[[personal-guide]]` wikilink to `02-projects/magnus-cli/_overview.md`
3. Update `_overview.md`'s `modified:` to today

---

## CKIS INTEGRATION REMINDER

- `.brain/decisions/` — ADR for every resolved open decision (write these in Task 0)
- `renderNote()` output must stay in sync with CKIS vault conventions
- Every commit triggers graph-report sync to CKIS automatically

Begin by executing Step 1 (Load Context) now.
Loading
Loading