Skip to content

feat(swarm): migration 071 — peer + signed-record storage (#88)#92

Merged
Dewinator merged 1 commit into
mainfrom
agent/issue-88-2026-04-28T08-38-56-316Z
Apr 28, 2026
Merged

feat(swarm): migration 071 — peer + signed-record storage (#88)#92
Dewinator merged 1 commit into
mainfrom
agent/issue-88-2026-04-28T08-38-56-316Z

Conversation

@Dewinator

Copy link
Copy Markdown
Owner

Summary

  • New supabase/migrations/071_swarm_storage.sql — extends nodes with peer + local-trust columns, adds swarm_lessons (§3.1) and swarm_hub_anchors (§3.2). File only — Reed runs migrations manually after merge.
  • Sibling SQL-text contract test mcp-server/src/__tests__/migration-071-swarm-storage.test.ts (11 tests) pins every column, CHECK, FK, and index so a later edit cannot silently weaken the schema.
  • HNSW vector_cosine_ops embedding indexes (matches the codebase-wide pattern from migrations 002, 004, 015, 017, 022). The issue body's ivfflat was inconsistent with the existing memory-embedding index, which is HNSW.

Research summary

  • Migration 070 (PR feat(swarm): migration 070 — nodes table for cryptographic identity (#75) #79) is the canonical pattern: header comment with phase/issue tag, CREATE TABLE IF NOT EXISTS, COMMENT ON TABLE referencing docs/SWARM_SPEC.md, no DROP / DELETE. Replicated structure here.
  • Embedding indexes in this repo all use HNSW: memories (002, 004), experiences/lessons/soul_traits (015), intentions/people (017), stimuli (022). No table uses ivfflat. Followed the established pattern; documented the choice in the migration header.
  • Test convention: SQL-text contract pin under mcp-server/src/__tests__/. Strip comments, lowercase + collapse whitespace, regex-pin structural contracts. Same pattern as migration-070-node-identity.test.ts — the autonomy loop is forbidden from executing migrations, so a runtime DB-side test wouldn't run; the SQL-text contract is what survives.
  • SWARM_SPEC §3.4 is explicit: trust never leaves the node. A trust_edges table would create the temptation to JOIN it into a wire response. Flat columns on nodes (trust_weight, trust_reason) satisfy the local-only contract; the contract test guards against re-introduction of a trust_edges table.
  • Column-level CHECKs map 1:1 to §5 rejection rules so a bug in the wire-validator (PR feat(swarm): wire-validator — rejection rules 1-13 (#86) #89) cannot silently land an out-of-spec record:
    • rule 4 → VECTOR(768) NOT NULL
    • rule 9 → table-level CHECK (signed_at >= created_at) on swarm_lessons
    • rule 11 → CHECK (synthesized_from_cluster_size >= 2) (wire floor, stricter than §3.1's >= 1)
    • rule 12 → CHECK (octet_length(content) <= 8192) and octet_length(topic_label) <= 256

What this does NOT do

  • No endpoints. /swarm/lessons and /swarm/hubs ship in later Phase 3/4 issues.
  • No wire-validator integration. Validator already lives in PR feat(swarm): wire-validator — rejection rules 1-13 (#86) #89; the polling job that calls it lives in Phase 4.
  • No data backfill. Only CREATE TABLE IF NOT EXISTS and ALTER TABLE … ADD COLUMN IF NOT EXISTS. The new nodes columns get safe defaults (or are nullable) so the bootstrap is_self row from migration 070 / phase 1b survives untouched.
  • No migration execution. File only. Reed runs bash scripts/migrate.sh manually after merge (matches Phase 1a / 1b precedent).
  • No trust_edges table. §3.4 forbids trust on the wire; flat columns on nodes are the deliberate choice.
  • No unique (origin_node_id, embedding) on swarm_hub_anchors. Issue body explicitly skips it as expensive and not required.

Constitution affirmation

This change touches:

  • Pillar 1 (Decentralized AI): reinforces it — every record carries origin_node_id provenance and a signature; storage knows who said what without a central registry.
  • Pillar 3 (Swarm intelligence): reinforces it — the two tables are the on-disk substrate that lets each node accumulate diverse lessons and hub-anchors from peers without flattening difference.
  • Pillar 6 (Cyber security): reinforces it — column-level CHECKs are defense in depth behind the wire-validator; FKs to nodes(node_id) make every signed record trace back to a known cryptographic identity; trust is structurally local-only (no trust_edges table on the wire path).

No pillar is weakened. Pillars 2, 4, 5 are not touched.

Test plan

  • cd mcp-server && npm run build — clean TypeScript build
  • cd mcp-server && npm test — full suite green: 365 tests pass, 0 fail (the 11 new tests for migration 071 ride along)
  • node --test dist/__tests__/migration-071-swarm-storage.test.js — 11/11 pass in isolation
  • After merge, Reed runs cd scripts && bash migrate.sh against a real Supabase to apply 071 (out of scope for this PR per Phase 1a precedent)
  • Reviewer can sanity-check: open supabase/migrations/071_swarm_storage.sql and verify it only contains CREATE TABLE IF NOT EXISTS, ALTER TABLE … ADD COLUMN IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, COMMENT ON TABLE — no DROP, no DELETE, no INSERT
  • Reviewer can grep the test file for the §5 rejection-rule numbers it pins (4, 9, 11, 12) to confirm coverage of the issue's acceptance criteria

Closes #88.

Storage floor for /swarm/lessons and /swarm/hubs (Phase 3/4 endpoints
land later). Schemas mirror docs/SWARM_SPEC.md §3.1 (Lesson) and §3.2
(HubAnchor); peer + local-trust columns extend `nodes` from migration
070. SWARM_SPEC §3.4 trust stays local — flat columns on `nodes`, no
`trust_edges` table that could be JOINed into a wire response.

Column-level CHECKs map to §5 rejection rules (defense in depth behind
the wire-validator from PR #89): rule 9 signed_at >= created_at, rule
11 synthesized_from_cluster_size >= 2, rule 12 content/topic_label
size caps, rule 4 embedding dim 768.

HNSW vector_cosine_ops indexes match every other embedding index in
the codebase (memories, experiences, lessons, soul_traits — migrations
002, 004, 015, 017). The issue body's ivfflat reference contradicted
the established pattern; HNSW keeps query plans uniform.

File only — Reed runs migrations manually after merge. Sibling
SQL-text contract test pins every column, CHECK, FK, and index so a
later edit cannot silently weaken the schema before the Phase 3/4
endpoints come online.

Closes #88.
@Dewinator Dewinator added the agent-opened Opened by the autonomy loop label Apr 28, 2026
@Dewinator Dewinator merged commit b0eca59 into main Apr 28, 2026
@Dewinator Dewinator deleted the agent/issue-88-2026-04-28T08-38-56-316Z branch April 28, 2026 08:50
Dewinator added a commit that referenced this pull request Apr 29, 2026
Updates the stale "spec-only" header (phases 0–3 have all merged via
PRs #79/#81/#82/#85/#89/#91/#92) and pins each phase to its issue +
merged commit so a reader can tell at a glance which sections are wired
on `main` vs. still paper.

Phases 4–9 are deliberately listed as "_not yet issued_" — the project's
current priority is *Gehirn perfektionieren* per CLAUDE.md § Roadmap
(Reed 2026-04-26), and the wire contract is frozen at v1.0 so an
independent implementer can build a phase-3-equivalent peer today.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dewinator added a commit that referenced this pull request Apr 29, 2026
The cryptographic foundation of the swarm (SWARM_SPEC v1, Ed25519
signing, JCS canonicalization, wire-validator, .well-known discovery,
peer/signed-record storage — PRs #78,#79,#81,#82,#85,#89,#91,#92) was
landing on main while the README/MANIFESTO still claimed
"pairing/swarm/federation deferred". This commit fixes that mismatch.

README (EN+DE):
- new "Swarm — federation in flight" section with merged-PR table
  and a "what is next" subsection pointing to the swarm label
- Roadmap rewritten: phase 4-5 from "deferred" to "Phase 1 shipped"
- existing /.well-known/mycelium-node block folded into the new section
- promo video as a clickable poster near the top, served from a
  v0.4-swarm-phase-1 GitHub release asset (14 MB H.264 1080p)

MANIFESTO (EN+DE):
- "What is built today" split into brain core + Swarm Phase 1
- aspirational Tailscale+mTLS / mutual-pairing claims removed; those
  pieces remain on archive/swarm-deferred as historical reference
- "What is not built yet" sharpened to the social layer (verification,
  reputation, banishment-by-consensus, Sybil resistance) plus
  micro-transactions

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-opened Opened by the autonomy loop

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Swarm Phase 3d: migration 071 — peer + signed-record storage (file only, do NOT run)

1 participant