Roll out eva-v0.42.56.5 to customer VM fleet with docs ingestion gate

[100yenadmin]

Summary

Roll out eva-v0.42.56.5 to the active customer VM fleet one VM at a time. Local canary is already green: CLI, doctor 100/100, Voyage 2048, Support KB search, workspace-docs search, Codex plugin rehearsal, OpenClaw plugin inspect, gateway inference, and an OpenClaw agent turn that called GBrain.

Durable Contract

• Release: https://github.com/electricsheephq/eva-brain/releases/tag/eva-v0.42.56.5
• Evidence root: /Volumes/LEXAR/Codex/evidence/eva-brain-fleet-rollout-0.42.56.5/
• VM inventory source: Supabase customer_vms, not memory.
• Mutation path: exact-tag repo updater only.
• Stop on: SSH/identity uncertainty, backup failure, updater failure after one retry, PGLite open failure, gateway restart failure, plugin load failure, Support KB search failure, workspace-docs failure when docs exist, schema stats empty while status shows pages, Jackie/proxy drift, or agent GBrain-call failure for runtime reasons.

Rollout Order

1. Golden
2. Eric
3. Jackie with proxy/gateway preflight
4. Remaining active VMs one at a time from Supabase inventory

Per-VM Acceptance

• gbrain version is 0.42.56.5
• gbrain doctor --scope=brain --json is 100/100, or classified with a non-runtime exception
• Voyage uses voyage:voyage-4-large with 2048 dimensions
• openclaw-support-kb is populated and searchable
• workspace-docs is searchable when docs exist
• AGENTS guidance mentions /root/.openclaw/workspace/docs and workspace-docs
• OpenClaw plugin exposes GBrain tools
• OpenClaw/Hermes agent records gbrain_search or gbrain_query
• No duplicate stale gbrain serve processes

Canonical VM Command

cd /root/eva-brain
export PATH="/root/.bun/bin:$PATH"
export GBRAIN_SKILLS_DIR=/root/eva-brain/skills
scripts/update-local-install.sh \
  --ref eva-v0.42.56.5 \
  --dir /root/eva-brain \
  --with-openclaw \
  --without-codex-plugin \
  --with-support-kb \
  --with-workspace-docs \
  --stop-stale-serve
systemctl restart openclaw-gateway

[100yenadmin]

[
{
"customer_id": "benjamin-kennedy",
"public_ip": "167.233.33.67",
"tailnet_ip": null,
"openclaw_port": 18789,
"status": "active",
"tier": "biz",
"region": "fsn1",
"updated_at": "2026-06-28T13:50:45.281634+00:00"
},
{
"customer_id": "david-poku",
"public_ip": "46.224.197.130",
"tailnet_ip": null,
"openclaw_port": 18789,
"status": "active",
"tier": "biz",
"region": "nbg1",
"updated_at": "2026-06-28T13:50:55.898139+00:00"
},
{
"customer_id": "eric-wilder",
"public_ip": "178.104.248.90",
"tailnet_ip": null,
"openclaw_port": 18789,
"status": "active",
"tier": "biz",
"region": "fsn1",
"updated_at": "2026-07-01T17:52:09.553278+00:00"
},
{
"customer_id": "evaos-support",
"public_ip": "178.104.123.213",
"tailnet_ip": null,
"openclaw_port": 18789,
"status": "active",
"tier": "go",
"region": "fsn1",
"updated_at": "2026-07-02T19:04:19.814226+00:00"
},
{
"customer_id": "golden",
"public_ip": "178.104.117.126",
"tailnet_ip": "100.64.0.7",
"openclaw_port": 18789,
"status": "active",
"tier": "go",
"region": "fsn1",
"updated_at": "2026-07-02T19:04:57.57377+00:00"
},
{
"customer_id": "holly-holmberg",
"public_ip": "178.104.8.18",
"tailnet_ip": null,
"openclaw_port": 18789,
"status": "active",
"tier": "go",
"region": "fsn1",
"updated_at": "2026-07-01T11:29:04.668411+00:00"
},
{
"customer_id": "jackie-david",
"public_ip": "46.224.160.253",
"tailnet_ip": null,
"openclaw_port": 18789,
"status": "active",
"tier": "biz",
"region": "fsn1",
"updated_at": "2026-07-02T15:02:11.671281+00:00"
},
{
"customer_id": "matt-bailey",
"public_ip": "94.130.229.51",
"tailnet_ip": null,
"openclaw_port": 18789,
"status": "active",
"tier": "biz",
"region": "nbg1",
"updated_at": "2026-07-02T19:47:47.85405+00:00"
},
{
"customer_id": "tarzan-sharif",
"public_ip": "178.105.33.176",
"tailnet_ip": null,
"openclaw_port": 18789,
"status": "active",
"tier": "biz",
"region": "fsn1",
"updated_at": "2026-06-28T13:51:27.035058+00:00"
}
]

[100yenadmin]

Golden canary update

Status: Eva Brain runtime PASS; agent-turn GBrain smoke AUTH-BLOCKED, not GBrain-blocked.

Evidence:

• Pre-upgrade version: gbrain 0.42.47.8
• Upgraded to: gbrain 0.42.56.5
• Backup: /root/eva-brain-backups/golden-20260706T090117Z
• Main audit: /root/eva-brain-upgrade-golden-20260706T090117Z.log
• Extraction repair audit: /root/eva-brain-upgrade-golden-20260706T090501Z-extract-repair.log
• Agent smoke audit: /root/eva-brain-upgrade-golden-20260706T090613Z-agent-smoke-default.log

Runtime checks:

• gbrain doctor --scope=brain --json: healthy, 100/100 after source-scoped gbrain extract --stale --source-id openclaw-support-kb
• Schema: version 122, schema stats sees 1099 typed pages
• Support KB: 1097 pages, 9384 chunks, 100% embedded, last commit 754beca5..., search passes
• Workspace docs: 2 pages, 100% embedded, AGENTS guidance present
• OpenClaw plugin: loaded, version 0.42.56.5, GBrain tools exposed
• Gateway: restarted and active

Non-release blocker:

• OpenClaw agent turn cannot complete on Golden because gateway reports scope upgrade pending, then embedded fallback reports missing/expired OpenAI auth for main.
• Classified as auth-not-configured/scope-approval-needed, not an Eva Brain/GBrain install failure.

Next: proceed to Eric as the first real customer canary.

[100yenadmin]

Eric canary update — rollout stopped on auth gate

Status: Eva Brain runtime PASS; required agent-turn GBrain smoke BLOCKED by expired OpenAI OAuth.

Evidence:

• Pre-upgrade version: gbrain 0.42.47.8
• Upgraded to: gbrain 0.42.56.5
• Backup: /root/eva-brain-backups/eric-wilder-20260706T090808Z
• Main audit: /root/eva-brain-upgrade-eric-wilder-20260706T090808Z.log
• Extraction repair audit: /root/eva-brain-upgrade-eric-wilder-20260706T091144Z-extract-repair.log
• Agent smoke audit: /root/eva-brain-upgrade-eric-wilder-20260706T091216Z-agent-smoke-default.log
• Auth status evidence: /Volumes/LEXAR/Codex/evidence/eva-brain-fleet-rollout-0.42.56.5/eric-wilder-auth-status-20260706T0913Z.log

Runtime checks:

• gbrain doctor --scope=brain --json: healthy, 100/100 after source-scoped extraction repair
• Schema: version 122
• Support KB: 1097 pages, 9375 chunks, 100% embedded, last commit 754beca5..., search passes
• Workspace docs: 99 pages, 100% embedded, search passes
• OpenClaw plugin: loaded, version 0.42.56.5, GBrain tools exposed
• Gateway: restarted and active

Stop condition:

• openclaw agent --agent main ... cannot run because OpenAI OAuth refresh fails with 401.
• openclaw models auth list shows openai:eric@wildertreecompany.com expired 2026-07-04T04:52:58.978Z.
• openclaw models list shows OpenAI models configured but Local Auth: no.

Decision:

• Stop fleet rollout before Jackie/remaining VMs. Eva Brain itself is healthy on Golden and Eric, but customer-ready proof requires an OpenClaw/Hermes agent turn that can actually call gbrain_search or gbrain_query.
• Next safe action is an OpenClaw auth refresh/repair lane for Eric, then rerun the agent GBrain-call smoke. After Eric passes, resume with Jackie.

[100yenadmin]

Eva Brain fleet rollout summary - eva-v0.42.56.5

Release: https://github.com/electricsheephq/eva-brain/releases/tag/eva-v0.42.56.5

Evidence root: /Volumes/LEXAR/Codex/evidence/eva-brain-fleet-rollout-0.42.56.5

Completed / green

VM	Version	Doctor	Sources / search	Plugin	Agent smoke	Notes
golden	0.42.56.5	100/100	Support KB searchable; workspace-docs searchable	loaded 0.42.56.5	blocked by local auth/scope	Golden runtime is healthy; model auth/scope remains separate.
jackie-david	0.42.56.5	100/100	Support KB 1097 pages; workspace-docs 28 pages; both embedded/searchable	loaded 0.42.56.5	pass via shell-backed gbrain search	Public readyz timeout was expected UFW/control-plane allowlist behavior; local gateway active.
benjamin-kennedy	0.42.56.5	100/100	Support KB 1097 pages; workspace-docs 10 pages; both embedded/searchable	loaded 0.42.56.5	pass via gbrain_search	No runtime blockers.
david-poku	0.42.56.5	100/100	Support KB 1097 pages; workspace-docs 51 pages; both embedded/searchable	loaded 0.42.56.5	pass via gbrain_search	No runtime blockers.
evaos-support	0.42.56.5	100/100	Support KB 1097 pages; workspace-docs 3 pages; searchable with content-aware query	loaded 0.42.56.5	pass via shell-backed gbrain search	Workspace has docs but no runbook docs; duplicate lossless-claw config warning is unrelated follow-up.
holly-holmberg	0.42.56.5	100/100	Support KB 1097 pages; workspace-docs 18 pages; both embedded/searchable	loaded 0.42.56.5	pass via gbrain_search	AGENTS guidance was absent before update and is now present. Duplicate lossless-claw warning is unrelated follow-up.
matt-bailey	0.42.56.5	100/100	Support KB 1101 pages; workspace-docs 65 pages; both embedded/searchable	loaded 0.42.56.5	pass via shell-backed gbrain search	Missing desktop runtime env warnings are unrelated to GBrain.
tarzan-sharif	0.42.56.5	100/100	Support KB 1097 pages; workspace-docs 6 pages; both embedded/searchable	loaded 0.42.56.5	pass via shell-backed gbrain search	Legacy plugin index warning is unrelated to GBrain runtime.

Intentionally deferred

VM	Current state	Reason
eric-wilder	Package upgraded to 0.42.56.5; doctor 100/100; Support KB/workspace-docs searchable; plugin loaded	User asked not to touch Eric further. Agent smoke is blocked by expired OpenAI OAuth profile (openai:eric@wildertreecompany.com, expired 2026-07-04).

What actually happened

The recurring post-upgrade issue was not database corruption and not missing embeddings. The updater refreshed the Support KB and created new chunks, leaving links_extraction_lag around 36-44% on most VMs. That made doctor report 95/100 even though embeddings were 100% and search worked.

Safe repair used on each affected VM:

gbrain extract --stale --source-id openclaw-support-kb
gbrain extract --stale --source-id workspace-docs

After that, doctor returned to 100/100. No PGLite rebuild was performed.

Follow-ups recommended

1. Add an Eva Brain updater health/repair mode that runs source-scoped extract --stale after large Support KB or workspace-docs updates, or clearly tells operators to run it before expecting 100/100 doctor.
2. Fix/clean duplicate lossless-claw plugin entries on evaos-support and holly-holmberg; this is unrelated to GBrain but noisy.
3. Decide whether to refresh Eric's OpenAI OAuth profile in the next turn.
4. Improve OpenClaw agent tool preference so it chooses named gbrain_search more consistently instead of falling back to shell gbrain search.