Skip to content

[Dream Cycle 2026-08-14] developer-experience: ledger signals' zeroMergeStreak is an unverified worst-case default + cli,tui scan #8

Description

@ruvnet

Rotation

DEEP=developer-experience, SCAN=cli,tui (slot 4, DAYINT % 5). No bonus deep dive triggered (DAYINT % 25 = 14, % 75 = 14).

Ledger Check

docs/dream-cycle/LEDGER.md has 1 real row (2026-08-13; demo seed rows were removed that night, see #6). dream-machine ledger signals (no --merged) reports zeroMergeStreak=true. Cross-checked against real GitHub state via mcp__github__pull_request_read/list_pull_requests: PR #7 (the only PR referenced) is in fact merged: true, merged_by: ruvnet, merged_at: 2026-08-13T22:05:44Z, and issue #6 is closed (state_reason: completed). That mismatch — the field's documented meaning ("no candidate PR merged") vs. real state — is tonight's deep dive, not a side note.

Deep Dive

Reproduced (grade A, first-hand, not sourced): node packages/cli/dist/bin.js ledger signals against this repo's real ledger returns zeroMergeStreak: true unconditionally whenever the trailing window contains a real PR reference. Root cause: packages/cli/src/index.ts's ledger signals handler calls learningSignals(rows) with no options, so packages/ledger/src/index.ts's opts.mergedPrNumbers is always undefined, forcing !merged || prsInWindow.every(...) to short-circuit true. The @dream-machine/ledger library itself is correct (already accepts mergedPrNumbers, already unit-tested for it) — the defect is CLI wiring only. renderDashboard/TUI has the identical gap but was deliberately left unfixed tonight (see PR body's Reward-Hack Check).

Also found live tonight (SCAN=tui, not fixed): tui.ts's pad() measures width with .length after stripping ANSI, not terminal display width — a ledger row with CJK/emoji text in Finding breaks the dashboard's box alignment (reproduced: an 80-column box renders an 85-column row, dropping the right border). Latent today (no existing row has non-ASCII content); flagged as a follow-up, not built.

Hypothesis

Given zeroMergeStreak, which this pipeline's own compiled STEP 1.1 uses to bias candidate selection, when the CLI is given an optional --merged "7,12" flag threading known-merged PR numbers into the existing learningSignals(rows, { mergedPrNumbers }) option, then the signal should correctly report false once the window's PR is in the supplied set, and must be byte-identical to today when --merged is omitted — subject to: no existing test changes, no change to the library's own behavior, no new network/exec surface in the CLI, and a malformed/value-less --merged must fail with a clear usage error, not a crash.

Frozen before implementation; not modified after evaluation began.

Evaluation Receipt

Real evaluator: npm test (vitest), this repo's own bench entrypoint — the only evaluator entrypoint live tonight.

Baseline (parent, 7bbb851) Candidate
Tests 96 101 (+5, 0 removed, 0 modified)
Result 96 passed 101 passed
Lint clean clean

Live receipt against the real ledger, cross-checked against real GitHub state — full transcript in docs/dream-cycle/2026-08-14-developer-experience-report.md ("Benchmarks / Evaluation"). Summary: ledger signals (no flag) → zeroMergeStreak: true (wrong); --merged 7zeroMergeStreak: false (matches GitHub); --merged (value-less) → clean usage error, not a crash.

Credentials reality check: OPENROUTER_API_KEY is present tonight — LLM_EVAL is not blocked. This candidate makes no model calls regardless; it's verified entirely by the real test suite and a live GitHub API check.

Darwin Results

Not run — DARWIN=not-applicable. A single CLI flag surfacing an already-correct, already-tested library option has no meaningful mutable population for bounded Darwin's search.

Evidence

OBSERVATION / MEASUREMENT / INFERENCE / DECISION trail in full in the committed report. Key: npm test 96/96 → 101/101, 0 regressions; ledger signals --merged 7 flips zeroMergeStreak from true to false, matching mcp__github__pull_request_read's ground truth for PR #7.

Reward-Hack Check

Self-critique pass (no second agent spawned tonight — disclosed as a real limitation, not hidden; see report's Security Review). Verdict: CLEAR of reward-hacking — no test/gold/threshold touched, 2 files changed, default (no-flag) behavior unchanged (pre-existing signals test still passes unmodified).

Found and fixed before finalizing: the first draft of --merged crashed (mergedFlag.split is not a function) on a value-less invocation, because parseArgs returns boolean true for a flag with nothing after it. Fixed with an explicit type guard + clear usage error; regression test added.

Deliberately left out of scope, not silently dropped: (1) renderDashboard/TUI is documented as a pure, dependency-free ledgerMd → framebuffer function — giving it the same --merged-style fix is a second conceptual change and a design question (accept an option vs. take a precomputed signals object) not resolved unilaterally tonight; (2) the CLI does not fetch merge state itself (no gh/API call added to bin.ts) — that would add a network dependency and stack a second network-facing responsibility onto a tool whose io.exec already carries an unsanitized-shell-exec risk flagged in PR #7.

Security Review

No new exec/network/file-write surface — --merged is pure string parsing into a Set. No credentials touched. No LLM calls (N/A for prompt injection). The self-critique limitation (no independent second agent tonight) is disclosed above rather than glossed over.

Scan Findings

cliledger signals' zeroMergeStreak was a worst-case default masquerading as verified; fixed tonight (see Deep Dive).
tuipad()'s ANSI-stripped .length isn't terminal display width; CJK/emoji Finding text breaks box alignment. Reproduced live, not fixed (latent today, recommended as a future deep=developer-experience, scan=tui candidate).

Competitors

SWE-agent (C, secondhand — resolve-rate drift with no dashboard visibility, same failure shape), OpenHands (B, official docs — recommend human review over trusting dashboard metrics), Sakana AI Scientist (B, official docs — no documented steering-signal-from-stale-data risk), DSPy/GEPA (B — no documented equivalent gap), AutoGPT-lineage (C, secondhand — unverified self-reported success). This repo's own PR #7 is the strongest evidence tonight — first-party, not a competitor claim. Full table in the committed report.

Gist

No gh gist create/gist tool available this session (same as the prior night). Report committed instead at docs/dream-cycle/2026-08-14-developer-experience-report.md. GIST=LOCAL.

Witness

report_sha256 : f488881084bea3ac273fe8e62710533eb1c225040e763cdc8b0ae26ef975017c
session_commit: 7bbb851d337067d16ede31175896bb5e587a7d22
witness       : e55f641310751223e98c72c55a72388cedfe47dffa64c4f60166d027ded095e9

Verify: sha256sum docs/dream-cycle/2026-08-14-developer-experience-report.md then printf '%s%s' "<hash>" "7bbb851d337067d16ede31175896bb5e587a7d22" | sha256sum must equal the witness above. Independently confirmed tonight via dream-machine witness verify (✓ VALID).

Recommendation

evaluated: accepted — human review of the draft PR. Not done tonight, flagged as follow-ups: (1) fix tui.ts's pad() to use real display-width instead of .length; (2) decide whether renderDashboard should accept merge-state input or a precomputed signals object; (3) consider whether the compiled nightly prompt's STEP 1.1 should instruct future nights to always pass --merged from this session's own GitHub check.

Metadata

Metadata

Assignees

No one assigned

    Labels

    dream-cycleOutput of a nightly Dream Cycle runresearchResearch finding

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions