Skip to content

fix: bill a failed stage for the LLM spend it already incurred#209

Merged
alphacrack merged 1 commit into
mainfrom
fix/stage-fail-cost-103
Jul 22, 2026
Merged

fix: bill a failed stage for the LLM spend it already incurred#209
alphacrack merged 1 commit into
mainfrom
fix/stage-fail-cost-103

Conversation

@alphacrack

Copy link
Copy Markdown
Owner

Closes #103.

stage_fail recorded no cost, so a stage that paid for LLM calls and then raised contributed $0.00 to total_cost_usd — the runs where you most want a spend figure reported zero.

The proven loss path is the distiller's grounding retry. run_distiller accumulates total_cost across up to two complete_json calls and then raises DistillError if the output is still ungrounded (distill.py) — _stage_distill never reaches stage_complete, so both paid calls vanished from the manifest. That's the failure mode this project is built around, billed at zero.

Fix, in three small pieces:

  • Manifest.stage_fail(name, error, cost_usd=0.0, **meta) accumulates and recomputes the total exactly like stage_complete; the 0.0 default keeps costless failures byte-identical.
  • DistillError carries cost_usd, so the spend survives the raise.
  • The orchestrator's two failure handlers thread getattr(e, "cost_usd", 0.0) — any future exception can opt in without touching run() again.

Scope correction worth recording: the issue named the agent stage as the victim, but that path was already correct — _stage_normalize calls stage_complete(cost_usd=cost) before raising on blocked/failed/budget, so stage_fail only flips the status. Ingest raises only before paying. Tutorial's spend genuinely isn't recoverable (it dies inside complete_json, which returns cost only on success) and is deliberately untouched — that would need llm.py to expose partial cost, which is a different change.

Verified: 397 passed, ruff clean, mypy clean, coverage 87.53%. Revert-detection: reverting src/ fails exactly 3 of the new/extended tests, so they pin the behavior rather than describing it.

The grounding invariant — no change to what reaches published artifacts. distill.py's only edit is the exception's payload; the grounding validator, its retry, and the raise condition are untouched.

Note: #204 also touches manifest.py (adds a stage_duration helper) — different function, no overlap, but whichever lands second may want a glance.

🤖 Generated with Claude Code

stage_fail recorded no cost, so a stage that paid for LLM calls and then
raised contributed $0.00 to total_cost_usd. The proven loss path is the
distiller's grounding retry: run_distiller can raise DistillError after
TWO paid complete_json calls (distill.py), and _stage_distill never
reaches stage_complete, so the spend vanished.

stage_fail now takes cost_usd and accumulates it exactly like
stage_complete; DistillError carries the spend to the raise site; and the
orchestrator's failure handlers thread it via getattr(e, "cost_usd", 0.0)
so any future exception can opt in the same way.

Scope note: the issue described the agent path as the victim, but that
one was already correct — _stage_normalize calls stage_complete with the
agent's cost BEFORE raising on blocked/failed/budget, so stage_fail only
flips the status. Ingest raises only before paying. Tutorial's spend is
unrecoverable (it dies inside complete_json, which returns cost only on
success) and is left alone.

Closes #103.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@alphacrack
alphacrack merged commit be5ff15 into main Jul 22, 2026
6 checks passed
@alphacrack
alphacrack deleted the fix/stage-fail-cost-103 branch July 22, 2026 20:37
alphacrack added a commit that referenced this pull request Jul 23, 2026
The distiller's grounding retry can raise after two paid LLM calls;
stage_fail now bills that spend instead of reporting $0.00 (#209,
closes #103). Docs: README outputs/config example (#119) and the
verify -> tutorial -> render stage order in the architecture diagram
and debug-run skill (#211).

Version bumped in lockstep across pyproject.toml, CITATION.cff
(0.7.3 / 2026-07-23), and CHANGELOG.md.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

manifest: stage_fail never records cost_usd — failed runs under-report spend

1 participant