fix: bill a failed stage for the LLM spend it already incurred#209
Merged
Conversation
stage_fail recorded no cost, so a stage that paid for LLM calls and then raised contributed $0.00 to total_cost_usd. The proven loss path is the distiller's grounding retry: run_distiller can raise DistillError after TWO paid complete_json calls (distill.py), and _stage_distill never reaches stage_complete, so the spend vanished. stage_fail now takes cost_usd and accumulates it exactly like stage_complete; DistillError carries the spend to the raise site; and the orchestrator's failure handlers thread it via getattr(e, "cost_usd", 0.0) so any future exception can opt in the same way. Scope note: the issue described the agent path as the victim, but that one was already correct — _stage_normalize calls stage_complete with the agent's cost BEFORE raising on blocked/failed/budget, so stage_fail only flips the status. Ingest raises only before paying. Tutorial's spend is unrecoverable (it dies inside complete_json, which returns cost only on success) and is left alone. Closes #103. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
alphacrack
added a commit
that referenced
this pull request
Jul 23, 2026
The distiller's grounding retry can raise after two paid LLM calls; stage_fail now bills that spend instead of reporting $0.00 (#209, closes #103). Docs: README outputs/config example (#119) and the verify -> tutorial -> render stage order in the architecture diagram and debug-run skill (#211). Version bumped in lockstep across pyproject.toml, CITATION.cff (0.7.3 / 2026-07-23), and CHANGELOG.md. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #103.
stage_failrecorded no cost, so a stage that paid for LLM calls and then raised contributed $0.00 tototal_cost_usd— the runs where you most want a spend figure reported zero.The proven loss path is the distiller's grounding retry.
run_distilleraccumulatestotal_costacross up to twocomplete_jsoncalls and then raisesDistillErrorif the output is still ungrounded (distill.py) —_stage_distillnever reachesstage_complete, so both paid calls vanished from the manifest. That's the failure mode this project is built around, billed at zero.Fix, in three small pieces:
Manifest.stage_fail(name, error, cost_usd=0.0, **meta)accumulates and recomputes the total exactly likestage_complete; the 0.0 default keeps costless failures byte-identical.DistillErrorcarriescost_usd, so the spend survives the raise.getattr(e, "cost_usd", 0.0)— any future exception can opt in without touchingrun()again.Scope correction worth recording: the issue named the agent stage as the victim, but that path was already correct —
_stage_normalizecallsstage_complete(cost_usd=cost)before raising on blocked/failed/budget, sostage_failonly flips the status. Ingest raises only before paying. Tutorial's spend genuinely isn't recoverable (it dies insidecomplete_json, which returns cost only on success) and is deliberately untouched — that would needllm.pyto expose partial cost, which is a different change.Verified: 397 passed, ruff clean, mypy clean, coverage 87.53%. Revert-detection: reverting
src/fails exactly 3 of the new/extended tests, so they pin the behavior rather than describing it.The grounding invariant — no change to what reaches published artifacts.
distill.py's only edit is the exception's payload; the grounding validator, its retry, and the raise condition are untouched.Note: #204 also touches
manifest.py(adds astage_durationhelper) — different function, no overlap, but whichever lands second may want a glance.🤖 Generated with Claude Code