Releases: attenlabs/hotato
Release list
hotato 0.8.0
Fixed
hotato connectcrashed on Windows withAttributeError: module 'os' has no attribute 'fchmod'when storing credentials.os.fchmodis POSIX only;
tempfile.mkstempalready creates the file owner-scoped, so the call is
now guarded. Found the same hour by the new cross-OS CI matrix, on its
first run.
Added
- Fresh-capture provenance guard for
hotato fix trial: every run
envelope now records anaudio_provenanceblock per event (a streamed
sha256 of the exact audio bytes scored, plus sample rate and frame count;
additive,schema_versionstays"1").hotato fix trialcompares this
identity, before vs. after, for every fixture theimprovedclaim rests
on: an identical digest (the after run re-scored the SAME recording the
before run scored, just under a different threshold) downgrades the
verdict torefused(the same exit code3hotato apply's own refusal
uses); a digest missing on either side (an older envelope, or one built by
hand) downgrades toinconclusive, never assumed fresh. Distinct, known
digests proceed exactly as before. Closes the exact forgery path where the
same recording is rescored with a looser threshold and passed off as a
verified fix. Seedocs/FIX-TRIAL.md#fresh-capture-provenance-guard-a-re-score-is-never-a-fix
anddocs/RECAPTURE.md#how-hotato-tells-a-recapture-from-a-re-score. - Report-facing claim-language cautions:
hotato contract verifynow
prints, in every text and HTML render, "This result re-measures stored
evidence. It does not test the current agent."hotato fix trial's
audio-provenance section now prints a matching provenance/revision
caution wherever that section renders (animprovedverdict, or a
refused/inconclusiveone the fresh-capture guard itself downgraded).
Both strings are drawn from a new claim-language table -- which evidence
you have, what it accurately lets you say, and the common overclaim it
does not support, for five levels of evidence from a historical contract
alone up to a production rerun after deploy -- and new audio-handling
rules covering what raw call audio may contain, why redaction does not
remove spoken content, and an audio-free evidence summary to prefer when
sharing proof without the recording. See
docs/RECAPTURE.md#claim-language-what-each-kind-of-evidence-lets-you-honestly-say
andSECURITY.md#audio-handling.
hotato 0.7.2
Fixed
hotato contract packrefuses a bundle containing any symlink (file or
directory) instead of silently following it. A planted link could
previously ship bytes from OUTSIDE the bundle inside the archive (for
example a linked secret file). A packed bundle is now self-contained by
construction; copy real files into the bundle instead of linking them.
Found by external diligence probing the published packages.
hotato 0.7.1
Fixed
hotato contract unpack --forceno longer deletes the existing output
directory before the archive has proven valid. In 0.6.0 and 0.7.0, a
corrupt or hostile archive combined with--forcedestroyed the
directory the user asked to replace and then failed. The destination is
now touched only on the success path, after every guard and the full
extraction into the temporary directory have passed. Found by external
diligence against the published package; regression-tested with the
exact reproduction.
hotato 0.7.0
Added
hotato explain, root cause by layer without guessing: reads the
evidence a run already produced (score results, fix plans, trust reports,
contract bundles, attached voice traces) and emits an attribution per
failure: the likely layer (turn-taking taxonomy: failure layer, type,
confidence, fixability, opposite risk), the evidence for and against it,
the explicit unknowns (for example, no client-side playout trace attached),
and one safe next action. When the evidence cannot support an attribution
(an unlabeled candidate, a not-scorable input, conflicting signals) it
REFUSES with the reason instead of guessing. Text,--format json
(schemahotato.explain.v1), and HTML report output. Explain reports
timing evidence; it does not prove root cause and does not infer intent.hotato fix trial, one command from candidate fix to before/after
proof: composes the shipped primitives (apply --clone, the verify
battery, contract verification, and the opposite-risk guardrails) into a
single fail-closed trial: baseline on the recorded failure, apply the
candidate change in a clone (production config is never touched), re-run,
and check neighbouring and opposite-risk cases. Verdicts: improved
(exit 0), regressed (exit 1, forced by ANY fixture or opposite-risk
regression even when everything else improved), inconclusive (exit 3,
refused rather than softened). The report embeds the explain attribution
for the failure under trial.- Operator-grade capture depth for LiveKit and Pipecat in the starter
templates and docs: where turn-taking configuration lives in each stack
and how to capture per-party audio for scoring.
Fixed
hotato contract unpacknow treats every archive as hostile input:
rejects path traversal (including backslash and drive-letter forms),
symlinked and encrypted members, duplicate member names, members not
declared in the manifest, oversized decompression (512 MiB default,
--max-bytes/HOTATO_CONTRACT_MAX_UNPACK_BYTESoverride, enforced
against actual streamed bytes, not just declared metadata), and
compression-ratio bombs. Extraction stays atomic: a refused archive
leaves nothing behind.packcross-platform byte determinism is now
explicit rather than incidental.
hotato 0.6.0
Added
hotato contract create/verify/inspect/pack/unpack, the portable failure
contract: turns one real call moment into a self-contained<id>.hotato/
bundle --contract.json(schemahotato.contract.v1), the (clipped) audio,
frame-level timing evidence, an input-health (trust) report, a shareable SVG
card, a CI policy, and the exact replay/CI commands.createwraps the SAME
round-trip scorability guaranteefixture creategives (via
--from-candidate FILE#N,--stereo,--caller+--agent, or the opt-in
--mono --diarizepath): a not-scorable moment or a mono recording is
refused with the honest reason (exit 2) and no bundle is written; the
diarized-mono path never silently upgrades an indicative-only verdict, and
honestly reports frame-level evidence as unavailable rather than fabricating
it.verifyre-scores a contracts directory (or one bundle) against each
contract's own recorded policy and is the CI gate: exit 0 every contract
passes, 1 a regression, 2 a usage error; emits text/JSON/HTML/JUnit.
inspectprints one contract.pack/unpackround-trip a bundle through
one deterministic single-file.hotato.packarchive with a sha256 manifest
checked on unpack (a corrupted or tampered archive is refused, exit 2,
nothing partial left behind).hotato cardalso renders a contract
directly (kindvoice-turn-taking-contract). Redacted by default (a
candidate ref / source recording name is hidden unless
--include-identifiers); no call ids, paths, or transcript text in any
artifact. New modulehotato.contract, new schema
schema/contract.v1.json, docsdocs/CONTRACTS.md.hotato trace ingest/attach/export, the voice-trace observability
bridge:ingest --otel FILE --out voice_trace.jsonlparses either a
standard OTel JSON export (resourceSpans, best-effort span/event
flattening) or hotato's own documented OTel bridge JSONL into schema
hotato.voice_trace.v1(caller/agent audio activity, TTS cancel/stop, ASR
partials, tool calls, ...).attach BUNDLE --trace voice_trace.jsonl
writes the trace into<bundle>/traces/voice_trace.jsonland re-renders
evidence/timeline.htmlwith the trace's events drawn as a scale-aligned
row, reading the bundle's OWNevidence/frames.jsonlandcontract.json
back in -- it never re-runs the VAD or diarizer, so it works on a
diarized-mono bundle (no frame-level evidence) without the diarization
extra installed, honestly noting the missing base timeline instead of
fabricating one.export BUNDLE --format otel --out FILEwrites the
attached trace back out as the same bridge JSONLingestreads, so
ingest -> attach -> export -> ingestround-trips the identical spans.
The evidence page states findings plainly -- "Evidence suggests TTS
cancellation delay: cancel requested at 2.60s, audio stopped at 2.90s
(delta 0.30s)." -- always followed by "Hotato does not prove root cause."
and an explicit "Unknowns: no client-side playout trace was attached."
line. Redacted by default (call id, agent id, ASR transcript text). New
modulehotato.trace, new schemaschema/voice_trace.v1.json, docs
docs/TRACE.mdanddocs/OTEL.md.hotato run --mono call.wav --diarize, the opt-in, quality-gated
mono-scorability front-end: a single-channel (mixed) recording -- until now
the hard coverage wall, rejected as not scorable -- becomes scorable by running
an off-the-shelf speaker diarizer over the mono to recover who was active when,
reconstructing two caller/agent tracks, and feeding the EXISTING dual-channel
scorer (zero engine edit). The two-channel path stays the gold reference; this
widens coverage, honestly labeled. A pluggable diarizer-backend seam
(mirroring the neural-VAD seam) ships three backends behind--diarizer:
pyannote(local, offline, CPU-viable, default;[diarize]),sortformer
(NVIDIA NeMo, local/GPU, best self-hostable on telephone;
[diarize-sortformer]), andpyannoteai(HOSTED, best absolute, requires
--egress-opt-in;[diarize-hosted]). A real per-file confidence gate
(extendingtrust.py's scorability model) reads six signals -- speaker count,
per-speaker activity, mean segmentation posterior, embedding cluster-separation
margin, overlap ratio, segment churn -- into aseparation_confidenceand a
tier:highscores normally (labeleddiarized-mono),lowscores but stamps
indicative_onlyand fires NO SLA gate,refuseis not scorable (exit 2).
Caller/agent assignment reuses the floor-dominance heuristic to PROPOSE a
mapping stated as an assumption, overridable with--caller-speaker/
--agent-speaker; a balanced mapping downgrades to indicative. On this path
signals.echo/ crosstalk is definitionally N/A (two slices of one microphone)
and the echo gate never fires.hotato trust --stereo mono.wav --diarize
reports the separability tier (high/low/refuse) WITHOUT scoring. No silent
fallback anywhere: a missing extra / token / model raises a clean
BackendUnavailable(exit 2) and NEVER scores raw mono. Honesty invariants:
the default path (no--mono/--diarize) stays byte-identical (a mono file is
still rejected as today), and a diarized-mono verdict is never presented as
equivalent to a true dual-channel measurement. New modulehotato.diarize
(DiarizationResult, the backend registry,reconstruct_tracks,
separation_confidence,assign_speakers,prepare_diarized_mono, the
pyannote/sortformer/pyannoteai adapters, and a hermetic stub backend); new
--mono/--diarize/--diarizer/--caller-speaker/--agent-speaker/
--egress-opt-inflags onhotato runand--diarize/--diarizeronhotato trust; documented indocs/DIARIZE.md. TheLIMITS.does_not_do/
METHODOLOGY.md"no diarization" framing is superseded: two channels is the
gold reference; mono is scorable via the opt-in, quality-gated front-end,
labeled indicative below the bar. Speaker IDENTIFICATION is still out of scope
(a diarizer assigns anonymous SPEAKER_00/01; it never says who a person is).- New optional extras
[diarize],[diarize-sortformer],[diarize-hosted]
(the[diarize]path raises the effective Python floor to >=3.10; the stdlib
core stays >=3.9). Dependency licenses are logged indocs/DIARIZE.mdand
carried in the envelope'sdiarization.licensesblock (pyannote-audio MIT;
community-1 weights CC-BY-4.0; Sortformer streaming v2 CC-BY-4.0 -- the offline
v1 is CC-BY-NC and is never shipped). hotato card INPUT[#REF] --out card.svg, a shareable card from any hotato
result: renders a DETERMINISTIC, stdlib-only, 1200x630 SVG with NO external
resource (no font, image, stylesheet, script, or link; all color inline), so
it drops straight into a PR, an issue, or a slide and looks the same forever.
Four kinds are auto-detected: a talk-over candidate and a false-stop
candidate (from asweep/analyzecandidate refFILE#N), the
threshold-funnel fix plan (decision: do_not_tune_single_threshold-- the
hero card: "NO SINGLE THRESHOLD CAN FIX THIS",fix class: engagement-control),
and a supported verify rollup ("FIX VERIFIED WITHOUT BREAKING
BACKCHANNELS"). Every card names the MEASURED timing moment and never a verdict
about intent, and carries no accuracy number. Redacted by default: a call
id, a filesystem path (only a basename is ever shown), and a vendor recording
name are hidden unless--include-identifiers. Exit 0 written / 2 bad input.
New modulehotato.card; three commit-ready assets under
docs/assets/cards/regenerated byscripts/render_card_assets.py; documented
indocs/CARDS.md.hotato start --demo, the guided, credential-less first run: one command,
no account, no network. It sweeps the two bundled real demo calls, writes the
sweep result (hotato-sweep.json), a self-contained HTML dashboard
(hotato-sweep.html), and the threshold-funnel card
(hotato-no-single-threshold.svg), then turns one real missed-interruption
candidate into a demo failure contract (contracts/demo-missed-interruption.hotato,
--expect yield) and verifies it immediately -- it genuinely fails, so the
loop is visible end to end in one command: a real failure becomes a
candidate, becomes a portable contract, andcontract verifycatches it.
It then prints the exact next commands: promote a candidate into a
permanent fixture, run those fixtures in CI, re-verify the demo contract,
and render a card. The--stack/--folder/--stereomodes are
placeholders in this build and route tohotato sweep/analyze/run.
Exit 0 done / 2 usage. New modulehotato.start; documented in
docs/START.md.
Fixed
hotato.diarizepyannote 4.x compatibility:Pipeline.from_pretrained
now tries thetoken=kwarg first and falls back to the removed
use_auth_token=name onTypeError, and a new_unpack_pipeline_output
branches on theDiarizeOutputobject pyannote.audio >=4.0 returns from a
pipeline call (previously unpacked as a 3.x(Annotation, embeddings)
tuple, raisingTypeError/AttributeErroragainst a 4.x install) so both
pyannote 3.x and 4.x load and score cleanly._embedding_marginno longer
divides by a zero norm and fabricatescos = 0(read by the confidence
gate as adequate separation) for a degenerate (zero-norm / non-finite)
speaker centroid; it now returnsNone-- "no margin available" -- the
same honest no-signal result a missing embeddings array already gave.- New yield-boundary confidence gate signal (
signals.yield_boundary):
benchmarked against a real pyannote community-1 backend over the AMI
corpus (in-repo harness:tools/bench_diarize/, dev-only, never shipped),
the existing six diarization-quality signals measured clean, well-separated,
stable speakers but were anti-cor...
hotato 0.5.0
hotato 0.5.0 - 2026-07-09
This release turns Hotato from a set of scoring commands into a passive regression
loop: connect a stack once, sweep recent real calls for candidate turn-taking
moments, promote the confirmed ones into fixtures, and prove a fix without an
unsafe threshold bandaid.
Added
hotato sweep --demoruns the full sweep over two bundled real calls with no
stack, no credentials, and no network, so the first sweep works before anything
is connected.hotato fixture promote FILE#N --expect yield|holdturns one sweep or analyze
candidate into a permanent regression fixture. The analyze and sweep HTML
reports now carry per-candidate copy-command promote and hold actions.hotato init webhook --stack vapi|retell|twilioscaffolds a self-hostable
ingestion worker that verifies the webhook, fetches the recording read-only,
runshotato ingest, and optionally posts a Slack or GitHub notification. By
construction it never mutates platform config and never labels intent.hotato issue createandhotato pr createrender a GitHub issue or a
fixtures PR from a sweep run. Both default to a dry run that prints the body and
the exact commands they would run, and touch a repo only with--repoand
--yes.pr createalways uses a feature branch and never force-pushes.hotato verify --out verify.htmlwrites a self-contained before and after proof
page, and--policy hotato.verify.yamlenforces guardrails as hard fail
conditions. A run cannot pass by improving one axis while regressing the other,
and a battery missing a hold or yield fixture fails the opposite-risk check.hotato apply PATCH_JSON --cloneapplies a patch only to a fresh staging
assistant, never the production source. It refuses a threshold funnel outright
and recommends engagement-control instead. There is no production-apply path.hotato trust --stereo call.wavreports whether a recording is even scorable
before you score it (channel swap, mono, sample rate, clipping, silence,
crosstalk, scorability), so a bad export is caught up front.
Fixed
hotato.__version__and thedescribeand--versionbanners are now covered
by a lockstep test, so a stale version string cannot ship again.llms-full.txtis built from the top-level docs only and is byte-reproducible
across a git checkout and an extracted sdist.
See the full entry list in CHANGELOG.md.
hotato 0.4.1
Hotato 0.4.1
Released 2026-07-08. Patch release; scores, envelopes, and reports are identical to 0.4.0.
Fixed
hotato --version, thehotato describemanifest, and stackbench provenance reported "0.3.1" from the 0.4.0 wheel. The 0.4.0 bump missed the__version__literal insrc/hotato/__init__.py, so every runtime self-report carried the previous release's number. Only the reported string was wrong; no score or envelope was affected.- The gap is now closed structurally:
tests/test_version_lockstep.pyanchorshotato.__version__, thedescribemanifest, and the installed distribution metadata againstpyproject.toml, and the release checklist enumerates every version lockstep site. The previous test compared the manifest to__version__itself, which agreed even when both were wrong.
Upgrade
pip install -U hotato
# or zero-install
uvx hotato --versionhotato 0.4.0
Hotato 0.4.0
Released 2026-07-08.
0.4.0 turns Hotato from a one-call scorer into a monitoring loop: connect a stack once, pull recent recordings, rank the candidate turn-taking moments across them, then carry a labeled fix to a paste-ready patch and a before/after proof.
Highlights
hotato connect/pull/sweep.connect <stack>runs a live auth check and stores credentials locally at~/.hotato/connections.json(mode 0600), sent nowhere but the vendor's own API.pullbulk-fetches recent recordings through each platform's verified list endpoint.sweeprunspulland then the zero-configanalyzeover the pulled folder, writing one self-contained offline dashboard of ranked candidate moments. Only verified endpoints are used; Retell pulls by explicit--call-id, and that limitation is documented rather than papered over.hotato analyze <folder>, zero-config folder analysis. Point it at a folder of dual-channel recordings and it ranks candidate turn-taking moments across all of them by measured salience, no scenarios, labels, onsets, or flags. Top moments embed the real audio inline with a playhead synced to the timeline, so you press play and hear the bug land where the chart marks it.--format jsonemits the ranked candidates for an agent to drive. Everything is framed as measured candidates you review and label; never a pass/fail, a failure count, or an intent claim.hotato patch/verify/loop, the closed loop.patch <fixplan.json>renders a fix plan into a literal, paste-ready artifact per platform: a JSON merge-patch body plus a ready curl for the platform's real config endpoint, or the exact constructor-kwarg source edit where config lives in agent code. It produces the change and never applies it.verify --before --afterreports battery-scale movement ("N of M fixtures that used to fail now pass, K of L hold fixtures still pass"), refuses the headline claim under low n, and reports coincidence, never causation.looporchestrates the whole cycle with state across runs and never auto-labels or auto-applies.hotato ingest, the passive on-ramp. Wire a platform webhook to invoke it and each completed call is fetched and scanned for candidate moments automatically. Discovery only: webhook payloads are treated strictly as untrusted data, exit 0 means it ran, and nothing is auto-labeled, auto-fixtured, or auto-tuned. Per-stack recipes indocs/INGEST.md.hotato describe, the generated capability manifest. One call (json or text) built from the live parser: every subcommand, argument, and exit code, so an agent learns the whole CLI without scraping--helpacross 18 subparsers. Exit-code epilogs are now uniform across every subcommand, anddoctor --format jsonemits the machine envelope.- Five more stacks behind
--allow-mono. Bland, ElevenLabs, Synthflow, Millis, and Cartesia capture and pull with exact verified endpoints. These platforms record one combined track, so scoring is degraded, gated behind the flag, and labeled indicative only.
Changed
hotato demoscores two real recorded calls. The packaged battery replaces synthetic audio with two operator-recorded probe calls against a scripted assistant on a provider's default interruption settings: a missed interruption (0.25 s talk-over) and a false stop on a soft backchannel. Full provenance and attestation ride in the scenario metadata, the only human voice is the recording operator's own, and two runs remain byte-identical.
Fixed
- Atomic writes for recording downloads, the sweep dashboard, and every
--out, so an interrupted run cannot truncate a previously good artifact. - Every JSON emitter refuses NaN and Infinity; a non-finite value surfaces as a clean exit 2, never invalid JSON.
.hotato/loop-state.jsonis validated on load; an incomplete or hand-edited state file is a clean error, never a KeyError.- Malformed WAV headers, identical caller/agent channels, invalid onsets, and bad flags are refused up front with plain reasons, and one bad file never aborts an
analyzeorloopbatch. - Wording in
docs/WHY.mdandREADME.mdsoftened where a frequency claim had no source.
Security
- Default-deny SSRF blocking on every vendor download URL: loopback, private, link-local, and reserved hosts are refused, re-checked on every redirect, and credential headers are stripped on cross-host redirects.
HOTATO_ALLOW_PRIVATE_URLSis the explicit opt-out. - MCP server input paths are sandboxed, mirroring the existing report-path sandbox.
ingestlocal recording paths requireHOTATO_INGEST_DIRand fail closed, so a forged payload cannot point the scanner at an arbitrary file.- Scenario ids are validated as safe path segments, closing a crafted-pack path that could read or exfiltrate an arbitrary local WAV.
CI and docs
- A new sdist guard job builds the sdist, installs only the extracted tree into a fresh venv, and runs the full suite from inside it, catching the green-wheel-broken-sdist failure mode that shipped unguarded in 0.2.3 and 0.3.0.
server.jsonMCP registry manifest,docs/MCP.mdwith copy-paste client configs,docs/CONNECT.md, an honest rewrite ofdocs/ADAPTER-STATUS.md,llms.txtreconciled to the real command surface, the new concatenatedllms-full.txt, andCITATION.cff.
v0.3.1
Fixed
- Self-consistent source distribution: the sdist now ships the small,
non-audio test dependencies (scenario labels, manifests, the corpus
validator, and the deterministic builders undercorpus/andexamples/),
so an extracted sdist collects and runs the full test suite instead of
hitting collection errors. The heavy real and rendered audio stays pruned;
suite and class audio is regenerated deterministically by
tests/conftest.py(seed = sha256(id)) when absent, and the tests that
depend on genuinely-absent heavy real audio skip cleanly rather than error. - Honest fix-plan wording in
README.mdanddocs/WHY.md: reworded the
claim that every failing event returns a fix that "names the exact setting to
change in your stack." Aplan()may correctly refuse to tune a single
threshold, report insufficient coverage, or emit a checklist, so the copy now
says a fix class is always returned and, when the failure maps cleanly to
stack config, the setting family and direction to investigate.
Added
hotato scanself-truncation candidate: a new candidate kind,
agent_stop_no_caller, surfaces the agent going from active to quiet with
zero caller energy anywhere nearby, a drop nothing on the caller channel
explains (not a barge-in, not a caller-driven handoff). Additive; existing
candidate kinds and scoring are unchanged.- Response-gap percentiles and a latency SLA gate:
hotato teamand
hotato exportnow also report p95 (alongside the existing mean/median/p90)
for talk-over and time-to-yield, and poolresponse_gap_sec(dead air
before the agent speaks) into the same distribution shape.--max-response-gap SECONDSon both commands gates the pooled p95 and fails (exit 1) exactly
when it is exceeded, the same pass/fail contract as--max-talk-over/
--max-time-to-yield. A plain export'senvelope.jsonstays byte-identical
to before; the new numbers live only in the printed summary and the
returned manifest. - Echo detector: a deterministic cross-channel coherence signal
(signals.echo:coherence,lag_sec,echo_suspected) on every scored
event, flagging when the caller channel looks like a lag-shifted copy of
the agent's own audio (leaked TTS), so a stop the agent makes because it
heard itself is no longer indistinguishable from a clean yield. Newscan
candidate kindecho_correlated_activity; a loud WARNING inhotato diagnoseand the single-run text output for every echo-suspected yield;
opt-in--echo-gateonhotato runholds a bleed-induced yield out of the
verdict (scorable: false) instead of counting it as a pass. Default run
behavior, and every existing golden verdict, is unchanged;--echo-gateis
off by default. - Resume/restart detector: on events where the agent yielded, an additive
signals.resumeblock (resumed,resume_gap_sec,restart_suspected)
measures from the agent's own VAD track whether it came back after the
yield, how fast, and whether the post-resume run is long enough to look
like a restart-from-the-top rather than a short continuation. Whether the
resumed words repeat the earlier ones is out of scope (a transcript
question, not a timing one). - Four corpus scenario classes under
corpus/classes/(deterministic,
additive, same seeded-render/--checkcontract ascorpus/suites/):
mid-utterance-pause,backchannel-multilingual,noise-hold, and
telephony-degraded. Detail:corpus/classes/README.md. - "Is this even a turn-taking bug?" triage in
docs/WHY.mdandREADME.md:
names the failure modes commonly conflated with turn-taking bugs (STT
hallucination, client-side audio buffering, LLM verbosity/tool-selection,
safety false-refusal, wrong-language STT) and which tool class to reach for
instead, alongside a plain statement of the flagship case Hotato covers
(agent-talks-over-caller and false-stop-on-backchannel). - Report accessibility: the generated HTML report carries aria-labels on
its inline SVGs and audio elements, amainlandmark, and measured titles,
so the artifact is navigable by assistive tech. Visual output is unchanged. - Startup lazy-imports:
importlib.resources, the report renderer, and the
numpy accelerator used only insidescanare deferred to first use, so plain
CLI paths import less at startup. Behavior and output are unchanged.
All of the above are additive optional fields or opt-in flags: did_yield
and passed for every existing fixture are unchanged, and the vendored
src/hotato/_engine is untouched (verify with python3 sync_engine.py --check).
v0.3.0
[0.3.0] - 2026-07-06
Added
hotato fixture create: the missing piece of the regression loop. One
bad call moment (a recording, an--onset, and YOUR--expect yield|hold
label) becomes a permanent fixture:scenarios/<id>.jsonin the existing
scenario schema shape (with provenance: source file, original onset,
created_by) plus a two-channelaudio/<id>.example.wav. Clips around the
event by default (--pre2.0 s /--post6.0 s) and re-bases the onset to
the clip;--no-clipkeeps the full recording. The created fixture is
validated by scoring it through the suite runner immediately; a not-scorable
input is refused with the honest reason (exit 2) and partial outputs are
removed unless--force. Overwrite refused without--force. Round-trips
throughhotato run --scenarios DIR --audio DIRas written.hotato compare: the shareable before/after on one fixed moment. Scores
both takes with the identical expectation, bounds, and reference config and
reports the per-signal movement plus one machine-stable result word:
fixed,regressed,improved,worse,unchanged,still_pass, or
not_scorable. Marks come from real measurements only; a not-scorable side
renders NOT SCORABLE (exit 2), never an invented verdict.
--before-onset/--after-onsethandle the moment shifting between takes;
--out report.htmlwrites the HTML report with the before take as the base
comparison;--fail-on-worseexits 1 onregressed/worse.hotato scan: candidate extraction across a WHOLE recording, as timing
facts only. Walks the two VAD activity tracks (windowed pass, so long files
never load fully into memory) and surfacesoverlap_while_agent_talking
(with overlap length and whether/when the agent went silent),
agent_start_during_caller, andlong_response_gapcandidates, sorted by
salience with--top(default 20). No intent claims anywhere: the output
header states that you decide the expected behavior and label the moment
withhotato fixture create.hotato diagnose/hotato inspect/hotato plan: the guarded,
read-only fix ladder (landed on main unreleased; first release here).
Diagnose explains a finished run per failing event plus a battery-level
decision; inspect reads and normalizes the live turn-taking config (GET or
static parse, never executed); plan combines them into a
hotato.fixplan.v1proposal: at most one bounded step on one setting,
refusal on the threshold funnel, checklist on ambiguous evidence,
insufficient_coveragewithout a passing opposite-risk fixture, and
production_applypinned to false.hotato planalso accepts the result JSON as a positional argument
(hotato plan result.json), and every plan now carrieskind: "fix-plan",
aplatform_mutationblock (performedalways false), the measured
evidence,risks,next_commands(including thehotato compare
verification step), and not-scorable events asinput_issues. New Twilio
rule:--stack twilionever yields agent-config advice; the plan points at
channel assignment and the upstream voice-agent stack.docs/BAD-CALL-TO-CI.md: the five-step loop from one bad call to a CI
gate, with the label semantics up top and explicit use/do-not-use lists;
runnable walkthrough inexamples/bad-call-to-ci/.
Changed
- Label wording, stated wherever the backchannel story is told (README,
METHODOLOGY, WHY, and therun/doctor/demo/fixture createhelp):
Hotato does not infer intent. You label the expected behavior for the
event: yield means the agent should stop for the caller. hold means the
agent should keep speaking through a backchannel/noise/acknowledgement.
Hotato then measures whether the timing matched that label. - README positioning line sharpened: Hotato turns bad voice-agent call
moments into offline regression tests. - Input wording unified: Hotato's main scorer requires separated caller and
agent tracks; a single mixed mono call is not enough to attribute talk-over
reliably.