Skip to content

Releases: attenlabs/hotato

hotato 0.8.0

Choose a tag to compare

@quantumCF quantumCF released this 10 Jul 06:33

Fixed

  • hotato connect crashed on Windows with AttributeError: module 'os' has no attribute 'fchmod' when storing credentials. os.fchmod is POSIX only;
    tempfile.mkstemp already creates the file owner-scoped, so the call is
    now guarded. Found the same hour by the new cross-OS CI matrix, on its
    first run.

Added

  • Fresh-capture provenance guard for hotato fix trial: every run
    envelope now records an audio_provenance block per event (a streamed
    sha256 of the exact audio bytes scored, plus sample rate and frame count;
    additive, schema_version stays "1"). hotato fix trial compares this
    identity, before vs. after, for every fixture the improved claim rests
    on: an identical digest (the after run re-scored the SAME recording the
    before run scored, just under a different threshold) downgrades the
    verdict to refused (the same exit code 3 hotato apply's own refusal
    uses); a digest missing on either side (an older envelope, or one built by
    hand) downgrades to inconclusive, never assumed fresh. Distinct, known
    digests proceed exactly as before. Closes the exact forgery path where the
    same recording is rescored with a looser threshold and passed off as a
    verified fix. See docs/FIX-TRIAL.md#fresh-capture-provenance-guard-a-re-score-is-never-a-fix
    and docs/RECAPTURE.md#how-hotato-tells-a-recapture-from-a-re-score.
  • Report-facing claim-language cautions: hotato contract verify now
    prints, in every text and HTML render, "This result re-measures stored
    evidence. It does not test the current agent." hotato fix trial's
    audio-provenance section now prints a matching provenance/revision
    caution wherever that section renders (an improved verdict, or a
    refused/inconclusive one the fresh-capture guard itself downgraded).
    Both strings are drawn from a new claim-language table -- which evidence
    you have, what it accurately lets you say, and the common overclaim it
    does not support, for five levels of evidence from a historical contract
    alone up to a production rerun after deploy -- and new audio-handling
    rules covering what raw call audio may contain, why redaction does not
    remove spoken content, and an audio-free evidence summary to prefer when
    sharing proof without the recording. See
    docs/RECAPTURE.md#claim-language-what-each-kind-of-evidence-lets-you-honestly-say
    and SECURITY.md#audio-handling.

hotato 0.7.2

Choose a tag to compare

@quantumCF quantumCF released this 10 Jul 03:48

Fixed

  • hotato contract pack refuses a bundle containing any symlink (file or
    directory) instead of silently following it. A planted link could
    previously ship bytes from OUTSIDE the bundle inside the archive (for
    example a linked secret file). A packed bundle is now self-contained by
    construction; copy real files into the bundle instead of linking them.
    Found by external diligence probing the published packages.

hotato 0.7.1

Choose a tag to compare

@quantumCF quantumCF released this 10 Jul 02:58

Fixed

  • hotato contract unpack --force no longer deletes the existing output
    directory before the archive has proven valid. In 0.6.0 and 0.7.0, a
    corrupt or hostile archive combined with --force destroyed the
    directory the user asked to replace and then failed. The destination is
    now touched only on the success path, after every guard and the full
    extraction into the temporary directory have passed. Found by external
    diligence against the published package; regression-tested with the
    exact reproduction.

hotato 0.7.0

Choose a tag to compare

@quantumCF quantumCF released this 10 Jul 01:58

Added

  • hotato explain, root cause by layer without guessing: reads the
    evidence a run already produced (score results, fix plans, trust reports,
    contract bundles, attached voice traces) and emits an attribution per
    failure: the likely layer (turn-taking taxonomy: failure layer, type,
    confidence, fixability, opposite risk), the evidence for and against it,
    the explicit unknowns (for example, no client-side playout trace attached),
    and one safe next action. When the evidence cannot support an attribution
    (an unlabeled candidate, a not-scorable input, conflicting signals) it
    REFUSES with the reason instead of guessing. Text, --format json
    (schema hotato.explain.v1), and HTML report output. Explain reports
    timing evidence; it does not prove root cause and does not infer intent.
  • hotato fix trial, one command from candidate fix to before/after
    proof
    : composes the shipped primitives (apply --clone, the verify
    battery, contract verification, and the opposite-risk guardrails) into a
    single fail-closed trial: baseline on the recorded failure, apply the
    candidate change in a clone (production config is never touched), re-run,
    and check neighbouring and opposite-risk cases. Verdicts: improved
    (exit 0), regressed (exit 1, forced by ANY fixture or opposite-risk
    regression even when everything else improved), inconclusive (exit 3,
    refused rather than softened). The report embeds the explain attribution
    for the failure under trial.
  • Operator-grade capture depth for LiveKit and Pipecat in the starter
    templates and docs: where turn-taking configuration lives in each stack
    and how to capture per-party audio for scoring.

Fixed

  • hotato contract unpack now treats every archive as hostile input:
    rejects path traversal (including backslash and drive-letter forms),
    symlinked and encrypted members, duplicate member names, members not
    declared in the manifest, oversized decompression (512 MiB default,
    --max-bytes / HOTATO_CONTRACT_MAX_UNPACK_BYTES override, enforced
    against actual streamed bytes, not just declared metadata), and
    compression-ratio bombs. Extraction stays atomic: a refused archive
    leaves nothing behind. pack cross-platform byte determinism is now
    explicit rather than incidental.

hotato 0.6.0

Choose a tag to compare

@quantumCF quantumCF released this 09 Jul 22:16

Added

  • hotato contract create/verify/inspect/pack/unpack, the portable failure
    contract
    : turns one real call moment into a self-contained <id>.hotato/
    bundle -- contract.json (schema hotato.contract.v1), the (clipped) audio,
    frame-level timing evidence, an input-health (trust) report, a shareable SVG
    card, a CI policy, and the exact replay/CI commands. create wraps the SAME
    round-trip scorability guarantee fixture create gives (via
    --from-candidate FILE#N, --stereo, --caller+--agent, or the opt-in
    --mono --diarize path): a not-scorable moment or a mono recording is
    refused with the honest reason (exit 2) and no bundle is written; the
    diarized-mono path never silently upgrades an indicative-only verdict, and
    honestly reports frame-level evidence as unavailable rather than fabricating
    it. verify re-scores a contracts directory (or one bundle) against each
    contract's own recorded policy and is the CI gate: exit 0 every contract
    passes, 1 a regression, 2 a usage error; emits text/JSON/HTML/JUnit.
    inspect prints one contract. pack/unpack round-trip a bundle through
    one deterministic single-file .hotato.pack archive with a sha256 manifest
    checked on unpack (a corrupted or tampered archive is refused, exit 2,
    nothing partial left behind). hotato card also renders a contract
    directly (kind voice-turn-taking-contract). Redacted by default (a
    candidate ref / source recording name is hidden unless
    --include-identifiers); no call ids, paths, or transcript text in any
    artifact. New module hotato.contract, new schema
    schema/contract.v1.json, docs docs/CONTRACTS.md.
  • hotato trace ingest/attach/export, the voice-trace observability
    bridge
    : ingest --otel FILE --out voice_trace.jsonl parses either a
    standard OTel JSON export (resourceSpans, best-effort span/event
    flattening) or hotato's own documented OTel bridge JSONL into schema
    hotato.voice_trace.v1 (caller/agent audio activity, TTS cancel/stop, ASR
    partials, tool calls, ...). attach BUNDLE --trace voice_trace.jsonl
    writes the trace into <bundle>/traces/voice_trace.jsonl and re-renders
    evidence/timeline.html with the trace's events drawn as a scale-aligned
    row, reading the bundle's OWN evidence/frames.jsonl and contract.json
    back in -- it never re-runs the VAD or diarizer, so it works on a
    diarized-mono bundle (no frame-level evidence) without the diarization
    extra installed, honestly noting the missing base timeline instead of
    fabricating one. export BUNDLE --format otel --out FILE writes the
    attached trace back out as the same bridge JSONL ingest reads, so
    ingest -> attach -> export -> ingest round-trips the identical spans.
    The evidence page states findings plainly -- "Evidence suggests TTS
    cancellation delay: cancel requested at 2.60s, audio stopped at 2.90s
    (delta 0.30s)." -- always followed by "Hotato does not prove root cause."
    and an explicit "Unknowns: no client-side playout trace was attached."
    line. Redacted by default (call id, agent id, ASR transcript text). New
    module hotato.trace, new schema schema/voice_trace.v1.json, docs
    docs/TRACE.md and docs/OTEL.md.
  • hotato run --mono call.wav --diarize, the opt-in, quality-gated
    mono-scorability front-end
    : a single-channel (mixed) recording -- until now
    the hard coverage wall, rejected as not scorable -- becomes scorable by running
    an off-the-shelf speaker diarizer over the mono to recover who was active when,
    reconstructing two caller/agent tracks, and feeding the EXISTING dual-channel
    scorer (zero engine edit). The two-channel path stays the gold reference; this
    widens coverage, honestly labeled. A pluggable diarizer-backend seam
    (mirroring the neural-VAD seam) ships three backends behind --diarizer:
    pyannote (local, offline, CPU-viable, default; [diarize]), sortformer
    (NVIDIA NeMo, local/GPU, best self-hostable on telephone;
    [diarize-sortformer]), and pyannoteai (HOSTED, best absolute, requires
    --egress-opt-in; [diarize-hosted]). A real per-file confidence gate
    (extending trust.py's scorability model) reads six signals -- speaker count,
    per-speaker activity, mean segmentation posterior, embedding cluster-separation
    margin, overlap ratio, segment churn -- into a separation_confidence and a
    tier: high scores normally (labeled diarized-mono), low scores but stamps
    indicative_only and fires NO SLA gate, refuse is not scorable (exit 2).
    Caller/agent assignment reuses the floor-dominance heuristic to PROPOSE a
    mapping stated as an assumption, overridable with --caller-speaker /
    --agent-speaker; a balanced mapping downgrades to indicative. On this path
    signals.echo / crosstalk is definitionally N/A (two slices of one microphone)
    and the echo gate never fires. hotato trust --stereo mono.wav --diarize
    reports the separability tier (high/low/refuse) WITHOUT scoring. No silent
    fallback anywhere: a missing extra / token / model raises a clean
    BackendUnavailable (exit 2) and NEVER scores raw mono. Honesty invariants:
    the default path (no --mono/--diarize) stays byte-identical (a mono file is
    still rejected as today), and a diarized-mono verdict is never presented as
    equivalent to a true dual-channel measurement. New module hotato.diarize
    (DiarizationResult, the backend registry, reconstruct_tracks,
    separation_confidence, assign_speakers, prepare_diarized_mono, the
    pyannote/sortformer/pyannoteai adapters, and a hermetic stub backend); new
    --mono/--diarize/--diarizer/--caller-speaker/--agent-speaker/
    --egress-opt-in flags on hotato run and --diarize/--diarizer on hotato trust; documented in docs/DIARIZE.md. The LIMITS.does_not_do /
    METHODOLOGY.md "no diarization" framing is superseded: two channels is the
    gold reference; mono is scorable via the opt-in, quality-gated front-end,
    labeled indicative below the bar. Speaker IDENTIFICATION is still out of scope
    (a diarizer assigns anonymous SPEAKER_00/01; it never says who a person is).
  • New optional extras [diarize], [diarize-sortformer], [diarize-hosted]
    (the [diarize] path raises the effective Python floor to >=3.10; the stdlib
    core stays >=3.9). Dependency licenses are logged in docs/DIARIZE.md and
    carried in the envelope's diarization.licenses block (pyannote-audio MIT;
    community-1 weights CC-BY-4.0; Sortformer streaming v2 CC-BY-4.0 -- the offline
    v1 is CC-BY-NC and is never shipped).
  • hotato card INPUT[#REF] --out card.svg, a shareable card from any hotato
    result
    : renders a DETERMINISTIC, stdlib-only, 1200x630 SVG with NO external
    resource (no font, image, stylesheet, script, or link; all color inline), so
    it drops straight into a PR, an issue, or a slide and looks the same forever.
    Four kinds are auto-detected: a talk-over candidate and a false-stop
    candidate (from a sweep/analyze candidate ref FILE#N), the
    threshold-funnel fix plan (decision: do_not_tune_single_threshold -- the
    hero card: "NO SINGLE THRESHOLD CAN FIX THIS", fix class: engagement-control),
    and a supported verify rollup ("FIX VERIFIED WITHOUT BREAKING
    BACKCHANNELS"). Every card names the MEASURED timing moment and never a verdict
    about intent, and carries no accuracy number. Redacted by default: a call
    id, a filesystem path (only a basename is ever shown), and a vendor recording
    name are hidden unless --include-identifiers. Exit 0 written / 2 bad input.
    New module hotato.card; three commit-ready assets under
    docs/assets/cards/ regenerated by scripts/render_card_assets.py; documented
    in docs/CARDS.md.
  • hotato start --demo, the guided, credential-less first run: one command,
    no account, no network. It sweeps the two bundled real demo calls, writes the
    sweep result (hotato-sweep.json), a self-contained HTML dashboard
    (hotato-sweep.html), and the threshold-funnel card
    (hotato-no-single-threshold.svg), then turns one real missed-interruption
    candidate into a demo failure contract (contracts/demo-missed-interruption.hotato,
    --expect yield) and verifies it immediately -- it genuinely fails, so the
    loop is visible end to end in one command: a real failure becomes a
    candidate, becomes a portable contract, and contract verify catches it.
    It then prints the exact next commands: promote a candidate into a
    permanent fixture, run those fixtures in CI, re-verify the demo contract,
    and render a card. The --stack/--folder/--stereo modes are
    placeholders in this build and route to hotato sweep/analyze/run.
    Exit 0 done / 2 usage. New module hotato.start; documented in
    docs/START.md.

Fixed

  • hotato.diarize pyannote 4.x compatibility: Pipeline.from_pretrained
    now tries the token= kwarg first and falls back to the removed
    use_auth_token= name on TypeError, and a new _unpack_pipeline_output
    branches on the DiarizeOutput object pyannote.audio >=4.0 returns from a
    pipeline call (previously unpacked as a 3.x (Annotation, embeddings)
    tuple, raising TypeError/AttributeError against a 4.x install) so both
    pyannote 3.x and 4.x load and score cleanly. _embedding_margin no longer
    divides by a zero norm and fabricates cos = 0 (read by the confidence
    gate as adequate separation) for a degenerate (zero-norm / non-finite)
    speaker centroid; it now returns None -- "no margin available" -- the
    same honest no-signal result a missing embeddings array already gave.
  • New yield-boundary confidence gate signal (signals.yield_boundary):
    benchmarked against a real pyannote community-1 backend over the AMI
    corpus (in-repo harness: tools/bench_diarize/, dev-only, never shipped),
    the existing six diarization-quality signals measured clean, well-separated,
    stable speakers but were anti-cor...
Read more

hotato 0.5.0

Choose a tag to compare

@quantumCF quantumCF released this 09 Jul 10:05

hotato 0.5.0 - 2026-07-09

This release turns Hotato from a set of scoring commands into a passive regression
loop: connect a stack once, sweep recent real calls for candidate turn-taking
moments, promote the confirmed ones into fixtures, and prove a fix without an
unsafe threshold bandaid.

Added

  • hotato sweep --demo runs the full sweep over two bundled real calls with no
    stack, no credentials, and no network, so the first sweep works before anything
    is connected.
  • hotato fixture promote FILE#N --expect yield|hold turns one sweep or analyze
    candidate into a permanent regression fixture. The analyze and sweep HTML
    reports now carry per-candidate copy-command promote and hold actions.
  • hotato init webhook --stack vapi|retell|twilio scaffolds a self-hostable
    ingestion worker that verifies the webhook, fetches the recording read-only,
    runs hotato ingest, and optionally posts a Slack or GitHub notification. By
    construction it never mutates platform config and never labels intent.
  • hotato issue create and hotato pr create render a GitHub issue or a
    fixtures PR from a sweep run. Both default to a dry run that prints the body and
    the exact commands they would run, and touch a repo only with --repo and
    --yes. pr create always uses a feature branch and never force-pushes.
  • hotato verify --out verify.html writes a self-contained before and after proof
    page, and --policy hotato.verify.yaml enforces guardrails as hard fail
    conditions. A run cannot pass by improving one axis while regressing the other,
    and a battery missing a hold or yield fixture fails the opposite-risk check.
  • hotato apply PATCH_JSON --clone applies a patch only to a fresh staging
    assistant, never the production source. It refuses a threshold funnel outright
    and recommends engagement-control instead. There is no production-apply path.
  • hotato trust --stereo call.wav reports whether a recording is even scorable
    before you score it (channel swap, mono, sample rate, clipping, silence,
    crosstalk, scorability), so a bad export is caught up front.

Fixed

  • hotato.__version__ and the describe and --version banners are now covered
    by a lockstep test, so a stale version string cannot ship again.
  • llms-full.txt is built from the top-level docs only and is byte-reproducible
    across a git checkout and an extracted sdist.

See the full entry list in CHANGELOG.md.

hotato 0.4.1

Choose a tag to compare

@quantumCF quantumCF released this 09 Jul 06:00

Hotato 0.4.1

Released 2026-07-08. Patch release; scores, envelopes, and reports are identical to 0.4.0.

Fixed

  • hotato --version, the hotato describe manifest, and stackbench provenance reported "0.3.1" from the 0.4.0 wheel. The 0.4.0 bump missed the __version__ literal in src/hotato/__init__.py, so every runtime self-report carried the previous release's number. Only the reported string was wrong; no score or envelope was affected.
  • The gap is now closed structurally: tests/test_version_lockstep.py anchors hotato.__version__, the describe manifest, and the installed distribution metadata against pyproject.toml, and the release checklist enumerates every version lockstep site. The previous test compared the manifest to __version__ itself, which agreed even when both were wrong.

Upgrade

pip install -U hotato
# or zero-install
uvx hotato --version

hotato 0.4.0

Choose a tag to compare

@quantumCF quantumCF released this 09 Jul 06:00

Hotato 0.4.0

Released 2026-07-08.

0.4.0 turns Hotato from a one-call scorer into a monitoring loop: connect a stack once, pull recent recordings, rank the candidate turn-taking moments across them, then carry a labeled fix to a paste-ready patch and a before/after proof.

Highlights

  • hotato connect / pull / sweep. connect <stack> runs a live auth check and stores credentials locally at ~/.hotato/connections.json (mode 0600), sent nowhere but the vendor's own API. pull bulk-fetches recent recordings through each platform's verified list endpoint. sweep runs pull and then the zero-config analyze over the pulled folder, writing one self-contained offline dashboard of ranked candidate moments. Only verified endpoints are used; Retell pulls by explicit --call-id, and that limitation is documented rather than papered over.
  • hotato analyze <folder>, zero-config folder analysis. Point it at a folder of dual-channel recordings and it ranks candidate turn-taking moments across all of them by measured salience, no scenarios, labels, onsets, or flags. Top moments embed the real audio inline with a playhead synced to the timeline, so you press play and hear the bug land where the chart marks it. --format json emits the ranked candidates for an agent to drive. Everything is framed as measured candidates you review and label; never a pass/fail, a failure count, or an intent claim.
  • hotato patch / verify / loop, the closed loop. patch <fixplan.json> renders a fix plan into a literal, paste-ready artifact per platform: a JSON merge-patch body plus a ready curl for the platform's real config endpoint, or the exact constructor-kwarg source edit where config lives in agent code. It produces the change and never applies it. verify --before --after reports battery-scale movement ("N of M fixtures that used to fail now pass, K of L hold fixtures still pass"), refuses the headline claim under low n, and reports coincidence, never causation. loop orchestrates the whole cycle with state across runs and never auto-labels or auto-applies.
  • hotato ingest, the passive on-ramp. Wire a platform webhook to invoke it and each completed call is fetched and scanned for candidate moments automatically. Discovery only: webhook payloads are treated strictly as untrusted data, exit 0 means it ran, and nothing is auto-labeled, auto-fixtured, or auto-tuned. Per-stack recipes in docs/INGEST.md.
  • hotato describe, the generated capability manifest. One call (json or text) built from the live parser: every subcommand, argument, and exit code, so an agent learns the whole CLI without scraping --help across 18 subparsers. Exit-code epilogs are now uniform across every subcommand, and doctor --format json emits the machine envelope.
  • Five more stacks behind --allow-mono. Bland, ElevenLabs, Synthflow, Millis, and Cartesia capture and pull with exact verified endpoints. These platforms record one combined track, so scoring is degraded, gated behind the flag, and labeled indicative only.

Changed

  • hotato demo scores two real recorded calls. The packaged battery replaces synthetic audio with two operator-recorded probe calls against a scripted assistant on a provider's default interruption settings: a missed interruption (0.25 s talk-over) and a false stop on a soft backchannel. Full provenance and attestation ride in the scenario metadata, the only human voice is the recording operator's own, and two runs remain byte-identical.

Fixed

  • Atomic writes for recording downloads, the sweep dashboard, and every --out, so an interrupted run cannot truncate a previously good artifact.
  • Every JSON emitter refuses NaN and Infinity; a non-finite value surfaces as a clean exit 2, never invalid JSON.
  • .hotato/loop-state.json is validated on load; an incomplete or hand-edited state file is a clean error, never a KeyError.
  • Malformed WAV headers, identical caller/agent channels, invalid onsets, and bad flags are refused up front with plain reasons, and one bad file never aborts an analyze or loop batch.
  • Wording in docs/WHY.md and README.md softened where a frequency claim had no source.

Security

  • Default-deny SSRF blocking on every vendor download URL: loopback, private, link-local, and reserved hosts are refused, re-checked on every redirect, and credential headers are stripped on cross-host redirects. HOTATO_ALLOW_PRIVATE_URLS is the explicit opt-out.
  • MCP server input paths are sandboxed, mirroring the existing report-path sandbox.
  • ingest local recording paths require HOTATO_INGEST_DIR and fail closed, so a forged payload cannot point the scanner at an arbitrary file.
  • Scenario ids are validated as safe path segments, closing a crafted-pack path that could read or exfiltrate an arbitrary local WAV.

CI and docs

  • A new sdist guard job builds the sdist, installs only the extracted tree into a fresh venv, and runs the full suite from inside it, catching the green-wheel-broken-sdist failure mode that shipped unguarded in 0.2.3 and 0.3.0.
  • server.json MCP registry manifest, docs/MCP.md with copy-paste client configs, docs/CONNECT.md, an honest rewrite of docs/ADAPTER-STATUS.md, llms.txt reconciled to the real command surface, the new concatenated llms-full.txt, and CITATION.cff.

v0.3.1

Choose a tag to compare

@quantumCF quantumCF released this 07 Jul 20:30

Fixed

  • Self-consistent source distribution: the sdist now ships the small,
    non-audio test dependencies (scenario labels, manifests, the corpus
    validator, and the deterministic builders under corpus/ and examples/),
    so an extracted sdist collects and runs the full test suite instead of
    hitting collection errors. The heavy real and rendered audio stays pruned;
    suite and class audio is regenerated deterministically by
    tests/conftest.py (seed = sha256(id)) when absent, and the tests that
    depend on genuinely-absent heavy real audio skip cleanly rather than error.
  • Honest fix-plan wording in README.md and docs/WHY.md: reworded the
    claim that every failing event returns a fix that "names the exact setting to
    change in your stack." A plan() may correctly refuse to tune a single
    threshold, report insufficient coverage, or emit a checklist, so the copy now
    says a fix class is always returned and, when the failure maps cleanly to
    stack config, the setting family and direction to investigate.

Added

  • hotato scan self-truncation candidate: a new candidate kind,
    agent_stop_no_caller, surfaces the agent going from active to quiet with
    zero caller energy anywhere nearby, a drop nothing on the caller channel
    explains (not a barge-in, not a caller-driven handoff). Additive; existing
    candidate kinds and scoring are unchanged.
  • Response-gap percentiles and a latency SLA gate: hotato team and
    hotato export now also report p95 (alongside the existing mean/median/p90)
    for talk-over and time-to-yield, and pool response_gap_sec (dead air
    before the agent speaks) into the same distribution shape. --max-response-gap SECONDS on both commands gates the pooled p95 and fails (exit 1) exactly
    when it is exceeded, the same pass/fail contract as --max-talk-over /
    --max-time-to-yield. A plain export's envelope.json stays byte-identical
    to before; the new numbers live only in the printed summary and the
    returned manifest.
  • Echo detector: a deterministic cross-channel coherence signal
    (signals.echo: coherence, lag_sec, echo_suspected) on every scored
    event, flagging when the caller channel looks like a lag-shifted copy of
    the agent's own audio (leaked TTS), so a stop the agent makes because it
    heard itself is no longer indistinguishable from a clean yield. New scan
    candidate kind echo_correlated_activity; a loud WARNING in hotato diagnose and the single-run text output for every echo-suspected yield;
    opt-in --echo-gate on hotato run holds a bleed-induced yield out of the
    verdict (scorable: false) instead of counting it as a pass. Default run
    behavior, and every existing golden verdict, is unchanged; --echo-gate is
    off by default.
  • Resume/restart detector: on events where the agent yielded, an additive
    signals.resume block (resumed, resume_gap_sec, restart_suspected)
    measures from the agent's own VAD track whether it came back after the
    yield, how fast, and whether the post-resume run is long enough to look
    like a restart-from-the-top rather than a short continuation. Whether the
    resumed words repeat the earlier ones is out of scope (a transcript
    question, not a timing one).
  • Four corpus scenario classes under corpus/classes/ (deterministic,
    additive, same seeded-render/--check contract as corpus/suites/):
    mid-utterance-pause, backchannel-multilingual, noise-hold, and
    telephony-degraded. Detail: corpus/classes/README.md.
  • "Is this even a turn-taking bug?" triage in docs/WHY.md and README.md:
    names the failure modes commonly conflated with turn-taking bugs (STT
    hallucination, client-side audio buffering, LLM verbosity/tool-selection,
    safety false-refusal, wrong-language STT) and which tool class to reach for
    instead, alongside a plain statement of the flagship case Hotato covers
    (agent-talks-over-caller and false-stop-on-backchannel).
  • Report accessibility: the generated HTML report carries aria-labels on
    its inline SVGs and audio elements, a main landmark, and measured titles,
    so the artifact is navigable by assistive tech. Visual output is unchanged.
  • Startup lazy-imports: importlib.resources, the report renderer, and the
    numpy accelerator used only inside scan are deferred to first use, so plain
    CLI paths import less at startup. Behavior and output are unchanged.

All of the above are additive optional fields or opt-in flags: did_yield
and passed for every existing fixture are unchanged, and the vendored
src/hotato/_engine is untouched (verify with python3 sync_engine.py --check).

v0.3.0

Choose a tag to compare

@quantumCF quantumCF released this 06 Jul 22:04

[0.3.0] - 2026-07-06

Added

  • hotato fixture create: the missing piece of the regression loop. One
    bad call moment (a recording, an --onset, and YOUR --expect yield|hold
    label) becomes a permanent fixture: scenarios/<id>.json in the existing
    scenario schema shape (with provenance: source file, original onset,
    created_by) plus a two-channel audio/<id>.example.wav. Clips around the
    event by default (--pre 2.0 s / --post 6.0 s) and re-bases the onset to
    the clip; --no-clip keeps the full recording. The created fixture is
    validated by scoring it through the suite runner immediately; a not-scorable
    input is refused with the honest reason (exit 2) and partial outputs are
    removed unless --force. Overwrite refused without --force. Round-trips
    through hotato run --scenarios DIR --audio DIR as written.
  • hotato compare: the shareable before/after on one fixed moment. Scores
    both takes with the identical expectation, bounds, and reference config and
    reports the per-signal movement plus one machine-stable result word:
    fixed, regressed, improved, worse, unchanged, still_pass, or
    not_scorable. Marks come from real measurements only; a not-scorable side
    renders NOT SCORABLE (exit 2), never an invented verdict.
    --before-onset / --after-onset handle the moment shifting between takes;
    --out report.html writes the HTML report with the before take as the base
    comparison; --fail-on-worse exits 1 on regressed/worse.
  • hotato scan: candidate extraction across a WHOLE recording, as timing
    facts only. Walks the two VAD activity tracks (windowed pass, so long files
    never load fully into memory) and surfaces overlap_while_agent_talking
    (with overlap length and whether/when the agent went silent),
    agent_start_during_caller, and long_response_gap candidates, sorted by
    salience with --top (default 20). No intent claims anywhere: the output
    header states that you decide the expected behavior and label the moment
    with hotato fixture create.
  • hotato diagnose / hotato inspect / hotato plan: the guarded,
    read-only fix ladder (landed on main unreleased; first release here).
    Diagnose explains a finished run per failing event plus a battery-level
    decision; inspect reads and normalizes the live turn-taking config (GET or
    static parse, never executed); plan combines them into a
    hotato.fixplan.v1 proposal: at most one bounded step on one setting,
    refusal on the threshold funnel, checklist on ambiguous evidence,
    insufficient_coverage without a passing opposite-risk fixture, and
    production_apply pinned to false.
  • hotato plan also accepts the result JSON as a positional argument
    (hotato plan result.json), and every plan now carries kind: "fix-plan",
    a platform_mutation block (performed always false), the measured
    evidence, risks, next_commands (including the hotato compare
    verification step), and not-scorable events as input_issues. New Twilio
    rule: --stack twilio never yields agent-config advice; the plan points at
    channel assignment and the upstream voice-agent stack.
  • docs/BAD-CALL-TO-CI.md: the five-step loop from one bad call to a CI
    gate, with the label semantics up top and explicit use/do-not-use lists;
    runnable walkthrough in examples/bad-call-to-ci/.

Changed

  • Label wording, stated wherever the backchannel story is told (README,
    METHODOLOGY, WHY, and the run/doctor/demo/fixture create help):
    Hotato does not infer intent. You label the expected behavior for the
    event: yield means the agent should stop for the caller. hold means the
    agent should keep speaking through a backchannel/noise/acknowledgement.
    Hotato then measures whether the timing matched that label.
  • README positioning line sharpened: Hotato turns bad voice-agent call
    moments into offline regression tests.
  • Input wording unified: Hotato's main scorer requires separated caller and
    agent tracks; a single mixed mono call is not enough to attribute talk-over
    reliably.