fix(video): tighten Seedance capability tables to token360's own schema (0.39.0) - #96
Conversation
VickyXAI
changed the title
Review pass (5 reviewers: testing, maintainability, api-contract, Claude adversarial, Codex)Applied in 61e6993: Mutation-proven test gaps (testing + adversarial, independently):
Codex: api-contract: deliberate breaking schema change was shipping as a patch → 0.38.2 becomes 0.39.0. CHANGELOG's "values that never produced output" was false for seedance-2.0 image-conditioned 360p; reworded to the real rationale. Cross-repo check: every table matches #353 and blockrun-sol#133 value-for-value; no deploy-order hazard in either direction. maintainability: stale comments fixed ("per generation mode", "cannot drift apart" — nothing mechanical enforced that; softened, and the probe matrix now covers each model's ceiling/floor so divergence surfaces in the free 402 sweep). 2.0-fast's 1080p advice now points at 1.5-pro ($0.80/5s) instead of only 2.0 ($2.56/5s). New live pins: 1.5-pro@1080p (the only above-720p tier with no live coverage) probed free: charge $0.797311 vs reserve $0.798311. 303 tests, 16 video probes, 0 under-reserving. |
Review round 3 (regression hunt over round 2's fixes)Round 2's code fixes all hold — a fresh-context reviewer ran four new mutations against the acceptance table, the image-conditioned rejections, the pointer assertion, and 2.5's set: all went red. The probe rows estimate cleanly; the 0.39.0 bump is consistent. What didn't hold was the prose — five doc-level fixes in 58f6598, including one error round 2 introduced while fixing another:
Also run this round: e2e through the real MCP protocol layer against the branch build — all six probe cases (enum-valid/model-invalid, removed enum values, kept ratios) behave exactly as documented. 303 tests · 17 video probes live · 0 under-reserving. Code-clean; ready to merge. |
…ma (0.38.2)
Sibling of blockrun#353 and blockrun-sol#133. Yesterday's 0.38.1 tightened
the guards toward the gateway's tables; live probing (2026-08-07) shows those
tables were themselves hand-typed drift. token360 publishes the answer —
GET /v1/models/{id} → parameter_schema — and nobody had ever read it.
Per-model truth, now enforced here: 2.0 = 480p/720p/1080p/4K in BOTH modes
(the t2v/i2v split is gone; 540p/1K hard-reject upstream even
image-conditioned), 1.5-pro = 480p/720p/1080p, 2.0-fast and 2.5 = 480p/720p.
360p/540p/1K leave the resolution enum and 9:21 leaves the ratio enum — no
model lists them. Off-schema values that merely pass upstream validation
(2.0-fast 1080p, 1.5-pro 540p) stay blocked: "passes validation" is not
"renders", and 1.5-pro historically echoed 2K/4K, billed the tier, rendered
720p.
All probes cost $0: token360 validates duration → resolution → ratio →
other, so an invalid LATER parameter guarantees rejection after the one
under test is judged, and a failed submit never creates a task or bills.
RESOLUTION_TOKEN_FACTOR drops the unreachable tiers (the estimator throws on
a missing key, so stale is loud, not a silent 720p-baseline reserve). The
verify:prices matrix keeps 2.5@1080p as a fleet-consistency probe — the
gateway still quotes $3.55 for a render token360 refuses until #353 deploys
— and drops 2.0@360p, which this client can no longer express.
302 tests pass. Live gate: 0 under-reserving.
…ertion matched itself Five-reviewer pass over this PR (testing, maintainability, api-contract, Claude adversarial, Codex). What survived verification: THE TESTS PINNED ONLY HALF THE TABLE — mutation-proven. Emptying 2.0-fast's resolution set left the suite green: every rejection was tested, almost no acceptance was, so an over-restrictive edit could brick a paid capability with no red test. And the first fix draft iterated the exported table in the test — mutating the table mutated the expectation, same two escapes. The acceptance sweep now hardcodes the expected pairs (the test IS the spec) and asserts deep-equality with the shipped table, both directions. A FOURTH VACUOUS ASSERTION, mutation-proven: `/seedance-2\.0/` intended to pin the advice pointer, but the message opens with "bytedance/seedance-2.0- fast", which contains the pattern — deleting the pointer entirely stayed green. Anchored on the note's phrasing now, and the note itself stops steering 1080p seekers to the $2.56 model when 1.5-pro renders it at $0.80. THE "BOTH MODES" CLAIM was carried by 360p — a value real callers can no longer send (it left the zod enum), tested on the one model whose set holds all four enum values. Re-adding an image-conditioned wide set in enum-valid values (the production-reachable bill-high/render-low trap) passed. Now pinned with 1080p/4K image-conditioned rejections per model; the 360p leg stays as labeled defense-in-depth. ASPECT_RATIO'S DESCRIPTION LIED (Codex): "Ignored by xAI/Sora" — the handler forwards it for every model and the gateway maps it to portrait/landscape on the per-second models. Description now says what happens. 0.38.2 -> 0.39.0 (api-contract): the enum removals are a deliberate breaking schema change and were shipping as a patch. CHANGELOG's justification also overclaimed — seedance-2.0 image-conditioned 360p DID produce billed output before this release; reworded to the real rationale (off-schema shapes can downscale while billing the requested tier). Comment sweep: "per generation mode" (the mode split this PR deleted), "cannot drift apart" (nothing mechanical enforces it — softened, and the probe matrix now covers each model's ceiling and floor tier so divergence shows up in the free 402 sweep: three new rows, including the only above-720p tier that had no live pin, 1.5-pro@1080p, charge $0.797311 vs reserve $0.798311), and a "2K" that never existed in this client. 303 tests, 6/6 mutations caught, live gate 0 under-reserving (16 video probes).
A fresh-context regression hunt over 61e6993 (round 2's fix commit) plus a manual pass. The code survived: four new mutations against the acceptance table, the image-conditioned guard, the pointer note, and 2.5's set all went red; the probe rows estimate cleanly; the semver bump is consistent. What did not survive was the WORDS around the code — including one error round 2 introduced while fixing another: - The aspect_ratio describe (and CHANGELOG) said "Sora/Grok use it to pick portrait vs landscape". Grok doesn't: submitXAIVideo builds its body from model/prompt/image_url/duration only — the gateway never forwards aspect_ratio to xAI. Round 2 replaced "both ignore it" (false for Sora) with "both use it" (false for Grok). Third try states each model's truth. - The CHANGELOG's "previously accepted no-ops" was false for (Sora, 9:21): azureSoraSize treats 9:21 as a portrait trigger, so the removed value changed output orientation. 9:16 is the exact replacement, now said. - "302 tests pass" was stale by round 2's own commit (303). - The probe-matrix comment claimed floor+ceiling coverage per model while seedance-2.0@480p — after round 2, the only schema pair with no live pin — had no row. Added: charge $0.56874, reserve $0.56974, the fee gap. - video.ts's table comment described unmerged blockrun PR #353 in the present tense. Until it deploys, production runs the old wide tables and this client is the only gate — the comment now says so. 303 tests, live gate 0 under-reserving (17 video probes).
…of the 2.5@1080p probe
58f6598 to
84a6aa1
Compare
Review round 4 — base drift caught and resolvedRounds 1-3 converged the diff itself; round 4's finding was outside it: main moved under this PR. #97 (the dual-gateway price sweep) landed after the branch point and rewrote Resolved by rebase onto Re-verified post-rebase: 303/303 tests · typecheck · Bonus datum the merged sweep surfaces for free: the tracked Also added a CHANGELOG note: #97 shipped to main without an entry, and merging this PR publishes 0.39.0 carrying it — the 0.39.0 entry now records what actually ships. CI re-running on the rebased head ( |
1 participant
Sibling of BlockRunAI/blockrun#353 and BlockRunAI/blockrun-sol#133 — same source of truth (
GET /v1/models/{id} → parameter_schema), client side.What changes for callers:
360p/540p/1Kleave the resolution enum,9:21leaves the ratio enum — no Seedance model lists them, and 540p/1K now hard-reject upstream even image-conditioned (the pay-then-lose path 0.38.1 only half-closed).RESOLUTION_TOKEN_FACTORtrims to match; the estimator throws on a missing key, so drift is loud.verify:priceskeeps a fleet-consistency probe: the gateway still quotes $3.55 for 2.5@1080p, which token360 refuses — fixed by #353; this probe reports the moment both sides agree.Probes cost $0 — token360 validates
duration → resolution → ratio → other, so an invalid later parameter guarantees rejection after the one under test is judged; failed submits never bill. Technique + full matrix recorded in #353.Breaking, deliberately: a stale caller sending
360p/9:21gets a zod enum error instead of a paid round trip or post-payment 400. These values never produced output.Verification: 302 tests pass ·
npm run verify:prices0 under-reserving · typecheck + build green.Merging to main auto-publishes 0.38.2 to npm + the MCP registry.