fix: SRT hour-field parsing — root cause of the tier-2 timeline contamination - #340
Merged
Conversation
robbiebyrd
changed the title
… keys Both found by actually running the summariser against production rather than fixtures, and neither could have shown up any other way. replace_minutes deleted every row for a source, but summarisation is windowed and paid-for: it will be run an hour at a time. Scoped only by source, running 14:00-15:00 would delete the 13:00-14:00 rows a previous run had just paid for, and nothing would report it -- the delete succeeds, the insert succeeds, the row count silently drops. replace_segments is right to clear the whole source because it rebuilds an entire SRT; this is the case that differs. The grouped bucket keys mix types by design -- TV carries an int channel and a null slug, radio the reverse -- so sorting them raised TypeError comparing None to int the instant both media appeared in one window, which is every real run. The unit tests used one medium at a time and never saw it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…mination
Root cause of the tier-2 contamination found yesterday: post-attack television
appearing on the morning of 9/11 across twelve channels.
_TIME required exactly two digits for the hour and matched with .search(). A
stitched channel spans nine days -- 216 hours -- so every timestamp past hour 99
grows a third digit, and "100:00:05,000" quietly matched the SUBSTRING
"00:00:05,000" starting one character in. Hour 100 was read as hour 0. No error,
no warning, no malformed output: just every cue after hour 99 landing exactly one
hundred hours early.
That is the whole contamination. 09-13 coverage was filed at 09-09, which is why
the anachronisms began at precisely 09-09 00:00 and spread evenly across every
hour. CNN alone had 114,038 affected timestamps, 60% of its file.
The writer was never wrong -- _fmt's {h:02d} is a minimum width and has always
emitted "100:00:05,000" correctly. Only reading was broken, so the SRTs on Wasabi
are sound and nothing needs regenerating; chat_transcript_segments does need
rebuilding from them.
Verified against the real 188,649-cue CNN file: backward jumps 2 -> 0, cues whose
end preceded their start present -> 0, parsed span 100.0h -> 216.4h (nine days,
as it should always have been).
Anchored with .match() as well as widening the quantifier. The quantifier alone
fixes today's bug; the anchor is what stops a partial match from ever again
succeeding silently, which is the property that let this run undetected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
robbiebyrd
force-pushed
the
fix/cleared-at-schema-def
branch
from
July 31, 2026 22:29
0fb640e to
be8d559
Compare
Contributor
|
Contributor
|
✅ Playwright E2E — 9 passed · 0 failed · 0 flaky · 0 skipped |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause found
Yesterday's finding — post-attack television appearing on the morning of 9/11 across twelve channels — traces to one regex.
A stitched channel spans nine days — 216 hours — so every timestamp past hour 99 grows a third digit.
"100:00:05,000"quietly matched the substring"00:00:05,000"starting one character in. Hour 100 was read as hour 0.No error, no warning, no malformed output. Just every cue after hour 99 landing exactly one hundred hours early.
That is the entire contamination. 09-13 coverage was filed at 09-09, which is why the anachronisms began at precisely
09-09 00:00and spread evenly across every hour rather than clustering. CNN alone: 114,038 affected timestamps, 60% of its file.The fix
Widen the quantifier and anchor the match:
The quantifier alone fixes today's bug. The anchor is what stops a partial match from ever again succeeding silently — that property is why this ran undetected for as long as it did.
Verified against the real production file
Re-parsing CNN's actual 188,649-cue
channel.srt:endpreceded theirstartFour regression tests, written failing first, including the exact production symptom (a cue spanning the 99→100 boundary parsing to
end=4.0, start=359985.0).Scope of remediation
The writer was never wrong.
_fmt's{h:02d}is a minimum width and has always emitted100:00:05,000correctly — so the SRTs on Wasabi are sound and nothing needs regenerating. Onlychat_transcript_segments, which is derived by parsing them, needs rebuilding. That rebuild is running.Also in this PR (found by running the summariser against production rather than fixtures):
replace_minutesdeleted a whole source rather than the window it wrote, so an incremental second run would silently destroy the first's paid-for rows.int/Noneacross TV and radio, sosorted()raised the instant both media appeared in one window.🤖 Generated with Claude Code