Skip to content

fix(backend): ground clip transcripts to real spans#29

Open
FujiwaraChoki wants to merge 1 commit into
mainfrom
fix/transcript-grounding
Open

fix(backend): ground clip transcripts to real spans#29
FujiwaraChoki wants to merge 1 commit into
mainfrom
fix/transcript-grounding

Conversation

@FujiwaraChoki

Copy link
Copy Markdown
Owner

Summary

  • split long diarized transcript utterances into smaller analysis spans before asking the LLM to pick clips
  • validate AI-selected clip ranges against real transcript boundaries and rebuild clip text from cached word timings
  • make clip writes retry-safe by replacing existing task clips and enforcing uniqueness on (task_id, clip_order) via migration

Why

Production task c81b75a1-4204-4fa0-b236-ab7b5d29ec97 showed a bad Clip 1 transcript: the saved generated_clips.text came from a different point in the lecture than the actual 24:51-26:11 clip audio. The root cause was coarse transcript spans plus loose AI validation. The same task also duplicated clip rows on retry.

Verification

  • cd backend && uv run pytest --no-cov tests/test_video_utils_diarization.py tests/unit/test_ai_grounding.py tests/unit/test_task_service.py

Notes

  • bumps the processing cache key version so old cached analysis payloads are bypassed
  • adds a migration to dedupe existing duplicate clip orders before creating the unique index

@vercel

vercel Bot commented Mar 19, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
supoclip Ready Ready Preview, Comment Mar 19, 2026 8:08am

Request Review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 07b4e6b9b8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +62 to +63
ON CONFLICT (task_id, clip_order) DO UPDATE SET
filename = EXCLUDED.filename,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Stop overwriting the next clip during split operations

With ON CONFLICT (task_id, clip_order) DO UPDATE, ClipRepository.create_clip() no longer creates a new row when that order is already occupied. TaskService.split_clip() in backend/src/services/task_service.py:697-717 always inserts the second half at old_clip_order + 1 and only reorders afterwards, so splitting any clip that is not already last will now overwrite the existing successor clip's file/timestamps/metadata instead of shifting it down.

Useful? React with 👍 / 👎.

Comment on lines +55 to +59
grounded_text = get_transcript_text_in_range(
transcript_data,
start_seconds,
end_seconds,
).strip()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Ground clip text by transcript spans, not rounded second ranges

This path rebuilds segment.text from parse_timestamp_to_seconds(MM:SS) and get_transcript_text_in_range(), but the analysis transcript timestamps come from format_ms_to_timestamp() in backend/src/video_utils.py:321-326, which floors millisecond timings to whole seconds. Because get_words_in_range() (backend/src/video_utils.py:736-768) includes any word that merely overlaps that second window, a clip starting at 00:10 can pick up the last word from the previous [..., 00:10] span if that word ends at 10.2s. In those common boundary cases the saved generated_clips.text is no longer the exact span the model selected.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant