assess-replies: media fixes, faster grading, debugging knobs#91
Merged
Conversation
…plies - _isValidMediaFile preflight skips students with missing/empty audio or image, replacing the reply_text fallback that was costing 5+ min per student on bogus inputs. - --choose-model (-M) prompts the instructor to pick from local gemma4 models at runtime, threaded through assessFollowUpReplies. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Saves each audio transcription to a sibling .txt file next to the .wav so the instructor can listen and read side-by-side when diagnosing noisy or mistranscribed recordings. Threaded through transcribe_audio → assess_replies / getMediaRecordings → the two CLI dispatchers. Empty transcripts still produce a zero-byte .txt (informative debug signal); write failures are logged and never block grading. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
_anonymizeCsvFile only redacts the standard 'name'/'id' column pair (and the pairing-style person<N>/id<N> shape). CSVs that use student_id / student_name columns — *_followup_*, *_reminder_sent_*, recordings, and conversations — pass through verbatim, along with free-text columns like transcript, reply_text, feedback, last_message, conversation_subject. A full fix is more work; for now the export prints a final NOTE telling the instructor to manually verify the CSVs before sharing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c4154181c3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Gemma 4's audio encoder has a ~30s window; longer recordings were silently truncated, producing transcripts that looped on the early-speech content and never reached the answer. transcribe_audio now probes duration with ffprobe and, when audio exceeds 30s, splits it into 30s segments via ffmpeg and joins per-segment transcripts. Even within a chunk, Gemma 4 can fall into runaway repetition loops on stuttered/noisy speech (observed: 640KB of looped output over 40 minutes on a single chunk). Adding repeat_penalty=1.3 + num_predict=512 to the chat options breaks most loops at formation and hard-caps any that slip through. End-to-end runtime on a 79s WAV: 40 min → 17 s, with the actual answer captured. Benefits both assess-replies and get-media-recordings without further changes since both flow through transcribe_audio. Adds 3 tests (long-audio chunks, short-audio single pass, probe-failure fallback). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Iterative hardening of the
assess-repliespipeline (with spillover improvements toget-media-recordingsandexport-anon-data). Five commits, each independently reviewable:d8c4342— fix two media-format failures, then cut per-call cost: route audio downloads through ffmpeg to 16 kHz mono PCM WAV (Canvas serves DASH manifests, not raw audio; Gemma misclassifies AAC m4a); rasterize PDF attachments to PNG via PyMuPDF (Gemma doesn't accept PDFs); embed the student's first name in reply filenames; cap image long edge at 1024 px (3.8 MB phone photos → 107 KB); wall-clock timer per student + total run time.554fcb3—_isValidMediaFilepreflight skips students with missing/empty audio or image so Gemma isn't fed garbage for 5+ minutes per student. Adds-M, --choose-modelso the instructor can interactively pick from localgemma4:*models at runtime (trade some assessment quality for throughput).60c00a6—-S, --save-transcriptwrites each audio transcription to a sibling.txtnext to the.wavso the instructor can listen and read side-by-side when diagnosing noisy or mistranscribed recordings. Applies to bothassess-repliesandget-media-recordings.c415418— print a manual-privacy-review reminder at the end ofexport-anon-data. The redactor only recognizesname/idcolumns;student_id/student_name/free-text columns (transcripts, reply_text, feedback, conversation subjects) pass through verbatim. A full fix is more work; for now we warn so the instructor doesn't share blindly.7db8c2e— chunk audio > 30s before transcription. Gemma 4's audio encoder has a ~30s window and silently truncates anything beyond it (observed: 79s WAV → transcript = 5x repetition of the student's opening; the actual answer in 30-79s was never seen).transcribe_audionow probes duration via ffprobe and chunks long audio into 30s segments with ffmpeg, transcribing each separately. Also addsrepeat_penalty=1.3+num_predict=512to break/cap runaway repetition loops Gemma falls into on stuttered speech (observed: a single chunk generating 640KB of looped output over 40 minutes; now capped at ~17s wall-clock for the 79s WAV). Bothassess-repliesandget-media-recordingsinherit the fix automatically.New dep:
pymupdf==1.27.2.3for PDF rasterization.Test coverage went 294 → 320 (added
TestFirstNameForFilename,TestRasterizePdfToPng,TestResizeImageInPlace,TestIsValidMediaFile,TestPickLocalGemmaModel,TestTranscribeAudio— the last grew by 3 cases covering the chunking path).Test plan
python -m pytest test_canvigator.py— all 320 passflake8both passes return 0python canvigator.py assess-replies --helpshows-M, --choose-modeland-S, --save-transcriptpython canvigator.py get-media-recordings --helpshows-S, --save-transcriptassess-replieson a quiz where at least one student didn't save their recording — confirm they're skipped immediately withno valid audio — skipped... processed in 0.0minassess-replies -S— confirm*.txtfiles appear next to*.wavfiles indata/<course>/replies/assess-replies -M— confirm interactive picker shows onlygemma4:*entriesassess-replieson a quiz with at least one >30s audio reply — confirm the transcript contains content from beyond the first 30s of the WAV (not a 5x loop of the opening sentence)export-anon-data— confirm the newNOTE: This export is NOT automatically privacy-safe ...line prints at the end🤖 Generated with Claude Code