IELTS speaking & reading trainer with on-device Whisper ASR#2
Merged
Conversation
Transform the TTS app into an IELTS English trainer (TTS retained as "Voice Studio"). Three tabs via Navigation Compose: Reading, Speaking, Voice Studio. ASR (on-device, offline): - WhisperAsrEngine: quantized whisper-tiny.en ONNX (encoder+decoder), greedy decode over 30s windows, bundled in assets/asr (~41 MB, LFS). - MelSpectrogram (80-bin log-mel) validated bit-exact vs HF feature extractor; byte-level BPE WhisperTokenizer; AudioRecorder (16 kHz). - Shared via AsrEngineProvider across both scored features. Reading practice: - 200 graded passages (bands 4.5-9, 15 topics) in assets/ielts. - WerScorer: Word Error Rate + word-level alignment/highlighting. - IeltsScorer: read-aloud four-criteria band estimate. - Listen button reuses KittenTTS to read the passage. Free speaking practice: - IELTS prompts (Parts 1-3) in assets/ielts/speaking_prompts.json. - FluencyAnalyzer: pace, long pauses, fillers from the audio. - FreeSpeechScorer: full four-criteria band estimate (Task Response, Fluency & Coherence, Lexical Resource, Grammatical Range) from the candidate's own words, with prompt-coverage feedback. Tooling: - tools/download_whisper_onnx.py: fetch pre-converted quantized model (no PyTorch); tools/export_whisper_onnx.py for optimum<2.0 export. - RECORD_AUDIO permission; navigation-compose dependency. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Transforms the TTS app into an IELTS English trainer (TTS retained as Voice Studio). Three tabs: Reading, Speaking, Voice Studio.
Features
assets/asr(~41 MB, LFS), greedy decode over 30 s windows, fully offline.Validation
WhisperFeatureExtractor.Tooling
tools/download_whisper_onnx.pyfetches the pre-converted quantized model (no PyTorch).🤖 Generated with Claude Code