Text-to-speech and speech recognition for Malagasy (low-resource language), built by fine-tuning pretrained models on a self-collected dataset.
Beginner-to-production learning project. Stack: Python, PyTorch, Coqui TTS, Whisper, audio DSP. Compute: 1× RTX 2070 (8 GB) via WSL2.
- Both TTS + ASR (matches the year goal).
- Standalone first; Vavaka Flutter app integration is deferred to a later phase.
- Dataset: to be recorded (Malagasy read speech, ~1–2h baseline).
The model is the easy 20%; the dataset is the hard 80%. Malagasy has no public TTS/ASR corpus, so the real work is data engineering + fine-tuning, not training from scratch. Fine-tuning is the correct choice on a single 8 GB GPU.
data/raw/ # original recordings (gitignored)
data/processed/ # aligned, cleaned, phonemized pairs (gitignored)
notebooks/ # exploration + DSP learning
src/ # reusable pipeline code
scripts/ # CLI entry points
checkpoints/ # model weights (gitignored)
outputs/ # generated audio / transcripts (gitignored)
- Env: Python venv/uv, PyTorch+CUDA, librosa, Coqui TTS, Whisper
- DL fundamentals (tensors, autograd, training loops) — see courses-site AI curriculum
- Audio DSP (STFT, mel-spectrograms, normalization)
- TTS architectures (VITS, XTTS/YourTTS, Piper)
- ASR (Whisper fine-tuning, CTC, decoding)
- Data pipeline ⭐ (forced alignment, Malagasy G2P, cleaning)
- Low-resource tricks (augmentation, transfer learning, few-shot cloning)
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python -c "import torch; print(torch.cuda.is_available())" # should print TrueSee the Notion "Malagasy TTS & Voice AI" project for the full task breakdown.