Development

Development

Setup

git clone https://github.com/ranafaraz/VoxTutor.git
cd VoxTutor
python -m venv .venv
# Linux/macOS:
source .venv/bin/activate
# Windows:
.venv\Scripts\activate

pip install -e ".[dev]"
pytest -q   # 56 tests, all should pass

Optional extras

pip install -e ".[scipy]"   # enables scipy cross-check tests (DTW + AUROC)

Running the benchmark

# Full table + RESULTS.md
python -m evals.harness

# Dissociation gate (used in CI)
python -m evals.gate

# Interactive CLI
voxtutor compare --regime warped
voxtutor score --method gop --regime noisy
voxtutor regimes

Repository structure

VoxTutor/
  voxtutor/
    synthesis/      -- phoneme inventory, utterance generator, mispronunciation injector
    scorers/        -- naive.py, aligned.py, normalized.py, gop.py, random_scorer.py
    regimes/        -- clean.py, warped.py, noisy.py
    eval/           -- auroc.py, gate.py
    cli.py          -- voxtutor CLI entry point
  evals/
    harness.py      -- writes evals/RESULTS.md
    gate.py         -- asserts the 2x2 dissociation
  tests/            -- 56 pytest tests
  docs/             -- ARCHITECTURE.md, DECISIONS.md, demo.gif
  .env.example
  Dockerfile
  pyproject.toml

How to add a new scoring method

1. Create voxtutor/scorers/my_scorer.py implementing a score(utterance, phonemes) -> np.ndarray function that returns a per-position score (higher = more likely mispronounced).
2. Register the scorer in voxtutor/scorers/__init__.py under a string key (e.g., "my_scorer").
3. Add tests in tests/test_scorers.py covering at least: output shape, output range, and that a clean utterance scores lower than a mispronounced one.
4. Run voxtutor compare --regime clean to check it appears in the table with a non-degenerate AUROC.
5. Add the key to VOXTUTOR_METHOD accepted values in Configuration.

How to add a new distortion regime

1. Create voxtutor/regimes/my_regime.py implementing apply(utterance, rng) -> utterance that returns a distorted copy of the utterance (frames array + metadata).
2. Register the regime in voxtutor/regimes/__init__.py under a string key.
3. Add a description string to voxtutor/regimes/descriptions.py (shown by voxtutor regimes).
4. Add tests in tests/test_regimes.py verifying: the regime does not change the ground-truth labels, a clean scorer degrades on this regime relative to clean, and the regime is reversible (or at least deterministic given the same RNG state).
5. Add the key to VOXTUTOR_REGIME accepted values in Configuration.

CI

GitHub Actions runs pytest -q and python -m evals.gate on Python 3.10, 3.11, and 3.12. No secrets are required -- the benchmark is fully offline.

Code style

• Format with black, lint with ruff (configured in pyproject.toml).
• Type annotations are encouraged but not enforced.
• All random state must flow through np.random.default_rng(seed) for reproducibility -- never use np.random.seed() or module-level state.