Problem
All tests mock LiteLLM responses. This gives confidence in internal logic but doesn't verify that the tool works against real APIs — response format changes, unexpected error codes, or LiteLLM breaking changes would go undetected.
Proposal
Add an integration test (in tests/test_integration.py or similar) that:
- Uses a cheap model (e.g.,
openai/gpt-4o-mini or anthropic/claude-haiku)
- Runs a minimal corpus (3-5 prompts, 1 batch)
- Verifies the full pipeline: executor → quality → decision → report output
- Gated behind a marker:
@pytest.mark.integration (skipped in CI by default, runnable with pytest -m integration)
Acceptance criteria
- Test passes with a live API key
- Skipped cleanly when no key is set
- Documented in
CLAUDE.md / README
Problem
All tests mock LiteLLM responses. This gives confidence in internal logic but doesn't verify that the tool works against real APIs — response format changes, unexpected error codes, or LiteLLM breaking changes would go undetected.
Proposal
Add an integration test (in
tests/test_integration.pyor similar) that:openai/gpt-4o-minioranthropic/claude-haiku)@pytest.mark.integration(skipped in CI by default, runnable withpytest -m integration)Acceptance criteria
CLAUDE.md/README