Skip to content

feat: add LiteLLM as LLM provider#291

Open
RheagalFire wants to merge 2 commits into
FujiwaraChoki:mainfrom
RheagalFire:feat/litellm-provider
Open

feat: add LiteLLM as LLM provider#291
RheagalFire wants to merge 2 commits into
FujiwaraChoki:mainfrom
RheagalFire:feat/litellm-provider

Conversation

@RheagalFire

Copy link
Copy Markdown

Summary

  • Add LiteLLM as an alternative LLM provider alongside Ollama, giving MoneyPrinter users access to 100+ LLM providers (OpenAI, Anthropic, AWS Bedrock, Vertex AI, Groq, Cohere, Mistral, Together, Fireworks, etc.) through the LiteLLM Python SDK
  • Activated by setting LLM_PROVIDER=litellm in .env
  • Uses litellm.completion() with drop_params=True, provider API keys read from standard environment variables

Changes

  • Backend/gpt.py -- add _litellm_generate_response() calling litellm.completion() with timeout and bounds checking; route via LLM_PROVIDER env var with validation; add _list_litellm_models() for model discovery; existing Ollama code untouched
  • Backend/main.py -- update /api/models to use provider-aware list_models() with correct error messages per provider
  • .env.example -- document LLM_PROVIDER, LITELLM_MODEL settings
  • pyproject.toml -- add litellm>=1.80.0,<1.87.0 as optional dependency (pip install moneyprinter[litellm])

Tests

Live E2E (Azure Foundry -> Claude Sonnet 4.6, through actual gpt.generate_response() code path):

# From Backend/gpt.py -- _litellm_generate_response()
response = litellm.completion(
    model=model_name,
    messages=[{"role": "user", "content": prompt}],
    stream=False,
    drop_params=True,
    timeout=LITELLM_TIMEOUT,
)
Response: pong

Example usage

# .env
LLM_PROVIDER=litellm
LITELLM_MODEL=anthropic/claude-sonnet-4-20250514
ANTHROPIC_API_KEY=sk-ant-...
pip install moneyprinter[litellm]
uv run python Backend/main.py

Risk / Compatibility

  • Additive only -- all existing Ollama code and behavior is untouched
  • LLM_PROVIDER defaults to "ollama", preserving existing behavior; unknown values log a warning and fall back to Ollama
  • litellm is an optional dependency -- existing users without it are unaffected (lazy import inside _litellm_generate_response)
  • drop_params=True ensures unsupported params are silently dropped per provider
  • Timeout matches existing Ollama timeout default (180s)
  • list_ollama_models remains as alias for backward compatibility

@RheagalFire

Copy link
Copy Markdown
Author

cc @FujiwaraChoki

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant