Skip to content

Add Ollama provider for local model support#217

Open
tusharadi25 wants to merge 1 commit into
FrankChen021:masterfrom
tusharadi25:feature/ollama-provider
Open

Add Ollama provider for local model support#217
tusharadi25 wants to merge 1 commit into
FrankChen021:masterfrom
tusharadi25:feature/ollama-provider

Conversation

@tusharadi25

@tusharadi25 tusharadi25 commented Jun 16, 2026

Copy link
Copy Markdown

Summary

  • Add an opt-in Ollama provider so DataStoria's AI features can run against locally hosted models (Llama, Qwen, Mistral, DeepSeek, etc.).
  • The provider is off by default and enabled by setting OLLAMA_BASE_URL. Optional OLLAMA_API_KEY supports deployments where Ollama sits behind an auth proxy.
  • Locally available models are discovered automatically via Ollama's native GET /api/tags endpoint and surfaced as system-backed models; chat/completions go through an OpenAI-compatible client (/v1).
  • Base URL accepts either the bare host (http://localhost:11434) or the OpenAI-compatible base (.../v1); discovery failures degrade gracefully (empty list, never throws) so the rest of the model catalog keeps loading.
  • GitHub Copilot and Ollama both expose models dynamically, so Ollama is exempted from static-catalog verification in createModel.
  • Docs/UX: provider link to the Ollama library in model settings, .env.example entry, and README/docs mentions.

Changes

  • src/lib/ai/llm/ollama-models.ts — host normalization, config detection, /api/tags discovery, model normalization.
  • src/lib/ai/llm/llm-provider-factory.ts — register Ollama provider (OpenAI-compatible), system key gated on OLLAMA_BASE_URL, skip catalog verification.
  • src/lib/ai/llm/provider-ids.ts — add PROVIDER_OLLAMA.
  • src/app/api/ai/models/available/route.ts — append discovered Ollama models to system models (fetched in parallel with GitHub models).
  • src/components/settings/models/models-edit.tsx — provider link for Ollama.
  • public/provider-logos/ollama.svg, .env.example, README.md, docs/index.md.

Test plan

  • npm run test — unit tests for Ollama discovery/normalization (ollama-models.test.ts), provider registration and dynamic model resolution (llm-provider-factory.test.ts), and the available-models route (route.test.ts).
  • npm run build / npm run typecheck — verify build and types.
  • Manual: with OLLAMA_BASE_URL=http://localhost:11434 and a pulled model, confirm the model appears in settings and chat works; with the env unset, confirm Ollama does not appear.

Note: npm run build was not run locally in this environment (system Node is v10, and the Docker build hits a pre-existing @openrouter/ai-sdk-provider vs ai@6 ERESOLVE conflict on master, unrelated to this change which does not touch package.json).

Introduce an opt-in Ollama provider so AI features can run against
locally hosted models. The provider is enabled by setting
OLLAMA_BASE_URL; available models are discovered automatically via
the native /api/tags endpoint and served through an OpenAI-compatible
client. Includes docs, env example, and unit tests.
@vercel

vercel Bot commented Jun 16, 2026

Copy link
Copy Markdown

@tusharadi25 is attempting to deploy a commit to the Frank Chen's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant