Skip to content

Add Ollama support as a local model provider #216

Description

@tusharadi25

Summary

Add Ollama as a supported model provider so users can run DataStoria's AI features (natural-language query generation, optimization, visualization, diagnostics) against local, self-hosted models instead of only cloud APIs.

Today the README lists OpenAI, Anthropic Claude, Google Gemini, GitHub Copilot, OpenRouter, Groq, Cerebras, and Nebius under Multiple Model Providers — but there is no option for a fully local/offline backend.

Motivation

  • Privacy / data residency — Many ClickHouse operators work with sensitive data and cannot send schema, queries, or results to a third-party cloud API. Ollama keeps everything on-prem.
  • Cost — No per-token billing for high-volume, multi-turn agent conversations.
  • Offline / air-gapped environments — Enables DataStoria in environments without outbound internet access.
  • Experimentation — Easy to try open models (Llama 3.x, Qwen, Mistral, DeepSeek, etc.) for SQL generation/optimization.

Proposed Solution

  • Add an Ollama provider in the model-provider configuration UI/settings.
  • Configurable base URL (default http://localhost:11434) to support remote/containerized Ollama servers.
  • Model selection/discovery via Ollama's GET /api/tags (list locally available models).
  • Use Ollama's OpenAI-compatible endpoint (/v1/chat/completions) where possible to minimize new client code, falling back to the native /api/chat API if needed.
  • No API key required (optional bearer token for proxied/secured deployments).
  • Ensure streaming responses and the existing Agent SKILLs flow work with the Ollama backend.

Additional Context

  • Ollama exposes an OpenAI-compatible API, which should make integration relatively low-effort if the existing OpenAI client can be pointed at a custom base URL.
  • References: Ollama API docs, Ollama OpenAI compatibility.

Happy to help test against a local Ollama instance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions