claude-local-stack

Give Claude Code a local model and local web search it can call, so routine fetch/extract/lookup jobs run on your own machine for zero cloud tokens and full privacy. Big model thinks; the local model fetches.

Built for Linux + an NVIDIA 4000-series laptop (8GB). NVIDIA is the easy case: Ollama ships CUDA and just works on the card.

What you get

• gemma-worker -- a local Gemma (4B, vision-capable) served by Ollama that Claude Code can hand bounded jobs to (extraction, classification, reading an image, private lookups over your own files). 0 cloud tokens; nothing leaves the box.
• SearXNG -- a self-hosted metasearch engine (in Docker) for web search that never touches a cloud AI tool or an API key.
• Two Claude Code skills (gemma-worker, web-search-local) that wire those into Claude Code, so you can just say "search locally for X" or "use the local gemma to pull the dates out of this file."

Prerequisites

1. NVIDIA driver (so the GPU is visible). Check: nvidia-smi.
2. Docker Engine (for SearXNG). Check: docker ps.
3. Claude Code itself -- install from https://claude.com/claude-code (this kit adds the local model + search; it does not install Claude Code).
4. curl, python3, openssl (all standard on a dev laptop).

The installer will install Ollama for you if it is missing.

Quick start

git clone <this-repo>          # or copy the folder
cd claude-local-stack
./install.sh

That one script: installs Ollama, pulls the 4B Gemma and builds the gemma-worker model, generates a SearXNG secret and starts SearXNG on localhost:8888, installs the two Claude Code skills into ~/.claude/skills/, and runs self-tests plus a live smoke test of each. Re-running it is safe (idempotent).

Using it

From a shell:

bin/gemma-worker.py "classify this as spam or not: ..."      # local model, 0 cloud tokens
bin/gemma-worker.py "read the total" -I receipt.png -x       # local VISION on an image
bin/searxng-search.py "some question 2026"                   # local web search
# fully local search-and-read-and-summarize (no cloud at any step):
bin/searxng-search.py --fetch-top 3 "<topic>" | bin/gemma-worker.py -x "Summarize with sources:"

From Claude Code: the skills trigger on natural language, e.g.

• "search locally for the latest Godot 4 release notes"
• "use the local gemma to extract every date from notes.md"
• "read this screenshot with the local model" (keeps the image on your machine)

Claude still does the thinking and judgment; the local model just does the grunt fetch/extract work when it is good enough, saving cloud tokens.

Swapping the model

The kit defaults to gemma3:4b (fits 8GB comfortably). To go bigger (needs the VRAM), edit the FROM line in Modelfile and rebuild:

# e.g. FROM gemma3:12b   (~8GB at Q4 -- tight on 8GB, good on 12GB+)
ollama create gemma-worker -f Modelfile

ollama list shows what you have; ollama pull <tag> gets a new one.

Troubleshooting

• Ollama unreachable (exit 3): systemctl status ollama (or ollama serve). The daemon serves localhost:11434.
• SearXNG "no results" / unreachable (exit 4): docker compose up -d in this dir; it can take ~30s to warm up. docker compose logs -f searxng to watch.
• First gemma call is slow: the model cold-loads into VRAM (~10-30s) on the first call, then stays warm.
• GPU busy while gaming: free the card with ollama stop gemma-worker.
• Verify anytime: bin/gemma-worker.py --selftest and bin/searxng-search.py --selftest (no network needed).

Notes

• Both scripts are standard-library-only Python (no pip installs) and carry a --selftest. They read $OLLAMA_HOST (default localhost:11434) and $SEARXNG_URL (default http://localhost:8888) if you ever move the services.
• Optional: a local chat UI (Open WebUI) over the same Gemma + SearXNG is easy to add later; ask if you want that too.

Ported from a working AMD/ROCm setup to Linux + NVIDIA/CUDA, 2026-07-21.