Privacy-first kitchen AI. Detect ingredients from a fridge photo entirely on-device, then stream a recipe generated by a local LLM — no image data ever leaves your machine.
- Scan — point your camera at the fridge or upload a photo
- Detect — YOLO26 runs in the browser (WebGPU or WASM SIMD) and identifies ingredients
- Generate — click "Generate Recipe" and watch a recipe stream back token-by-token from a local Ollama model
Everything runs locally: vision inference in a browser Web Worker, language generation via Ollama on your machine. No cloud calls, no image uploads.
| Layer | Technology |
|---|---|
| Orchestration | .NET Aspire 9 |
| Backend API | ASP.NET Core Minimal API + SignalR |
| AI / LLM | Semantic Kernel + Ollama (llama3.2) |
| Vector search | PostgreSQL + pgvector |
| Frontend | Blazor WebAssembly (PWA) |
| Edge vision | ONNX Runtime Web — YOLO26 nano, WebGPU → WASM SIMD fallback |
| Streaming | SignalR IAsyncEnumerable<string> server → client |
- .NET 10 SDK — the repo pins to
10.0.200viaglobal.json(do not use the .NET 11 preview SDK, it ships a mismatched native WASM runtime that causesMONO_WASM: index out of boundsat startup) - Docker Desktop (Aspire spins up Postgres automatically)
- Ollama with
llama3.2pulled:ollama pull llama3.2
wasm-toolsdotnet workload:dotnet workload install wasm-tools
- A YOLO26 nano ONNX model (NMS-free, INT8/Q4 quantized) placed at:
Kitchen.Web/wwwroot/models/yolo26n.onnx
git clone https://github.com/your-org/KitchenCopilot
cd KitchenCopilot
# Start everything (Aspire launches Postgres, Ollama, API, and the web app)
dotnet run --project Kitchen.AppHostOn first run, EF Core migrations are applied automatically when the API starts in Development mode. If you need to create a new migration manually, do so after Aspire has started Postgres:
# In a second terminal, while AppHost is running:
dotnet ef migrations add <MigrationName> --project Kitchen.Api --startup-project Kitchen.Api
dotnet ef database update --project Kitchen.Api --startup-project Kitchen.ApiThe Aspire dashboard opens automatically and shows all resources (api, web, postgres, ollama). Navigate to the web endpoint to use the app.
Note: Ollama will pull
llama3.2(~2 GB) on first startup. Theollamaresource in the Aspire dashboard will show as starting until the pull completes.
KitchenCopilot.sln
├── Kitchen.AppHost/ # Aspire orchestrator — wires all resources
├── Kitchen.ServiceDefaults/ # Shared telemetry, health checks, service discovery
├── Kitchen.Api/ # Minimal API + SignalR hub + EF Core + Semantic Kernel
│ ├── Data/ # AppDbContext, IngredientEmbedding (pgvector)
│ ├── Endpoints/ # InventoryEndpoints (MapGroup /api/v1/inventory)
│ ├── Hubs/ # RecipeHub — streams recipe tokens via SignalR
│ └── Services/ # RecipeService — SK prompt + pgvector fuzzy match
├── Kitchen.Web/ # Blazor WASM PWA
│ ├── Components/Vision/ # FridgeScan.razor — camera + YOLO detection UI
│ ├── Components/ # RecipeDisplay.razor — token streaming "typing" effect
│ ├── Services/ # YoloInferenceService, RecipeStreamService
│ └── wwwroot/
│ ├── js/ort-interop.js # ONNX Runtime Web Worker (WebGPU → WASM SIMD)
│ ├── js/camera.js # getUserMedia + canvas capture helpers
│ └── models/ # Place yolo26n.onnx here
└── Kitchen.Shared/ # Shared records: Ingredient, RecipeRequest, RecipeChunk
| Method | Path | Description |
|---|---|---|
GET |
/api/v1/inventory |
List all stored ingredient embeddings |
DELETE |
/api/v1/inventory/{name} |
Remove an ingredient embedding by name |
GET |
/openapi/v1.json |
OpenAPI spec (dev only) |
| SignalR | /hubs/recipe |
Stream recipe tokens — invoke StreamRecipe(RecipeRequest) |
- Camera frames and uploaded images are processed entirely in the browser by the ONNX model running in a Web Worker.
- Only ingredient labels and confidence scores are sent to the API — never raw pixel data.
- The API communicates only with the local Ollama instance; no external network calls are made.
MIT