Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ PORT=3000
NODE_ENV=development

# AI Service Configuration
# AI provider selection: "openrouter" (default) or "ollama"
# AI provider selection: "openrouter" (default), "ollama", or "mock" (local/demo only)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass AI_PROVIDER through Compose

When a local/demo user copies this env template, sets AI_PROVIDER=mock, and starts the stack with Docker Compose, the gateway container still will not see the setting because docker-compose.yml only lists the variables that get placed in the container environment and omits AI_PROVIDER; Docker’s Compose docs describe the environment attribute as what sets container environment variables. In that scenario the gateway falls back to openrouter and fails startup without OPENROUTER_API_KEY, so please add AI_PROVIDER (and any provider-specific mock/ollama variables as needed) to the gateway service environment.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass AI_PROVIDER through Compose

When a developer follows this new AI_PROVIDER=mock option with docker-compose up, Compose still only passes OPENROUTER_API_KEY and OPENROUTER_MODEL into the gateway (docker-compose.yml lines 18-20), so the container never sees AI_PROVIDER=mock, defaults back to OpenRouter, and fails startup without an OpenRouter key. Please add AI_PROVIDER to the gateway environment passthrough so the documented offline/demo mode works in the Compose stack too.

Useful? React with 👍 / 👎.

AI_PROVIDER=openrouter

# OpenRouter Configuration (when AI_PROVIDER=openrouter)
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ flowchart TB
Gateway["gateway/ Go Gin API :3000"]
Verifier["verifier/ Rust Axum :3002"]
Redis["Redis 7 receipt store by default\noptional response cache"]
AI["AI provider: OpenRouter or Ollama"]
AI["AI provider: OpenRouter, Ollama, or Mock"]
Wallet["EVM wallet on configured chain\nBase Sepolia default"]

Client --> Web
Expand Down Expand Up @@ -107,7 +107,7 @@ flowchart TB
Summarize["POST /api/ai/summarize"]
PaymentContext["Create PaymentContext\nrecipient, token, amount, chainId, nonce, timestamp"]
VerifyClient["Verifier HTTP client\nVERIFIER_URL"]
AIClient["AI provider client\nOpenRouter or Ollama"]
AIClient["AI provider client\nOpenRouter, Ollama, or Mock"]
ReceiptSigner["Receipt signer\nserver wallet key"]
ReceiptLookup["GET /api/receipts/{id}"]
Health["/healthz and /readyz"]
Expand All @@ -132,6 +132,7 @@ flowchart TB
subgraph Providers["AI providers"]
OpenRouter["OpenRouter chat completions"]
Ollama["Local Ollama generate API"]
Mock["Mock offline provider"]
end

Browser --> Gin
Expand All @@ -148,6 +149,7 @@ flowchart TB
Summarize -->|verified cache miss| AIClient
AIClient --> OpenRouter
AIClient --> Ollama
AIClient --> Mock
Summarize --> ReceiptSigner --> Receipts
ReceiptSigner --> MemoryStore
Cache -->|cached response receipt| ReceiptSigner
Expand Down Expand Up @@ -316,7 +318,7 @@ Core local variables live in [.env.example](.env.example). Production placeholde

| Variable | Service | Notes |
| --- | --- | --- |
| `AI_PROVIDER` | Gateway | `openrouter` by default, `ollama` for local Ollama experiments. |
| `AI_PROVIDER` | Gateway | `openrouter` by default, `ollama` for local Ollama experiments, or `mock` for offline local/demo use. |
| `OPENROUTER_API_KEY` | Gateway | Required when using OpenRouter. Never commit a real key. |
| `OPENROUTER_MODEL` | Gateway | OpenRouter model name. Demo docs use `z-ai/glm-4.5-air:free` unless overridden. |
| `OLLAMA_URL`, `OLLAMA_MODEL` | Gateway | Used only when `AI_PROVIDER=ollama`. |
Expand Down
7 changes: 7 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,13 @@ services:
- METRICS_ENABLED=${METRICS_ENABLED:-true}
- METRICS_PATH=${METRICS_PATH:-/metrics}
# Pass through from .env file
- AI_PROVIDER

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Pass the mock-enabling environment into Compose

When a user follows the Docker Compose quick start and sets AI_PROVIDER=mock in the copied .env, the gateway container receives AI_PROVIDER but none of NODE_ENV, APP_ENV, or ALLOW_MOCK_PROVIDER. Docker documents that values from .env are used for interpolation and only enter the container through an explicit environment/env_file entry (Docker Docs); consequently IsMockAllowed sees three empty values and the gateway exits during startup. Forward at least NODE_ENV=${NODE_ENV:-development} (and preferably the two override variables) so the advertised offline Compose setup works.

Useful? React with 👍 / 👎.

- OLLAMA_URL=${OLLAMA_URL:-http://host.docker.internal:11434}
- OLLAMA_MODEL
- OPENROUTER_API_KEY
- OPENROUTER_MODEL

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass through the OpenRouter URL override

This Compose env list now exposes the provider knobs, but it still omits OPENROUTER_URL, even though NewOpenRouterProvider reads that variable and .env.example documents it for custom OpenRouter-compatible endpoints. In Docker runs that set OPENROUTER_URL in .env, the gateway container will silently fall back to the public OpenRouter endpoint, breaking local/stubbed providers; add OPENROUTER_URL alongside the other OpenRouter settings.

Useful? React with 👍 / 👎.

- OPENROUTER_URL
- RUNNING_IN_DOCKER=true
- SERVER_WALLET_PRIVATE_KEY
- RECIPIENT_ADDRESS
- PAYMENT_AMOUNT
Expand All @@ -37,6 +42,8 @@ services:
start_period: 10s
networks:
- microai-net
extra_hosts:
- "host.docker.internal:host-gateway"

verifier:
build: ./verifier
Expand Down
6 changes: 3 additions & 3 deletions gateway/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ The verifier route `POST /verify` is not a gateway route. It belongs to the inte
| `ratelimit.go` | Token bucket implementation. |
| `metrics.go` | Prometheus metric definitions and metrics endpoint config helpers. |
| `middleware.go` | Request timeout, correlation ID, and request metrics middleware. |
| `internal/ai/` | OpenRouter and Ollama provider implementations. |
| `internal/ai/` | OpenRouter, Ollama, and Mock provider implementations. |
| `openapi.yaml` | Public gateway API contract. |

## Configuration
Expand All @@ -88,7 +88,7 @@ Common optional variables:
| Variable | Default | Notes |
| --- | --- | --- |
| `PORT` | `3000` | Gateway listen port. |
| `AI_PROVIDER` | `openrouter` | Supported values: `openrouter`, `ollama`. |
| `AI_PROVIDER` | `openrouter` | Supported values: `openrouter`, `ollama`, `mock` (local/demo only). |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Document mock in root setup docs

When readers follow the root README instead of gateway/README.md, the env table still says AI_PROVIDER is only openrouter by default or ollama for local experiments (README.md:319), so the newly added local/demo mock path is not discoverable from the main setup flow. Please update the root README alongside this gateway-specific entry to keep the documented supported providers aligned. Nice addition overall; this just avoids setup confusion.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update root provider docs for mock mode

Adding mock here leaves the top-level README's architecture/configuration sections still describing only OpenRouter/Ollama (README.md lines 69 and 319-322), so users following the main setup docs won't discover the new offline provider or that an OpenRouter key is unnecessary in mock mode. Since this PR changes the supported gateway configuration, please update the root docs alongside this gateway-specific table.

Useful? React with 👍 / 👎.

| `OPENROUTER_MODEL` | `z-ai/glm-4.5-air:free` in code/docs unless overridden | OpenRouter model. |
| `OPENROUTER_URL` | `https://openrouter.ai/api/v1/chat/completions` provider default | Used by tests and custom OpenRouter-compatible endpoints. |
| `OLLAMA_URL` | `http://localhost:11434` | Used when `AI_PROVIDER=ollama`. |
Expand Down Expand Up @@ -129,7 +129,7 @@ cd gateway
RECEIPT_STORE=memory CACHE_ENABLED=false go run .
```

The verifier must be reachable at `VERIFIER_URL` for signed requests. OpenRouter startup requires `OPENROUTER_API_KEY` unless `AI_PROVIDER=ollama`.
The verifier must be reachable at `VERIFIER_URL` for signed requests. OpenRouter startup requires `OPENROUTER_API_KEY` unless `AI_PROVIDER` is set to `ollama` or `mock`.

## Testing

Expand Down
2 changes: 2 additions & 0 deletions gateway/cache.go
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,8 @@ func CacheMiddleware() gin.HandlerFunc {
if model == "" {
model = "llama2"
}
} else if os.Getenv("AI_PROVIDER") == "mock" {
model = "mock"
} else if model == "" {
model = "z-ai/glm-4.5-air:free"
}
Expand Down
73 changes: 73 additions & 0 deletions gateway/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -474,3 +474,76 @@ func TestGetReceiptTTL(t *testing.T) {
func stringPtr(value string) *string {
return &value
}

func TestValidateConfig_MockProvider(t *testing.T) {
t.Run("allowed in dev environment", func(t *testing.T) {
t.Setenv("AI_PROVIDER", "mock")
t.Setenv("NODE_ENV", "development")
t.Setenv("APP_ENV", "")
t.Setenv("ALLOW_MOCK_PROVIDER", "")
t.Setenv("CACHE_ENABLED", "false")
t.Setenv("RECIPIENT_ADDRESS", "")
t.Setenv("SERVER_WALLET_PRIVATE_KEY", "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef")
t.Setenv("VERIFIER_URL", "http://127.0.0.1:3002")
t.Setenv("RECEIPT_STORE", "memory")

err := validateConfig()
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
})

t.Run("allowed with ALLOW_MOCK_PROVIDER", func(t *testing.T) {
t.Setenv("AI_PROVIDER", "mock")
t.Setenv("NODE_ENV", "")
t.Setenv("APP_ENV", "")
t.Setenv("ALLOW_MOCK_PROVIDER", "true")
t.Setenv("CACHE_ENABLED", "false")
t.Setenv("RECIPIENT_ADDRESS", "")
t.Setenv("SERVER_WALLET_PRIVATE_KEY", "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef")
t.Setenv("VERIFIER_URL", "http://127.0.0.1:3002")
t.Setenv("RECEIPT_STORE", "memory")

err := validateConfig()
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
})

t.Run("rejected in production", func(t *testing.T) {
t.Setenv("AI_PROVIDER", "mock")
t.Setenv("NODE_ENV", "production")
t.Setenv("APP_ENV", "")
t.Setenv("ALLOW_MOCK_PROVIDER", "true")
t.Setenv("CACHE_ENABLED", "false")
t.Setenv("RECIPIENT_ADDRESS", "")
t.Setenv("SERVER_WALLET_PRIVATE_KEY", "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef")
t.Setenv("VERIFIER_URL", "http://127.0.0.1:3002")
t.Setenv("RECEIPT_STORE", "memory")

err := validateConfig()
if err == nil {
t.Fatal("expected error, got nil")
}
if !strings.Contains(err.Error(), "disabled in production") {
t.Fatalf("expected error to mention disabled in production, got %v", err)
}
})

t.Run("rejected outside allowed env", func(t *testing.T) {
t.Setenv("AI_PROVIDER", "mock")
t.Setenv("NODE_ENV", "")
t.Setenv("APP_ENV", "")
t.Setenv("ALLOW_MOCK_PROVIDER", "")
t.Setenv("CACHE_ENABLED", "false")
t.Setenv("RECIPIENT_ADDRESS", "")
t.Setenv("SERVER_WALLET_PRIVATE_KEY", "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef")
t.Setenv("VERIFIER_URL", "http://127.0.0.1:3002")
t.Setenv("RECEIPT_STORE", "memory")

err := validateConfig()
if err == nil {
t.Fatal("expected error, got nil")
}
})
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
18 changes: 18 additions & 0 deletions gateway/internal/ai/mock.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
package ai

import "context"

type MockProvider struct {
Name string
}

func NewMockProvider() *MockProvider {
return &MockProvider{Name: "Mock"}
}

func (m *MockProvider) Generate(ctx context.Context, text string) (string, error) {
if err := ctx.Err(); err != nil {
return "", err
}
return "This is a deterministic mock summary of the input text for local/demo testing.", nil
}
7 changes: 7 additions & 0 deletions gateway/internal/ai/ollama.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@ func NewOllamaProvider() *OllamaProvider {
url = "http://localhost:11434"
}

if os.Getenv("RUNNING_IN_DOCKER") == "true" {
if url == "http://localhost:11434" || url == "http://localhost:11434/" ||
url == "http://127.0.0.1:11434" || url == "http://127.0.0.1:11434/" {
url = "http://host.docker.internal:11434"
}
}

model := os.Getenv("OLLAMA_MODEL")
if model == "" {
model = "llama2"
Expand Down
31 changes: 31 additions & 0 deletions gateway/internal/ai/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"context"
"fmt"
"os"
"strings"
)

// Provider defines the interface for AI service providers
Expand All @@ -25,7 +26,37 @@ func NewProvider() (Provider, error) {
return NewOpenRouterProvider(), nil
case "ollama":
return NewOllamaProvider(), nil
case "mock":
if err := IsMockAllowed(os.Getenv("NODE_ENV"), os.Getenv("APP_ENV"), os.Getenv("ALLOW_MOCK_PROVIDER")); err != nil {
return nil, err
}
return NewMockProvider(), nil
Comment on lines +29 to +33

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Block mock provider in production

Because mock is described as local/demo-only but is accepted unconditionally here, a production deployment with AI_PROVIDER=mock will start cleanly, /readyz reports the AI provider as ok, and paid summarize requests return the canned mock response instead of contacting an AI provider. Please gate this provider to development/test environments (or fail startup when production mode selects it) so a misconfigured paid gateway cannot charge users for fake summaries.

Useful? React with 👍 / 👎.

Comment on lines +29 to +33

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Block mock provider in production

Because the mock provider is documented as local/demo-only but this branch accepts AI_PROVIDER=mock unconditionally, any production deploy that carries this env var (for example from a copied demo .env) will pass startup and /readyz while serving the deterministic fake summary for paid requests. Consider rejecting mock when NODE_ENV=production or requiring an explicit dev-only override so this test provider cannot be enabled accidentally in a real deployment.

Useful? React with 👍 / 👎.

default:
return nil, fmt.Errorf("unsupported AI provider: %s", providerType)
}
}

// IsMockAllowed evaluates NODE_ENV, APP_ENV, and ALLOW_MOCK_PROVIDER to see if the mock provider can be run.
// It normalizes checks by lowercasing and trimming spaces, evaluating production prefixes,
// and treating truthy values for ALLOW_MOCK_PROVIDER.
func IsMockAllowed(nodeEnv, appEnv, allowMock string) error {
nodeEnv = strings.ToLower(strings.TrimSpace(nodeEnv))
appEnv = strings.ToLower(strings.TrimSpace(appEnv))
allowMock = strings.ToLower(strings.TrimSpace(allowMock))

isProd := strings.HasPrefix(nodeEnv, "prod") || strings.HasPrefix(appEnv, "prod")
if isProd {
return fmt.Errorf("mock AI provider is disabled in production environments")
}

isAllowedEnv := nodeEnv == "development" || nodeEnv == "local" || nodeEnv == "demo" || nodeEnv == "test" || nodeEnv == "dev" ||
appEnv == "development" || appEnv == "local" || appEnv == "demo" || appEnv == "test" || appEnv == "dev"

isExplicitlyAllowed := allowMock == "true" || allowMock == "1" || allowMock == "yes"

if !isAllowedEnv && !isExplicitlyAllowed {
return fmt.Errorf("mock AI provider is disabled. Enable it by setting ALLOW_MOCK_PROVIDER=true, or set NODE_ENV/APP_ENV to development/local/demo/test")
}

return nil
}
Loading
Loading