Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,17 @@ surface is governed by [`COMPATIBILITY.md`](COMPATIBILITY.md).
## [Unreleased]

### Added
- **Native AWS Bedrock provider.** Run Bedrock-hosted models through RiskKernel with
full budgets / approvals / audit / OTel — set the standard AWS env vars
(`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, optional `AWS_SESSION_TOKEN`,
`AWS_REGION`) and `RISKKERNEL_DEFAULT_PROVIDER=bedrock`, then call with a Bedrock
model id (e.g. `anthropic.claude-3-5-sonnet-20240620-v1:0`). Implemented against
the Bedrock Runtime **Converse** API and signed with hand-rolled AWS SigV4 — **no
AWS SDK dependency** (the signer is verified against AWS's published SigV4
known-answer test vector). Registered only when AWS credentials are present;
`RISKKERNEL_BEDROCK_BASE_URL` overrides the endpoint for a VPC/PrivateLink setup.
Long-tail Bedrock model ids meter `priced:false` until a `RISKKERNEL_PRICING_FILE`
rate is added (token budget yes, dollar budget no). See [`docs/PROVIDERS.md`](docs/PROVIDERS.md#aws-bedrock-native).
- **Importable Datadog dashboard.** [`examples/otel/datadog`](examples/otel/datadog)
ships a ready-made Datadog dashboard (spend per run, budget halts by reason,
tool-call outcomes, latency and token burn by model) built from the same
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ It is **not** another gateway (LiteLLM/Portkey own routing), **not** another obs

## Three ways to adopt — pick the one that fits

1. **Proxy (zero code).** Set one env var: `OPENAI_BASE_URL=http://localhost:7070/v1` (or `ANTHROPIC_BASE_URL` for `/v1/messages`). Every call — streaming or not — is intercepted, budgeted, logged, checkpointed, and forwarded to the real provider with your key. Native providers: Anthropic, OpenAI, and Ollama (local); front the long tail (Gemini, Cohere, Mistral, Bedrock, …) with [LiteLLM upstream](docs/PROVIDERS.md).
1. **Proxy (zero code).** Set one env var: `OPENAI_BASE_URL=http://localhost:7070/v1` (or `ANTHROPIC_BASE_URL` for `/v1/messages`). Every call — streaming or not — is intercepted, budgeted, logged, checkpointed, and forwarded to the real provider with your key. Native providers: Anthropic, OpenAI, Ollama (local), and AWS Bedrock; front the long tail (Gemini, Cohere, Mistral, …) with [LiteLLM upstream](docs/PROVIDERS.md).
2. **SDK (deep control).** `pip install riskkernel` (Python) or `npm install @riskkernel/sdk` (TypeScript), then governed runs, per-step loop/time budgets, checkpoints, and approval gates. Framework adapters for the Claude Agent SDK, OpenAI Agents SDK, LangChain, LlamaIndex, CrewAI, AutoGen, and PydanticAI (Python), and the Vercel AI SDK (TypeScript).
3. **OpenTelemetry (universal).** RiskKernel is an OTLP endpoint *and* emitter — ingest GenAI spans (`POST /v1/traces`) to meter apps already instrumented with OpenLLMetry / the OpenAI Agents SDK / the Vercel AI SDK, and export cost/halt/tool spans to the backend you already run.

Expand Down
10 changes: 4 additions & 6 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,17 @@ The core runtime is built and released:
([`docs/RESUME.md`](docs/RESUME.md)).
- **OpenAI- and Anthropic-compatible proxy** — point one env var at RiskKernel and
every call (streaming or not) is metered, priced, and budget-enforced (BYO key).
Native providers: Anthropic, OpenAI, and Ollama (local, key-free).
Native providers: Anthropic, OpenAI, Ollama (local, key-free), and AWS Bedrock; the
long tail is fronted via LiteLLM upstream ([`docs/PROVIDERS.md`](docs/PROVIDERS.md)).
- **Human-in-the-loop approval** — gate side-effecting tools; resolve from the CLI,
a local web page, a webhook, or **Slack** ([`docs/APPROVALS_SLACK.md`](docs/APPROVALS_SLACK.md)).
- **Policy-as-code, enforced per-run** — named policy bundles via `POST /v1/policies`
or a reviewed `riskkernel.yaml`, with a dry-run against recorded runs; a run created
under a bundle is governed by its tool allowlist and approval rules, not just its
budget ([`docs/POLICY.md`](docs/POLICY.md)).
- **OpenTelemetry GenAI — export and ingress** — emit cost/halt/tool spans into your
existing backend (ready-made **Grafana + Tempo** and **SigNoz** dashboards), *and*
existing backend (ready-made **Grafana + Tempo**, **SigNoz**, and **Datadog**
dashboards), *and*
ingest GenAI spans (`POST /v1/traces`) to meter apps RiskKernel never proxied
([`docs/OTLP_INGRESS.md`](docs/OTLP_INGRESS.md)).
- **Spend attribution** — roll cost up across runs by team/user/feature
Expand All @@ -51,10 +53,6 @@ The core runtime is built and released:

Where the work is heading near-term:

- **More native providers** — AWS Bedrock ([#24](https://github.com/prashar32/riskkernel/issues/24));
the long tail via LiteLLM-as-upstream.
- **More backend dashboards** — a Datadog dashboard to join the Grafana and SigNoz
examples.
- **Easier install** — a Homebrew tap for `brew install riskkernel`
([#97](https://github.com/prashar32/riskkernel/issues/97)).

Expand Down
48 changes: 43 additions & 5 deletions docs/PROVIDERS.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Providers — native, and the long tail via LiteLLM

RiskKernel implements the top providers **natively** in Go: Anthropic, OpenAI, and
Ollama (local). For those, point your app at the proxy and you're done — no extra
moving parts.
RiskKernel implements the top providers **natively** in Go: Anthropic, OpenAI,
Ollama (local), and AWS Bedrock. For those, point your app at the proxy and you're
done — no extra moving parts. (Bedrock setup is in its own section, [below](#aws-bedrock-native).)

The other 100+ providers (Google Gemini, Cohere, Mistral, Groq, Together, Azure
OpenAI, AWS Bedrock, OpenRouter, …) are not reimplemented inside RiskKernel — that
OpenAI, OpenRouter, …) are not reimplemented inside RiskKernel — that
isn't the product. Instead you front them with **[LiteLLM](https://github.com/BerriAI/litellm)**,
which already speaks all of them through one OpenAI-compatible endpoint, and put
RiskKernel **in front of LiteLLM**. RiskKernel governs every call; LiteLLM does the
Expand All @@ -29,7 +29,8 @@ upstream. LiteLLM is just the real-world version of that upstream.
| Anthropic (`claude-*`) | Native. Set `ANTHROPIC_API_KEY` on the daemon; use a `claude-*` model. |
| OpenAI (`gpt-*`, `o1`, `o3`) | Native. Set `OPENAI_API_KEY` on the daemon; use a `gpt-*`/`o1`/`o3` model. |
| A local Ollama model | Native. Set `RISKKERNEL_OLLAMA_BASE_URL`; use a model the routing sends to Ollama. |
| Anything else (Gemini, Cohere, Mistral, Groq, Bedrock, …) | Front it with **LiteLLM** as described below. |
| AWS Bedrock | Native. Set AWS credentials + `RISKKERNEL_DEFAULT_PROVIDER=bedrock`; see [below](#aws-bedrock-native). |
| Anything else (Gemini, Cohere, Mistral, Groq, …) | Front it with **LiteLLM** as described below. |

You only need LiteLLM for the long tail. If your stack is purely Anthropic/OpenAI/
Ollama, skip this page.
Expand Down Expand Up @@ -203,3 +204,40 @@ section of [`docs/BUDGETS.md`](BUDGETS.md) for the full format and stability pro
Anthropic/OpenAI/Ollama stacks, native providers need no LiteLLM at all.
- **Streaming** works end-to-end (LiteLLM is OpenAI-compatible SSE), and the call is
metered from the stream's final usage chunk — same as a native streamed call.

## AWS Bedrock (native)

Bedrock is implemented natively against the Bedrock Runtime **Converse** API, signed
with hand-rolled AWS SigV4 — **no AWS SDK dependency**. BYO AWS credentials, read
from the standard AWS environment variables and never stored.

```bash
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=... # optional, for STS temporary credentials
export AWS_REGION=us-east-1 # or AWS_DEFAULT_REGION
export RISKKERNEL_DEFAULT_PROVIDER=bedrock
riskkernel serve
```

Then call with a **Bedrock model id** (a foundation-model id or an inference-profile
id), e.g. `anthropic.claude-3-5-sonnet-20240620-v1:0` or
`us.anthropic.claude-3-5-sonnet-20241022-v2:0`.

**Routing.** RiskKernel routes by model-name prefix: `claude-*` → native Anthropic,
`gpt-*`/`o1`/`o3` → native OpenAI, everything else → the **default provider**. A
Bedrock model id starts with a vendor namespace (`anthropic.`, `amazon.`, `meta.`,
`us.`, …), so it doesn't match those prefixes and falls to the default — which is why
you set `RISKKERNEL_DEFAULT_PROVIDER=bedrock`. (Bare `claude-*`/`gpt-*` names still
route to the native Anthropic/OpenAI providers if those are also configured.)

Bedrock is registered only when `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are
set; with `RISKKERNEL_DEFAULT_PROVIDER=bedrock` and no credentials, the daemon
refuses to start with a clear error. For a VPC/PrivateLink endpoint, override the
runtime URL with `RISKKERNEL_BEDROCK_BASE_URL`.

**Cost caveat (same as the long tail).** The built-in pricing table only knows
`claude-*` / `gpt-*` names, so a Bedrock model id meters `priced: false` (cost `$0`)
until you add a rate for it via `RISKKERNEL_PRICING_FILE` — its tokens still count
toward the **token** budget, just not the **dollar** budget. Cost is metered from
Bedrock's own returned token usage; RiskKernel only needs the per-token rate.
13 changes: 8 additions & 5 deletions internal/app/bootstrap.go
Original file line number Diff line number Diff line change
Expand Up @@ -198,22 +198,25 @@ func OpenStore(cfg *config.Config, log *slog.Logger) (storage.Store, error) {
}

// BuildRegistry constructs the provider registry from config. Anthropic, OpenAI,
// and Ollama are implemented natively; Bedrock is a stub config can reference.
// The default provider must be usable.
// Ollama, and AWS Bedrock are implemented natively. The default provider must be
// usable.
func BuildRegistry(cfg *config.Config) (*provider.Registry, error) {
// Anthropic is always registered (native provider). When the key is absent the
// daemon still boots — health and routing work — and only an actual Chat call
// returns a clear "missing API key" error. Ollama is native and key-free
// (local models); OpenAI is registered (native) when a key is present; Bedrock
// is a stub config can name before it's built out.
// (local models); OpenAI and Bedrock are registered (native) when their
// credentials are present.
ps := []provider.Provider{
provider.NewAnthropic(cfg.AnthropicAPIKey).WithBaseURL(cfg.AnthropicBaseURL),
provider.NewBedrock(),
provider.NewOllama(cfg.OllamaBaseURL), // empty → local default
}
if cfg.OpenAIAPIKey != "" {
ps = append(ps, provider.NewOpenAI(cfg.OpenAIAPIKey).WithBaseURL(cfg.OpenAIBaseURL))
}
if cfg.AWSAccessKeyID != "" && cfg.AWSSecretAccessKey != "" {
ps = append(ps, provider.NewBedrock(cfg.AWSAccessKeyID, cfg.AWSSecretAccessKey,
cfg.AWSSessionToken, cfg.AWSRegion).WithBaseURL(cfg.BedrockBaseURL))
}

return provider.NewRegistry(cfg.DefaultProvider, ps...)
}
Expand Down
46 changes: 32 additions & 14 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,19 @@ type Config struct {
// OllamaBaseURL points the native Ollama provider at a server. Empty uses the
// local default (http://localhost:11434). Read from RISKKERNEL_OLLAMA_BASE_URL.
OllamaBaseURL string
// BedrockBaseURL overrides the Bedrock runtime endpoint (a VPC/PrivateLink
// endpoint, or a mock). Empty uses the regional default. Read from
// RISKKERNEL_BEDROCK_BASE_URL.
BedrockBaseURL string

// AWS credentials + region for the native Bedrock provider, read from the
// standard AWS env vars so existing setups need no new config. Bedrock is
// registered only when an access key and secret are present. Never stored or
// logged. AWSSessionToken is optional (STS temporary credentials).
AWSAccessKeyID string // AWS_ACCESS_KEY_ID
AWSSecretAccessKey string // AWS_SECRET_ACCESS_KEY
AWSSessionToken string // AWS_SESSION_TOKEN
AWSRegion string // AWS_REGION, then AWS_DEFAULT_REGION

// DefaultBudget is applied to runs created without an explicit budget — e.g.
// proxy calls that supply only a run-id. Any zero field is unlimited. When no
Expand Down Expand Up @@ -216,20 +229,25 @@ func Load() (*Config, error) {
}

cfg := &Config{
Port: port,
DataDir: getenvDefault("RISKKERNEL_DATA_DIR", "./data"),
DatabaseURL: os.Getenv("RISKKERNEL_DATABASE_URL"),
APIToken: os.Getenv("RISKKERNEL_API_TOKEN"),
DefaultProvider: getenvDefault("RISKKERNEL_DEFAULT_PROVIDER", "anthropic"),
AnthropicAPIKey: os.Getenv("ANTHROPIC_API_KEY"),
OpenAIAPIKey: os.Getenv("OPENAI_API_KEY"),
AnthropicBaseURL: os.Getenv("RISKKERNEL_ANTHROPIC_BASE_URL"),
OpenAIBaseURL: os.Getenv("RISKKERNEL_OPENAI_BASE_URL"),
OllamaBaseURL: os.Getenv("RISKKERNEL_OLLAMA_BASE_URL"),
DefaultBudget: budget,
PricingFile: os.Getenv("RISKKERNEL_PRICING_FILE"),
PolicyFile: os.Getenv("RISKKERNEL_POLICY_FILE"),
OTel: loadOTel(),
Port: port,
DataDir: getenvDefault("RISKKERNEL_DATA_DIR", "./data"),
DatabaseURL: os.Getenv("RISKKERNEL_DATABASE_URL"),
APIToken: os.Getenv("RISKKERNEL_API_TOKEN"),
DefaultProvider: getenvDefault("RISKKERNEL_DEFAULT_PROVIDER", "anthropic"),
AnthropicAPIKey: os.Getenv("ANTHROPIC_API_KEY"),
OpenAIAPIKey: os.Getenv("OPENAI_API_KEY"),
AnthropicBaseURL: os.Getenv("RISKKERNEL_ANTHROPIC_BASE_URL"),
OpenAIBaseURL: os.Getenv("RISKKERNEL_OPENAI_BASE_URL"),
OllamaBaseURL: os.Getenv("RISKKERNEL_OLLAMA_BASE_URL"),
BedrockBaseURL: os.Getenv("RISKKERNEL_BEDROCK_BASE_URL"),
AWSAccessKeyID: os.Getenv("AWS_ACCESS_KEY_ID"),
AWSSecretAccessKey: os.Getenv("AWS_SECRET_ACCESS_KEY"),
AWSSessionToken: os.Getenv("AWS_SESSION_TOKEN"),
AWSRegion: getenvDefault("AWS_REGION", os.Getenv("AWS_DEFAULT_REGION")),
DefaultBudget: budget,
PricingFile: os.Getenv("RISKKERNEL_PRICING_FILE"),
PolicyFile: os.Getenv("RISKKERNEL_POLICY_FILE"),
OTel: loadOTel(),
Approval: ApprovalConfig{
DefaultSafe: envBoolDefault("RISKKERNEL_APPROVAL_DEFAULT_SAFE", true),
WebhookURL: os.Getenv("RISKKERNEL_APPROVAL_WEBHOOK"),
Expand Down
Loading