A pluggable infrastructure gateway for AI agents. Agents call a stable REST API for memory, browser automation, code execution, observability, identity, tools, LLM routing, policy enforcement, and evaluations. Platform operators choose backends via YAML configuration. The two concerns are fully decoupled.
Documentation | API Reference | Examples
AI agents need infrastructure: memory, identity, sandboxed code execution, browsers, observability, tools, policies, and evaluations. Today, every agent framework hard-codes these to specific vendors. Switching memory from mem0 to AWS Bedrock AgentCore means rewriting agent code. Running the same agent in different environments means maintaining multiple configurations inside the agent itself.
The gateway extracts infrastructure into a standalone service. Agent developers code against the API. Platform operators choose backends via configuration. Switching from Langfuse to AgentCore for observability is a YAML config change, not a code change.
# Prerequisites: Python 3.14+, AWS credentials (aws configure)
git clone <repo-url>
cd agentic-primitives-gateway
pip install -e .
./run.shThe gateway starts at http://localhost:8000 with Bedrock for LLM and in-memory storage. A declarative agent (assistant with memory) is included — no Python code needed.
Chat with the agent:
curl -X POST http://localhost:8000/api/v1/agents/assistant/chat \
-H "Content-Type: application/json" \
-d '{"message": "Hello! Remember that my favorite color is blue."}'
curl -X POST http://localhost:8000/api/v1/agents/assistant/chat \
-H "Content-Type: application/json" \
-d '{"message": "What is my favorite color?"}'Use the Python client:
from agentic_primitives_gateway_client import AgenticPlatformClient, Memory
client = AgenticPlatformClient("http://localhost:8000", aws_from_environment=True)
memory = Memory(client, namespace="agent:my-agent")
await memory.remember("api-limit", "100 requests per minute")
results = await memory.search("rate limiting")Auto-build tools for any framework:
# Strands
tools = client.get_tools_sync(["memory", "browser"], namespace="agent:demo", format="strands")
agent = Agent(model="us.anthropic.claude-sonnet-4-20250514-v1:0", tools=tools)
# LangChain
tools = await client.get_tools(["memory", "browser"], namespace="agent:demo", format="langchain")
agent = create_agent(llm, tools=tools)Open the web UI at http://localhost:8000/ui/ (after cd ui && npm install && npm run build). Interactive API docs at /docs.
See the Quickstart Guide for complete setup instructions and the Building Tools Guide for all integration approaches.
| Config | Command | What you get |
|---|---|---|
| quickstart | ./run.sh |
Bedrock LLM + in-memory storage. AWS creds only. |
| agentcore | ./run.sh agentcore |
All AWS managed (AgentCore + Bedrock). Needs Redis + AGENTCORE_MEMORY_ID. |
| selfhosted | ./run.sh selfhosted |
Open-source: mem0/Milvus, Langfuse, Jupyter, Selenium. Needs Redis. |
| mixed | ./run.sh mixed |
Both backends + JWT auth + Cedar policies + OIDC credentials. |
The selfhosted and mixed configs require open-source infrastructure (Milvus, Langfuse, Selenium, Jupyter, Redis). You can run these locally with Docker, or deploy them on Kubernetes using the Agents on EKS infrastructure which provisions everything with a single command.
See the Configuration Guide for a full reference of every config option.
+------------------------------------------------------------------------+
| Agentic Primitives Gateway |
| |
| +------------------------------------------------------------------+ |
| | Web UI (React SPA at /ui/) -- Dashboard, Agents, Teams, Chat | |
| +------------------------------------------------------------------+ |
| |
| +------------------------------------------------------------------+ |
| | Agents Subsystem | |
| | (Declarative specs, CRUD API, LLM tool-call loop, auto-hooks) | |
| | POST /api/v1/agents/{name}/chat --> AgentRunner --> primitives | |
| +----+------------------------------+------------------------------+ |
| | | |
| +---------+ +---------+ +---------+ +---------+ +---------+ |
| | Memory | |Identity | | Code | | Browser | | Tools | |
| | Routes | | Routes | |Interpret| | Routes | | Routes | |
| +----+----+ +----+----+ | Routes | +----+----+ +----+----+ |
| | | +----+----+ | | |
| +----+----+ +----+----+ | +----+----+ +----+----+ |
| |Observ. | | LLM | | | Policy | | Evals | |
| | Routes | | Routes | | | Routes | | Routes | |
| +----+----+ +----+----+ | +----+----+ +----+----+ |
| | | | | | | | |
| +----v-----------v-----------v-------v---------v-v---------v--------+ |
| | PolicyEnforcementMiddleware (Cedar) | |
| +------------------------------------------------------------------+ |
| | CredentialResolutionMiddleware (OIDC) | |
| +------------------------------------------------------------------+ |
| | AuthenticationMiddleware (JWT/API key/noop) | |
| +------------------------------------------------------------------+ |
| | AuditMiddleware (http.request, auth/policy/cred emits) | |
| +------------------------------------------------------------------+ |
| | RequestContextMiddleware (AWS + correlation_id) | |
| +------------------------------------------------------------------+ |
| | Provider Registry (MetricsProxy) | |
| +--+-------+-------+-------+-------+--------+-------+------+-------+ |
+-----+-------+-------+-------+-------+--------+-------+------+---------+
| | | | | | | |
+----v---+ +-v-------+ +v------+ +v----+ +v-----+ +v------+ +v------+ +v------+ +v----------+ +v-----------+
| Memory | |Identity | |Code | |Brwsr| |Obsrv.| | LLM | |Policy | | Evals | | Tools | | Knowledge |
|--------| |---------| |Interp | |-----| |------| |-------| |-------| |-------| |----------| |------------|
| Noop | |Noop | |Noop | |Noop | |Noop | |Noop | |Noop | |Noop | | Noop | | Noop |
| InMem | |AgntCore | |AgntCr | |Agnt | |Lang | |Bedrock| |Agnt | |Agnt | | AgntCore | | LlamaIndex |
| Mem0 | |Keycloak | |Juptyr | |Core | |fuse | |Convrs | |Core | |Core | | MCP | | +FalkorDB |
| Agnt | |Entra | | | |Seln | |Agnt | |OpenAI | | | | | | Registry | | AgntCore |
| Core | |Okta | | | |Grid | |Core | |Compat | | | | | | | | |
+--------+ +---------+ +-------+ +-----+ +------+ +-------+ +-------+ +-------+ +----------+ +------------+
Governance fan-out (from AuditRouter)
+-------------+ +--------+ +----------------+ +---------------+
| stdout_json | | file | | redis_stream | | observability |
+-------------+ +--------+ +----------------+ +---------------+
| Primitive | Description | Backends |
|---|---|---|
| Memory | Key-value storage, semantic search, conversation history, session management | Noop, InMemory, mem0/Milvus, AgentCore |
| Identity | Workload tokens, OAuth2 exchange, API keys, credential management | Noop, AgentCore, Keycloak, Entra, Okta |
| Code Interpreter | Sandboxed code execution with persistent sessions | Noop, AgentCore, Jupyter |
| Browser | Headless browser automation (navigate, click, screenshot) | Noop, AgentCore, Selenium Grid |
| Observability | Trace/log ingestion, LLM generation tracking, scoring | Noop, Langfuse, AgentCore |
| LLM | LLM routing with tool_use support | Noop, Bedrock Converse |
| Tools | Tool registration, discovery, and invocation (MCP) | Noop, AgentCore, MCP Registry |
| Policy | Cedar policy engine and policy CRUD | Noop, AgentCore |
| Evaluations | LLM-as-a-judge evaluator management and evaluation | Noop, Langfuse, AgentCore |
| Knowledge | RAG / graph retrieval over an ingested corpus, with optional native retrieve-and-generate | Noop, LlamaIndex (vector + FalkorDB graph), AgentCore Knowledge Bases |
See the Primitives Guide for details on each primitive and its backends.
The gateway emits structured audit events, Prometheus metrics, and JSON logs for every request so compliance, operational debugging, and security detection are one coherent story.
| Subsystem | What it records | Default backend |
|---|---|---|
| Audit events | auth.*, policy.*, credential.*, agent.run.*, team.run.*, agent.version.*, team.version.*, {agent,team}.fork, tool.call, llm.generate, http.request, provider.call, resource.access.denied |
stdout_json (always on); pluggable sinks: file, redis_stream, observability |
| Metrics | gateway_auth_events_total, gateway_policy_decisions_total, gateway_credential_operations_total, gateway_agent_runs_total, gateway_team_runs_total, gateway_{agent,team}_versions_created_total, gateway_{agent,team}_forks_total, gateway_{agent,team}_version_approvals_total, gateway_tool_calls_total, gateway_llm_{requests,tokens}_total, gateway_access_denials_total, plus audit-pipeline health |
Prometheus scrape at /metrics |
| Logs | Structured JSON with request_id, correlation_id, principal_id on every line; secret scrubbing filter enabled by default |
stdout (opt-in via logging.format: json) |
| Versioning + approval | Immutable AgentVersion / TeamVersion records, fork lineage DAG, optional admin-approval gate on deploy |
Redis or file-backed spec stores; /ui/agents/{name}/lineage + /ui/admin/proposals |
See the Governance Guide for the architecture, Observability Guide for SIEM/Loki/Datadog/CloudWatch recipes, and Compliance Guide for SOC 2 / GDPR alignment.
Agents are defined in YAML — no framework code needed. The gateway runs the LLM tool-call loop server-side.
agents:
specs:
research-assistant:
model: "us.anthropic.claude-sonnet-4-20250514-v1:0"
system_prompt: "You are a research assistant with long-term memory..."
primitives:
memory: { enabled: true }
browser: { enabled: true }
hooks:
auto_memory: true
auto_trace: trueKey capabilities:
- Token streaming via SSE
- Agent-as-tool delegation — agents call other agents (coordinator pattern)
- Meta-agents — create specialist agents at runtime
- Teams — multi-agent collaboration with shared task board and parallel execution
- User-scoped memory — automatic per-user isolation in multi-tenant deployments
- Durable execution — Redis checkpointing with cross-replica recovery
- Background runs — agent/team runs continue if the client disconnects
- Versioning + fork + lineage — every edit produces an immutable
AgentVersion; identities are owner-scoped so users can fork shared agents into their own namespace with auto-qualified sub-refs;/ui/agents/{name}/lineagerenders the fork DAG - Admin-approval gate (opt-in) —
governance.require_admin_approval_for_deploy: trueforces deploys through a propose → approve → deploy workflow with/ui/admin/proposals
See the Agents Guide and Teams Guide for full documentation.
| Feature | Description |
|---|---|
| Authentication | Pluggable: noop (dev), static API keys, JWT/OIDC (Keycloak, Cognito, Auth0, Okta) |
| Policy enforcement | Cedar policies evaluated on every request. Default-deny when active. |
| Resource ownership | Agents/teams have owner_id + shared_with. Mutations require ownership. |
| User-scoped memory | Automatic :u:{user_id} namespace isolation for declarative agents |
| Session ownership | Browser/code_interpreter sessions are tied to their creator |
| Credential isolation | Per-request contextvars — no credential leakage between concurrent requests |
| Per-user credentials | OIDC-resolved credentials from user attributes (Langfuse keys, MCP tokens, etc.) |
See the Authentication, Policy Enforcement, and Credentials sections of the Configuration Guide.
# Server
pip install -e ".[dev]"
python -m pytest tests/ -v # 1800+ tests
# Client (separate package)
cd client && pip install -e ".[dev]"
python -m pytest tests/ -v # 100+ tests
# Lint & format
make format # Auto-fix
make lint # Check
make typecheck # mypy
make check # All three + tests
# Web UI
cd ui && npm install && npm run dev # Dev server at :5173
cd ui && npm run build # Production buildPrebuilt images are available on ECR Public:
docker pull public.ecr.aws/ai-registry/agentic-primitives-gateway/gateway:latestOr build your own:
docker build -t agentic-primitives-gateway:latest .Deploy with Helm:
cd deploy/helm
helm install apg ./agentic-primitives-gateway -f my-values.yamlThe Helm chart renders provider config as a ConfigMap, sets up health probes, and triggers rolling restarts on config changes. See deploy/helm/ for the full chart.
Add a custom provider by implementing the primitive's ABC and registering it in config:
# my_company/providers/redis_memory.py
from agentic_primitives_gateway.primitives.base import MemoryProvider
class RedisMemoryProvider(MemoryProvider):
def __init__(self, redis_url: str = "redis://localhost:6379", **kwargs):
self._redis = Redis.from_url(redis_url)
async def store(self, namespace, key, content, metadata=None): ...
async def retrieve(self, namespace, key): ...
async def search(self, namespace, query, top_k=10, filters=None): ...
async def healthcheck(self): return self._redis.ping()providers:
memory:
default: "redis"
backends:
redis:
backend: "my_company.providers.redis_memory.RedisMemoryProvider"
config:
redis_url: "redis://redis:6379/0"agentic-primitives-gateway/
├── src/agentic_primitives_gateway/ # Server package
│ ├── main.py # FastAPI app, lifespan, router registration
│ ├── middleware.py # Request context (AWS creds, provider routing)
│ ├── config.py # Pydantic settings, YAML config loading
│ ├── registry.py # Provider registry (dynamic loading, per-request resolution)
│ ├── routes/ # FastAPI routers (one per primitive + agents + teams)
│ ├── agents/ # Declarative agent orchestration (runner, stores, tools, teams)
│ ├── auth/ # Authentication (JWT, API key, noop)
│ ├── enforcement/ # Cedar policy enforcement
│ ├── credentials/ # Per-user credential resolution (OIDC)
│ ├── models/ # Pydantic request/response models
│ └── primitives/ # Provider ABCs + backend implementations
├── ui/ # React + Vite + TypeScript + Tailwind CSS
├── client/ # Python client (separate package)
├── tests/ # 1800+ server tests
├── configs/ # YAML presets (quickstart, agentcore, selfhosted, mixed)
├── examples/ # Example agents (Strands, LangChain)
└── deploy/helm/ # Kubernetes Helm chart