diff --git a/CONTEXT.md b/CONTEXT.md index f04c718..421cff2 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -7,8 +7,8 @@ following the skillimage.io/v1alpha1 SkillCard format. A SkillCard with `type: skill` (default) is on-demand — only its name and description are loaded at startup; the full content activates when the agent invokes it. A SkillCard with `type: rule` is always-loaded — its full -content is injected into every agent turn and counts toward the LLM's -context budget. A SkillCard CR supports three source types: an OCI +content is injected into every agent turn. A SkillCard CR supports +three source types: an OCI image ref (pre-built artifact), a git source URL (controller clones, builds, and pushes the OCI artifact), or inline markdown content (controller builds and pushes). All three converge to a resolved OCI @@ -25,25 +25,24 @@ Examples: "konveyor-quarkus-skills" (a collection of 15 migration skills from a git repo), "enterprise-rules" (a curated set of rules as OCI images). -**LLMProvider** — An LLM service endpoint together with credentials and -the set of models it serves. Each model declares its context window -size and an optional tier label (e.g. premium, efficient). The context -window is used to validate that an Agent's always-loaded rules fit -within budget before execution begins. +**LLMProvider** — *Deprecated, to be removed.* Replaced by OpenShell +gateway inference routing. See ADR-0004. **Agent** — A capability definition declaring what is available for execution. References zero or more SkillCards and SkillCollections, -one or more LLMProviders (the set of providers and models available -for runs), a container image (carrying the agent runtime and language -toolchains), a prompt (standing instructions for how the agent -operates), and optionally a memory service for accumulating domain -knowledge across executions. An Agent does not select a specific -model — it declares what is available. Model selection happens at -execution time in the AgentRun. The Agent controller validates -referenced resources, computes context budget (always-loaded rule -content vs. available model context windows), and reports readiness. +one or more OpenShell gateways (each representing a provider/model +combination available for runs), a container image (carrying the +agent runtime and language toolchains), a prompt (standing +instructions for how the agent operates), and optionally a memory +service for accumulating domain knowledge across executions. An Agent +does not select a specific model — it declares what is available. +Gateway selection happens at execution time in the AgentRun. The +Agent controller validates that referenced gateways exist as Services +in the namespace and that SkillCards/SkillCollections are ready. Subagent delegation is a runtime concern — the agent runtime may spawn subagents internally but this is not modeled in the CRD. +_Avoid_: conflating Agent with AgentRun — Agent is a template, +AgentRun is an invocation. **AgentPlaybook** — A reusable playbook combining a high-level guide with an ordered sequence of stages. Each stage references an Agent and @@ -57,31 +56,32 @@ creating one does not execute anything. A future enhancement may introduce phases within stages for session continuity via shared PVCs. **AgentRun** — A request to execute a single Agent with specific -selections. References an Agent (or inlines the spec), selects which -providers and models to use for this run (from the Agent's available -set), carries instructions and generic parameters (key-value pairs -injected as environment variables into the Sandbox). Each model -selection includes a role (e.g. primary, efficient) that the base -image entrypoint maps to runtime-specific configuration. The -controller validates that selected providers/models are in the -Agent's available set, creates a Sandbox, and tracks status to -completion. Parameters are domain-agnostic — the Konveyor UI knows -to populate Hub-specific params (APP_ID, HUB_BASE_URL, etc.) for -migration use cases. +selections. References an Agent, selects which OpenShell gateway to +use for this run (from the Agent's available set), carries +instructions and generic parameters (key-value pairs injected as +environment variables into the sandbox). The controller validates +that the selected gateway is in the Agent's gateway list and that the +gateway Service exists, creates a sandbox through the OpenShell +gateway API, and tracks status to completion. Parameters are +domain-agnostic — the Konveyor UI knows to populate Hub-specific +params (APP_ID, HUB_BASE_URL, etc.) for migration use cases. **AgentPlaybookRun** — A request to execute an AgentPlaybook. References an AgentPlaybook (or inlines the spec) and carries generic parameters, -model selections, and env/envFrom. The controller orchestrates the -execution: creates an AgentRun per stage sequentially, all sharing the -same target branch. Cross-stage continuity comes from committed handoff -files (e.g. `.konveyor/handoff.md`). Tracks per-stage status (Pending, -Running, Succeeded, Failed). +a gateway selection, and env/envFrom. The controller orchestrates the +execution: creates an AgentRun per stage sequentially (each using the +selected gateway), all sharing the same target branch. Cross-stage +continuity comes from committed handoff files (e.g. +`.konveyor/handoff.md`). Tracks per-stage status (Pending, Running, +Succeeded, Failed). ## Personas **Platform Admin** — Creates and manages SkillCards, SkillCollections, -and LLMProviders. Responsible for what capabilities and infrastructure -are available to agents. +and OpenShell gateways. Deploys gateways via Helm, configures +providers and inference routing on each gateway via the OpenShell CLI. +Responsible for what capabilities and infrastructure are available to +agents. **Architect / PM** — Creates Agents and AgentPlaybooks. Defines the playbook for how migrations (or other agentic work) should be @@ -107,12 +107,37 @@ claude, cursor, windsurf, opencode, openclaw. agent workloads: Sandbox, SandboxTemplate, SandboxClaim, and SandboxWarmPool. Single-container design. Handles pod lifecycle, stable identity, network isolation, and warm pool pre-allocation. +The controller does not interact with Agent Sandbox directly — +OpenShell manages Sandbox CRs on our behalf. **OpenShell** — NVIDIA's secure-by-design runtime for autonomous -agents. A policy enforcement and governance layer that runs on top of -Agent Sandbox. Provides kernel-level isolation, declarative YAML-based -security policies, and inference routing. Complementary to Agent -Sandbox, not a replacement. +agents. Runs on top of Agent Sandbox. The controller's primary +execution interface — sandboxes are created through the OpenShell +gateway API (via the Go SDK), not by creating Sandbox CRs directly. +Each gateway is deployed via Helm as a Kubernetes Service, configured +with one provider and one model. The gateway's supervisor is +sideloaded into sandbox pods and provides: kernel-level isolation, +declarative YAML-based security policies, credential injection via +privacy proxy, and inference routing through `inference.local`. The +agent inside the sandbox calls `inference.local` and never sees real +LLM credentials. Providers, inference routes, and policies are +configured on the gateway by the Platform Admin — they are not +Kubernetes CRDs. +_Avoid_: treating OpenShell as optional — it is a hard dependency +for sandbox creation, replacing the direct Agent Sandbox dependency. + +**OpenShell Gateway** — An instance of the OpenShell control plane +deployed as a Kubernetes Service. Each gateway serves exactly one +provider/model combination via `inference.local`. Platform Admin +deploys multiple gateways (one per provider/model combo) into the +shared namespace via Helm. All gateways must live in the same +namespace as the controller and Hub — OpenShell's gateway, sandbox +pods, TLS Secrets, and database are all namespace-scoped. An Agent +references one or more gateways; an AgentRun selects one. The +controller connects to the gateway using the OpenShell Go SDK and +the gateway's client TLS credentials. +_Avoid_: gateway (lowercase) when referring to Kubernetes Gateway API +resources — always capitalize when referring to an OpenShell Gateway. **Hub** — The Konveyor application inventory and analysis engine. In the agentic platform Hub serves as a data service: agents call its API @@ -135,10 +160,10 @@ service instance. **SkillCard** CRs. - An **Agent** references zero or more **SkillCards** and zero or more **SkillCollections**. -- An **Agent** references one or more **LLMProviders** — declaring the - set of providers and models available for execution. -- An **AgentRun** references one **Agent** (or inlines it) and selects - specific providers, models, and roles from the Agent's available set. +- An **Agent** references one or more **OpenShell Gateways** — + declaring the set of provider/model combinations available for runs. +- An **AgentRun** references one **Agent** and selects one **OpenShell + Gateway** from the Agent's available set. - An **AgentPlaybook** organizes work into stages. Each stage references an **Agent** and carries instructions. - An **AgentPlaybookRun** references one **AgentPlaybook** (or inlines it) diff --git a/docs/adr/0004-openshell-as-execution-interface.md b/docs/adr/0004-openshell-as-execution-interface.md new file mode 100644 index 0000000..5981911 --- /dev/null +++ b/docs/adr/0004-openshell-as-execution-interface.md @@ -0,0 +1,221 @@ +# ADR 0004: OpenShell as Execution Interface + +**Status:** Accepted +**Date:** 2026-07-20 +**Authors:** David Zager + +The controller replaces its direct Agent Sandbox dependency with +OpenShell as the execution interface. Instead of creating +`agents.x-k8s.io` Sandbox CRs directly, the controller creates +sandboxes through the OpenShell gateway API using the OpenShell Go SDK. +This eliminates the LLMProvider CRD entirely — inference routing, +credential management, and security policy enforcement move to +OpenShell, which already solves these problems. + +## Context + +OpenShell (NVIDIA) is a secure runtime that layers on top of Agent +Sandbox. It provides inference routing through a sandbox-local +`inference.local` HTTPS endpoint, a privacy router that strips +sandbox-supplied credentials and injects real backend credentials, and +declarative security policies. OpenShell's Go SDK +(github.com/NVIDIA/OpenShell, in-progress) provides typed clients for +sandbox lifecycle management following client-go conventions, including +watch primitives suitable for controller use. + +### Current state + +The controller creates `agents.x-k8s.io` Sandbox CRs directly and +has RBAC permissions for that API group. The AgentRun controller +manages LLM credentials through the LLMProvider CRD: validates +Secrets, runs connectivity verification Jobs, and builds +`KONVEYOR_MODEL_*` environment variables to inject provider endpoints, +model names, and API keys into sandbox containers. The agent inside +the sandbox receives real LLM credentials in its environment. + +### What changes + +The controller stops creating Sandbox CRs directly and instead +creates sandboxes through the OpenShell gateway API using the Go SDK. +The LLMProvider CRD, its controller, and all credential plumbing are +removed. Agent references OpenShell Gateways instead of LLMProviders. +AgentRun selects a single gateway instead of provider/model/role +combinations. The controller's `agents.x-k8s.io` RBAC is no longer +needed — the OpenShell gateway owns Sandbox CR lifecycle. + +### Prerequisites + +- **OpenShell Go SDK** (NVIDIA/OpenShell#2270): PR A is in review, + 5 more PRs follow. Must land before implementation can begin. +- **OpenShell gateways deployed in the namespace**: Platform Admin + installs and configures gateways via Helm + OpenShell CLI. + +## Decision + +**The controller becomes an OpenShell gateway API client.** The AgentRun +controller creates sandboxes through the OpenShell Go SDK instead of +creating Sandbox CRs directly. OpenShell creates the Sandbox CRs on +our behalf and injects its supervisor, which handles credential +isolation, inference routing, and policy enforcement. + +**LLMProvider is removed.** The CRD, the controller, and all credential +plumbing (`buildEnvVars`, `validateModels`, verification Jobs) are +eliminated. Inference routing is OpenShell's problem — each gateway is +configured with one provider and one model by the Platform Admin. + +**Agent references OpenShell Gateways, not LLMProviders.** Each +OpenShell Gateway is a Kubernetes Service deployed via Helm, configured +with exactly one provider/model combination. An Agent declares one or +more gateways (the set available for runs). An AgentRun selects one +gateway from the Agent's list. The controller validates the gateway +Service exists and creates the sandbox through it. + +**No Gateway CRD.** OpenShell Gateways are admin-managed infrastructure +deployed independently. The controller discovers them as Services in +the shared namespace — it does not create, manage, or mirror them as +custom resources. + +**Multi-model per run is dropped.** ADR-0001 introduced per-run model +selection with roles (e.g. primary, efficient) via +`KONVEYOR_MODEL__*` env vars. OpenShell's `inference.local` +serves exactly one model per gateway, and each sandbox lives on one +gateway. Multi-role model selection within a single run is not +possible through OpenShell's routing. An AgentRun selects one +gateway = one model. The multi-role design was speculative and not +exercised in practice. + +**Agent Sandbox remains a transitive dependency.** OpenShell uses Agent +Sandbox under the hood. The controller no longer imports the Agent +Sandbox Go module or creates Sandbox CRs directly. Agent Sandbox must +still be installed in the cluster as an OpenShell prerequisite. + +## Deployment Model + +OpenShell deploys one gateway per Helm release. Each gateway is a +StatefulSet (or Deployment with external Postgres) with its own +Service, database, TLS Secrets, and RBAC — all namespace-scoped. +Sandboxes are created in the same namespace as the gateway +(`server.sandboxNamespace` defaults to the release namespace). + +**This means all OpenShell gateways must be installed in the same +namespace as our controller and Hub** (e.g. the konveyor namespace). +Multiple gateways coexist by using different Helm release names: + +```sh +helm install anthropic-gw oci://ghcr.io/.../helm-chart -n konveyor +helm install openai-gw oci://ghcr.io/.../helm-chart -n konveyor +``` + +Each produces a separate Service (`anthropic-gw`, `openai-gw`) that +the controller can reach without cross-namespace access. Each gateway +is fully independent — its own database, TLS material, provider +config, and inference route. They do not share state. + +**This is not multi-tenancy.** OpenShell has open upstream work on +multi-tenant deployments (NVIDIA/OpenShell#1722), a Kubernetes +operator (NVIDIA/OpenShell#1719), and cross-namespace sandbox +creation (NVIDIA/OpenShell#1795). Our design does not depend on any +of these. We are running multiple independent single-user gateway +instances in one namespace — this is supported today via Helm and +does not require upstream multi-tenancy features. + +OpenShell is explicit about its current scope: "Alpha software — +single-player mode. One developer, one environment, one gateway. We +are building toward multi-tenant enterprise deployments." There are +no Kubernetes CRDs for providers, inference routes, or policies — +configuration is imperative (CLI/gRPC against the gateway). + +### UX impact + +LLM provider configuration moves from "create an LLMProvider CR" to +"Platform Admin pre-provisions OpenShell gateways, users select from +them." The per-persona experience: + +- **Platform Admin**: deploys gateways via Helm, configures providers + and inference routing via the OpenShell CLI. This is infrastructure + setup comparable to deploying the Agent Sandbox controller today. +- **Architect**: references available gateways by Service name in Agent + CRDs. Does not interact with OpenShell directly. +- **Developer**: picks an Agent and a gateway, runs it. Does not + configure providers, credentials, or OpenShell. Hub presents + available gateways through its REST API / UI. + +Hub could eventually provide a UI for managing OpenShell gateways +(creating providers, setting inference routes) so the admin does not +need the CLI. That is a Hub feature, not a controller concern, and +not in scope for this decision. + +### Multi-tenancy limitations + +OpenShell's namespace-scoped gateway model means all gateways, all +sandbox pods, and all our CRs live in one namespace today. This works +for our current single-namespace deployment but does not scale to +multi-tenant scenarios where teams need namespace-level isolation +between gateway configurations. Relevant upstream work: + +- NVIDIA/OpenShell#1719 — Kubernetes operator (status: Idea) +- NVIDIA/OpenShell#1722 — Multi-tenant deployments (milestone: + OpenShell Beta, status: Idea) +- NVIDIA/OpenShell#1795 — Cross-namespace sandbox CRs (open) + +Multi-namespace support would require either: + +- OpenShell adding cross-namespace or cluster-scoped gateway support +- Us contributing upstream to OpenShell to expand their deployment model +- Running separate controller + OpenShell stacks per namespace + +This is a known constraint, not a blocker for the current phase. +Our single-namespace, multi-gateway design works today without any +upstream changes. + +## Alternatives Considered + +### Controller creates Sandbox CRs directly, OpenShell deployed alongside + +The controller continues creating Sandbox CRs and OpenShell is an +optional operational layer. Rejected because OpenShell's supervisor is +only injected into sandboxes created through its gateway API — Sandbox +CRs created by third parties bypass OpenShell entirely. There is no +webhook or sidecar injection model, and OpenShell's architecture does +not point toward one. + +### Hub as OpenShell gateway client instead of the controller + +Hub already mediates between the UI and Kubernetes. It could create +sandboxes through the OpenShell gateway. Rejected because this would +hollow out the controller — its core job is reconciling AgentRun CRs +into running workloads. Moving that to Hub eliminates the controller's +reason to exist. + +### New Gateway CRD to represent OpenShell Gateways + +A lightweight CRD mapping gateway names to endpoints and credentials, +with a controller that validates connectivity. Rejected because it +mirrors an external concept for no reason. Gateways are admin-managed +infrastructure — the controller just needs to verify the Service +exists, not own its lifecycle. + +### Keep LLMProvider alongside OpenShell + +Maintain LLMProvider for credential validation while using OpenShell +for execution. Rejected because OpenShell handles credential injection +through its privacy router — there is nothing left for LLMProvider to +do. Keeping it would mean maintaining dead validation logic. + +## Consequences + +- The controller gains a runtime dependency on OpenShell gateways + being deployed and reachable. This is the same shape of dependency as + the existing Agent Sandbox requirement. +- The `sigs.k8s.io/agent-sandbox` Go module dependency is replaced by + the OpenShell Go SDK. The controller's RBAC for `agents.x-k8s.io` + Sandboxes is removed — the gateway owns that lifecycle. +- Hub discovers gateways by listing annotated Services and exposes + them through its curated REST API. The UI shows available gateways + to users. +- Multi-model profiling (running the same agent against different + models to compare results) is supported by listing multiple gateways + on an Agent. +- RBAC for gateway access (restricting which users can use which + gateways) is a Hub concern, not a controller concern. The controller + validates structural correctness only.