Skip to content
This repository was archived by the owner on Jun 25, 2026. It is now read-only.

architecture.md

Latest commit

 

History

History
382 lines (312 loc) · 16 KB

architecture.md

File metadata and controls

382 lines (312 loc) · 16 KB
Raw

Architecture

Rascal has three runtime parts and a small set of internal abstractions that keep control-plane orchestration, execution, and persistence separate.

Core Model

The easiest way to read Rascal is to separate control-plane responsibilities from execution-plane responsibilities.

  • Control plane: rascal and rascald
  • Execution plane: detached runner containers launched via the Docker launcher
  • Runtimes: goose-codex, codex, claude, and goose-claude
  • Harnesses (derived from runtime): goose and direct
  • Model providers (derived from runtime): codex and anthropic
  • Packaging: separate runner images per runtime (Goose-Codex, Codex, Claude, Goose-Claude)

In simple terms:

  • A Task is the long-lived unit of work.
  • A Run is one attempt to advance that task.
  • A Task tracks the runtime selected for its latest run.
  • A Task may have one current task-scoped session record.
  • A Run uses the runtime recorded on that run and may resume the task's session when the runtime still matches.
  • A detached container is RunExecution state for a run, not the run itself.

This split is important during deploys and restarts:

  • Blue/green is control-plane topology for rascald.
  • Active work keeps running in detached containers in the execution plane.
  • After cutover or restart, the active slot recovers and adopts detached run supervision.

High-Level Flow

rascal (CLI) or GitHub webhook
            |
            v
      rascald control plane
  - create/update task
  - create run
  - persist state
  - schedule/supervise
            |
            v
   Docker launcher / execution plane
  - start detached runner container
  - inspect / stop / remove container
            |
            v
   rascal-runner in container
  - clone repo
  - run goose or codex
  - write artifacts and meta.json
  - push branch / update PR
            |
            v
      rascald finalization
  - read artifacts
  - update run/task state
  - post GitHub status/comments

Runtime Components

  1. rascal (CLI)
  • Local operator interface.
  • Handles init, deploy, config, run creation, logs, and control commands.
  • Lives in cmd/rascal.
  1. rascald (orchestrator server)
  • Receives API requests and GitHub webhooks.
  • Persists task, run, lease, cancellation, and detached execution state.
  • Schedules runs serially per task and concurrently across different tasks.
  • Starts detached runner containers and supervises them via persisted execution handles.
  • Supports blue/green slot handoff by letting the previous slot drain for one generation and reclaiming the oldest drainer on the next deploy when needed.
  • Lives in cmd/rascald.
  1. Runner container (rascal-runner)
  • Clones the repository and checks out the target branches.
  • Executes the selected runtime (goose-codex, codex, claude, or goose-claude).
  • Commits changes, pushes the head branch, and creates or reuses a PR.
  • Writes canonical artifacts into mounted /rascal-meta.
  • Runtime logic lives in Go in cmd/rascal-runner.
  • runner/entrypoint.sh is a thin shim that only executes /usr/local/bin/rascal-runner.

Code-Level Abstractions

These are the main layers in the Go codebase.

  1. Entry points
  • cmd/rascal: operator-facing CLI.
  • cmd/rascald: HTTP API, webhook handling, scheduling, supervision, recovery.
  • cmd/rascal-runner: in-container task executor.
  1. Agent abstraction
  • internal/runtime defines Runtime, Harness, ModelProvider, and SessionMode.
  • Runtime is the user-facing selection; Harness and ModelProvider are derived from it via Runtime.Harness() and Runtime.Provider() methods.
  1. Execution abstraction
  • internal/runner defines the Runner launcher interface and Spec/ExecutionHandle contract.
  • Current production implementation is Docker; noop exists for non-runtime/test scenarios.
  • Session mounting is harness-aware: Goose uses GOOSE_PATH_ROOT, Codex uses CODEX_HOME, Claude uses CLAUDE_CONFIG_DIR.
  1. Control-plane and client boundaries
  • internal/github is the single deep GitHub boundary for API calls, webhook payload interpretation helpers, and comment rendering helpers.
  • internal/apiclient owns the CLI transport layer for HTTP and SSH-backed requests to rascald.
  • internal/clientconfig owns client config load/save/effective-resolution behavior.
  • internal/remote owns shared SSH/SCP/shell-quoting primitives reused by CLI and deploy flows.
  1. Persistence abstraction
  • internal/state owns SQLite-backed persistence and state transitions.
  • It stores runs, tasks, run leases, detached run executions, cancel requests, webhook deliveries, task agent session records, and encrypted stored credentials plus credential leases.
  • SQL schema lives in embedded migrations and typed queries are generated under internal/state/sqlitegen.
  1. Supporting integrations
  • internal/runsummary: PR body and completion comment formatting.
  • internal/logs: tailing run log files.

Packaging Model

  • Rascal builds and deploys one orchestrator binary: rascald.
  • Rascal also builds one runner binary: rascal-runner.
  • That runner binary is packaged into separate Docker images for Goose and Codex.
  • rascald selects the runner image based on the task/run runtime.
  • Blue/green deploy replaces the control plane, while runner containers remain detached in the execution plane.

Execution Flow

  1. User triggers a run from the CLI or via GitHub webhook.
  2. rascald creates or updates task context, writes run artifacts, and queues the run.
  3. Scheduler claims a queued run, enforces per-task serialization, and records a run lease.
  4. rascald resolves runtime/session settings and persists a deterministic detached execution handle.
  5. internal/runner starts a detached Docker container for rascal-runner.
  6. Active slot supervises the detached execution by inspect/stop/remove operations and lease heartbeats.
  7. On slot rotation or process restart, a new slot can recover the persisted handle and adopt supervision.
  8. rascal-runner finalizes meta.json; rascald reads that artifact, updates run/task state, posts GitHub reactions/comments, and removes the container.
  9. User monitors via ps, logs, and open.

Lifecycle Summary

Task and run lifecycle:

Task created or reused
    |
    +--> Run queued --> Run running --> review | succeeded | failed | canceled
                    |
                    +--> detached RunExecution created and supervised

Deploy and recovery lifecycle:

active slot A running
    |
    +--> if slot B is still draining from an earlier deploy, reclaim B
    +--> deploy prepares slot B
    +--> slot B passes readiness
    +--> traffic flips to B
    +--> slot A enters deploy-drain and may keep supervising active runs
    +--> later deploy or restart may reclaim/adopt remaining executions

System Invariants

  • A run belongs to exactly one task.
  • A run uses the runtime recorded on that run.
  • A task may have at most one current task-scoped session record.
  • Changing a task runtime must discard incompatible task-scoped session resume state before the next run starts.
  • At most one orchestrator instance should own a run lease at a time.
  • run_executions store detached execution metadata, not user-visible business progress.
  • Only the active slot should process webhook traffic during blue/green overlap.

Persistence Model

Persistent state is stored on the server in a SQLite database under the Rascal data directory.

By default, task-scoped session state is also stored on disk under ${RASCAL_DATA_DIR}/agent-sessions/<task-key>/.

Runs stay short-lived. Each run mounts its run directory plus, when session resume is enabled, a task-scoped session directory. There is no always-on background worker.

Key persisted entities:

  • runs: user-visible execution records and final outcome.
  • tasks: long-lived task identity across retries and follow-up feedback; API responses include a derived pending_input flag (computed from queued runs, not stored as a task column).
  • run_leases: supervision ownership and heartbeat expiry.
  • run_executions: detached execution handle metadata for adoption and cleanup.
  • run_cancels: persisted cancel intent.
  • task sessions: stable harness session identifiers and mounted session roots. In the current SQLite schema this data lives in the task_sessions table.
  • credentials: encrypted stored credential payloads and allocation metadata.
  • credential_leases: per-run credential assignments and lease expiry state.
  • deliveries: webhook dedupe/claim bookkeeping.

Where State Lives

Location What lives there Notes
SQLite state DB tasks, runs, leases, execution handles, sessions, credentials Primary control-plane source of truth
Run directory per-run artifacts, logs, meta.json, transient auth material Short-lived execution artifacts
Task session directory resumable harness session state Optional and task-scoped
Docker runtime detached runner container process state Execution-plane state, not the system of record
Caddy and systemd config on host active slot routing and service activation Deployment/control-plane topology

Source of Truth by Object

Object Source of truth Why
Task tasks table Durable unit of work across iterations
Run runs table User-visible attempt and final outcome
Active supervision owner run_leases table Coordinates which rascald instance supervises
Detached container identity run_executions table Enables adoption and cleanup across restarts
Session resume state task session records plus mounted session directory Tracks harness session identity and storage root
Run artifacts run directory on disk Execution outputs consumed during finalization
Live container process Docker runtime Actual execution process while the run is active

Run Artifacts

Each run directory stores metadata and artifacts such as:

  • context.json
  • instructions.md
  • runner.log
  • agent.ndjson (canonical agent stream log path for all runtimes)
  • agent_output.txt (structured/fallback agent output, especially for Codex)
  • commit_message.txt
  • pr_body.md
  • meta.json
  • SQLite-backed run response targets and completion-comment state, with file fallbacks only for legacy runs flows are used

Session Behavior

  • Session policy is configured at the orchestrator via off, pr-only, or all.
  • pr-only currently resumes for pr_comment, pr_synchronize, pr_review, pr_review_comment, pr_review_thread, retry, and issue_edited.
  • Goose resumes by named Goose session plus mounted session storage.
  • Codex resumes by reusing a task-scoped CODEX_HOME and the discovered harness session id.
  • If a task switches runtime between runs, Rascal starts a fresh session for the new runtime and replaces the stored task session record.
  • If a Goose resume attempt fails because the stored session is missing or invalid, the runner falls back to a fresh session.

Failure and Recovery

Common failure and recovery cases:

  • rascald restart: persisted run execution handles let the restarted process recover and re-adopt detached runs.
  • Blue/green deploy during active work: detached containers keep running while the new active slot adopts supervision.
  • Missing detached container during adoption: Rascal marks the run failed because execution disappeared before finalization.
  • Lease ownership loss: the local instance stops supervision so another instance can take over safely.
  • Credential lease renewal failure: Rascal requests cancellation and attempts to stop the detached run.
  • Cancel during slot rotation: cancel intent is persisted, and the active slot after cutover should still stop/finalize the run.

Credential Handling

Rascal uses stored credentials tagged by provider and managed by rascald.

  • Stored credentials are encrypted before being persisted in SQLite.
  • Each credential has a provider tag (codex or anthropic) that determines which runtimes can use it:
    • codex credentials (default, including legacy credentials with empty provider): used by codex and goose-codex runtimes via auth.json.
    • anthropic credentials: used by claude and goose-claude runtimes via OAuth token.
  • Each credential is either personal (owned by a user) or shared.
  • When a run starts, rascald asks the credential broker to lease a credential matching the run's runtime and records the selected credential id in state.
  • The broker chooses from eligible credentials using the configured allocation strategy, provider compatibility filter, and tracks lease assignment per run.
  • The leased auth blob is written into a per-run secrets directory outside the broad /rascal-meta mount and then mounted read-only into the container at /run/rascal-secrets (codex_auth.json for codex/goose runs, claude_oauth_token for claude/goose-claude runs). Legacy run-local auth paths remain as fallback for older runs.
  • While a run is active, rascald renews the credential lease. If renewal is lost, the run is canceled.
  • Bootstrap and deploy can seed an initial shared stored credential from a local Codex auth file.
  • Operators can manage credentials with rascal auth credentials ... and use --provider codex|anthropic to tag credentials for specific providers.

Runner Environment Contract

Required:

  • RASCAL_RUN_ID
  • RASCAL_TASK_ID
  • RASCAL_REPO
  • GH_TOKEN or GH_TOKEN_FILE

Common optional:

  • RASCAL_INSTRUCTION
  • RASCAL_AGENT_RUNTIME (goose-codex, codex, claude, or goose-claude; defaults to goose-codex when unset; goose is accepted as an alias)
  • RASCAL_BASE_BRANCH (default: main)
  • RASCAL_HEAD_BRANCH (runner fallback default: rascal/<run_id> when unset; rascald normally sets a task-derived branch and may reuse the previous head branch for PR comment/review follow-ups)
  • RASCAL_ISSUE_NUMBER (default: 0)
  • RASCAL_PR_NUMBER (default: 0)
  • RASCAL_TRIGGER (default: cli)
  • RASCAL_GOOSE_DEBUG (default: true)
  • RASCAL_CONTEXT
  • RASCAL_META_DIR (default: /rascal-meta)
  • RASCAL_WORK_ROOT (default: /work)
  • RASCAL_REPO_DIR (default: ${RASCAL_WORK_ROOT}/repo)
  • RASCAL_TASK_SESSION_MODE (off, pr-only, all; orchestrator default: all)
  • RASCAL_TASK_SESSION_RESUME (set by orchestrator per run)
  • RASCAL_TASK_SESSION_KEY (stable task-scoped key when resume is enabled)
  • RASCAL_TASK_SESSION_ID (runtime session id when known)
  • CODEX_HOME (run-scoped /rascal-meta/codex in stateless mode, or task-scoped mount in resume mode for Codex)
  • CODEX_AUTH_FILE (default secure mode path: /run/rascal-secrets/codex_auth.json)
  • CLAUDE_CODE_OAUTH_TOKEN_FILE (default secure mode path: /run/rascal-secrets/claude_oauth_token)
  • GOOSE_PATH_ROOT (run-scoped /rascal-meta/goose in stateless mode, or task-scoped mount in resume mode for Goose)

Further Reading