[1/6][rollout] feat: add TtlCache utility - #283
Merged
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
mathewjhan
changed the title
JoyboyBrian
approved these changes
Aug 6, 2026
mathewjhan
added a commit
that referenced
this pull request
Aug 6, 2026
Part of splitting #272 into reviewable pieces (4/6). Builds on #284 (container contract) and #285 (packaging). ## What this adds `HarborBackendV2`: runs each rollout as a Harbor trial. The agent can be either a user workflow (packaged into a wheel and installed in the container at trial start) or a registered native Harbor agent (`terminus-2`, `mini-swe-agent`, `oracle`) with the rollout endpoint injected into its environment. How a rollout flows through it: 1. **Task selection** (`tasks.py`): template mode uses one task directory for every rollout; dataset mode routes by `metadata["harbor_task_id"]` to a folder under `tasks_dir` (path escapes rejected); `metadata["harbor_task"]` fetches a task from a local path, git checkout, or registry package, with per-ref locks so concurrent rollouts download once. 2. **Materialization**: the task is copied into a per-rollout directory; the rollout's input file is staged; if the task has no `tests/` and a grader exists, a `test.sh` is generated that installs and runs the grader. The ground-truth label is staged only into `tests/`, which Harbor uploads at verification time — the agent phase cannot read it. 3. **Image preparation**: `patch_dockerfile_with_sdk` appends a block to the task's Dockerfile that installs a static `uv` binary and creates `/opt/osmosis/venv` with the bundle's dependencies pre-installed. Per-trial installs then only add the user's own code (`--no-deps`), which cuts container startup from minutes to seconds. The patch is deterministic, so identical tasks keep identical image content hashes and share builds. 4. **Execution** (`harness_agent.py`): the installed agent uploads the wheel, installs it into the venv, backfills an empty prompt from the task's `instruction.md`, runs the agent script, and returns the result through the trial's agent metadata. 5. **Callbacks**: the workflow-complete callback fires when verification starts (agent phase over); the grader-complete callback fires at trial end with the reward parsed from Harbor's verifier result. Callback delivery failures are logged and never abort trial archival. 6. **Observability and lifecycle**: per-phase timings and failure phases in every result (`diagnostics.py`), native-agent ATIF parsing with secret redaction, artifact relocation, `prewarm()` / `prewarm_lifespan()` to build task images before serving traffic, `cancel_rollouts(ids | prefix | all)`, `rollout_status()` with terminal outcomes retained in a `TtlCache` (#283), and admission control via `max_queue_depth`. ## Example ```python backend = HarborBackendV2( orchestrator=TrialQueue(n_concurrent=100), tasks_dir=Path("tasks"), # 300 task folders task_mode="dataset", agent=MyWorkflow, # or agent="mini-swe-agent" workflow_config=my_config, environment_config=EnvironmentConfig(type=EnvironmentType.SKYPILOT), ) app = create_rollout_server( backend=backend, lifespan=backend.prewarm_lifespan(task_ids=["task-0000"]), ) ``` A trainer then POSTs rollouts with `metadata={"harbor_task_id": "task-0042"}`; each one runs in its own sandbox and reports back through the callbacks. Co-authored-by: Cursor <cursoragent@cursor.com>
github-actions
Bot
added
enhancement
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Why
How to Test
Checklist
[module] type: descriptionformatenhancement,bug,breaking)ruff check .andruff format --check .passpyright osmosis_ai/passespytestpasses (new tests added if applicable)Summary by cubic
Adds
TtlCachetorolloutfor uniform TTL-based caching with amortized O(1) expiry using tombstones. Provides simple set/get semantics with TTL refresh on overwrite and is fully unit tested.time.monotonicwith a single TTL for all entries.getreturns None for missing/expired keys;__len__reflects live entries.Written for commit ac3cf16. Summary will update on new commits.