[3/6][misc] feat: package rollout projects into installable wheel bundles - #285
Merged
Conversation
mathewjhan
changed the title
mathewjhan
force-pushed
the
mathew/packaging
branch
from
August 6, 2026 22:29
ad6f921 to
80c19ad
Compare
JoyboyBrian
approved these changes
Aug 6, 2026
Co-authored-by: Cursor <cursoragent@cursor.com>
mathewjhan
force-pushed
the
mathew/packaging
branch
from
August 6, 2026 22:31
80c19ad to
a0e5a1f
Compare
mathewjhan
added a commit
that referenced
this pull request
Aug 6, 2026
Part of splitting #272 into reviewable pieces (4/6). Builds on #284 (container contract) and #285 (packaging). ## What this adds `HarborBackendV2`: runs each rollout as a Harbor trial. The agent can be either a user workflow (packaged into a wheel and installed in the container at trial start) or a registered native Harbor agent (`terminus-2`, `mini-swe-agent`, `oracle`) with the rollout endpoint injected into its environment. How a rollout flows through it: 1. **Task selection** (`tasks.py`): template mode uses one task directory for every rollout; dataset mode routes by `metadata["harbor_task_id"]` to a folder under `tasks_dir` (path escapes rejected); `metadata["harbor_task"]` fetches a task from a local path, git checkout, or registry package, with per-ref locks so concurrent rollouts download once. 2. **Materialization**: the task is copied into a per-rollout directory; the rollout's input file is staged; if the task has no `tests/` and a grader exists, a `test.sh` is generated that installs and runs the grader. The ground-truth label is staged only into `tests/`, which Harbor uploads at verification time — the agent phase cannot read it. 3. **Image preparation**: `patch_dockerfile_with_sdk` appends a block to the task's Dockerfile that installs a static `uv` binary and creates `/opt/osmosis/venv` with the bundle's dependencies pre-installed. Per-trial installs then only add the user's own code (`--no-deps`), which cuts container startup from minutes to seconds. The patch is deterministic, so identical tasks keep identical image content hashes and share builds. 4. **Execution** (`harness_agent.py`): the installed agent uploads the wheel, installs it into the venv, backfills an empty prompt from the task's `instruction.md`, runs the agent script, and returns the result through the trial's agent metadata. 5. **Callbacks**: the workflow-complete callback fires when verification starts (agent phase over); the grader-complete callback fires at trial end with the reward parsed from Harbor's verifier result. Callback delivery failures are logged and never abort trial archival. 6. **Observability and lifecycle**: per-phase timings and failure phases in every result (`diagnostics.py`), native-agent ATIF parsing with secret redaction, artifact relocation, `prewarm()` / `prewarm_lifespan()` to build task images before serving traffic, `cancel_rollouts(ids | prefix | all)`, `rollout_status()` with terminal outcomes retained in a `TtlCache` (#283), and admission control via `max_queue_depth`. ## Example ```python backend = HarborBackendV2( orchestrator=TrialQueue(n_concurrent=100), tasks_dir=Path("tasks"), # 300 task folders task_mode="dataset", agent=MyWorkflow, # or agent="mini-swe-agent" workflow_config=my_config, environment_config=EnvironmentConfig(type=EnvironmentType.SKYPILOT), ) app = create_rollout_server( backend=backend, lifespan=backend.prewarm_lifespan(task_ids=["task-0000"]), ) ``` A trainer then POSTs rollouts with `metadata={"harbor_task_id": "task-0042"}`; each one runs in its own sandbox and reports back through the callbacks. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of splitting #272 into reviewable pieces (3/6).
What this adds
osmosis_ai/packaging.py: builds a standard Python wheel from a user's rollout project so the project can be installed inside a task container with onepip install.build_bundle(project_dir, workflow=..., grader=...)produces a wheel containing the project's package plus a generatedbundle_main.pyshim. The shim imports the user's classes directly (from my_harness.solver import MyWorkflow) and exposes two console scripts,<package>-agentand<package>-grade, which call the runner entrypoints from [2/6][rollout] feat: container contract, workflow output types, in-container runner #284. Nothing is resolved dynamically at runtime; the class binding happens at build time.platformdirs), so rebuilding an unchanged project is free.inspect_bundle(wheel)reads the wheel's metadata withimportlib.metadataand returns the declared dependencies (keeping environment markers, dropping extras-gated entries) plus the two script names. The Harbor backend (next in the stack) uses this list to pre-install dependencies into the task image.Also includes the
bench_harnessfixture project the packaging tests build against, and theplatformdirsdependency.Example
Inside a container,
pip install my_harness-0.1.0-py3-none-any.whlfollowed by runningmy-harness-agentexecutes the user's workflow with no other setup.