Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
531 changes: 531 additions & 0 deletions .github/scripts/verify-wheel-install.py

Large diffs are not rendered by default.

86 changes: 67 additions & 19 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
python-version: "3.12"
enable-cache: true
- name: Install dependencies
run: uv sync --locked --extra dev
run: uv sync --locked --all-extras --group dev
- name: Run pyright
run: uv run pyright osmosis_ai/
- name: Verify public API types
Expand All @@ -50,13 +50,10 @@ jobs:
# Known --verifytypes baselines we intentionally skip:
# * agent_adapter: inherits from harbor's BaseInstalledAgent which
# has no type stubs ("base class unknown").
# * convert_sample_to_trajectory: returns harbor's Trajectory which
# has no type stubs ("return type is unknown").
unexpected=$(echo "$output" | awk '
/^osmosis_ai\./ { sym = $0 }
/error:/ {
if (sym ~ /agent_adapter/) next
if (sym ~ /convert_sample_to_trajectory/) next
print $0
}
')
Expand All @@ -82,7 +79,7 @@ jobs:
enable-cache: true

- name: Install dependencies
run: uv sync --locked --extra dev
run: uv sync --locked --all-extras --group dev

- name: Run tests with coverage
if: matrix.python-version == '3.12'
Expand Down Expand Up @@ -150,22 +147,73 @@ jobs:
sys.exit(1)
PY

- name: Smoke-test the built wheel
run: |
uv venv --python 3.12 /tmp/osmosis-wheel-smoke
uv pip install --python /tmp/osmosis-wheel-smoke/bin/python dist/*.whl
/tmp/osmosis-wheel-smoke/bin/python - <<'PY'
import osmosis_ai
from agents.usage import Usage
from osmosis_ai.rollout.integrations.agents.openai_agents import OsmosisLitellmModel

assert Usage().input_tokens_details.cached_tokens == 0
print(osmosis_ai.__version__, OsmosisLitellmModel.__name__)
PY
/tmp/osmosis-wheel-smoke/bin/osmosis --version

- name: Upload dist artifact
uses: actions/upload-artifact@v7
with:
name: dist
path: dist/

wheel-smoke:
name: Wheel smoke (${{ matrix.scenario }})
needs: build
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- scenario: bare
extra: ""
- scenario: server
extra: "[server]"
- scenario: strands
extra: "[strands]"
- scenario: openai-agents
extra: "[openai-agents]"
- scenario: harbor
extra: "[harbor]"
- scenario: rubric
extra: "[rubric]"
- scenario: parquet
extra: "[parquet]"
- scenario: full
extra: "[full]"
steps:
- uses: actions/checkout@v7

- uses: astral-sh/setup-uv@v7
with:
python-version: "3.12"
enable-cache: true

- name: Download built wheel
uses: actions/download-artifact@v8
with:
name: dist
path: dist/

- name: Install and verify wheel
env:
INSTALL_EXTRA: ${{ matrix.extra }}
SMOKE_SCENARIO: ${{ matrix.scenario }}
run: |
mapfile -t wheels < <(find "$GITHUB_WORKSPACE/dist" -maxdepth 1 -type f -name '*.whl' -print | sort)
if [ "${#wheels[@]}" -ne 1 ]; then
echo "::error::Expected exactly one wheel, found ${#wheels[@]}"
exit 1
fi

smoke_root=$(mktemp -d)
python="$smoke_root/venv/bin/python"
install_spec="${wheels[0]}${INSTALL_EXTRA}"

uv venv --python 3.12 "$smoke_root/venv"
(
cd "$smoke_root"
env -u PYTHONPATH uv pip install --python "$python" "$install_spec"
env -u PYTHONPATH uv pip check --python "$python"
env -u PYTHONPATH "$python" -I \
"$GITHUB_WORKSPACE/.github/scripts/verify-wheel-install.py" \
--scenario "$SMOKE_SCENARIO" \
--source-root "$GITHUB_WORKSPACE"
)
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@

This file records changes to `osmosis-ai`. For earlier versions, see [GitHub Releases](https://github.com/Osmosis-AI/osmosis-sdk-python/releases).

## Unreleased

### Breaking Changes

- Runtime features now use independent installation extras: `server`, `strands`, `openai-agents`, `harbor`, `rubric`, and `parquet`; `full` installs all of them. The former `platform` extra is replaced by `parquet`.
- Development tools are no longer published through the `dev` extra. From a source checkout, install the PEP 735 dependency group with `uv sync --all-extras --group dev` or `python -m pip install -e ".[full]" --group dev`.
- Rollout feature imports moved out of `osmosis_ai.rollout`:
- Server: `from osmosis_ai.rollout.server import create_rollout_server, ControllerAuth`
- Strands: `from osmosis_ai.rollout.integrations.agents.strands import OsmosisStrandsAgent, OsmosisRolloutModel`
- OpenAI Agents: `from osmosis_ai.rollout.integrations.agents.openai_agents import OsmosisAgent`
- Harbor: `from osmosis_ai.rollout.backend.harbor import HarborBackend`
- Harbor workflow context: `from osmosis_ai.rollout.context import HarborAgentWorkflowContext`
- `evaluate_rubric` is no longer included by `from osmosis_ai import *`. Import it explicitly from `osmosis_ai.eval.rubric` and install the `rubric` extra.
- The Harbor extra no longer installs Daytona or SkyPilot. Daytona support is retired; the rollout runtime must provide SkyPilot when it is used.

## 0.3.0rc1 - 2026-07-28

### Breaking Changes
Expand Down
11 changes: 7 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
```bash
git clone https://github.com/Osmosis-AI/osmosis-sdk-python
cd osmosis-sdk-python
uv sync --extra dev
uv sync --all-extras --group dev
pre-commit install
uv run pytest
```
Expand All @@ -18,7 +18,8 @@ uv run pytest
git clone https://github.com/Osmosis-AI/osmosis-sdk-python
cd osmosis-sdk-python
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
python -m pip install --upgrade pip
python -m pip install -e ".[full]" --group dev
Comment thread
JoyboyBrian marked this conversation as resolved.
pre-commit install
pytest
```
Expand Down Expand Up @@ -52,9 +53,11 @@ Ruff is pinned to one version across `pyproject.toml`, `.pre-commit-config.yaml`

## Type Checking

[Pyright](https://microsoft.github.io/pyright/) is the type checker, included in the `dev` extras.
[Pyright](https://microsoft.github.io/pyright/) is the type checker, included in
the `dev` dependency group.

- **Pyright** — must pass. All errors must be resolved before merging.
- **Pyright** — must pass. All errors must be resolved before merging. It is
installed from the `dev` dependency group.
- **Pyright `--verifytypes`** — must pass. Ensures all public API symbols have complete type annotations.

Configuration lives in `pyproject.toml` under `[tool.pyright]`.
Expand Down
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,18 @@ Python SDK and CLI for [Osmosis AI](https://platform.osmosis.ai), a platform for
Requires **Python 3.12+**.

```bash
pip install osmosis-ai # Core SDK
pip install osmosis-ai[server] # + FastAPI rollout server
pip install osmosis-ai # CLI + framework-neutral rollout core
pip install "osmosis-ai[server]" # + generic FastAPI rollout server
pip install "osmosis-ai[strands]" # + Strands integration
pip install "osmosis-ai[openai-agents]" # + OpenAI Agents integration
pip install "osmosis-ai[harbor]" # + Harbor backend (uses an externally provided SkyPilot runtime)
pip install "osmosis-ai[rubric]" # + LLM-as-judge rubric evaluation
pip install "osmosis-ai[parquet]" # + Parquet dataset support
pip install "osmosis-ai[full]" # every optional feature
# or with uv: uv add osmosis-ai
```

See [Installation](https://docs.osmosis.ai/cli/installation) for the full extras matrix and [CONTRIBUTING.md](CONTRIBUTING.md) for development setup.
There is one distribution, `osmosis-ai`. The `harbor` extra installs plain Harbor only: Daytona is retired, and Harbor's `skypilot` extra must not be installed because the rollout runtime provides SkyPilot. See [Installation](https://docs.osmosis.ai/cli/installation) for product setup and [CONTRIBUTING.md](CONTRIBUTING.md) for development setup.

## Documentation

Expand Down
4 changes: 3 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@ The package (`osmosis_ai/`) is organized into top-level domains. See [architectu
| Eval helpers | [../osmosis_ai/eval/](../osmosis_ai/eval/) | Rubric (LLM-as-judge) + workflow/grader loader | `from osmosis_ai.eval.rubric import evaluate_rubric` |
| Workspace templates | [../osmosis_ai/templates/](../osmosis_ai/templates/) | `osmosis template` recipe catalog + source resolution | (internal) |

The single `osmosis-ai` distribution always includes the CLI and framework-neutral rollout core. Install extras only for the feature you use: `server`, `strands`, `openai-agents`, `harbor`, `rubric`, `parquet`, or `full`. The Harbor extra installs plain Harbor for an externally provided SkyPilot runtime; Daytona is retired, and Harbor's `skypilot` extra is intentionally unsupported.

## Pages

- [architecture.md](./architecture.md) — package layout, domain boundaries, import paths, lazy-loading rules, and the remote rollout protocol (controller <-> rollout server). Start here.
- [rollout-sdk.md](./rollout-sdk.md) — the library API you implement against: `AgentWorkflow`, `Grader`, contexts, configs, `create_rollout_server`, execution backends, and framework integrations.
- [rollout-sdk.md](./rollout-sdk.md) — the library API you implement against: `AgentWorkflow`, `Grader`, contexts, configs, server/backends, and framework integrations.
- [eval.md](./eval.md) — the `osmosis eval submit` config contract (SDK-vs-backend validation, submit flow), plus a brief note on the `evaluate_rubric` / `osmosis eval rubric` LLM-as-judge API.
- [datasets.md](./datasets.md) — the dataset row contract enforced by the SDK validator.
- [troubleshooting.md](./troubleshooting.md) — engineering issues (rollout timeouts, event-loop blocking, concurrency tuning).
Expand Down
16 changes: 10 additions & 6 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ osmosis_ai/
│ ├── context.py # RolloutContext / AgentWorkflowContext / GraderContext
│ ├── driver.py # RolloutDriver — eval-facing execution contract
│ ├── validator.py # Static backend validation
│ ├── server/ # create_rollout_server (FastAPI) + ControllerAuth
│ ├── backend/ # ExecutionBackend ABC + Local / Harbor backends
│ ├── server/ # optional generic FastAPI server (`[server]`)
│ ├── backend/ # ExecutionBackend ABC + Local / optional Harbor backend
│ ├── types/ # protocol.py, config.py, sample.py
│ └── integrations/ # Strands / OpenAI Agents adapters
│ └── integrations/agents/ # Strands / OpenAI Agents adapters
├── eval/ # Eval helpers
│ ├── rubric/ # evaluate_rubric() LLM-as-judge engine
│ └── common/cli.py # Workflow + grader loader (used by cloud submit preflight)
Expand All @@ -39,7 +39,7 @@ osmosis_ai/

- `cli/` — the CLI framework layer plus every command group. Files in [../osmosis_ai/cli/commands/](../osmosis_ai/cli/commands/) are thin shells that delegate to business logic; see [cli.md](./cli.md).
- `platform/` — anything that calls the Osmosis Platform API. Business-logic helpers (no Typer registration) live in [../osmosis_ai/platform/cli/](../osmosis_ai/platform/cli/).
- `rollout/` — the remote rollout protocol SDK: the `AgentWorkflow` + `Grader` abstraction, execution backends, and the FastAPI server. See [rollout-sdk.md](./rollout-sdk.md).
- `rollout/` — the remote rollout protocol SDK: the `AgentWorkflow` + `Grader` abstraction and framework-neutral execution core. The generic FastAPI server and framework/back-end adapters are explicit optional modules; see [rollout-sdk.md](./rollout-sdk.md).
- `eval/` — `rubric/` powers `osmosis eval rubric` (see [eval.md](./eval.md)); `common/cli.py` exposes the workflow + grader loader that cloud `eval submit` / `train submit` preflight uses.

## Key import paths
Expand All @@ -49,10 +49,14 @@ from osmosis_ai.cli.errors import CLIError
from osmosis_ai.cli.console import Console
from osmosis_ai.platform.auth import load_credentials
from osmosis_ai.eval.rubric import evaluate_rubric, RubricResult
from osmosis_ai.rollout import AgentWorkflow, Grader, create_rollout_server
from osmosis_ai.rollout import AgentWorkflow, Grader, LocalBackend, SampleSource
from osmosis_ai.rollout.server import create_rollout_server
from osmosis_ai.rollout.backend.harbor import HarborBackend
from osmosis_ai.rollout.integrations.agents.strands import OsmosisStrandsAgent
from osmosis_ai.rollout.integrations.agents.openai_agents import OsmosisAgent
```

`osmosis_ai.rollout` is **not** re-exported at the package top level — import it directly.
`osmosis_ai.rollout` is **not** re-exported at the package top level — import it directly. Its public surface is framework-neutral core only; it does not export the server or Strands integration. `server`, `harbor`, `strands`, and `openai-agents` each require their matching installation extra. The generic server has no Harbor dependency.

## Lazy loading

Expand Down
Loading