feat(scenario-up): source-pinned platform/KAS/Go-SDK in instance mode (DSPX-3356 repro)#460
Draft
dmihalcik-virtru wants to merge 16 commits into
Draft
feat(scenario-up): source-pinned platform/KAS/Go-SDK in instance mode (DSPX-3356 repro)#460dmihalcik-virtru wants to merge 16 commits into
dmihalcik-virtru wants to merge 16 commits into
Conversation
`install scenario` could not run as written: it iterated `ScenarioSdks.union()` as a dict (it returns a list) and passed a `source=` kwarg `install_release` does not accept. The emitted `installed.json` shape also did not match what `scenario_to_pytest_sdks` reads (per-role lists, not sdk-name-keyed dict), so even the platform-only path produced a manifest no downstream tool could consume. Source fixes: - cli_scenario.py: iterate `union()` as the list it is, cache installs by (sdk, version, source), emit role-keyed lists matching the reader's expected shape; on failure write a partial manifest with `status=partial` so half-installed dist trees are diagnosable. Catch YAMLError in `_peek_kind` to surface a clean typer error. - platform_installer.py: `_git_rev_parse` raises on failure instead of silently writing an empty `sha=` into `.version`. Missing `scripts/` raises instead of warning-and-continuing. SHA passthrough heuristic tightened from `>=7` chars to exactly 40 (SHA-1) or 64 (SHA-256), so ambiguous short tags like `abc1234` no longer skip the `service/` prefix. Dropped a docstring fragment pointing to a planning doc that won't exist post-merge. - cli_install.py: dropped a docstring whose "deferred import" claim was false (the registration runs at module import). `lts platform` with no pinned version now exits 1 instead of warning-and-exit-0. Tests: - test_platform_installer.py: parametrized cases for `_resolve_platform_ref` covering version normalization, branch passthrough, the tightened hex heuristic, and SHA-1/SHA-256 passthrough. - test_cli_scenario.py: end-to-end smoke that mocks the installers and asserts the produced manifest is round-trip consumable by `scenario_to_pytest_sdks`. This is the gating test that would have caught the original bug. 79 passing (was 67). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- platform_installer: fix worktree update from bare repo (no `origin` remote exists), use `git reset --hard <branch>` instead of `git pull` - platform_installer: stop swallowing subprocess output so long-running `go build`/`git clone` progress is visible to the user - cli_install: extract `_install_platform_or_exit` to dedupe platform handling across `lts`, `tip`, and `release` - cli_scenario: parse manifest YAML once and dispatch by `kind`, instead of peeking + re-parsing in each loader Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…typing - AGENTS.md: add "Before Committing Python Changes" section requiring `uv run ruff check`, `uv run ruff format`, and `uv run pyright` on any touched Python package before commit. Explicitly call out that `uvx` must NOT be used for pyright (isolated env can't see project deps, so every project import becomes a spurious "could not be resolved" error). - cli_scenario: split the single `dict[str, object]` install record into per-section typed containers (`installed_platform`, `installed_kas`, `installed_sdks`) assembled at write time via a `_snapshot()` helper. Fixes pre-existing pyright `__setitem__ ... not defined on object` errors at the nested writes; on-disk JSON shape is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Root AGENTS.md: add a Repository Layout table near the top, correct the `platform/` description (it's installed by `otdf-sdk-mgr install`, not committed source), and trim the duplicated "Summary → Preferred Workflow" block that restated the body. - otdf-local/AGENTS.md: lead with the dependency on `otdf-sdk-mgr` (otdf-local launches the binaries the installer produces). Mark the manual-keys YAML block as an emergency fallback that may drift. - otdf-sdk-mgr/AGENTS.md (new): operational guide for the installer — subcommand layout, bare-clone-worktree gotchas (no `origin` remote, namespaced `service/vX.Y.Z` tags, unbuffered subprocess output), pattern for adding a new subcommand. - xtest/AGENTS.md (new): test-suite layout, custom pytest options, audit-log fixture quick reference, authoring guidance. - otdf-sdk-mgr/CLAUDE.md, xtest/CLAUDE.md: symlinks to AGENTS.md to match the repo convention. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Refactors otdf-local from a single-instance CLI (one platform checkout,
fixed ports, hardcoded six KAS instances) into a multi-instance harness
where each named instance under tests/instances/<name>/ owns its own
opentdf.yaml, keys, KAS configs, and port range.
Why
---
A single bug report often describes a *combination* — platform v0.9.0
with Java SDK 0.7.8 and a KAS at a pre-release. Today a developer has
to hand-edit configs and re-checkout the platform to reproduce. After
this change:
otdf-local instance init java-078 --from-scenario .../scenario.yaml
otdf-local --instance java-078 up
brings up exactly the topology the scenario describes, using platform
binaries that otdf-sdk-mgr already provisioned (each instance, and each
KAS within an instance, can reference a different pinned version). Two
instances on disjoint ports.base can coexist on a developer laptop.
What changes
------------
otdf-local now depends on otdf-sdk-mgr via a uv path source so both
tools share the canonical Scenario/Instance schema.
Settings (otdf_local.config.settings):
- New instance_name (env-overridable via OTDF_LOCAL_INSTANCE_NAME),
instance_dir, instances_root, instance_yaml properties.
- platform_dir becomes optional; legacy sibling-discovery only kicks
in when no per-instance configuration is present.
- platform_binary_for(dist) resolves to the otdf-sdk-mgr-managed
xtest/platform/dist/<dist>/service binary.
- keys_dir, logs_dir, config_dir, platform_config, and
get_kas_config_path switch to per-instance paths whenever
instance.yaml exists; legacy behavior is preserved otherwise.
- load_instance() reads the per-instance manifest via the shared
Pydantic model.
Ports (otdf_local.config.ports):
- KAS_OFFSETS exposes the offset table (alpha=+101, beta=+202, ...,
km2=+606) so multiple instances on different bases get disjoint
port ranges. The legacy 8080-based constants are preserved as
defaults.
- get_kas_port(name, base=...) computes the port relative to base.
Services (otdf_local.services.platform / .kas):
- PlatformService.start() and KASService.start() use the pinned dist
binary at xtest/platform/dist/<dist>/service when an instance is
loaded, with cwd set to the recorded worktree so the binary finds
its embedded resources. Legacy `go run ./service` path runs
unchanged when no instance is active.
- KASService.is_key_management defers to the manifest's `mode` field
instead of the legacy name-based heuristic; per-KAS features (e.g.
ec_tdf_enabled) pass through to opentdf.yaml.
- KASManager constructs only the KAS instances listed in
instance.yaml's kas: map. start_standard / start_km filter on
is_key_management so subset topologies still work.
utils.keys.setup_golden_keys:
- Writes key files into the target directory (per-instance keys_dir
or legacy platform_dir) and uses absolute paths in the generated
keys_config so the binary finds them regardless of cwd.
CLI:
- New top-level --instance option threads through every command via
OTDF_LOCAL_INSTANCE_NAME.
- New `instance` subcommand group: init [--from-scenario PATH],
ls --json, rm.
- New `scenario` subcommand: `run <path>` translates the scenario's
suite block into `pytest --sdks-encrypt ... --sdks-decrypt ...
--containers ...` under xtest/ with OTDF_LOCAL_INSTANCE_NAME set.
Tests (otdf-local/tests/test_multi_instance.py):
- Port arithmetic at default and alternate bases.
- Settings round-trip with and without an instance.yaml.
- platform_binary_for resolves under the otdf-sdk-mgr-managed
xtest/platform/ tree.
.gitignore additions:
- tests/instances/ (per-instance config and logs)
- xtest/scenarios/*.installed.json (provisioning records)
- .claude/tmp/
Backward compatibility:
- `otdf-local up` with no --instance flag keeps working against a
sibling platform/ checkout.
Refs: https://virtru.atlassian.net/browse/DSPX-3302
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds two new pytest CLI options so xtest can be driven by a scenarios.yaml
and run against a specific otdf-local instance.
--scenario PATH When set, defaults --sdks-encrypt, --sdks-decrypt,
and --containers from the scenario's `sdks` and
`suite` blocks. Options explicitly passed on the
CLI always override.
--instance NAME Propagated to OTDF_LOCAL_INSTANCE_NAME so child
`otdf-local` invocations within the test see the
same instance the scenario expects.
If otdf-sdk-mgr is not installed (minimal pytest environments), the
--scenario flag silently no-ops via an ImportError guard. The flag
shape is invariant either way so CI configs don't fork.
This is the consumer side of the PR 3 / scenario-driven flow: the
authoritative entry point remains `otdf-local scenario run <path>`,
which sets these flags for you; this PR lets pytest accept them
directly when running scenario-aware sessions outside the wrapper.
Refs: https://virtru.atlassian.net/browse/DSPX-3302
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds five Claude Code skills under tests/.claude/skills/ that together
turn a Jira bug ticket into a running reproduction, plus a downstream-
installable plugin manifest under .claude/plugin/.
Why
---
The end-to-end goal of DSPX-3302 is to make bug reproduction approachable
for QA, downstream-product engineers, and CI. PRs 1-4 build the plumbing
(shared schema, platform installer, multi-instance otdf-local, xtest
conftest hooks). This PR is the user-facing surface: a Claude can pull
context from Jira, draft an xtest/scenarios/<jira-key>.yaml (and, when
needed, an xtest/bug_<jira_key>_test.py), bring the environment up at
the right version pins, run the scenario's pytest selection, and tear
down.
Skills
------
scenario-from-bug-report
Pulls the Jira issue and its comments via `acli jira workitem view
--fields '*all' --json` and `acli jira workitem comment list`,
extracts version pins / KAS topology / container type / feature
flags, then writes xtest/scenarios/<jira-key-lowercased>.yaml
validated against otdf_sdk_mgr.schema.Scenario. Drafts a new
xtest/bug_<id>_test.py only when no existing pytest covers the
case; never silently lands assertions.
scenario-up
Runs `otdf-sdk-mgr install scenario`, then `otdf-local instance
init --from-scenario`, then `otdf-local --instance <name> up`, and
polls status until healthy. Surfaces logs rather than retrying
blindly when something stays unhealthy.
scenario-run
Invokes `otdf-local scenario run <path>` and classifies the
result: "bug reproduced" / "not reproduced" / "unrelated failure".
Cites the evidence line and points at per-service logs.
scenario-tear-down
Stops the instance and optionally removes the directory after
explicit user confirmation.
instance-status
Lists known instances, their port bases, health, and flags port
collisions.
Jira-safety
-----------
Permissions in both .claude/settings.json and the plugin manifest
allow only read+comment via acli jira: workitem view, workitem search,
workitem comment list, workitem comment create, plus a handful of
read-only project/board/sprint queries. edit, delete, transition,
assign, archive, link create, watcher add are all denied. The
plugin.json carries a permission_notes block explaining the policy.
Plugin manifest
---------------
.claude/plugin/plugin.json declares the skill names, runtime
requirements (uv, go, git, docker, acli), and the canonical permission
allowlist, so downstream first/third-party integrators can install
this plugin into their own Claude Code setups.
Refs: https://virtru.atlassian.net/browse/DSPX-3302
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…m-ticket (DSPX-3302) Headless dogfooding (run-1 on DSPX-2719) showed the bug-only framing was too narrow — the common workflow is writing tests for new features first (TDD), not reproducing version-pinned bugs. - Rename and rewrite the skill to branch on Jira Issue Type. Bug follows the old expected/actual flow; Story/Task uses ref pins (`main`, feature branch, PR SHA via `gh pr view --json headRefOid`) for forward-looking regression gates; Spike bails out rather than fabricating. Mandates `acli workitem comment list` and steers away from cli.sh greps (both were run-1 gaps). - New `scenario-matrix` sibling skill: write N scenario files from a base × N refs (PRs/branches/releases). Schema/installer support was already there via `PlatformPin.source.ref` and `install_platform_source(ref)` — no other changes needed. - `scenario-run` output classification generalized from "bug reproduced / not reproduced" to "expected / unexpected outcome", with explicit branches for bug-repro vs TDD interpretations. - `scenario-up` description and `plugin.json` (description, skills array, requirements) updated to match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
For features (or bugs) that touch more than one OpenTDF repo — platform
plus the Go / Java / JS SDKs — feature-design captures the work as a
single spec at xtest/features/<name>.yaml plus the tests-side artifacts
that land first (feature_type entry in tdfs.py, scenario, draft test).
The model matches the team's existing pattern: tests-side artifacts
merge first, dormant under a `supports("<feature>")` gate, and each
per-repo PR activates the gate by adding `supports <feature>` to its
cli.sh. PRs land async, in any order; no cross-PR lockstep needed.
- `feature-design` SKILL: propose-then-iterate authoring from a Jira
ticket (or free-form description). Drafts a complete spec on the
first pass, asks one composite redirect question, then writes the
spec + patches tdfs.py + invokes scenario-from-ticket internally
to produce the dormant scenario and draft test. Bails on Spike or
unclear tickets rather than fabricating.
- `xtest/features/{README,CLAUDE}.md`: progressive-disclosure docs —
human-facing README and agent-facing CLAUDE.md.
- `xtest/README.md` gains a brief "Test artifact directories" section
pointing at scenarios/ and features/.
- `settings.json` + `plugin.json`: Write(xtest/features/**) allowlist,
feature-design added to plugin skills array.
The complementary feature-orchestrate skill (fanning out per-repo
subagents to draft impl PRs in each touched repo) is a follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…PX-3302) Headless dogfooding (runs 1 and 2 of scenario-from-ticket on DSPX-2719) surfaced two real gaps: - The `Skill` tool was denied on both runs because the allowlist didn't cover it, so the body of SKILL.md wasn't injected on invocation; the agent had to manually `Read` the skill file ~25 turns in, wasting time and biasing exploration toward grepping unrelated files first. Add `Skill(*)` to settings.json and per-skill `Skill(<name>)` entries to plugin.json (the latter enumerates exactly what downstream installs get, since they shouldn't inherit a wildcard). - `acli jira workitem comment list` requires `--key <KEY>` (the subcommand differs from `view`, which takes the key positionally). Both scenario-from-ticket and feature-design had the wrong form; corrected, with a one-line note about the asymmetry so the next agent doesn't paraphrase. Verified via run-3 on DSPX-2719: 41 turns / 5m16s / $1.07 (vs run-1's 48 turns / 6m44s / $1.27). Skill tool returned success on first call, both acli commands ran cleanly, the Story/Task branch produced `source.ref: main` pins correctly (no more incorrectly defaulting to `dist: lts`), and the agent's `actual:` field correctly enumerated all three test-infrastructure prerequisites including a `with_ecdsa_binding` parameter that run-1's scenario missed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…emas (DSPX-3302)
Headless runs of scenario-from-ticket kept trying `python3 -c "from
otdf_sdk_mgr.schema import Scenario; ..."` to introspect Pydantic model
shape while authoring scenarios. That form isn't in the plugin's Bash
allowlist (deliberately — it's arbitrary code execution), so the agent
fell back to Reading schema.py source. Static, committed JSON Schemas
give the same information declaratively without needing a python verb
in the allowlist at all.
- `otdf-sdk-mgr schema dump [--out-dir]`: writes
`xtest/schema/{scenario,instance}.schema.json` from
`Model.model_json_schema()`, sorted-keys + trailing newline so output
is byte-stable. Add new models to `SCHEMAS` in cli_schema.py and they
get picked up automatically.
- `xtest/schema/` is committed with the generated files plus brief
README/CLAUDE.md (progressive-disclosure, mirroring xtest/features/).
- `test_schema_sync.py` parametrizes over `SCHEMAS` and fails if any
committed file drifts from the live model — the safety net for
"someone edited a Pydantic model without regenerating."
- `scenario-from-ticket` SKILL.md Step 5 now points at
`xtest/schema/scenario.schema.json` as the canonical field list.
- `xtest/README.md` lists the new directory alongside `scenarios/` and
`features/`.
No allowlist changes needed — `Bash(uv run otdf-sdk-mgr *)` already
covers the dump subcommand, and `Read` is unrestricted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nce mode (DSPX-3356)
Previously `otdf-local --instance <id> up` and `otdf-sdk-mgr install scenario`
only handled `dist:` pins. Source pins (`source: { ref: main }`) failed with
"No sibling platform/ directory found" and Go SDK at `version: main` failed with
a module-proxy 404.
Changes:
- **otdf-sdk-mgr**: add `install_go_from_platform(ref, dist_dir)` — builds
otdfctl from the platform monorepo worktree (needed when `version: main` can't
be resolved via the Go module proxy). `cli_scenario.py` dispatches to it when
`sdks.*.source == "platform"`.
- **otdf-local/settings**: add `platform_source_dir` property that returns the
platform src worktree in instance+source mode. `docker_compose_file` uses it
to find docker-compose.yaml in the worktree.
- **otdf-local/platform**: `_instance_dist_paths` handles `source` pins (not
just `dist`). `_generate_config` falls back to `opentdf-dev.yaml` when the
worktree has no `opentdf.yaml`.
- **otdf-local/kas**: `_instance_paths` handles `source` pins.
- **otdf-local/provisioner**: use `platform_source_dir` as cwd.
- **otdf-local/cli** (env): emit `PLATFORM_VERSION` by reading it from the
built binary directly (fast path), with `go run ./service version` fallback.
Fix `service version` printing to stderr. Fix `PLATFORM_DIR` NoneType crash.
- **otdf-local/cli_scenario** (run): fix `suite.select` → `suite.targets`;
strip leading `xtest/` from target paths; set `OTDFCTL_HEADS` from the
installed.json Go dist name so `load_otdfctl()` finds the right binary.
- **xtest/scenarios**: add `dspx-3356.yaml` — reproduces X-Wing wrappedKey
size bug (1190 bytes vs expected 1120) at platform@main.
Bug confirmed: `test_xwing_roundtrip` FAILS with
`AssertionError: X-Wing wrappedKey should be 1120 bytes, got 1190`.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a multi-instance refactor for the OpenTDF test harness, enabling concurrent and isolated test environments. Key updates include new instance management commands in otdf-local, platform service installation via git worktrees in otdf-sdk-mgr, and scenario-driven test execution integrated with pytest. Feedback highlights a critical merge conflict marker left in xtest/AGENTS.md that needs to be resolved.
| | `fixtures/` | Module-scoped pytest fixtures: `attributes.py`, `keys.py`, `audit.py`, `assertions.py`, `kas.py`, `encryption.py`, `obligations.py`. | | ||
| | `tdfs.py` | SDK abstraction layer — wraps the `cli.sh` shims under `sdk/<lang>/dist/<version>/`. | | ||
| | `sdk/{go,java,js}/dist/<version>/` | SDK CLI builds. Installed by `otdf-sdk-mgr install` (see `../otdf-sdk-mgr/README.md`). | | ||
| <<<<<<< HEAD |
Contributor
There was a problem hiding this comment.
A merge conflict marker (<<<<<<< HEAD) has been accidentally left in the file. This should be removed to maintain the integrity of the documentation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
source:pins inotdf-localandotdf-sdk-mgr— previously onlydist:pins worked.install_go_from_platformtootdf-sdk-mgrso Go SDK built from the platform monorepo (source: platform) can be used in scenarios._require_platform_dir()crashes inotdf-localwhen no siblingplatform/checkout exists.xtest/scenarios/dspx-3356.yamlto reproduce DSPX-3356.Bug confirmed
test_pqc.py::test_xwing_roundtripFAILS at platformmain(commit1209acc2):To reproduce
Infrastructure changes
otdf-sdk-mgr/installers.pyinstall_go_from_platform(ref, dist_dir)builds otdfctl from platform monorepo worktreeotdf-sdk-mgr/cli_scenario.pysdks.*.source == "platform"otdf-local/settings.pyplatform_source_dirproperty;docker_compose_fileresolves from src worktreeotdf-local/platform.py_instance_dist_pathshandlessourcepins; fallback toopentdf-dev.yamlotdf-local/kas.py_instance_pathshandlessourcepinsotdf-local/provisioner.pyplatform_source_diras cwdotdf-local/cli.pyPLATFORM_VERSIONfrom built binary;PLATFORM_DIRNoneType fixotdf-local/cli_scenario.pysuite.select→suite.targets; stripxtest/prefix; setOTDFCTL_HEADSTest plan
uv run otdf-sdk-mgr install scenario xtest/scenarios/dspx-3356.yaml --skip-scriptscompletes (builds platform + Go SDK)uv run otdf-local --instance dspx-3356 upreaches "Environment is up!"uv run otdf-local scenario run xtest/scenarios/dspx-3356.yamlexits 1 withAssertionError: X-Wing wrappedKey should be 1120 bytes, got 1190🤖 Generated with Claude Code