v1.5.0

• ci(release): fetch full git history in release job (c4101aa)
• ci(release): align test job with ci.yml — drop pytest + framework scenarios (ddf72cf)
• ci: bind secrets to env vars before referencing in nightly-gate guard (77b41b2)
• Merge pull request #17 from accelerate-data/feature/vd-2174-multi-sdk-plugin-contract (0b716f5)
• ci: consolidate release.yml + publish.yml into a single release.yml (4a2b015)
• ci(vd-2174-12): merge release.yml into publish.yml (7813d47)
• fix(openhands-sdk): align event_extractor with live SDK event fields (9b24513)
• feat(dir-walk): forward trailing promptfoo args to fan-out children (39df834)
• feat(config): default shipped tiers to openhands_sdk (9e3cd8e)
• feat(resolver): auto-route v0 multi-turn packages to the SDK bridge (0dea36e)
• docs(setup): add Step 1.5 — configure secrets (11fefbe)
• docs(setup): add unpublished-branch setup guide (eee1453)
• docs(setup): clarify npm aliases vs ad-evals shortcuts (3f04e8b)
• fix(ci): repair CI green — fetch-depth for byte-identity test + ref'd sleep timer (ddfd540)
• docs(agents): make AGENTS.md + CLAUDE.md LLM-runnable for teammate onboarding (b8bc3f0)
• docs(setup): align Provider Key Matrix + secret redactor with code (0f967a4)
• fix(harness-smoke): make agent-server smoke fixture self-contained (dcec048)
• fix(openhands_agent_server): bump pair 1.21.1 → 1.23.1 and pin sdk explicit (1531b47)
• fix(ad-evals): honor AD_EVALS_ROOT in resolveHarnessPaths (dff0bd2)
• fix(ad-evals): resolve discovered configs against evalRoot; move helpers test into scripts/ (0c8b658)
• fix(openhands_agent_server): address codex review findings (lockfile, scope, ws close) (de6c393)
• Merge branch 'feature/vd-2174-12-glue-providers' into feature/vd-2174-multi-sdk-plugin-contract (ea36ede)
• feat(framework): WIP openhands_agent_server provider with daemon lifecycle (63cf853)
• docs(framework): expand setup.md with consumer-porting sections (codex-reviewed) (0d1e630)
• docs(framework): document plugin glue providers and bump to v1.5.0 (2a0c158)
• test(framework): add phase-06 contract tests for ported glue providers (57d6843)
• feat(framework): add opencode-cli-plugin SIBLING provider (Shape B) (63a94f9)
• feat(framework): port claude-agent-sdk Node-side provider into harness (4605958)
• feat(framework): port codex-sdk thin wrapper provider into harness (2575614)
• fix(bridge): honor wrapper-supplied workspace_root over mkdtemp fallback (222a7d3)
• feat(framework): port provider-run-metadata utility into harness (8958251)
• feat(runtime): extend allowlist with 14 optional fields (29ae9b4)
• fix(providers): nest openhands-sdk-provider.js under scripts/framework/ (d64f819)
• refactor(providers): relocate openhands-sdk-provider.js into harness (c722576)
• feat(templates): scaffold prompts/ and fixtures/ in harness-smoke starter (eed794b)
• fix(bridge): spawn subprocess providers with framework root as cwd (a94e5da)
• fix(dir-walk): inject tier providers for consumer scenarios in EVAL_ROOT (bd79791)
• feat(openhands_sdk): .env overrides for base_url and model (967bbcd)
• fix(openhands_sdk): expose sibling modules for bare imports (c1b2aac)
• fix(adapter): isolate stdout for NDJSON IPC (08e20cd)
• fix(openhands_sdk): pass conversation callbacks via constructor (ab474c6)
• fix(openhands_sdk): align tool registry with sdk 1.22+ / openhands-tools 1.23 (18dacf9)
• fix(ci): skip ensureDepsInstalled when AD_EVALS_ROOT has no package.json (2b91e54)
• fix(ci): restore missing transitive entries in package-lock.json (fe9f19e)
• docs(vd-2174-14): document openhands_sdk gateway mode in AGENTS.md + CLAUDE.md (02e3a73)
• feat(vd-2174-14): openhands_sdk gateway mode — base_url + model (v1.4.0) (2774c0d)
• fix(vd-2174-13): opencode_sdk guaranteed kill on signal/exit (v1.3.3) (9dc0de2)
• docs(vd-2174-12): document multi-SDK setup, provider selection, and run commands (fd4bf52)
• fix(vd-2174-10): opencode_sdk SDK v1.15.10 response shape sync (v1.3.2) (13aff21)
• fix(vd-2174-11): codex_sdk uses ESM dynamic import for @openai/codex-sdk (v1.3.1) (14be756)
• chore(vd-2174-11): release v1.3.0 — codex_sdk in-proc provider (4460047)
• ci(vd-2174-11): wire codex_sdk mock scenario into nightly workflow (f647094)
• chore(vd-2174-11): pin @openai/codex-sdk + @openai/codex (57da3ce)
• feat(vd-2174-11): register codex_sdk kind in bridge KIND_REGISTRY (2fef1f9)
• feat(vd-2174-11): add codex_sdk in-proc provider + require.cache mock (b6f70a3)
• chore(vd-2174-10): release v1.2.0 — opencode_sdk in-proc provider (965a844)
• ci(vd-2174-10): wire opencode_sdk mock scenario into nightly workflow (1a61677)
• chore(vd-2174-10): pin @opencode-ai/sdk + bump engines.node to >=20 (5584d58)
• feat(vd-2174-10): register opencode_sdk kind in bridge KIND_REGISTRY (e3c4db9)
• feat(vd-2174-10): add opencode_sdk in-proc provider + module-level SDK stub (3d963c4)
• chore(vd-2174-9): release v1.1.0 — claude_agent_sdk provider (5a59f8e)
• ci(vd-2174-9): wire claude_agent_sdk mock scenario into nightly workflow (75f4aed)
• feat(vd-2174-9): dispatch claude_agent_sdk through bridge subprocess + thread total_turns (97328f4)
• feat(vd-2174-9): land claude_agent_sdk async provider + Layer 3 mock-SDK tests (f1861c9)
• feat(vd-2174-9): pin claude-agent-sdk and register adapter kind (2a3cedd)
• chore(vd-2174-12): release v1.0.1 — bridge in-proc + hierarchical concurrency + async adapter foundation (fad5aa0)
• feat(vd-2174-12): generic mode==='inproc' dispatch in _node_bridge.js (fc11c7d)
• feat(vd-2174-12): async-aware lifecycle in _python_adapter.py (03b3204)
• feat(vd-2174-12): hierarchical acquire(kind?) in concurrency.js (9348c41)
• docs(vd-2174-12): amend spec §2.5 for in-proc Node SDK dispatch (Phase 9.5) (a1d738d)
• docs(vd-2174-12): rework phase-10/11/12 + add phase-09.5 foundation (c478a2d)
• docs(vd-2174): SDK shape spikes + codex adversarial review for phase 2-4 (db6dd2b)
• chore(vd-2174-8): bump to 1.0.0 + release-readiness verification (0f9330b)
• feat(vd-2174-8): consumer dependabot template + init drop logic (H.36) (e82d740)
• ci(vd-2174-8): publish workflow with nightly-gate + npm tag derivation (H.35) (33c3952)
• docs(vd-2174-8): provider matrix + scenario index + release-readiness checklist (H.34) (f2c6007)
• docs(vd-2174-8): CHANGELOG entry for 1.0.0 (H.33) (bf9de62)
• fix(vd-2174-7): isolate Promptfoo state DB per scenario to restore parallelism (e32b339)
• feat(ci): add nightly-scenarios workflow for mock-mode scenario gate (F.27) (4736cc7)
• feat(scenarios): add minimal-smoke and openhands-mock-multi-turn scenarios (F.26) (f7f727e)
• feat(vd-2174-7): doctor --install-providers uv-run pre-warm + offline mode (G.32) (716e0b9)
• feat(vd-2174-7): eval-harness-init scaffolds new layout + --upgrade --migrate-from-v0 flags (G.31) (db05792)
• docs(vd-2174-7): migration guide v0 to v1 (G.30) (84ac8b7)
• feat(vd-2174-7): eval-harness-init --upgrade --migrate-from-v0 in-place rewrite (G.29) (316e88d)
• feat(vd-2174-7): ad-evals run with outer p-limit fan-out (G.28) (9b8ae37)
• feat(vd-2174-7): scenario registry + harness-scenarios contract (F.25) (c7ac5e8)
• chore(vd-2174-6): apply ruff format to _python_adapter.test.py (731cc52)
• chore(vd-2174-6): mark phase-07 todos complete (4812b9f)
• feat(vd-2174-6): post-run workspace assertion in run-promptfoo-with-guard (E.24) (90ba5af)
• feat(vd-2174-6): NDJSON structured logger with run/case metadata (E.23) (20b3abe)
• fix(vd-2174-6): remove provider-shim.js workaround now that bridge exposes string label (5791b25)
• fix(vd-2174-6): expose bridge label as string property (post-phase-05 finding) (494a6a2)
• feat(vd-2174-6): land Node + Python secret redactor with shared patterns (E.22) (4c9f28b)
• chore(vd-2174-4): mark phase-05 todos complete + ignore coverage artifacts (3dc67fd)
• chore(vd-2174-5): mark phase-06 todo list complete (44109b8)
• feat(vd-2174-5): dispatch OpenHands SDK through bridge subprocess (D.21) (6738d6e)
• feat(vd-2174-4): add opencode-cli-compatibility scenario (C.15) (d003aa8)
• feat(vd-2174-5): land OpenHands SDK provider + Layer 3 tests (D.20) (8af4fe8)
• feat(vd-2174-5): assemble OpenHands agent factory (D.19) (9a464bc)
• feat(vd-2174-5): land OpenHands event extractor + tests (D.18) (06471d6)
• refactor(vd-2174-4): adopt §1 plugin contract for OpenCode CLI provider (C.14) (2ab1fa1)
• feat(vd-2174-5): wire OpenHands SDK model resolver (D.17) (2058a91)
• test(vd-2174-4): lock OpenCode CLI provider behavior with contract test (C.13) (cf61c0b)
• feat(vd-2174-5): seed OpenHands SDK tool registry (D.16) (6f7d369)
• chore(vd-2174-3): mark phase-04 complete (3731cd4)
• feat(vd-2174-3): validate package config + fail fast on bad shapes (B.11) (67d4fff)
• feat(vd-2174-3): clamp bridge concurrency via global p-limit (B.10) (c45710f)
• feat(vd-2174-3): emit single bridge URL per tier×provider×scenario (B.9) (84b1f58)
• feat(vd-2174-3): support v0 + v1 tier config shapes (B.8) (6c9fa0c)
• feat(vd-2174-2): cold-spawn cost gate + parallel-throughput bench + CI wiring (A.7) (f88a4c6)
• test(vd-2174-2): close L1/L2 error-envelope gaps + document IPC wire format (A.6) (aed129c)
• chore(vd-2174-2): scope markdownlint exemption to plans/spikes only (8a6afec)
• feat(vd-2174-2): Node bridge + KIND_REGISTRY + concurrency gates (A.5) (26bf2bc)
• feat(vd-2174-2): Python NDJSON adapter + mock provider + L2 test (A.4) (bd296f5)
• feat(vd-2174-2): hand-write provider contract in Python + TypeScript (A.3) (02ae8a2)
• feat(vd-2174-2): pin openhands-sdk 1.22.1 + opencode_cli min version (A.2) (846430f)
• chore(vd-2174-2): add uv-managed Python toolchain (A.1) (746ee96)
• chore(vd-2174-1): promote OpenHands SDK shape spike to live-key PASS (4fff803)
• fix(vd-2174-1b): apply spec v9 edits from spike A.0.B verdict (7d4d999)
• docs(vd-2174-1): apply spec edits from OpenHands SDK shape spike (666bfed)
• feat(vd-2174-1): land OpenHands SDK shape spike + verdict (0eb7087)
• feat(vd-2174-1b): land Promptfoo file-provider option-shape spike + verdict (04beab6)
• docs(vd...

v0.1.3

• chore: bump to 0.1.3 and fix repository URL in package.json (d61801a)

Full Changelog: v0.1.2...v0.1.3

v0.1.2

• chore: bump version to 0.1.2 (409c13b)

Full Changelog: v0.1.1...v0.1.2

v0.1.1

• feat: scaffold AGENTS.md into tests/evals/ on init (939ccf5)

Full Changelog: v0.1.0...v0.1.1

Initial release

Initial release.

What's Changed

• feat: implement promptfoo eval harness framework package by @hbanerjee74 in #1
• build(deps): Bump the npm_and_yarn group across 1 directory with 3 updates by @dependabot[bot] in #7
• build(deps-dev): Bump markdownlint-cli from 0.42.0 to 0.48.0 by @dependabot[bot] in #6
• build(deps): Bump actions/checkout from 4 to 6 by @dependabot[bot] in #4
• build(deps): Bump softprops/action-gh-release from 2 to 3 by @dependabot[bot] in #2
• build(deps): Bump github/codeql-action from 3 to 4 by @dependabot[bot] in #3
• build(deps): Bump actions/setup-node from 4 to 6 by @dependabot[bot] in #5
• fix: add missing gcp-metadata@7.0.1 to lockfile by @hbanerjee74 in #8

New Contributors

• @hbanerjee74 made their first contribution in #1
• @dependabot[bot] made their first contribution in #7

Full Changelog: https://github.com/accelerate-data/promptfoo-eval-harness/commits/v0.1.0