diff --git a/CHANGELOG.md b/CHANGELOG.md index 66b1a0b..ffe949f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,42 @@ # Changelog +## 0.4.1 + +This patch release synchronizes the public installation documentation with the +current package line. + +### Changed + +- Updated repository, SDK, runtime, quickstart, and CI installation examples. +- Clarified that Streamable HTTP agents and their clients run in separate terminals. + +## 0.4.0 + +This release establishes a stricter, backward-compatible protocol contract and adds +first-class support for async Python agents. + +### Added + +- Added method-specific conformance validation for initialize, step, and reset responses. +- Added machine-readable conformance reports with `--json` and `--json-out`. +- Added synchronous and asynchronous `@on_step` and `@on_reset` support across stdio + and Streamable HTTP. +- Added configurable RPC timeouts through `--timeout` and `ECP_RPC_TIMEOUT`. +- Added a runnable async Python example covering both supported transports. +- Expanded automated test coverage across Python 3.9 through 3.13. + +### Changed + +- Aligned runtime models, SDK result normalization, JSON Schemas, and protocol validation. +- Enforced JSON-RPC response identifier matching and preserved evaluator-visible logs. +- Improved timeout and RPC failure diagnostics with method, scenario, and step context. + +### Compatibility + +- Existing valid synchronous agents remain supported without migration. +- Invalid manifests and malformed protocol payloads that previously passed through are + now rejected with actionable diagnostics. + ## 0.3.1 This release makes ECP easier to share with developers as a usable portable evaluation contract. diff --git a/README.md b/README.md index b1edd56..e69c6d3 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ ECP is a vendor-neutral protocol and reference runtime for testing agent outputs MCP gives agents a common way to use tools. ECP gives evaluators a common way to inspect what an agent returned, what tools it used, and what audit evidence it exposed. -> Status: experimental but usable. The current package line is `0.3.1`. +> Status: experimental but usable. The current package line is `0.4.1`. ## Why ECP @@ -28,7 +28,7 @@ Use ECP when you want to: ```bash py -m venv .venv .\.venv\Scripts\Activate.ps1 -pip install "ecp-runtime==0.3.1" "ecp-sdk==0.3.1" +pip install "ecp-runtime==0.4.1" "ecp-sdk==0.4.1" ecp init ecp validate ecp_eval/manifest.yaml ecp run --manifest ecp_eval/manifest.yaml --json diff --git a/docs/ci.md b/docs/ci.md index 05cf126..6868443 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -20,7 +20,7 @@ jobs: - uses: actions/setup-python@v5 with: python-version: "3.11" - - run: pip install "ecp-runtime==0.3.1" "ecp-sdk==0.3.1" + - run: pip install "ecp-runtime==0.4.1" "ecp-sdk==0.4.1" - run: ecp validate examples/customer_support_demo/manifest.yaml - run: ecp run --manifest examples/customer_support_demo/manifest.yaml --json-out ecp-report.json - uses: actions/upload-artifact@v4 diff --git a/docs/examples.md b/docs/examples.md index dbea5eb..91dcc5e 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -154,10 +154,15 @@ Run it over stdio: ecp run --manifest examples/async_python_demo/manifest.yaml --timeout 10 ``` -Or start the same agent over Streamable HTTP: +Or start the same agent over Streamable HTTP in one terminal: ```bash ECP_TRANSPORT=http ECP_HTTP_PORT=8765 python examples/async_python_demo/agent.py +``` + +Run the conformance checks from another terminal: + +```bash ecp conformance --target http://127.0.0.1:8765/ecp --timeout 10 ``` diff --git a/docs/index.md b/docs/index.md index f5ee92a..55c96b8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -36,7 +36,7 @@ Most evals start with the final answer. ECP also checks the behavior behind that ## Developer Path ```bash -pip install "ecp-runtime==0.3.1" "ecp-sdk==0.3.1" +pip install "ecp-runtime==0.4.1" "ecp-sdk==0.4.1" ecp init ecp validate ecp_eval/manifest.yaml ecp run --manifest ecp_eval/manifest.yaml --json diff --git a/docs/quickstart.md b/docs/quickstart.md index caca434..11f3b84 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -7,16 +7,16 @@ ```bash py -m venv .venv .\.venv\Scripts\Activate.ps1 -pip install "ecp-runtime==0.3.1" "ecp-sdk==0.3.1" +pip install "ecp-runtime==0.4.1" "ecp-sdk==0.4.1" ``` For framework demos, install the matching SDK extra: ```bash -pip install "ecp-sdk[langchain]==0.3.1" langchain-openai -pip install "ecp-sdk[crewai]==0.3.1" crewai -pip install "ecp-sdk[pydanticai]==0.3.1" pydantic-ai -pip install "ecp-sdk[llamaindex]==0.3.1" llama-index llama-index-llms-openai llama-index-tools-yahoo-finance +pip install "ecp-sdk[langchain]==0.4.1" langchain-openai +pip install "ecp-sdk[crewai]==0.4.1" crewai +pip install "ecp-sdk[pydanticai]==0.4.1" pydantic-ai +pip install "ecp-sdk[llamaindex]==0.4.1" llama-index llama-index-llms-openai llama-index-tools-yahoo-finance ``` ## 2. Create A Starter Eval @@ -115,7 +115,7 @@ ecp conformance --target "python agent.py" --timeout 60 ## Notes -- The current release line is `0.3.1`. +- The current release line is `0.4.1`. - New agents should use `evaluation_context`; `private_thought` remains a deprecated compatibility alias. - `--timeout` controls the RPC timeout for `run` and `conformance`. It overrides `ECP_RPC_TIMEOUT`; the default is 30 seconds. - Python SDK `@on_step` and `@on_reset` hooks may be synchronous or `async def` functions. diff --git a/sdk/python/README.md b/sdk/python/README.md index cfbe58a..ac264b1 100644 --- a/sdk/python/README.md +++ b/sdk/python/README.md @@ -7,16 +7,16 @@ ECP is a vendor-neutral protocol for testing agent outputs, tool calls, and eval ## Install ```bash -pip install "ecp-sdk==0.3.1" +pip install "ecp-sdk==0.4.1" ``` Framework extras: ```bash -pip install "ecp-sdk[langchain]==0.3.1" -pip install "ecp-sdk[crewai]==0.3.1" -pip install "ecp-sdk[llamaindex]==0.3.1" -pip install "ecp-sdk[pydanticai]==0.3.1" +pip install "ecp-sdk[langchain]==0.4.1" +pip install "ecp-sdk[crewai]==0.4.1" +pip install "ecp-sdk[llamaindex]==0.4.1" +pip install "ecp-sdk[pydanticai]==0.4.1" ``` ## Usage