Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,42 @@
# Changelog

## 0.4.1

This patch release synchronizes the public installation documentation with the
current package line.

### Changed

- Updated repository, SDK, runtime, quickstart, and CI installation examples.
- Clarified that Streamable HTTP agents and their clients run in separate terminals.

## 0.4.0

This release establishes a stricter, backward-compatible protocol contract and adds
first-class support for async Python agents.

### Added

- Added method-specific conformance validation for initialize, step, and reset responses.
- Added machine-readable conformance reports with `--json` and `--json-out`.
- Added synchronous and asynchronous `@on_step` and `@on_reset` support across stdio
and Streamable HTTP.
- Added configurable RPC timeouts through `--timeout` and `ECP_RPC_TIMEOUT`.
- Added a runnable async Python example covering both supported transports.
- Expanded automated test coverage across Python 3.9 through 3.13.

### Changed

- Aligned runtime models, SDK result normalization, JSON Schemas, and protocol validation.
- Enforced JSON-RPC response identifier matching and preserved evaluator-visible logs.
- Improved timeout and RPC failure diagnostics with method, scenario, and step context.

### Compatibility

- Existing valid synchronous agents remain supported without migration.
- Invalid manifests and malformed protocol payloads that previously passed through are
now rejected with actionable diagnostics.

## 0.3.1

This release makes ECP easier to share with developers as a usable portable evaluation contract.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ ECP is a vendor-neutral protocol and reference runtime for testing agent outputs

MCP gives agents a common way to use tools. ECP gives evaluators a common way to inspect what an agent returned, what tools it used, and what audit evidence it exposed.

> Status: experimental but usable. The current package line is `0.3.1`.
> Status: experimental but usable. The current package line is `0.4.1`.

## Why ECP

Expand All @@ -28,7 +28,7 @@ Use ECP when you want to:
```bash
py -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install "ecp-runtime==0.3.1" "ecp-sdk==0.3.1"
pip install "ecp-runtime==0.4.1" "ecp-sdk==0.4.1"
ecp init
ecp validate ecp_eval/manifest.yaml
ecp run --manifest ecp_eval/manifest.yaml --json
Expand Down
2 changes: 1 addition & 1 deletion docs/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: pip install "ecp-runtime==0.3.1" "ecp-sdk==0.3.1"
- run: pip install "ecp-runtime==0.4.1" "ecp-sdk==0.4.1"
- run: ecp validate examples/customer_support_demo/manifest.yaml
- run: ecp run --manifest examples/customer_support_demo/manifest.yaml --json-out ecp-report.json
- uses: actions/upload-artifact@v4
Expand Down
7 changes: 6 additions & 1 deletion docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,10 +154,15 @@ Run it over stdio:
ecp run --manifest examples/async_python_demo/manifest.yaml --timeout 10
```

Or start the same agent over Streamable HTTP:
Or start the same agent over Streamable HTTP in one terminal:

```bash
ECP_TRANSPORT=http ECP_HTTP_PORT=8765 python examples/async_python_demo/agent.py
```

Run the conformance checks from another terminal:

```bash
ecp conformance --target http://127.0.0.1:8765/ecp --timeout 10
```

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Most evals start with the final answer. ECP also checks the behavior behind that
## Developer Path

```bash
pip install "ecp-runtime==0.3.1" "ecp-sdk==0.3.1"
pip install "ecp-runtime==0.4.1" "ecp-sdk==0.4.1"
ecp init
ecp validate ecp_eval/manifest.yaml
ecp run --manifest ecp_eval/manifest.yaml --json
Expand Down
12 changes: 6 additions & 6 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@
```bash
py -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install "ecp-runtime==0.3.1" "ecp-sdk==0.3.1"
pip install "ecp-runtime==0.4.1" "ecp-sdk==0.4.1"
```

For framework demos, install the matching SDK extra:

```bash
pip install "ecp-sdk[langchain]==0.3.1" langchain-openai
pip install "ecp-sdk[crewai]==0.3.1" crewai
pip install "ecp-sdk[pydanticai]==0.3.1" pydantic-ai
pip install "ecp-sdk[llamaindex]==0.3.1" llama-index llama-index-llms-openai llama-index-tools-yahoo-finance
pip install "ecp-sdk[langchain]==0.4.1" langchain-openai
pip install "ecp-sdk[crewai]==0.4.1" crewai
pip install "ecp-sdk[pydanticai]==0.4.1" pydantic-ai
pip install "ecp-sdk[llamaindex]==0.4.1" llama-index llama-index-llms-openai llama-index-tools-yahoo-finance
```

## 2. Create A Starter Eval
Expand Down Expand Up @@ -115,7 +115,7 @@ ecp conformance --target "python agent.py" --timeout 60

## Notes

- The current release line is `0.3.1`.
- The current release line is `0.4.1`.
- New agents should use `evaluation_context`; `private_thought` remains a deprecated compatibility alias.
- `--timeout` controls the RPC timeout for `run` and `conformance`. It overrides `ECP_RPC_TIMEOUT`; the default is 30 seconds.
- Python SDK `@on_step` and `@on_reset` hooks may be synchronous or `async def` functions.
10 changes: 5 additions & 5 deletions sdk/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,16 @@ ECP is a vendor-neutral protocol for testing agent outputs, tool calls, and eval
## Install

```bash
pip install "ecp-sdk==0.3.1"
pip install "ecp-sdk==0.4.1"
```

Framework extras:

```bash
pip install "ecp-sdk[langchain]==0.3.1"
pip install "ecp-sdk[crewai]==0.3.1"
pip install "ecp-sdk[llamaindex]==0.3.1"
pip install "ecp-sdk[pydanticai]==0.3.1"
pip install "ecp-sdk[langchain]==0.4.1"
pip install "ecp-sdk[crewai]==0.4.1"
pip install "ecp-sdk[llamaindex]==0.4.1"
pip install "ecp-sdk[pydanticai]==0.4.1"
```

## Usage
Expand Down
Loading