examples: add local ART-E email search task#679
Conversation
|
I pushed an update to strengthen this ART-E example PR. New addition:
This should make the example easier to review because the task contract can be checked without API keys or training infrastructure. Note: I still could not run Python locally because this Windows environment does not have python/uv in PATH, but git diff --check passes. |
|
Validation update from my local Windows checkout: PS D:\BOUNTY\ART> & "$env:LOCALAPPDATA\Programs\Python\Python314\python.exe" -m pytest tests/test_art_e_example.py -q
...... [100%]
6 passed, 1 warning in 0.07s
PS D:\BOUNTY\ART> & "$env:LOCALAPPDATA\Programs\Python\Python314\python.exe" examples\art_e\evaluate.py
quarterly-budget-owner: reward=1.00 answer=1 citations=1 refs=[msg-budget-2]
customer-escalation-time: reward=1.00 answer=1 citations=1 refs=[msg-northwind-2]
contract-renewal-discount: reward=1.00 answer=1 citations=1 refs=[msg-finch-3]
average_reward=1.00
PS D:\BOUNTY\ART> git diff --check
# no outputNote: this machine has Python 3.14 available directly at the user install path rather than on PATH. The pytest run emits one global config warning because |
|
Clean validation update after installing the repo's pytest async plugin locally: PS D:\BOUNTY\ART> & "$env:LOCALAPPDATA\Programs\Python\Python314\python.exe" -m pip install pytest-asyncio
Successfully installed pytest-asyncio-1.3.0
PS D:\BOUNTY\ART> & "$env:LOCALAPPDATA\Programs\Python\Python314\python.exe" -m pytest tests/test_art_e_example.py -q
...... [100%]
6 passed in 0.03s
PS D:\BOUNTY\ART> & "$env:LOCALAPPDATA\Programs\Python\Python314\python.exe" examples\art_e\evaluate.py
quarterly-budget-owner: reward=1.00 answer=1 citations=1 refs=[msg-budget-2]
customer-escalation-time: reward=1.00 answer=1 citations=1 refs=[msg-northwind-2]
contract-renewal-discount: reward=1.00 answer=1 citations=1 refs=[msg-finch-3]
average_reward=1.00
PS D:\BOUNTY\ART> git diff --check
# no outputThe previous pytest config warning is gone now that |
Summary
examples/art_eMotivation
The README points users to the ART-E email-search agent, but the main repo did not include a small local example that can be inspected without external email infrastructure. This adds a portable starter task for experimenting with ART-style email search rollouts before scaling to the full ART-E setup.
What changed
examples/art_e/scenarios.py: local inbox fixtures, search/read helpers, JSON command parsing, and reward scoringexamples/art_e/rollout.py: multi-turn ART trajectory using<search>,<read>, and<answer>commandsexamples/art_e/train.py: LocalBackend training loop for the ART-E style taskexamples/art_e/README.md: quickstart and file overviewtests/test_art_e_example.py: helper-level tests for search, read, scoring, and command parsingREADME.md: adds the local ART-E example to the notebooks/examples tableTesting
Not run locally because this Windows environment does not currently have
pythonoruvavailable in PATH. The added tests are intentionally small and should run with the repo test suite once dependencies are installed.