test: production-harden the suite (error contract, tasks, async, real URLs, coverage gate)#18
Conversation
Khurdhula-Harshavardhan
left a comment
There was a problem hiding this comment.
please help resolve the following.
4905c07 to
a60f60e
Compare
|
|
Khurdhula-Harshavardhan
left a comment
There was a problem hiding this comment.
please help resolve the following.
…ngchain tests (review #18)
- Error contract: status->exception mapping (400-503), retry behavior, and timeouts, sync + async (were completely untested). - All 8 task helpers unit-tested (sync + async): asserts the <task> tag on the wire AND result extraction; forecast precontext/fallback/skip-unrelated paths. - Async parity, streaming failure modes (empty stream, streamed tool calls, double-consume, malformed SSE), input edge cases (from_path), guard/schema. - Real Interfaze/JigsawStack doc asset URLs (tests/assets.py); dropped the fake x.com placeholders. - Coverage gate: pytest-cov + branch coverage + --cov-fail-under=90 (suite hits 100% line+branch). 24 -> 119 tests.
…ngchain tests (review #18)
7bd894c to
86a8949
Compare
Turns the happy-path suite (24 tests, ~81% line coverage) into a production-grade one: 119 tests, 100% line+branch coverage, gate floor at 90%.
What was missing (and is now covered)
test_errors.py) — status→exception mapping 400/401/403/404/429/500/503 → the rightopenaitype (assertingstatus_codeand the error body reaches the caller); retries fire on 429/5xx (respxside_effect+ monkeypatched sleep, assertcall_count); timeouts →APITimeoutError. Sync + async. (Previously zero error tests.)test_tasks.py) — all 8 (ocr/object_detection/gui_detection/transcribe/web_search/scrape/translate/forecast), sync + async: assert the<task>tag on the wire and result extraction; forecast precontext / fallback / skip-unrelated-entries paths;_extractedge cases. (Was ~48% covered.)test_async.py) — real async coverage, not one smoke test.get_final_completion.from_path(real temp file), blacklist viafrom_path, unknown-extension,auto_partrouting.test_guard.py,test_schema.py) edge cases.Real URLs
Added
tests/assets.pywith the canonical Interfaze/JigsawStack doc asset URLs (OCR receipt, STT wav, forecast CSV, YOLO bus.jpg, arXiv PDF, HN, …) and replaced the fakehttps://x.com/...placeholders.Coverage gate (self-contained)
pytest-covin thedevextra +[tool.coverage.run] branch=true+addopts=--cov --cov-fail-under=90inpyproject.toml. Travels with the tests, so any CI runningpytestenforces it. Tests only — nosrc/changes.Findings the hardened tests surfaced (documented in the tests; flagging for follow-up)
gui_detectionsilently mis-routes extensionless image URLs. A CDN URL with no.ext(e.g. Unsplash) makesauto_partfall through to a genericfilepart instead ofimage_url(mime detection is extension-only), andtasks.gui_detection()has nofilename/formathint to fix it. Real silent-failure gap for a common URL shape — worth an SDK follow-up.data:lines raise a rawjson.JSONDecodeErrorfrom openai's SSE decoder (before interfaze code runs), rather than a typed error — a stream robustness gap.InternalServerError(a subclass ofAPIStatusError), not a distinct type per code — tests assert actual behavior.ruff formatdrift in 6src/files (untouched here; deferred format-check from the CI PR).