fix: nightly hardening - http empty-content error surfacing#23
Open
mouse-value-add wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The HTTP provider could return a successful with empty content when upstream endpoints returned unsupported payload shapes or structured error payloads. This made failures look like normal empty generations and hid root-cause diagnostics.
Approach
Verification
platform darwin -- Python 3.12.13, pytest-9.0.3, pluggy-1.6.0
rootdir: /Users/mouse/.openclaw/workspace/PromptLens
configfile: pyproject.toml
plugins: cov-7.1.0, anyio-4.13.0
collected 7 items
tests/test_http_provider_empty_content_guard.py ... [ 42%]
tests/test_http_provider_response_parsing.py .... [100%]
=============================== warnings summary ===============================
promptlens/models/tools.py:14
/Users/mouse/.openclaw/workspace/PromptLens/promptlens/models/tools.py:14: PydanticDeprecatedSince20: Support for class-based
configis deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.13/migration/class ToolParameter(BaseModel):
promptlens/models/test_case.py:10
/Users/mouse/.openclaw/workspace/PromptLens/promptlens/models/test_case.py:10: PydanticDeprecatedSince20: Support for class-based
configis deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.13/migration/class TestCase(BaseModel):
promptlens/models/test_case.py:66
/Users/mouse/.openclaw/workspace/PromptLens/promptlens/models/test_case.py:66: PydanticDeprecatedSince20: Support for class-based
configis deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.13/migration/class GoldenSet(BaseModel):
promptlens/models/config.py:100
/Users/mouse/.openclaw/workspace/PromptLens/promptlens/models/config.py:100: PydanticDeprecatedSince20: Support for class-based
configis deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.13/migration/class RunConfig(BaseModel):
promptlens/providers/google.py:8
/Users/mouse/.openclaw/workspace/PromptLens/promptlens/providers/google.py:8: FutureWarning:
All support for the
google.generativeaipackage has ended. It will no longer be receivingupdates or bug fixes. Please switch to the
google.genaipackage as soon as possible.See README for more details:
https://github.com/google-gemini/deprecated-generative-ai-python/blob/main/README.md
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
______________ coverage: platform darwin, python 3.12.13-final-0 _______________
Name Stmts Miss Cover
promptlens/init.py 3 0 100%
promptlens/main.py 3 3 0%
promptlens/cli.py 169 169 0%
promptlens/exporters/init.py 6 6 0%
promptlens/exporters/base.py 15 15 0%
promptlens/exporters/csv_exporter.py 26 26 0%
promptlens/exporters/html_exporter.py 38 38 0%
promptlens/exporters/json_exporter.py 16 16 0%
promptlens/exporters/markdown_exporter.py 77 77 0%
promptlens/judges/init.py 3 3 0%
promptlens/judges/base.py 15 15 0%
promptlens/judges/llm_judge.py 71 71 0%
promptlens/judges/parser.py 107 107 0%
promptlens/judges/prompts.py 30 30 0%
promptlens/loaders/init.py 4 4 0%
promptlens/loaders/base.py 14 14 0%
promptlens/loaders/json_loader.py 23 23 0%
promptlens/loaders/yaml_loader.py 34 34 0%
promptlens/models/init.py 4 0 100%
promptlens/models/config.py 41 0 100%
promptlens/models/result.py 60 13 78%
promptlens/models/test_case.py 25 0 100%
promptlens/models/tools.py 74 34 54%
promptlens/providers/init.py 3 0 100%
promptlens/providers/anthropic.py 52 32 38%
promptlens/providers/base.py 23 6 74%
promptlens/providers/factory.py 21 10 52%
promptlens/providers/google.py 47 28 40%
promptlens/providers/http.py 84 32 62%
promptlens/providers/openai.py 57 36 37%
promptlens/providers/you.py 57 39 32%
promptlens/runners/init.py 2 2 0%
promptlens/runners/runner.py 96 96 0%
promptlens/utils/init.py 1 0 100%
promptlens/utils/cost.py 15 11 27%
promptlens/utils/diff.py 25 25 0%
promptlens/utils/retry.py 21 15 29%
promptlens/utils/timing.py 24 13 46%
TOTAL 1386 1043 25%
Coverage HTML written to dir htmlcov
======================== 7 passed, 5 warnings in 1.51s =========================
Risks
Rollback Plan