Skip to content

Add capacity status edge fixture#79

Merged
0xzr merged 3 commits into
0xzr:mainfrom
HarperZ9:agent/capacity-status-fixture
Jul 20, 2026
Merged

Add capacity status edge fixture#79
0xzr merged 3 commits into
0xzr:mainfrom
HarperZ9:agent/capacity-status-fixture

Conversation

@HarperZ9

@HarperZ9 HarperZ9 commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a deterministic capacity status --all --no-catalog-sync CLI fixture for issue Add a capacity-status fixture for quota edge cases #54.
  • Isolate config, cache, quota, home, key-inventory, catalog, and environment lookups under tmp_path / monkeypatches so local developer credentials cannot affect the result.
  • Assert the semantic capacity output: healthy keyless provider, low-quota configured provider, missing-key provider, summary, warning, and action recommendation without depending on full table whitespace.
  • Add a small Windows guard around os.fchmod in upsert_config_key; the focused Windows test command otherwise crashed in unrelated keys add tests before this PR's fixture could be validated. POSIX still enforces 0600; Windows asserts successful write/content without pretending POSIX mode bits apply.

Closes #54.

Validation

Passed:

  • PYTHONPATH=src python -m pytest tests/test_cli.py::test_cli_capacity_status_all_reports_quota_edges_without_local_state -q
  • PYTHONPATH=src python -m pytest tests/test_capacity.py tests/test_cli.py -q
  • PYTHONPATH=src python -m pytest tests/test_capacity.py tests/test_cli.py tests/test_key_inventory.py tests/test_robustness.py -q
  • PYTHONPATH=src python -m ruff check .

Full coverage gate attempted but not green in this Windows checkout:

  • Native Windows initially exposed unrelated platform assumptions outside this PR's requested scope.
  • WSL/Linux run reached 748 pass / 10 fail; the remaining failures are all tests/test_metaswarm_integration.py caused by the Windows checkout materializing integrations/metaswarm/freellmpool-review-adapter.sh with CRLF, producing /usr/bin/env: bash\r and case ... in\r shell parse failures.

I left those metaswarm line-ending policy issues untouched to keep this PR focused.

Summary by Sourcery

Add a deterministic CLI capacity status fixture that isolates local state and adjust key config permissions handling for cross-platform compatibility.

New Features:

  • Add a CLI test fixture that exercises capacity status --all --no-catalog-sync across keyless, low-quota, and missing-key providers without relying on local configuration.

Bug Fixes:

  • Prevent crashes on platforms without os.fchmod by guarding key config permission changes in upsert_config_key.

Tests:

  • Add a capacity status edge-case test that asserts semantic output for provider health, quota warnings, and action recommendations while remaining robust to formatting differences.
  • Relax key permission tests to only enforce 0600 mode when os.fchmod is available, aligning expectations with platform behavior.

@sourcery-ai

sourcery-ai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adds a deterministic CLI test fixture for capacity status --all --no-catalog-sync that isolates environment/config state and validates semantic capacity/quota reporting, and makes key configuration writing tolerant of platforms without os.fchmod while adjusting related permission tests for cross-platform behavior.

File-Level Changes

Change Details Files
Add deterministic capacity status CLI test that isolates local state and asserts semantic quota edge reporting.
  • Extend CLI tests to import Model and Provider so a synthetic catalog can be constructed in tests.
  • Introduce a new test that builds an in-memory provider catalog and fake quota store, monkeypatches environment/config paths and catalog/key/quota loaders under tmp_path, and runs capacity status --all --target 3 --no-catalog-sync.
  • Assert output lines for healthy/low-quota/missing-key providers, capacity summary, external catalog cache size, warning, and action recommendation using content-based rather than whitespace-sensitive checks.
tests/test_cli.py
Make upsert_config_key robust on platforms without os.fchmod while preserving POSIX 0600 enforcement where available.
  • Change upsert_config_key to look up os.fchmod via getattr and only call it when present, still narrowing permissions to 0600 and ignoring OSError as before.
  • Ensure key config files are still written with the same TOML content format.
src/freellmpool/key_inventory.py
Relax key permission tests to only enforce 0600 when os.fchmod exists, improving cross-platform compatibility.
  • Import os in key inventory tests and guard the mode assertion in test_upsert_config_key_creates_new_file behind a hasattr(os, "fchmod") check.
  • Import os in robustness tests and similarly guard the 0o600 mode assertion in test_upsert_config_key_is_0600 on the presence of os.fchmod while still verifying the secret key content.
tests/test_key_inventory.py
tests/test_robustness.py

Assessment against linked issues

Issue Objective Addressed Explanation
#54 Add a no-network CLI test for freellmpool capacity status --all --no-catalog-sync that exercises quota edge cases with at least one healthy provider, one low-quota or exhausted provider, and one missing provider.
#54 Make the new CLI test deterministic by using fake quota, environment, catalog, and key-inventory data so that local developer state does not affect results.
#54 Ensure the test suite (pytest tests/test_capacity.py tests/test_cli.py) and linting (ruff check .) pass after adding the fixture.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@HarperZ9
HarperZ9 marked this pull request as ready for review July 20, 2026 03:16

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The new CLI capacity test heavily monkeypatches global functions (e.g., load_catalog, QuotaStore, effective_env); consider extracting a small helper or fixture to construct this synthetic environment so future changes to these APIs don’t require updating multiple inline monkeypatch calls.
  • In upsert_config_key, using getattr(os, "fchmod", None) is a good Windows guard, but you might want to centralize this permission handling (e.g., a small helper that documents the platform-specific behavior) so other code paths don’t accidentally assume POSIX semantics in the future.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The new CLI capacity test heavily monkeypatches global functions (e.g., `load_catalog`, `QuotaStore`, `effective_env`); consider extracting a small helper or fixture to construct this synthetic environment so future changes to these APIs don’t require updating multiple inline monkeypatch calls.
- In `upsert_config_key`, using `getattr(os, "fchmod", None)` is a good Windows guard, but you might want to centralize this permission handling (e.g., a small helper that documents the platform-specific behavior) so other code paths don’t accidentally assume POSIX semantics in the future.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@HarperZ9

Copy link
Copy Markdown
Contributor Author

Addressed the Sourcery review cleanup in 2d99d1d without changing runtime behavior:

  • extracted the synthetic capacity-status monkeypatch setup into _install_capacity_status_edge_fixture(...) so the test's assertions stay readable
  • centralized the platform-specific os.fchmod guard in _restrict_owner_read_write(fd) and kept upsert_config_key focused on the write path

Fresh validation on the PR branch:

  • $env:PYTHONPATH='src'; python -m pytest tests/test_cli.py::test_cli_capacity_status_all_reports_quota_edges_without_local_state tests/test_key_inventory.py::test_upsert_config_key_creates_new_file tests/test_robustness.py::test_upsert_config_key_is_0600 -q -> passed
  • $env:PYTHONPATH='src'; python -m pytest tests/test_capacity.py tests/test_cli.py tests/test_key_inventory.py tests/test_robustness.py -q -> passed
  • $env:PYTHONPATH='src'; python -m ruff check . -> passed
  • git diff --check -> passed

@HarperZ9

Copy link
Copy Markdown
Contributor Author

Updated the branch with current origin/main in 5629067 so the PR includes #80/#81 and is no longer behind the base branch.

Fresh validation on the updated head:

  • $env:PYTHONPATH='src'; python -m pytest tests/test_cli.py::test_cli_capacity_status_all_reports_quota_edges_without_local_state tests/test_key_inventory.py::test_upsert_config_key_creates_new_file tests/test_robustness.py::test_upsert_config_key_is_0600 -q -> passed
  • $env:PYTHONPATH='src'; python -m pytest tests/test_capacity.py tests/test_cli.py tests/test_key_inventory.py tests/test_robustness.py -q -> passed
  • $env:PYTHONPATH='src'; python -m ruff check . -> passed
  • git diff --check -> passed
  • GitHub/Sourcery check on 5629067 -> passed

I also attempted the repo coverage command locally on Windows. The coverage threshold script passed, but the full pytest run exposed broader Windows/environment assumptions outside this PR's changed path (missing bash / direct .sh execution, CRLF JSONL assertion, Windows path separator expectation, and a cp1252 console encoding issue in freellmpool --help). I am not counting that as a full-suite pass for this PR; just recording it as separate hardening signal.

@0xzr 0xzr left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed against issue #54 and current main. The fixture is deterministic and no-network, covers the required capacity states, and the Windows permission guard preserves POSIX behavior. Focused tests, Ruff, count checks, and all CI lanes pass. Thank you @HarperZ9 for the contribution.

@0xzr
0xzr merged commit d83b43e into 0xzr:main Jul 20, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a capacity-status fixture for quota edge cases

2 participants