Skip to content

Add endpoint probe, rename cost_probe, and add PR/probe interval throttling#106

Merged
BillJr99 merged 2 commits into
mainfrom
claude/free-script-provider-probing-tw1x2u
Jun 19, 2026
Merged

Add endpoint probe, rename cost_probe, and add PR/probe interval throttling#106
BillJr99 merged 2 commits into
mainfrom
claude/free-script-provider-probing-tw1x2u

Conversation

@BillJr99

Copy link
Copy Markdown
Owner

Summary

  • Rename probecost_probe everywhere: source file (scripts/sources/cost_probe.py), class (CostProbeSource), config key (free_tier.cost_probe), state file (cost_probe_state.json), CLI flags (--cost-probe, --cost-probe-max). Back-compat migration copies old probe_state.json on first run.
  • New endpoint_probe source (scripts/sources/endpoint_probe.py): actively calls each configured provider's GET /v1/models to discover new :free-suffix models and flag disappeared ones — gated by existing sync_on_startup / update_on_startup flags, throttled by free_tier.endpoint_probe.frequency_minutes (default 30).
  • Interval checks on startup and per-request: _maybe_fire_interval_probes() (60 s in-memory gate) fires endpoint probe, cost probe, and PR creation as daemon threads when their respective intervals elapse.
  • PR creation throttle: providers_pr.frequency_days config key; state tracked in pr_state.json next to the user config.
  • 429 retry logic in DocsScraperBase.fetch(): up to 3 retries with Retry-After support, returns [] on exhaustion. Adds two new tests for the retry path.
  • Race-condition fix in _build_models_list: startup warmup (only_if_empty=True) no longer overwrites _models_list_cache when a concurrent request or a cross-test daemon thread already populated it.

New config keys

"providers_pr": {
  "enabled": true,
  "frequency_days": 7
},
"free_tier": {
  "sync_on_startup": true,
  "update_on_startup": false,
  "endpoint_probe": {
    "frequency_minutes": 30,
    "timeout_sec": 10
  },
  "cost_probe": {
    "enabled": false,
    "autoremove": false,
    "frequency_days": 0
  }
}

State files (auto-created next to config.json):

  • endpoint_probe_state.json — tracks last_probe_at for the endpoint probe
  • cost_probe_state.json — tracks last_probe_at for the cost probe (migrated from probe_state.json)
  • pr_state.json — tracks last_pr_at for PR throttling

Test plan

  • All existing tests pass (no regressions)
  • test_probe_source.py uses CostProbeSource from scripts.sources.cost_probe
  • test_probe_throttle.py uses cost_probe_state.json path helpers
  • test_docs_huggingface.py covers 429 retry and exhaustion paths
  • test_real_model_name_unchanged no longer flaky (startup daemon thread race fixed)
  • test_admin_api.py::test_put_maintenance_sets_flags checks free_tier.cost_probe key

🤖 Generated with Claude Code

https://claude.ai/code/session_01RkRhX4Znv73oPcZy41bnyS


Generated by Claude Code

claude added 2 commits June 19, 2026 11:53
…tale data

_build_models_list now skips the _models_list_cache write when called
with only_if_empty=True and the cache is already populated. This prevents
two races:

1. Within-test: the startup daemon thread (step 5 of _run_startup_tasks_once)
   overwrote a fresh cache built by list_models() on a concurrent request.
2. Cross-test: a daemon thread left over from a prior test (after monkeypatches
   were torn down) wrote an empty cache to the module dict after a module reload
   reset it to None, causing the next test's list_models() to see a stale cache hit.

Also suppress startup daemon threads and interval probe checks in the
test_proxy_display_id_format server fixture to eliminate remaining cross-test
contamination from prior tests' threads writing with old lock instances.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RkRhX4Znv73oPcZy41bnyS
… fixtures

_maybe_fire_interval_probes() spawns background daemon threads that run the
full update pipeline. Under pytest-cov with concurrent threads, Python's
coverage/collector.py:lock_data crashes with SIGSEGV (exit 139). After each
importlib.reload(server_mod) the module globals reset, but old daemon threads
from prior tests still run against the old lock instances — causing races.

Patch both _run_startup_tasks_once and _maybe_fire_interval_probes to no-ops
immediately after every reload in all 15 server-loading test helpers/fixtures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RkRhX4Znv73oPcZy41bnyS
@BillJr99 BillJr99 merged commit 96e3a35 into main Jun 19, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants