Add endpoint probe, rename cost_probe, add PR/probe interval throttling#105
Merged
Merged
Conversation
- Rename scripts/sources/probe.py → cost_probe.py; class ProbeSource →
CostProbeSource; config key free_tier.probe → free_tier.cost_probe;
state file probe_state.json → cost_probe_state.json (with auto-migration).
Back-compat aliases preserved in config.py and admin.py so existing
configs and admin API keep working.
- Add scripts/sources/endpoint_probe.py (EndpointProbeSource): calls each
configured provider's GET /v1/models, emits positive Evidence for new
:free-suffixed models not yet in believed_free and negative Evidence for
:free models that disappeared. Gated by sync_on_startup or
update_on_startup (no separate enabled flag needed). Throttled by
free_tier.endpoint_probe.frequency_minutes (default 30).
- Add providers_pr.frequency_days throttle: _maybe_open_providers_pr()
checks pr_state.json before opening a PR; saves last_pr_at on success.
Accumulated sidecar changes are bundled into one PR per interval.
- Add _maybe_fire_interval_probes() in server.py: debounced (60 s gate)
check called on startup and every request. Fires endpoint probe
(sync_on_startup / update_on_startup), cost probe (update_on_startup +
cost_probe.enabled), and PR creation (frequency_days elapsed) as
single-flight daemon threads when their intervals have elapsed.
- New config keys: free_tier.endpoint_probe.{frequency_minutes,timeout_sec},
free_tier.cost_probe (renamed), providers_pr.frequency_days.
State files: cost_probe_state.json, endpoint_probe_state.json, pr_state.json.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RkRhX4Znv73oPcZy41bnyS
- Rename ProbeSource→CostProbeSource throughout test_probe_source.py (module path, monkeypatch targets, class usage, source name assertion) - Sort cost_probe/endpoint_probe imports to correct alphabetical position in scripts/sources/__init__.py (ruff I001) - Rename unused loop variable prov_name→_prov_name in endpoint_probe.py (ruff B007) - Fix server.py: split `import os, sys` into separate lines (ruff E401), separate import groups with blank lines per isort (ruff I001), and remove unused `save_pr_state as _save_pr_state` alias (ruff F401) - Regenerate config.example.json with cost_probe/endpoint_probe sections Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RkRhX4Znv73oPcZy41bnyS
…n order - test_admin_api: check free_tier.cost_probe (not free_tier.probe) after PUT - test_probe_throttle: import/use cost_probe_* state helpers; --source cost_probe - test_fusion: use sorted() comparison for backfill panel order (parallel threads make request order non-deterministic, only set membership matters) - docs/base.py: retry GET up to 3 times on 429 with exponential-ish backoff, honouring Retry-After; return [] on exhaustion instead of raising - test_docs_huggingface: add tests for 429 retry-then-succeed and exhausted-429 returns-empty, monkeypatching time.sleep to keep tests fast Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RkRhX4Znv73oPcZy41bnyS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rename scripts/sources/probe.py → cost_probe.py; class ProbeSource →
CostProbeSource; config key free_tier.probe → free_tier.cost_probe;
state file probe_state.json → cost_probe_state.json (with auto-migration).
Back-compat aliases preserved in config.py and admin.py so existing
configs and admin API keep working.
Add scripts/sources/endpoint_probe.py (EndpointProbeSource): calls each
configured provider's GET /v1/models, emits positive Evidence for new
:free-suffixed models not yet in believed_free and negative Evidence for
:free models that disappeared. Gated by sync_on_startup or
update_on_startup (no separate enabled flag needed). Throttled by
free_tier.endpoint_probe.frequency_minutes (default 30).
Add providers_pr.frequency_days throttle: _maybe_open_providers_pr()
checks pr_state.json before opening a PR; saves last_pr_at on success.
Accumulated sidecar changes are bundled into one PR per interval.
Add _maybe_fire_interval_probes() in server.py: debounced (60 s gate)
check called on startup and every request. Fires endpoint probe
(sync_on_startup / update_on_startup), cost probe (update_on_startup +
cost_probe.enabled), and PR creation (frequency_days elapsed) as
single-flight daemon threads when their intervals have elapsed.
New config keys: free_tier.endpoint_probe.{frequency_minutes,timeout_sec},
free_tier.cost_probe (renamed), providers_pr.frequency_days.
State files: cost_probe_state.json, endpoint_probe_state.json, pr_state.json.
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01RkRhX4Znv73oPcZy41bnyS