Skip to content

feat: Add coordination scripts for managing gap slices and state#1

Open
slurrr wants to merge 1 commit into
mainfrom
telemetry
Open

feat: Add coordination scripts for managing gap slices and state#1
slurrr wants to merge 1 commit into
mainfrom
telemetry

Conversation

@slurrr

@slurrr slurrr commented Mar 18, 2026

Copy link
Copy Markdown
Owner
  • Introduced coordination_config.py for reading configuration values from TOML files.
  • Added coordination_state.py to maintain and update the state of gap selections and responses.
  • Created gap_slice.py to generate current gap slices based on defined criteria.
  • Implemented shell scripts for pulling and pushing model runner responses and gap slices.
  • Developed reconcile-gap-ledger.sh to integrate responses and update the gap ledger.
  • Added run-gap-loop.sh to automate the gap processing loop.
  • Enhanced ingestion monitoring in live.py to track telemetry events and errors.
  • Updated server and store modules to support ingestion status reporting.

- Introduced `coordination_config.py` for reading configuration values from TOML files.
- Added `coordination_state.py` to maintain and update the state of gap selections and responses.
- Created `gap_slice.py` to generate current gap slices based on defined criteria.
- Implemented shell scripts for pulling and pushing model runner responses and gap slices.
- Developed `reconcile-gap-ledger.sh` to integrate responses and update the gap ledger.
- Added `run-gap-loop.sh` to automate the gap processing loop.
- Enhanced ingestion monitoring in `live.py` to track telemetry events and errors.
- Updated server and store modules to support ingestion status reporting.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e19d3f3864

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/lab_llm/live.py
Comment on lines 131 to +133
if previous is not None:
delay = max((current - previous).total_seconds() / self._speed, 0.0)
if delay > 0:
time.sleep(delay)
if delay > 0:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Initialize delay before the first replay event

When previous is None on the first loop iteration, delay is never assigned before if delay > 0. In the default fixture workflow (lab_llm.main enables --replay-fixtures by default), the replay thread now dies with UnboundLocalError on its very first event, so the standard local dashboard startup path comes up without ingesting any fixture data.

Useful? React with 👍 / 👎.

Comment thread scripts/push-gap-slice.sh
Comment on lines +73 to +74
## Verification run
-

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add a parseable verified session id to the response template

The new coordination loop only recognizes verified runs through the exact Verified session id: block parsed in coordination_state.py/audit-dashboard.sh, and check-loop-stop aborts with missing-verified-session-id whenever addressed gaps are reported without one. Because the generated response.md template leaves ## Verification run as a free-form bullet, a model-runner response that follows this template but phrases the session id differently will not be parsed, causing reconciliation to stop even after successful upstream fixes.

Useful? React with 👍 / 👎.

Comment thread frontend/src/App.tsx
Comment on lines +196 to +198
source.addEventListener("connected", () => {
setError(null);
setIngestion((current) => (current ? { ...current, connected: true } : current));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep ingestion connectivity separate from SSE connectivity

connected in /api/status comes from IngestionMonitor and is supposed to describe whether the telemetry source itself is available, but this handler forces it to true as soon as the browser opens the SSE stream. In JSONL mode with a missing or rotated file, the backend correctly reports connected: false; the frontend then overwrites that to true on the SSE handshake and shows jsonl · connected even though no telemetry source is actually being tailed.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant