feat(data-analyst) csv-analyst agent for UP Police (Phase 1 spec)#23
Open
madhyamakist wants to merge 21 commits into
Open
feat(data-analyst) csv-analyst agent for UP Police (Phase 1 spec)#23madhyamakist wants to merge 21 commits into
madhyamakist wants to merge 21 commits into
Conversation
…tes over prose) Forensic mining of all Jul-20 runs (5 lenses, 44 findings) showed step-3.7's output is clean and its question content the best of any model — it fails only where the SKILL relies on prose it under-executes, and on the nvidia serving route (240s dead-streams = 43% of agent-active time, zero prompt cache). hy3's wins trace to endpoint reliability (0 429s / 2,613 calls) + clean turn economy. Changes (all: convert prose to numbered gates / verbatim blocks): - Stage 1: five numbered intake mechanics rules — batch each round as parallel clarify calls in ONE turn (18m10s -> ~10m measured), flat string choices (3/20 calls silently dropped options), no SKILL-text leak, round-exit question-count check (Round 3 dropped 2/4), recap habit. - API key: promoted from prose to a numbered HARD GATE — Stage 2 is invalid until the transcript contains the validation script's OK (live run skipped it entirely and built Phase 1 on an unvalidated key). - Stage 2 scaffold: verbatim script block (base captured BEFORE branching; live run invented `rev-parse HEAD~2` and PR'd against main), mandatory git.md read, NOTES.md in the first commit, and a fresh-clone guard: when base == default branch, open the PR as DRAFT titled [build — DO NOT MERGE]. - Stage 2->3 seam: after `gh pr create`, IMMEDIATELY start Phase 1 — no permission stop (un-asked pause idled a live run 72 minutes). - Stage 3: numbered slice loop (ONE slice -> tests -> implement -> gate), never "all surfaces in one pass" (live run shipped 11 type errors, zero tests). - Router (.hermes.md + AGENTS.md): a build-shaped message IS $ARGUMENTS — start intake same turn, never bounce the idea back (live run wasted a round-trip). - Pitfall #21: serving route matters as much as the model (nvidia vs OpenRouter step-3.7: 0% vs 83.6% cache); reliability beats latency; gates over prose; sessions.model lies (attribute via per-call log lines). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…minutes User constraint update: OpenRouter step-3.7 is not free — nvidia route is forced, so the route-switch item is replaced by CONTEXT DIET as the primary transport mitigation. New live evidence (session 66c652, 00:30-00:55): - 3rd dead-stream (300s at ~118k ctx); zero cache confirmed on all 132 calls; 11.6M input tokens re-prefilled. Stalls correlate with large-context requests. - Phase 1 ran 35+ min with 10 source files on disk but ZERO commits — invisible in git/PR and one crash from loss. Slice loop now has step 5: commit+push the slice AND post a one-line progress note (the user sent four "are you done?" nudges; each answer cost a full ~135k re-prefill). - Refined delegation rule: parallel fan-out stays forbidden on shared keys, but sequential DELEGATION beats inline once root context exceeds ~60k on a no-cache route (fresh ~20k worker context ≈ 7x cheaper per call, below the stall zone). - Pitfall #21 extended: context-diet playbook + discovery that Hermes's background skill-review runs in-session on the MAIN model with full context (own call counter at 135-150k on the same endpoint, amplifying 429s). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t diet) into the build branch
…ermes config edits assumed Clone-and-go philosophy: users never edit ~/.hermes config, so 429/stall resilience lives in the SKILL: set the 'just type continue' expectation at build start, resume-don't-replan on continue, and structural defenses (batched turns, per-slice commit+push, state on disk). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…on' into feature/up-datanalyst-20260720-1848-v0.1
…ories Strip run anecdotes/measurements from SKILL.md (they live in commit messages, PR #24, and the pitfalls reference). Net: leaner injected context. Also fold the provider-hiccup guidance into one rule and add turn-pacing: do a slice's related writes as parallel tool calls in one turn — fewer, fuller turns = fewer per-minute rate-limit hits at the same throughput. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…on' into feature/up-datanalyst-20260720-1848-v0.1
…any rate-limit mention Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…on' into feature/up-datanalyst-20260720-1848-v0.1
…ed parallelism Three surgical, instruction-only changes to the build SKILL (main otherwise untouched), designed for providers that cap REQUESTS per minute: - Intake: batch each round's questions as parallel clarify calls in ONE turn. - Slice rhythm: one slice ≈ 3 requests (parallel writes turn; single chained pytest+add+commit+push command; progress note). Never two terminal calls where one && chain works. 429 mention → sleep 30 prefix + batch harder. - Delegation: replace the blanket free-key inline preference with a request BUDGET — 2-3 parallel workers on disjoint slices while total request rate stays under ~half the provider cap; drop to one worker after any rate limit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…feature/up-datanalyst-20260720-1848-v0.1
… on free tiers Main harness's resilience was accidental: fat context made calls slow (~2 rpm), so the loop rarely rolled the dice during a rate-limited pool's empty windows. The trim's lean-fast calls rolled 4x more often and died 4x more often. This makes the armor deliberate: keep the trim's batching (fewer requests per unit of work) but hold the cadence at main's pace via a sleep-15 floor on terminal commands, escalating to 45s after any rate-limit mention. Steady beats fast. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…feature/up-datanalyst-20260720-1848-v0.1
…oral guards Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…feature/up-datanalyst-20260720-1848-v0.1
…r the nemotron test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Natural-language data analyst for UP Police. Multi-CSV upload + NL questions → plan + query (DuckDB SQL) + table + charts + KPIs. Phase 2 extends to live MsSQL via DuckDB cache layer with zero additional load.
Spec files added/changed
Assumptions captured