Skip to content

Evidence-first AI bake-off completion and fallback hardening - #8

Merged
sev7enITA merged 3 commits into
mainfrom
codex/policywatcher-ai-bakeoff-completion
Aug 14, 2026
Merged

Evidence-first AI bake-off completion and fallback hardening#8
sev7enITA merged 3 commits into
mainfrom
codex/policywatcher-ai-bakeoff-completion

Conversation

@sev7enITA

Copy link
Copy Markdown
Owner

Summary

  • preserve deterministic evidence-first acquisition and publication gates
  • complete provider JSON Schema, privacy-minimized telemetry and frozen golden-set evaluation
  • make Qwen3 a real embedding-then-reranking track and add loopback-only local adapters
  • replace three polynomial regex paths with bounded linear parsers
  • classify network resets as transient fallback events and use monotonic latency
  • contain golden-set file access before loopback transmission

Observed bake-off

  • BM25: frozen retrieval gates pass
  • Gemini 3.5 Flash-Lite: 5/5 completed, all extraction gates pass
  • Gemini 3.7 Flash: 0/3 completed in final run, provider returned 503, no promotion
  • Qwen3/BGE-M3: not scored because anonymous Hugging Face downloads could not complete; no substitute results
  • RAGFlow, LightRAG, Kimi K3 and GraphRAG remain research-only

Verification

  • 139 Vitest files / 736 tests
  • ESLint and TypeScript pass
  • production build passes with 159 pages
  • production dependency audit reports zero vulnerabilities
  • local Prisma schema is current
  • golden-set validation and deterministic baseline pass

Comment thread src/lib/aiRetrievalAdapter.ts Fixed
@sev7enITA
sev7enITA merged commit 381dee3 into main Aug 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants