Skip to content

PolicyWatcher 3.9.0 Beta 41 — Evidence-first AI evaluation and adaptive experience - #7

Merged
sev7enITA merged 59 commits into
mainfrom
codex/policywatcher-beta41-ai-evaluation
Aug 14, 2026
Merged

PolicyWatcher 3.9.0 Beta 41 — Evidence-first AI evaluation and adaptive experience#7
sev7enITA merged 59 commits into
mainfrom
codex/policywatcher-beta41-ai-evaluation

Conversation

@sev7enITA

@sev7enITA sev7enITA commented Aug 14, 2026

Copy link
Copy Markdown
Owner

What changed

  • completes the PolicyWatcher 3.9.0 Beta 41 adaptive experience, global Civic directory, canonical SEO alignment and public evidence navigation
  • replaces the retired Gemini fallback, upgrades the SDK and supply-chain pins, and enforces provider JSON Schema plus local semantic validation
  • adds privacy-minimized AI invocation telemetry, its additive SQLite migration, protected admin aggregate and explainability panel
  • introduces a frozen EN/IT synthetic golden set and a reproducible bake-off harness for BM25, loopback-only Qwen3/BGE adapters, Gemini 3.5 extraction and Gemini 3.7 escalation
  • closes CodeQL security boundaries for renderer routing, admin/scraper logging, archive parsing, iframe messaging and evaluation-adapter egress
  • adds release audits, research reports, paper sources, speeches and media assets while excluding local artifacts, build outputs, extracted packages and temporary files

Why

The prior branch and PR names no longer represented the accumulated Beta 41 scope. The AI audit also identified a retired fallback, missing structured-output enforcement, missing evaluation lineage and vulnerable transitive dependencies. This PR consolidates the current evidence-first release on a coherent branch without adopting RAGFlow, LightRAG, Kimi K3 or GraphRAG wholesale.

User and operator impact

  • production evidence and publication gates remain unchanged
  • AI telemetry stores model/outcome/latency/token/schema lineage only; prompts, responses, policy text, identities, URLs and provider messages are not persisted
  • deployment adds one append-only AiModelInvocation migration and requires a verified database backup
  • Qwen3/BGE evaluation traffic is restricted to loopback; a remote VPS adapter requires an authenticated SSH tunnel
  • no new environment variable and no Renderer or Operations Agent update is required

Validation

  • 136 Vitest files and 728 tests passed
  • 7 renderer security tests passed
  • ESLint and TypeScript passed
  • Next.js production build passed with 159 generated pages
  • npm audit --omit=dev --audit-level=high reported 0 vulnerabilities
  • golden-set validation passed; BM25 baseline: Hit@1 0.8571, Hit@3 1.0, MRR 0.9286, answerability accuracy 0.8889, abstention F1 0.8
  • GitHub Quality Gate, Sonar, reliability coverage, CodeQL workflow and Advanced Security PR check passed
  • Hostinger source archive inventory, secret exclusions and SHA-256 verification passed

Known evaluation boundary

Qwen3/BGE and paid Gemini challenger tracks are wired but not represented as completed results because no loopback adapter endpoints or evaluation API key were available. Promotion remains gated on measured golden-set results and human evidence review.

Fabrizio Degni and others added 30 commits July 21, 2026 16:13
Comment thread renderer/server.mjs Fixed
Comment thread src/lib/evidenceCollection.ts Fixed
Comment thread docs/media/policywatcher-social-shorts/script.js Fixed
Comment thread scripts/ai-bakeoff.ts
const response = await fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
Comment thread src/app/developers/webhook-readiness/WebhookReadinessClient.tsx Fixed
expect(existsSync('src/app/api/pulse/launch-kit/route.ts')).toBe(false);
expect(existsSync('src/app/api/og/launch/[asset]/route.tsx')).toBe(false);
expect(change).toContain('/api/evidence-packet/${id}?format=pdf');
expect(change).not.toContain('/api/report/${change.policy.id}');
Comment thread src/lib/__tests__/seoCanonical.test.ts Fixed
Comment thread src/lib/scraper.ts Fixed
Comment thread src/lib/scraper.ts Fixed
@sev7enITA
sev7enITA marked this pull request as ready for review August 14, 2026 06:35
@sev7enITA
sev7enITA merged commit 817b251 into main Aug 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants