Skip to content

Tighten merged-summary grounding for AI news bullets#126

Draft
Copilot wants to merge 2 commits into
mainfrom
copilot/fix-news-article-summary
Draft

Tighten merged-summary grounding for AI news bullets#126
Copilot wants to merge 2 commits into
mainfrom
copilot/fix-news-article-summary

Conversation

Copilot AI commented May 27, 2026

Copy link
Copy Markdown
Contributor

The summary writer could synthesize related articles into a single bullet with unsupported claims, which led to an incorrect F1 result being stated as fact. This change narrows the summary contract so merged bullets stay anchored to what cited items explicitly say.

  • Prompt guardrails

    • Updated the Copilot summary prompt to forbid inferring winners, outcomes, timelines, or cause-and-effect details unless they are explicit in the cited inputs.
    • Kept the existing merge behavior for clearly related items, but made “source-grounded” stricter where synthesis can overreach.
  • Style guide contract

    • Added the same constraint to build_prompt_payload() so the runtime payload and prompt text enforce the same editorial rule.
    • This keeps merged bullets allowed while reducing the chance of invented narratives across multiple related articles.
  • Regression coverage

    • Added focused tests to lock the new rule into both:
      • the prompt payload style guide, and
      • the exact CLI prompt sent to the summary provider.
"Do not infer winners, outcomes, timelines, or cause-and-effect details "
"that are not explicitly stated in the cited input items."

Copilot AI linked an issue May 27, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Fix incorrect news summary for Canadian GP article Tighten merged-summary grounding for AI news bullets May 27, 2026
Copilot AI requested a review from DevSecNinja May 27, 2026 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(news): incorrect news summarized

2 participants