refactor(grouper): split into 2a extractor + 2b classifier with dedup chokepoint#32
Merged
Merged
Conversation
Reduces digest duplication and noise by addressing three sources of low-signal output observed in production: 1. Channel summary header line (🚀 ...) was being extracted as a separate bullet by the grouper, duplicating Section 1 key points. Strip it before feeding to classification. 2. Section 2 (📎 Also/Также) low-priority items were promoted to cross-channel grouping where they competed with high-signal bullets. Strip the section before classification — Also is per-channel by design. 3. The grouper occasionally emitted the same (group, source, point) twice. Add deterministic dedup in _parse_grouped_response keyed on (group, source, normalized_point) so prompt flakes can no longer produce verbatim duplicates. Also tightens the per-channel summarizer prompt with an explicit QUALITY GATE that drops low-signal posts (photo-only, meta-empty, expired local invites, internal admin, AI speculation) and caps Section 1 at 5 bullets. Measured impact (Approach A config-only baseline already cut items 80→44): these changes target the remaining 5 issue categories identified after that baseline.
Adding deterministic dedup pushed _parse_grouped_response complexity to 12, over the project's flake8 max-complexity=10. Extract the per-group inner loop (validation + dedup + GroupedPoint construction) into a helper, restoring complexity under threshold. No behavior change.
… chokepoint Architecturally rewires the grouper from a single combined AI call (extract + classify + dedup at once) into a two-pass pipeline with a deterministic chokepoint between, addressing the architect's review of PR #31. Why: - Single combined call asks an LLM to do three jobs at once on input from 30+ channels. At low temperature this still produces flakes: same-topic bullets from different channels stay separate, cross-group duplicates leak, and the per-channel header line gets echoed as a bullet. - The grouper prompt was getting heavier and heavier trying to compensate. PR #31 added a deterministic post-dedup; this PR splits the pipeline so the LLM can focus on one task at a time and the deterministic step has more leverage. New flow: Pass 2a — `_extract_all_bullets`: Per-channel parallel AI calls (Semaphore-bounded at concurrency 10), each turning one channel summary into a flat list of ExtractedBullet. Each extractor sees only its own channel — no cross-channel context to confuse classification with. Chokepoint — `_dedup_extracted` (only when `dedup_topics: true`): Cross-channel deterministic merge by normalized point text. Same story from N channels → 1 bullet with sources joined as comma-separated list, longest text variant kept. The LLM never sees the duplicates. Pass 2b — `_classify_bullets`: Single AI call over the dedup'd flat list, classifies each bullet into a user-defined topic group. Smaller, focused prompt — much shorter than the old combined prompt. Public API unchanged: `group_summaries(channel_summaries, channel_urls)` still returns `Dict[str, List[GroupedPoint]]`. Also fixes core.py double-print bug: When a group's formatted message exceeds split_message threshold (4000 chars), it produces multiple chunks. The old code passed every chunk's group name into the summary header, producing duplicates like "📰 News, 📰 News" in the stats line. Dedupe with `dict.fromkeys` to preserve order. Tests: 33 grouper tests (all passing), full suite 400/0 with 79% coverage.
…d-stats-fix # Conflicts: # src/grouper.py # tests/test_grouper.py
7 tasks
belaytzev
added a commit
that referenced
this pull request
May 9, 2026
#33) * feat(grouper): deterministic QUALITY GATE filter + broader noise strip Stops trusting the LLM to follow negative drop rules. Adds a regex-based filter that runs on the extracted bullet list before dedup/classification, catching the noise patterns the prompt-level QUALITY GATE was failing to suppress in practice (gpt-5.4-mini ignores negative rules at low temp). Architect's recommendation: "use grep for what grep can do." Three changes: 1. Broader noise strip in `_strip_channel_summary_noise`: - 📌 Key points / Ключевые моменты / Puntos clave / etc — template header was leaking into bullets as `📌 Key points: (channel list)` - 1️⃣-9️⃣ section numbering prefixes — cosmetic clutter - [emoji], [brief fact] template token placeholders 2. New `_quality_gate_filter` deterministic pass between extract and dedup. Drops: - Admin chatter (new chat members, joins/leaves) - Meta-empty bullets ("без подробностей", "no details", "just a poll") - Short bullets (<30 chars) lacking concrete entity (digits/@/URL/proper) - Speculation/hedging without concrete entity ("вероятно", "probably") 3. QUALITY GATE moved into extractor system prompt as belt-and-suspenders. Framed as "DROP rules OVERRIDE extract-verbatim" so the model isn't torn between conflicting instructions. Real safety still comes from #2. Architecture: Pass 2a extract → QUALITY GATE filter → cross-channel dedup → Pass 2b classify ^^^^^^^^^^^^^^^^^^ NEW deterministic chokepoint Tests: 410 passed, 7 skipped, coverage 79.18%. Real-world digest motivation (from PR #32 post-merge run): - Tech section had `🆕 В чате появился новый участник Denis Nogtev` (admin chatter) - Другое had `📌 Key points: (Вастрик Селфхостеры🔧, kyrillic🌍, ...)` (template leak) - Tech had `🛸 В подборке упомянуты темы, но без дополнительных деталей` (meta-empty) - Другое had `📊 Парк выглядит как сильный фотоспот` (speculation, no entity) * style(tests): collapse line for black 24.10.0 compatibility CI pins black 24.10.0 (per CLAUDE.md note); local newer black accepted the multi-line form. CI black wants the call on one line. Verified locally with: uvx --from "black==24.10.0" black --check src/ tests/
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two changes in one PR (per request to bundle):
📰 News, 📰 Newsbug).Builds on top of the now-merged PR #31.
Why split the grouper
PR #31 cut the digest from 80 → 44 items but five problem categories remained, three of which traced back to the same root cause: the grouper was being asked to extract, classify, and dedup all in one AI call across 30+ channels. The architect review on #31 flagged this — at low temperature with long input, LLMs struggle with 3-axis decisions.
This PR's structure follows the architect's recommendation:
Same-topic bullets from N channels collapse to 1 entry before the classifier sees them. The classifier no longer has to decide "are these the same story?" — that's already handled.
Why the stats fix lives here
Stats double-print (
📰 News, 📰 Newsin the summary header) was identified during PR #31 analysis as a separate 1-line bug incore.py. Bundled per request.Root cause: when
format_group_messagereturns a message longer than 4000 chars,split_messagechunks it. Each chunk gets appended as(name, part). The oldgroup_names = [name for name, _ in group_messages]then produced duplicates. Fix:list(dict.fromkeys(...))preserves order while deduping.Trade-offs
Cost: Pass 2a is N parallel AI calls (one per channel) instead of 1. For 33 channels with
gpt-5.4-mini:Rate limits:
Semaphore(10)caps concurrency. If you hit OpenAI tier limits with very large channel lists, lower_EXTRACTOR_CONCURRENCYingrouper.py:31.Backward compat:
group_summaries()signature unchanged. Internal restructure only.What's preserved from PR #31
_strip_channel_summary_noise(🚀 header + 📎 Also strip) — now invoked from extractor prompt builder_normalize_pointhelper — now used by both_collect_group_pointsand_dedup_extracted_collect_group_pointsdeterministic dedup in classifier output (extra safety)base_summary.txtTest plan
test_summary_message_dedupes_split_group_names)uv run pytest tests/— 400 passed, 7 skipped, 78.89% coverageuv run mypy src/— no issuesuv tool run ruff check src/ tests/— cleanuv run black --check src/ tests/— cleanuvx flake8 --max-complexity=10 src/ tests/— clean (lesson from feat(grouper): strip noise + deterministic dedup before classification #31 CI)/digeston real bot, verify Касперская no longer appears in both Tech and News, UAP not splitFiles
src/grouper.py— major: ExtractedBullet, _dedup_extracted, _build_extractor_prompt, _build_classifier_prompt, _extract_bullets_from_channel, _extract_all_bullets (asyncio.gather + Semaphore), _classify_bullets, _warn_missing_channels; group_summaries rewiredsrc/core.py— 1-line dedupe fix at line ~364tests/test_grouper.py— 12 new tests (extractor, parallel extraction, cross-channel dedup, classifier), existing tests migrated to new APItests/test_core.py— new test for stats dedupe