feat(pipeline): Voikko-based kirjakieli filter

[laurigates]

Summary

Implements the core discriminator from the README: frequent in colloquial corpus + rejected by libvoikko = puhekieli/slang.

• New Lexicon trait in niinku-pipeline and a pure apply_kirjakieli_filter(table, lexicon, allowlist) function. Trait-typed so pipeline tests stay hermetic with a StaticLexicon mock.
• New VoikkoLexicon wrapper in niinku-sources using voikko-rs 1.1. SpellOk → accepted; internal/charset errors → not accepted (conservative — never drop a word we couldn't analyse).
• niinku-sources/build.rs uses pkg-config to find libvoikko on macOS Homebrew (/opt/homebrew/lib) and other non-default locations; falls back silently when pkg-config / libvoikko.pc isn't present.
• CLI wires the filter between min-count and scoring, loads data/allowlist.txt, adds --no-voikko and --voikko-dict-path flags.
• CI installs libvoikko-dev + voikko-fi before cargo runs.
• README documents the new prerequisite.

Local end-to-end run

loading opensubtitles-fi from data/cached/opensubtitles-fi.txt
  50000 tokens
denylist: 0 entries
allowlist: 0 entries
merged: 50000 tokens
after denylist + min-count>=5: 50000 tokens
after voikko kirjakieli filter: 7435 tokens (dropped 42565)
wrote 7435 entries to data/out/niinku.combined

Spot-check — the puhekieli vocabulary now lands cleanly:

word=mä,f=219     word=sä,f=215     word=sun,f=207    word=mun,f=209
word=tää,f=201    word=oon,f=198    word=mua,f=200    word=sua,f=200
word=niinku,f=186 word=sit,f=182    word=ois,f=182    word=mee,f=174
word=onks,f=181   word=nää,f=181    word=noi,f=170    word=moro,f=169
word=emmä,f=165   word=just,f=194   word=kaa,f=181    word=tuu,f=182
word=törppö,f=172 word=sulta,f=170

Known noise at the head of the list: English proper nouns from translated subtitles (john, jack, michael, fbi). These are valid OOV tokens that Voikko correctly rejects but they aren't useful as puhekieli. A future commit can either add common English names to the denylist or pre-filter by character-set heuristics — not in scope here.

Test plan

• just lint (fmt + clippy -D warnings) — green
• just test (17 unit tests across pipeline + sources, including 2 new filter tests) — green
• just generate end-to-end — produces 7435-entry filtered .combined
• Spot-check: puhekieli vocabulary present at expected freq band
• --no-voikko flag bypasses the filter (back to original 50000 entries)

Remaining open items (not in this PR)

• .combined header + .dict compile via dicttool_aosp.jar
• Mastodon adapter
• Suomi24 adapter
• Filtering English proper-noun noise from subtitle corpora

🤖 Generated with Claude Code