feat(kbc): coverage v2 — media assets auto-attach via citing docs#439
Open
LikiosSedo wants to merge 3 commits into
Open
feat(kbc): coverage v2 — media assets auto-attach via citing docs#439LikiosSedo wants to merge 3 commits into
LikiosSedo wants to merge 3 commits into
Conversation
Media assets (images under assets/) are document attachments, not first-class
sources. Coverage v2 auto-accounts an image against any document that embeds it
in its body, once that document is itself cited or excluded — so compiled_from
no longer needs a token per image and the exclusion ledger no longer needs a
row per image.
Motivation
- compiled_from bloat: aggregated cf drowned in image tokens (UHI 昇腾.md: 139
entries = 35 docs + 104 images), unreadable to a human.
- exclusion-ledger noise: images as first-class sources forced ~563 of UHI's
603 exclusion rows to be mechanical asset entries.
- attribution gap: the schema had no way to express "which document embeds this
image", so mediaverify and the collapse UI each guessed with heuristics.
Math (v2)
auto = { a in media_assets | exists d in (cited | excluded): edge(d->a) }
unaccounted = sources - cited - excluded - auto
- edges are derived from the raw tree (each document's body ![](), [](), and
<img src> links), resolved relative to the document + posixpath.normpath +
URL-decoded; a link to a nonexistent/non-media path yields no edge, no error.
- an ORPHAN image (embedded by no accounted document — upload residue) is NOT
auto-attached; it stays unaccounted and must be excluded with a reason so a
human still sees it (anti fail-open).
- a directly-cited asset still counts as cited (v1 compatibility).
Monotonicity (old green libraries stay green)
v2 only ADDS an accounting path, so unaccounted can only shrink. All pre-existing
selfcheck tests pass unchanged — none asserted "unreferenced asset => unaccounted"
over an assets/ path (the media-asset predicate requires an assets/ segment,
which no existing test used).
Sheet-placeholder carve-out
assets/sheets/*.md table placeholders are content files, not media
(is_media_asset requires an image extension). They stay first-class sources —
auto-attaching them would launder "the table data was never compiled" into
"covered".
Observability
coverage adds auto_attached (count) and auto_attached_sample (<=20 asset<-via
edges), so auto-accounting is never a silent fail-open.
Two-repo fixture
kbc/platform/pod/fixtures/asset-provenance/ is the shared, byte-for-byte
contract (raw tree + candidate + EXCLUSIONS + expected.json). sicore's adoption
ledger will assert the SAME expected.json. Cases: relative link, ../ cross-dir,
HTML <img>, URL-encoded path, missing target (no edge), 0-byte download-failed
placeholder, assets/sheets/*.md (not media), image shared by a cited and an
unaccounted document, orphan image, image inheriting a document's exclusion,
directly-cited asset.
S1b: box constitution now tells the compile agent to cite documents only —
images auto-attach; do not cite or exclude them one-by-one; orphan images still
need a reasoned exclusion; assets/sheets/ placeholders handled as usual.
Follow-up to coverage v2. media_citing_pages() gains a third discovery path: a candidate page that cites a DOCUMENT d in its compiled_from now inherits the image numeric-fidelity check of every image d embeds in its body (via asset_attribution_edges), unioned with the existing two paths (directly-cited compiled_from images + body (source: img) citations), which legacy pages keep using. Why: S1b tells the compile agent to stop citing images one-by-one — they auto-attach. media_citing_pages() drove the blind fresh-eyes transcription check off cited images only, so without this the numeric check would silently stop covering embedded charts — a silent fail-open on the exact fidelity risk it exists to catch, in the window before the image-audit dimension lands. Edge assets are intersected with IMAGE_SOURCE_EXTS (the transcription surface), so a media asset like .tiff is accounted by coverage but not numerically re-read. Additive only: no existing media test used an assets/ raw path, so all prior assertions pass unchanged.
Two implementation-level details locked byte-for-byte with the sicore adoption ledger (shared fixture, DESIGN §六), after cross-checking the two implementations: 1. is_media_asset segment match is now case-INSENSITIVE (`Assets/`, `report.ASSETS/` count too). The platform always writes lowercase `assets/`, so this only affects hand-authored trees, but both repos must agree. Locked by a unit test, NOT the shared fixture: an uppercase directory is not portable on a case-insensitive filesystem (it collides with `assets/`). 2. document_link_targets truncates a `#fragment` AND `?query` from the target, and does so on the STILL-ENCODED string BEFORE percent-decoding (new helper _strip_fragment_query). This matches the ledger's parse order (strip angle → truncate #/? → unescape), so an encoded %23/%3F inside a real filename survives while an actual delimiter is removed. Everything else already matched (ext set incl. tiff / no pdf, both quote styles for HTML <img>, resolve-then-membership skip rule, single unquote, Clean(Join) keeping leading ..). The shared fixture gains a `?query`-suffixed target case (raw/extras.md → assets/q.png?v=2). Neither change alters any prior fixture output; expected.json is regenerated from the reference implementation with an in-generator sanity assert on the invariants. All selfcheck + mediaverify tests green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Media assets (images under
assets/) are document attachments, not first-class sources. Coverage v2 auto-accounts an image against any document that embeds it in its body, once that document is itself cited or excluded.compiled_fromno longer needs a token per image and the exclusion ledger no longer needs a row per image.Implements S1 (siclaw/kbc) of the raw reference contract v2. Design:
DESIGN-kb-asset-provenance-2026-07-22.md§四 (normative).Motivation
compiled_fromdrowns in image tokens (UHI昇腾.md: 139 entries = 35 docs + 104 images), unreadable to a human.mediaverifyand the collapse UI each guessed with heuristics.Math (v2)
,[](…), and HTML<img src>links — resolved relative to the document +posixpath.normpath+ URL-decoded. A link to a nonexistent / non-media path yields no edge, no error.unaccountedand must be excluded with a reason, so a human still sees it (anti fail-open).cited(v1 compatibility).Monotonicity — old green libraries stay green
v2 only adds an accounting path, so
unaccountedcan only shrink. All pre-existingselfchecktests pass unchanged — none asserted "unreferenced asset ⇒ unaccounted" over anassets/path (the media-asset predicate requires anassets/segment, which no existing test used, so no assertion needed updating).Sheet-placeholder carve-out
assets/sheets/*.mdtable placeholders are content files, not media (is_media_assetrequires an image extension). They stay first-class sources — auto-attaching them would launder "the table data was never compiled" into "covered".Observability
coveragegainsauto_attached(count) andauto_attached_sample(≤20asset ← viaedges), so auto-accounting is never a silent fail-open.Two-repo fixture (byte-for-byte contract)
kbc/platform/pod/fixtures/asset-provenance/= shared raw tree + candidate +EXCLUSIONS.json+expected.json. The sicore adoption ledger (S2) will assert the sameexpected.json. Cases: relative link,../cross-dir, HTML<img>, URL-encoded path, missing target, 0-byte download-failed placeholder,assets/sheets/*.md(not media), image shared by a cited + an unaccounted document, orphan image, image inheriting a document's exclusion, directly-cited asset.S1b — compile discipline
The box constitution (
prompts/{en,zh}/box_role.md) now tells the compile agent: cite documents only; images auto-attach — do not cite or exclude them one-by-one; orphan images still need a reasoned exclusion;assets/sheets/placeholders handled as usual.Test
python test_selfcheck.py— all green (4 new:is_media_asset,document_link_targets,coverage_v2_auto_attach,asset_provenance_fixture).test_mediaverify/test_incremental/test_redblue/test_compile_boxalso green.