fix: enrichment now reaches the agent and the resolver - #18
Merged
Conversation
querypad enrich --apply wrote descriptions and synonyms that nothing ever read
back. Four separate links in the chain were broken:
1. There was no readGlossary, and prepareDataset rebuilt the semantic model from
profiles unconditionally, so enrichment was discarded before the next ask.
glossary.json is now a curation input re-applied over the derived model,
exactly as verdicts.json is re-applied over inferred relationships. Same
contract: honored by every surface, preserved across re-runs.
2. mergeGlossary dropped any entry naming a column that was not a dimension. A
numeric column always becomes a measure, so every money term in a glossary hit
that continue and vanished. Column-level entries now resolve against measures
too.
3. SemanticDimension and SemanticMeasure had no synonyms field, and there was no
code path that could add one to a measure.
4. buildTermCatalog indexed only names; descriptions went into the embed-only
text field and were never scored lexically.
Net effect, and the reason this matters for the trap dataset work: a business
word can now reach an opaque column. resolve_terms("net revenue") returns
sum_amt_c where it previously returned "No matching terms" - verified against
the real engine both ways.
The grounding context also renders descriptions and inline dimension/measure
annotations, so enrichment is visible to the agent instead of only to the YAML.
All 131 existing tests pass untouched (the change is additive); 5 new tests pin
the measure-mapping path, synonym indexing, the resolution flip, the context
rendering, and glossary.json curation through prepareDataset.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Found while designing the harder trap dataset:
enrich --applywas writing descriptions and synonyms that nothing ever read back. Any eval trap depending on a glossary would have failed in both arms, so this had to be fixed before the dataset work could mean anything.Four separate links in the chain were broken:
readGlossaryexisted, andprepareDatasetrebuilt the semantic model from profiles unconditionally - so enrichment was discarded before the nextask.mergeGlossarydropped any entry naming a column that is not a dimension. A numeric column always becomes a measure, so every money term in a glossary hit thatcontinueand vanished.SemanticDimension/SemanticMeasurehad nosynonymsfield, and no code path could add one to a measure.buildTermCatalogindexed only names; descriptions went into the embed-onlytextfield and were never scored lexically.What changed
glossary.jsonbecomes the second curation layer, symmetric with the one that already exists:verdicts.jsonglossary.jsonPlus: column-level glossary entries resolve against measures as well as dimensions; dimensions and measures carry
synonymsand the term catalog indexes them; and the grounding context renders descriptions and inline annotations so enrichment is visible to the agent rather than only to the YAML file.The observable difference
Verified against the real engine, same dataset, with and without a glossary mapping "revenue" to a numeric column:
Verification
npm run checkexit 0npm run test:cli136 pass / 0 fail - all 131 existing tests untouched, which is the proof this is additivenpm run eval:engine18/18, unchangedglossary.jsoncuration end-to-end throughprepareDataset(including that an eval-styleoutDirreads no curation and the explicit override is the way back in)Stage 1 of 3 for the trap-dataset work; shipped separately so it can be judged on its own and so a later bisect can separate it from the dataset change.
🤖 Generated with Claude Code
https://claude.ai/code/session_01CKZ9UVgw2Mhiu6T5YAG7k6