Skip to content

fix: a unique price column is not a foreign-key target - #22

Merged
kiyeonjeon21 merged 1 commit into
mainfrom
fix/phantom-fk-targets
Jul 25, 2026
Merged

fix: a unique price column is not a foreign-key target#22
kiyeonjeon21 merged 1 commit into
mainfrom
fix/phantom-fk-targets

Conversation

@kiyeonjeon21

Copy link
Copy Markdown
Owner

Why

Relationship inference treated any unique, non-null column as a valid foreign-key target. A small price list has unique prices, so every money column whose values happened to coincide got "explained" as a foreign key into it. Measured at 81% / 68% / 54% on a realistic fixture when the hard eval dataset was first built - which is how this was found.

The damage is worse than a wrong edge in the graph. Both endpoints of a relationship are excluded from the semantic model, so a phantom edge silently deletes the table's real measure:

before:  Sale:      measures = sales_count             <- sum_line_amt is gone
         PriceList: measures = price_list_count        <- sum_list_amt is gone
after:   Sale:      measures = sales_count, sum_line_amt
         PriceList: measures = price_list_count, sum_list_amt

Nothing in the output says why. On the hard dataset this removed sum_net_amt from the invoice table entirely.

What

A non-id target is credible only when the foreign column names it outright (nameSimilarity >= STRONG_NAME_SIMILARITY), which is what a genuine natural key looks like: sku -> sku, region_cd -> region_cd. Uniqueness alone no longer qualifies a column as a join target.

isIdLike moves to signals.ts as the single shared definition. The two halves of the engine had quietly disagreed: semantic-model.ts already carried the comment "a unique, non-null column like amount is a real measure, not a key" - and inference did not know that.

Verification

  • Isolated fixture (a 4-row price list whose prices coincide with a sales table's line amounts): the phantom edge disappears, the legitimate sku -> sku join survives at 100%, and both measures come back. A regression test pins exactly this.
  • npm run eval:engine 18/18 and npm run eval:engine:hard 25/25, both unchanged - so no real edge was lost, including the three natural-key joins the hard dataset depends on.
  • npm run check exit 0; npm run test:cli 142 pass / 0 fail

ROADMAP step 6.1. Found by building a harder eval dataset, which is the second engine defect that work has surfaced.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CKZ9UVgw2Mhiu6T5YAG7k6

Relationship inference treated any unique, non-null column as a valid join
target. A small price list has unique prices, so every money column whose values
coincided was explained as a foreign key into it - measured at 81/68/54% on a
realistic fixture when the hard eval dataset was first built.

The damage was worse than a wrong edge. Both endpoints of a relationship are
excluded from the semantic model, so a phantom edge silently deleted the table's
real measure: the invoice table stopped exposing sum_net_amt at all, with
nothing in the output to say why.

A non-id target is now credible only when the foreign column names it outright
(nameSimilarity >= STRONG_NAME_SIMILARITY), which is what a genuine natural key
looks like: sku -> sku, region_cd -> region_cd. Uniqueness alone no longer
qualifies a column as a target.

isIdLike moves to signals.ts as the single shared definition. The two halves of
the engine had disagreed about it - semantic-model.ts already carried the comment
'a unique, non-null column like amount is a real measure, not a key', and
inference did not know that.

Verified on an isolated fixture (a 4-row price list whose prices coincide with a
sales table's line amounts): the phantom edge disappears, the legitimate sku join
survives at 100%, and both sum_line_amt and sum_list_amt come back. Both engine
suites are unchanged (18/18 and 25/25), so no real edge was lost.
@kiyeonjeon21
kiyeonjeon21 merged commit d4b9ad3 into main Jul 25, 2026
1 check passed
@kiyeonjeon21
kiyeonjeon21 deleted the fix/phantom-fk-targets branch July 25, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant