Skip to content

search: R2 salience ranking boost needs a config gate and an outcome gate (or removal) #692

Description

@itlackey

Problem

The R2 boost (asset_salience.rank_score → search/curate ranking, ×[1.0–1.2]) shipped default-on in 0.9.0-beta.53 with no config opt-out — only the internal salienceRankScores: null API param can disable it. On live data it is both weak and biased:

  • asset_salience has zero pack refs (%//% = 0 rows), while externally-installed packs dominate actual consumption (top shows AND curates). The boost can only ever favor self-generated personal assets — worst case ~+7% for an unread knowledge note over an equally-relevant pack asset.
  • For 5,560/5,601 non-content-scored rows the encoding term is a per-type constant, so the boost mostly reduces to "is a personal skill/agent".
  • Its dominant input (retrieval, w_r=0.60) counts search impressions as retrievals with no source filter — a second impression→rank loop on top of the utility EMA.
  • The utility EMA contributor (weight 0.5) already delivers usage reinforcement from the same usage_events, with full pack coverage — R2 double-counts it for personal assets only.
  • Live effect is ~noise: max rank_score 0.356 → max multiplier ×1.071, mean ×1.016.

Fix options (in preference order)

  1. Remove the salience contributor from user-facing search ranking (keep rank_score for improve-internal selection) until a measured curate-golden-bench delta justifies it.
  2. If kept: add a config gate (default off), and gate the boost on retrieval-backed outcome signal (asset_outcome.outcome_score > 0 — column exists, ~541 assets qualify) so it stops double-counting impressions and favoring the unread corpus.

Either way, a config opt-out should exist for a term that touches every query on the hottest read path.

🤖 Generated with Claude Code

https://claude.ai/code/session_018YjU1mGiShdDP4qFW7p4mv

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions