Feature: Don't early-exit filtered search before `wanted` results are found

[Jeadie]

Describe what you are looking for

When using filtered_search with a predicate, the search often returns fewer than wanted results even when plenty of passing nodes exist in the graph.

I think the issue is in search_to_find_in_base_:

if ((-candidate.distance) > radius)
    break;

radius comes from top.top().distance, but only predicate-passing nodes enter top. So with a selective predicate, radius collapses to the nearest passing node and the traversal exits almost immediately — before finding wanted results.

Without a predicate every node enters top, so radius tracks the true exploration frontier and this works fine. With a predicate they diverge.

This would keep traversing through non-passing nodes until enough passing results are found (or the graph is exhausted), while preserving the existing early-exit when no predicate is used.

Can you contribute to the implementation?

• I can contribute

Is your feature request specific to a certain interface?

C++ implementation

Contact Details

jack@spice.ai

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

[Jeadie]

Motivation for this is adding pre-filtering to duckdb-vss, see commit.

[Jeadie]

duckdb-vss is running 2.12.0. Looks like these previous commits address it (for separate reasons):

• Early break without enough results (ffa5986)
• radius updated for non-passing nodes (369a553)

Will update duckdb-vss and confirm before closing issue.

[ashvardanian]

Feel free to open a PR if there are more patches needed, @Jeadie 🤗