Skip to content

add bulk intoBitSet for SortedDocValues ordinal ranges - #16180

Merged
jimczi merged 3 commits into
apache:mainfrom
iprithv:ordinal-into-bitset
Jul 29, 2026
Merged

add bulk intoBitSet for SortedDocValues ordinal ranges#16180
jimczi merged 3 commits into
apache:mainfrom
iprithv:ordinal-into-bitset

Conversation

@iprithv

@iprithv iprithv commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

extends the unified two-phase strategy from #16177 to SortedDocValues. Ordinal range queries now bulk-evaluate through intoBitSet instead of confirming matches() one doc at a time.

@jimczi

jimczi commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

I wonder if this is the right tradeoff for ANN-backed indices in general. Losing the ANN structure for the whole segment after a small vector update and falling back to exact search until merge feels fairly expensive and potentially surprising.

Maybe this only really makes sense for flat indices where there is no ANN structure to invalidate?

The use case that feels more important to me is actually the opposite one: updating other fields/doc values without having to rewrite vectors or rebuild the ANN structure.

I saw the “change of model” use case mentioned in the PR, but in that situation you typically need to reindex everything anyway, so I’m not sure this optimization helps much there.

Wrong PR, sorry for the noise

@iverase

iverase commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

@jimczi I think your comment is referring to this other PR?

@jimczi

jimczi commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Oups thanks @iverase and sorry for the noise @iprithv , I was reviewing this change at the same time

@github-actions

Copy link
Copy Markdown
Contributor

This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!

@github-actions github-actions Bot added the Stale label Jun 24, 2026
@romseygeek romseygeek modified the milestones: 10.5.0, 10.6.0 Jun 26, 2026
@github-actions github-actions Bot removed the Stale label Jun 27, 2026
@iprithv
iprithv force-pushed the ordinal-into-bitset branch from 0dfb31c to 2523c45 Compare July 3, 2026 22:55
@iprithv

iprithv commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

moved entry of changes.txt under 10.6

@iprithv
iprithv force-pushed the ordinal-into-bitset branch from d9c8117 to bc72389 Compare July 4, 2026 08:13
@iprithv
iprithv force-pushed the ordinal-into-bitset branch from bc72389 to f5a83a8 Compare July 8, 2026 15:14
@github-actions

Copy link
Copy Markdown
Contributor

This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!

@github-actions github-actions Bot added the Stale label Jul 23, 2026
Comment thread lucene/core/src/java/org/apache/lucene/index/SortedDocValues.java Outdated
@iprithv
iprithv requested a review from jimczi July 27, 2026 18:46
@iprithv
iprithv force-pushed the ordinal-into-bitset branch 2 times, most recently from a442d4a to f3e8b21 Compare July 28, 2026 03:39
@github-actions github-actions Bot removed the Stale label Jul 29, 2026
@jimczi
jimczi merged commit 04d33b4 into apache:main Jul 29, 2026
23 checks passed
jimczi pushed a commit that referenced this pull request Jul 29, 2026
add bulk intoBitSet for SortedDocValues ordinal ranges

(cherry picked from commit 04d33b4)
@jimczi

jimczi commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Merged and backported to 10x, thanks @iprithv !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants