Skip to content

Fix VectorScorer#bulk eagerly advancing its iterator - #16377

Merged
tteofili merged 1 commit into
apache:mainfrom
michaeljmarshall:improve-default-vector-bulk-scorer
Jul 20, 2026
Merged

Fix VectorScorer#bulk eagerly advancing its iterator#16377
tteofili merged 1 commit into
apache:mainfrom
michaeljmarshall:improve-default-vector-bulk-scorer

Conversation

@michaeljmarshall

@michaeljmarshall michaeljmarshall commented Jul 8, 2026

Copy link
Copy Markdown
Member

Description

The default bulk() implementation advanced its iterator as a side effect of construction, unlike Bulk.fromRandomScorerDense/Sparse, which defer positioning to the first nextDocsAndScores call. Move the positioning into the returned Bulk to match, and update the javadoc.

The current behavior is problematic when combining the iterator with others via conjunction. Given that this API is evolving and the behavior is split I propose adopting the lazy behavior.

In one case, I hit the following error:

  Caused by: java.lang.IllegalArgumentException: Sub-iterators of ConjunctionDISI are not on the same document!
        at ConjunctionDISI.throwSubIteratorsNotOnSameDocument(ConjunctionDISI.java:149)
        at ConjunctionDISI.createConjunction(ConjunctionDISI.java:104)
        at ConjunctionUtils.intersectScorers(ConjunctionUtils.java:44)
        at ConjunctionScorer.<init>(ConjunctionScorer.java:36)
        at BooleanScorerSupplier.requiredBulkScorer(BooleanScorerSupplier.java:453)
        at BooleanScorerSupplier.booleanScorer(BooleanScorerSupplier.java:219)
        at BooleanScorerSupplier.bulkScorer(BooleanScorerSupplier.java:177)

The tests confirm the bulk call is side effect free, which will satisfy the ConjunctionDISI requirements.

The default bulk() implementation advanced its iterator as a side
effect of construction, unlike Bulk.fromRandomScorerDense/Sparse,
which defer positioning to the first nextDocsAndScores call. Move
the positioning into the returned Bulk to match, and update the
javadoc.

The current behavior is problematic when combining the iterator
with others via conjunction. Given that this API is evolving
and the behavior is split I propose adopting the lazy behavior.

In one case, I hit the following error:

  Caused by: java.lang.IllegalArgumentException: Sub-iterators of ConjunctionDISI are not on the same document!
        at ConjunctionDISI.throwSubIteratorsNotOnSameDocument(ConjunctionDISI.java:149)
        at ConjunctionDISI.createConjunction(ConjunctionDISI.java:104)
        at ConjunctionUtils.intersectScorers(ConjunctionUtils.java:44)
        at ConjunctionScorer.<init>(ConjunctionScorer.java:36)
        at BooleanScorerSupplier.requiredBulkScorer(BooleanScorerSupplier.java:453)
        at BooleanScorerSupplier.booleanScorer(BooleanScorerSupplier.java:219)
        at BooleanScorerSupplier.bulkScorer(BooleanScorerSupplier.java:177)

The tests don't hit this error because it's trivial to confirm
that the bulk call is side effect free.
michaeljmarshall added a commit to carlosdelest/elasticsearch that referenced this pull request Jul 8, 2026
Build the bulk scorer lazily; building it eagerly broke conjunctions
with sibling filter clauses.

Test added in this commit failed without the change.

See apache/lucene#16377.

@tteofili tteofili left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tteofili
tteofili merged commit 9d09aa7 into apache:main Jul 20, 2026
12 checks passed
@michaeljmarshall
michaeljmarshall deleted the improve-default-vector-bulk-scorer branch July 20, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants