Skip to content

Ensure S3PartitionConsumer throws OffsetOutOfRangeException if position is earlier than earliest s3 offset - #45

Merged
jeffxiang merged 6 commits into
mainfrom
offset_reset_bug_fix
Dec 8, 2025
Merged

Ensure S3PartitionConsumer throws OffsetOutOfRangeException if position is earlier than earliest s3 offset#45
jeffxiang merged 6 commits into
mainfrom
offset_reset_bug_fix

Conversation

@jeffxiang

@jeffxiang jeffxiang commented Nov 25, 2025

Copy link
Copy Markdown
Contributor

There is a bug in S3PartitionConsumer where if the desired fetch offset is smaller than the earliest available offset on S3, it will automatically reset position to earliest on s3. This is due to the fact that we perform a binary search on the S3 log segments to locate the first S3 segment which might contain the desired offset, meaning that the found segment when desiredOffset < earliest is just the earliest log segment. The effect of this is that S3PartitionConsumer does not honor the user-supplied auto.offset.reset config, especially if the user has supplied latest. This undesirable behavior happens when the TieredStorageConsumer is constructed with auto.offset.reset=latest and group.id is set to a group where the committed offsets exist, but are expired.

In the case that the group.id is entirely new and there does not exist any previously-committed offsets for that group, this PR did not change any behavior. This is because AssignmentAwareConsumerRebalanceListener resets consumer positions according to auto.offset.reset config proactively when committed group offsets do not exist. However, if they do exist, the RebalanceListener will set fetch positions to the committed offsets even if they are expired. When these fetch positions are passed down to S3PartitionConsumer, the current behavior before this PR is that the S3PartitionConsumer will skip the "lost" offsets and reset to earliest, no matter what the auto.offset.reset config is.

This PR addresses the issue by throwing an OffsetOutOfRangeException when the above scenario is detected, and bubbling up the exception to TieredStorageConsumer where the offset reset logic already exists. TieredStorageConsumer is then responsible for handling the exception according to auto.offset.reset.

This PR also includes additional refactoring, such as deduplicating some S3 poll logic code in TieredStorageConsumer.

@jeffxiang
jeffxiang marked this pull request as ready for review November 26, 2025 21:04
@jeffxiang
jeffxiang requested a review from a team as a code owner November 26, 2025 21:04
@jeffxiang
jeffxiang merged commit 8116003 into main Dec 8, 2025
1 check passed
@jeffxiang
jeffxiang deleted the offset_reset_bug_fix branch December 8, 2025 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants