CBG-5225: sg-bucket KV range scan implementations#8084
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request adds KV (Key-Value) range scan functionality to Sync Gateway through the sg-bucket abstraction layer, enabling both Couchbase Server (via gocb) and Rosmar to perform efficient range scans over document keys.
Changes:
- Adds
Scan()method implementation forCollectionthat bridges sg-bucket'sRangeScanStoreinterface to gocb's range scan API - Adds
AsRangeScanStore()helper function following the existing pattern for feature-checking datastore capabilities - Implements
RangeScanStoreinterface inLeakyDataStoreto support testing with leaky bucket
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| base/collection_rangescan.go | New file implementing the Scan() method for Collection, converting sg-bucket scan types to gocb scan types, and providing an iterator wrapper for scan results |
| base/collection_rangescan_test.go | Comprehensive test coverage for range scan functionality including full range, partial range, IDs-only, prefix scans, empty ranges, and tombstone exclusion |
| base/leaky_datastore.go | Implements RangeScanStore.Scan() by delegating to underlying datastore, adds interface assertion for RangeScanStore |
| base/collection.go | Adds feature flag BucketStoreFeatureRangeScan with version check (7.6+) |
| base/bucket.go | Adds AsRangeScanStore() helper function to check if a datastore supports range scan operations |
bbrks
force-pushed
the
rosmar_kv_range_scan
branch
2 times, most recently
from
ea06bac to
07ed2f8
Compare
torcolvin
reviewed
Mar 6, 2026
|
|
||
| allDocIDs := []string{"doc_a", "doc_b", "doc_c", "doc_d", "doc_e"} | ||
|
|
||
| // CBS range scan may not immediately reflect recent writes (requires persistence). |
Collaborator
There was a problem hiding this comment.
Is this true? This seems perilous if so. Does this depend of the type of storage used by the backing bucket?
Member
Author
There was a problem hiding this comment.
there's no simple scan consistency setting available to use to guard against this. We could pass in vb snapshots to avoid - but I don't think it's worth it given we're not expecting to one-shot the range scan to migrate all data in one go
it will be an iterative process until no eligible data is remaining to move (i.e. guard against any writes that may be coming in underneath a running range scan)
bbrks
force-pushed
the
rosmar_kv_range_scan
branch
from
69ae861 to
639b7a6
Compare
Wrap gocb.Collection.Scan() in the SG collection adapter, converting sgbucket types to gocb types with RawJSONTranscoder. Add AsRangeScanStore helper, LeakyDataStore passthrough, and IsSupported for CBS 7.5+. Includes dual-backend test covering full range, partial range, IDsOnly, sampling, empty range, prefix, and tombstone exclusion.
bbrks
force-pushed
the
rosmar_kv_range_scan
branch
from
18021a8 to
089bdba
Compare
…d around with an explicit maximum term in sgbucket.NewRangeScanForPrefix
gregns1
previously approved these changes
May 19, 2026
gregns1
left a comment
gregns1
left a comment
Contributor
There was a problem hiding this comment.
LGTM, will let you update dependencies where and will re-approve
bbrks
changed the title
bbrks
force-pushed
the
rosmar_kv_range_scan
branch
from
6694dab to
54032d9
Compare
gregns1
approved these changes
May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note: This was coded primarily using Opus 4.6 - with manual guidance/verification/review.
Adds a test that can do a KV range scan on both Couchbase via Gocb and Rosmar via an sg-bucket interface.
Dependencies
Integration Tests