Optimize REST calls for single-page virtual directory listings#2295
Open
syeleti-msft wants to merge 1 commit into
Open
Optimize REST calls for single-page virtual directory listings#2295syeleti-msft wants to merge 1 commit into
syeleti-msft wants to merge 1 commit into
Conversation
syeleti-msft
requested review from
anmolanmol1234,
jainakanksha-msft and
souravgupta-msft
as code owners
July 24, 2026 06:12
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes Block Blob virtual directory listings by eliminating per-directory REST GetProperties marker checks when a hierarchical listing is confirmed to be single-page (no incoming marker and no outgoing continuation marker). This reduces REST call volume for directories containing many markerless virtual directories while preserving the existing paginated-listing correctness logic.
Changes:
- Detect single-page hierarchical listings via a new
isSinglePageListing(marker, nextMarker)helper and pass the result intoprocessBlobPrefixes. - For single-page listings, create directory attributes directly from
BlobPrefixentries instead of performing per-prefix marker lookups. - Add unit tests for
isSinglePageListingand for skipping marker lookups on single-page listings; update the changelog.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| component/azstorage/block_blob.go | Adds single-page detection and uses it to skip per-prefix marker REST checks when safe. |
| component/azstorage/block_blob_test.go | Adds unit tests for the single-page logic and for skipping marker lookups. |
| CHANGELOG.md | Notes the REST call reduction optimization in the unreleased changelog. |
Comment on lines
827
to
828
| // marker file not found in current iteration, so we need to manually check attributes via REST | ||
| _, err := bb.getAttrUsingRest(*blobInfo.Name) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Optimize Block Blob directory listings by avoiding per-directory marker lookups when the hierarchical listing completes in a single page.
BlobPrefixentries.dirListmarker-deduplication logic.Motivation
Previously, listing a directory containing multiple markerless virtual directories triggered one
GetPropertiesrequest for everyBlobPrefix.For example, 200 markerless virtual directories could require: