feat: v5 vs v7 benchmark - #112
Open
woutervanranst wants to merge 25 commits into
Open
Conversation
Azure blob name prefixes for chunk-index shards are not segment-aligned (chunk-index/aa must match aa, aa0, aa3f). The existing ListAsync appends a trailing slash, so add ListAsync(directory, namePrefix) with a default client-side-filter implementation, a native Azure override, a recording override on the in-memory fake, and a ChunkIndexLists counter on the counting decorator. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Shards start at the 2-char prefix layout and split 16-way by the next hex char when they exceed MaxShardEntryCount (10k) at flush time, recursively and unevenly per subtree. Only non-empty shards are written; the layout is self-describing from blob existence. - Routing: parent-wins walk — the shallowest existing shard on a hash's prefix path is authoritative; a crashed split leaves the parent intact and unpublished children shadowed, so reads stay correct at every crash point and a flush retry re-splits and converges. - Lookup: per-root coverage claims in loaded_prefixes (validated at the latest snapshot, zero remote calls when warm); cold path = one subtree listing per root deciding empty-vs-descend, etag revalidation from the listed etag (no download). - Flush: per-root gates; split uploads all non-empty leaves before deleting the parent and stale blobs in its range. - Repair: recomputes a balanced layout from staged entries (splits and coarsens); stale-shard deletion unchanged. - Local store schema v2: entries keyed by content_hash BLOB ranges (no routed prefix column), recreate-on-version-mismatch, non-overlapping coverage claims. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lance Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…load Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ShardSerializer.Deserialize and SerializeAsync now require an ICompressionService argument (added by the zstd branch). ChunkIndexService and SnapshotService constructors also gained a compression parameter. Tests added by the dynamic-shard-prefix branch were written against the pre-zstd signatures; this commit threads TestCompression.Instance through all the affected call sites. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
|
Warning Review limit reached
More reviews will be available in 7 minutes and 13 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (48)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #112 +/- ##
==========================================
+ Coverage 80.77% 81.00% +0.23%
==========================================
Files 99 99
Lines 6340 6534 +194
Branches 852 887 +35
==========================================
+ Hits 5121 5293 +172
- Misses 986 1003 +17
- Partials 233 238 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.