bound vector deserialization total allocation in persistence - #134
Merged
Conversation
add validate_vector_total(dim, count) check after individual dim/count validation in snapshot deserialization. without this, a crafted snapshot with 65536 dims x 10M vectors would attempt ~2.6 TB of allocation. the new cap limits total f32 elements to 1 billion (~4 GB).
kacy
added a commit
that referenced
this pull request
Feb 19, 2026
add validate_vector_total(dim, count) check after individual dim/count validation in snapshot deserialization. without this, a crafted snapshot with 65536 dims x 10M vectors would attempt ~2.6 TB of allocation. the new cap limits total f32 elements to 1 billion (~4 GB).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
summary
addresses security audit finding #14: vector deserialization unbounded memory.
snapshot deserialization validates
dim <= 65536andcount <= 10Mindividually, but not the product. a crafted snapshot with 65536 dims x 10M vectors would attempt ~2.6 TB of f32 allocation.this adds
validate_vector_total(dim, count)informat.rsthat caps total f32 elements at 1 billion (~4 GB), called at both snapshot deserialization sites (encrypted and plaintext paths).AOF VADD records store single vectors (not vector sets), so dim * count is not relevant there — no change needed.
what was tested
cargo test -p ember-persistence --features vector— 75/75 passcargo clippy -p ember-persistence --features vector -- -D warnings— cleancargo fmt --all --check— cleandesign considerations
the 1 billion total float budget (~4 GB) is generous enough for real workloads (e.g. 1024 dims x 1M vectors = 1B floats) while preventing the worst-case product of the two individual limits.