Add fixed-size binary support - #8865
Conversation
Merging this PR will not alter performance
|
dde9039 to
e13e63c
Compare
e13e63c to
5dd799a
Compare
Polar Signals Profiling ResultsLatest Run
Previous Runs (9)
Powered by Polar Signals Cloud |
Benchmarks: Vortex queries 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.085x ➖, 0↑ 1↓)
datafusion / parquet (1.023x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.989x ➖, 0↑ 0↓)
duckdb / parquet (1.009x ➖, 0↑ 0↓)
File Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
Totals:
|
5dd799a to
8dedc68
Compare
8dedc68 to
8ef7bb2
Compare
8ef7bb2 to
8dedc68
Compare
8dedc68 to
0592a4f
Compare
0592a4f to
97c8e02
Compare
97c8e02 to
c924a06
Compare
c924a06 to
d36c772
Compare
d36c772 to
11a4e4f
Compare
460d6dc to
14e1fb1
Compare
758c3e0 to
5b1b039
Compare
| } | ||
| DType::Utf8(_) => DataType::Utf8View, | ||
| DType::Binary(_) => DataType::BinaryView, | ||
| DType::FixedSizeBinary(byte_width, _) => DataType::FixedSizeBinary( |
5b1b039 to
1767981
Compare
39ea0b9 to
c645586
Compare
c645586 to
53a4c9d
Compare
53a4c9d to
cc89366
Compare
3fd82c6 to
e364d90
Compare
7dbc50b to
12792b7
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
12792b7 to
3935ac9
Compare
Stacked on #8864.
Why
FixedSizeBinary has the same one-buffer fixed-width structure, so it should inherit the shared structural compute rather than introduce a third take/filter implementation. It remains a distinct logical type and stores an explicit row count because zero-width values cannot derive length from the values buffer.
What
FixedWidthArrayadapter used by Primitive and Decimal; FSB has no local take or filter pipelineIssue provenance
ArrayPartsvalidation, zero-width Arrow handling, and full-source cloning for uncommon-width filters.Validation
cargo test -p vortex-array --lib(3,064 passed, 1 ignored)cargo clippy -p vortex-array --all-targets --all-featurescargo test -p vortex-arrow fixed_size_binarycargo test -p vortex-file test_round_trip_many_typescargo test --no-run --target x86_64-apple-darwin -p vortex-array --libgit diff --check