Improve take on Chunked(FSL) and FSL(Chunked) array alongside implementation of take using PiecewiseSequence by robert3005 · Pull Request #8881 · vortex-data/vortex

robert3005 · 2026-07-21T16:15:05Z

Take on Chunked(FSL) and FSL(Chunked) can be made faster for primitive type
elements

github-actions · 2026-07-21T16:27:14Z

Polar Signals Profiling Results

Latest Run

Status	Commit	Job	Attempt	Link
🟢 Done	`f81e287`		1	Explore Profiling Data

Previous Runs (2)

Status	Commit	Job	Attempt	Link
🟢 Done	`682ffe9`		1	Explore Profiling Data
🟢 Done	`0112c06`		1	Explore Profiling Data

Powered by Polar Signals Cloud

github-actions · 2026-07-21T16:28:08Z

Benchmarks: Vortex queries 📖

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +2.1%
Engines: DataFusion No clear signal (+5.7%, low confidence) · DuckDB No clear signal (-1.4%, low confidence)
Vortex (geomean): 0.984x ➖
Parquet (geomean): 0.974x ➖
Shifts: Parquet (control) -2.6% · Median polish -0.9%

How to read Verdict and Engines

Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.004x ➖, 0↑ 0↓)

name	PR `f81e287` (ns)	base `69f5bc5` (ns)	ratio (PR/base)
vortex_q00/datafusion:vortex-file-compressed	9528268	9640865	0.99
vortex_q01/datafusion:vortex-file-compressed	6204717	6087569	1.02

datafusion / parquet (0.950x ➖, 0↑ 0↓)

name	PR `f81e287` (ns)	base `69f5bc5` (ns)	ratio (PR/base)
vortex_q00/datafusion:parquet	19531086	20804028	0.94
vortex_q01/datafusion:parquet	4458812	4638739	0.96

duckdb / vortex-file-compressed (0.984x ➖, 0↑ 0↓)

name	PR `f81e287` (ns)	base `69f5bc5` (ns)	ratio (PR/base)
vortex_q00/duckdb:vortex-file-compressed	9764105	9980322	0.98
vortex_q01/duckdb:vortex-file-compressed	5839266	5899636	0.99

duckdb / parquet (0.998x ➖, 0↑ 0↓)

name	PR `f81e287` (ns)	base `69f5bc5` (ns)	ratio (PR/base)
vortex_q00/duckdb:parquet	23159102	23225257	1.00
vortex_q01/duckdb:parquet	9354709	9364034	1.00

No file size changes detected.

codspeed-hq · 2026-07-23T14:09:02Z

Merging this PR will improve performance by ×2.1

⚡ 13 improved benchmarks
✅ 1828 untouched benchmarks
🆕 6 new benchmarks
⏩ 55 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	Simulation	`take_chunked_fsl_random[32, 64]`	517.1 µs	166.3 µs	×3.1
⚡	Simulation	`take_chunked_fsl_sorted[32, 64]`	515.2 µs	170.5 µs	×3
⚡	Simulation	`take_fsl_chunked_elements_random[32, 64]`	428.2 µs	152.4 µs	×2.8
⚡	Simulation	`take_fsl_chunked_elements_sorted[32, 64]`	434.5 µs	158.4 µs	×2.7
⚡	Simulation	`take_chunked_fsl_random[16, 64]`	355.6 µs	152.7 µs	×2.3
⚡	Simulation	`take_chunked_fsl_sorted[16, 64]`	350.1 µs	161.7 µs	×2.2
⚡	Simulation	`take_fsl_chunked_elements_random[16, 64]`	283.7 µs	137.3 µs	×2.1
⚡	Simulation	`fsl_large`	89.2 µs	44.5 µs	×2
⚡	Simulation	`take_chunked_fsl_random[8, 64]`	279.6 µs	144 µs	+94.11%
⚡	Simulation	`take_fsl_chunked_elements_sorted[16, 64]`	270.7 µs	147.5 µs	+83.61%
⚡	Simulation	`take_chunked_fsl_sorted[8, 64]`	271 µs	153.6 µs	+76.5%
⚡	Simulation	`take_fsl_chunked_elements_random[8, 64]`	198.4 µs	129.4 µs	+53.33%
⚡	Simulation	`take_fsl_chunked_elements_sorted[8, 64]`	189.5 µs	140.1 µs	+35.28%
🆕	Simulation	`take_chunked_compressed_fsl[16, 64]`	N/A	226.6 µs	N/A
🆕	Simulation	`take_chunked_compressed_fsl[32, 64]`	N/A	242.1 µs	N/A
🆕	Simulation	`take_chunked_compressed_fsl[8, 64]`	N/A	220.2 µs	N/A
🆕	Simulation	`take_chunked_compressed_fsl_piecewise[16, 64]`	N/A	150.4 µs	N/A
🆕	Simulation	`take_chunked_compressed_fsl_piecewise[32, 64]`	N/A	161.3 µs	N/A
🆕	Simulation	`take_chunked_compressed_fsl_piecewise[8, 64]`	N/A	142.4 µs	N/A

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing rk/chunkedfsltake (11b7ae6) with develop (f3d6795)}

55 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Depends on #8881. Use the existing chunked FixedSizeList canonicalization path before applying the specialized take. This removes the physical FSL encoding requirement and lets logically FSL-typed chunks use the optimization. --------- Signed-off-by: Dimitar Dimitrov <dimitar@spiraldb.com> Signed-off-by: Robert Kruszewski <github@robertk.io> Co-authored-by: Robert Kruszewski <github@robertk.io> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

dimitarvdimitrov · 2026-07-28T15:45:59Z

rebasing this on top of develop

Depends on #8881. Use the existing chunked FixedSizeList canonicalization path before applying the specialized take. This removes the physical FSL encoding requirement and lets logically FSL-typed chunks use the optimization. --------- Signed-off-by: Dimitar Dimitrov <dimitar@spiraldb.com> Signed-off-by: Robert Kruszewski <github@robertk.io> Co-authored-by: Robert Kruszewski <github@robertk.io> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

robert3005 · 2026-07-28T23:05:40Z

I have undone your change @dimitarvdimitrov since I was wrong about the flow here. I think for what I wanted we would need #8964

alongside implementation of take using PiecewiseSequence Signed-off-by: Robert Kruszewski <github@robertk.io>

Signed-off-by: Robert Kruszewski <github@robertk.io>

Signed-off-by: Dimitar Dimitrov <dimitar@spiraldb.com>

Depends on #8881. Use the existing chunked FixedSizeList canonicalization path before applying the specialized take. This removes the physical FSL encoding requirement and lets logically FSL-typed chunks use the optimization. --------- Signed-off-by: Dimitar Dimitrov <dimitar@spiraldb.com> Signed-off-by: Robert Kruszewski <github@robertk.io> Co-authored-by: Robert Kruszewski <github@robertk.io> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Signed-off-by: Robert Kruszewski <github@robertk.io>

robert3005 added the changelog/performance A performance improvement label Jul 21, 2026

robert3005 force-pushed the rk/chunkedfsltake branch from 682ffe9 to f81e287 Compare July 21, 2026 21:11

dimitarvdimitrov force-pushed the rk/chunkedfsltake branch from f81e287 to d72c4e8 Compare July 23, 2026 14:03

dimitarvdimitrov mentioned this pull request Jul 27, 2026

Execute chunked FSL before take #9012

Merged

dimitarvdimitrov force-pushed the rk/chunkedfsltake branch from d85fca4 to b7b10d1 Compare July 28, 2026 15:53

robert3005 force-pushed the rk/chunkedfsltake branch from 140e5f4 to ff8c308 Compare July 28, 2026 23:04

robert3005 marked this pull request as ready for review July 28, 2026 23:04

robert3005 commented Jul 29, 2026

View reviewed changes

Comment thread vortex-array/src/arrays/chunked/compute/take.rs Outdated

robert3005 and others added 5 commits July 29, 2026 15:00

Improve take on Chunked(FSL) and FSL(Chunked) array

711d736

alongside implementation of take using PiecewiseSequence Signed-off-by: Robert Kruszewski <github@robertk.io>

simpler

0b4bf83

Signed-off-by: Robert Kruszewski <github@robertk.io>

bench: add compressed FSL take coverage

031b39f

Signed-off-by: Dimitar Dimitrov <dimitar@spiraldb.com>

less

f375f7f

Signed-off-by: Robert Kruszewski <github@robertk.io>

robert3005 force-pushed the rk/chunkedfsltake branch from ff8c308 to f375f7f Compare July 29, 2026 14:00

more

11b7ae6

Signed-off-by: Robert Kruszewski <github@robertk.io>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve take on Chunked(FSL) and FSL(Chunked) array alongside implementation of take using PiecewiseSequence - #8881

Improve take on Chunked(FSL) and FSL(Chunked) array alongside implementation of take using PiecewiseSequence#8881
robert3005 wants to merge 6 commits into
developfrom
rk/chunkedfsltake

robert3005 commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Jul 23, 2026 •

edited

Loading

Uh oh!

dimitarvdimitrov commented Jul 28, 2026

Uh oh!

robert3005 commented Jul 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

robert3005 commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Polar Signals Profiling Results

Latest Run

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks: Vortex queries 📖

Uh oh!

codspeed-hq Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by ×2.1

Performance Changes

Footnotes

Uh oh!

dimitarvdimitrov commented Jul 28, 2026

Uh oh!

robert3005 commented Jul 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

codspeed-hq Bot commented Jul 23, 2026 •

edited

Loading