Skip to content

add test to catch mis-aligned ipc buffers - #10405

Merged
Jefffrey merged 3 commits into
apache:mainfrom
Rich-T-kid:rich-T-kid/add-alignment-test
Jul 23, 2026
Merged

add test to catch mis-aligned ipc buffers#10405
Jefffrey merged 3 commits into
apache:mainfrom
Rich-T-kid:rich-T-kid/add-alignment-test

Conversation

@Rich-T-kid

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

see #10394

What changes are included in this PR?

Adds a single test test_misaligned_buffers_error() in arrow-ipc/src/reader.rs that verifies misaligned IPC buffers are caught by require_alignment = true. The test loops over byte offsets 0..128 across both fixed-width (Int32, Int64) and variable-width (StringArray, LargeStringArray) array types, asserting that any offset not satisfying the type's alignment requirement returns a "Misaligned buffers" error, and that aligned offsets decode successfulldy.

Are these changes tested?

yes

type=LargeUtf8 offset=105: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 1) (expected Err)
type=LargeUtf8 offset=106: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 2) (expected Err)
type=LargeUtf8 offset=107: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 3) (expected Err)
type=LargeUtf8 offset=108: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 4) (expected Err)
type=LargeUtf8 offset=109: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 3) (expected Err)
type=LargeUtf8 offset=110: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 2) (expected Err)
type=LargeUtf8 offset=111: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 1) (expected Err)
type=LargeUtf8 offset=112: Ok (expected Ok)
type=LargeUtf8 offset=113: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 1) (expected Err)
type=LargeUtf8 offset=114: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 2) (expected Err)
type=LargeUtf8 offset=115: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 3) (expected Err)
type=LargeUtf8 offset=116: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 4) (expected Err)
type=LargeUtf8 offset=117: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 3) (expected Err)
type=LargeUtf8 offset=118: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 2) (expected Err)
type=LargeUtf8 offset=119: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 1) (expected Err)
type=LargeUtf8 offset=120: Ok (expected Ok)
type=LargeUtf8 offset=121: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 1) (expected Err)
type=LargeUtf8 offset=122: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 2) (expected Err)
type=LargeUtf8 offset=123: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 3) (expected Err)
type=LargeUtf8 offset=124: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 4) (expected Err)
type=LargeUtf8 offset=125: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 3) (expected Err)
type=LargeUtf8 offset=126: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 2) (expected Err)
type=LargeUtf8 offset=127: Err(Invalid argument error: Misaligned buffers[0] in array of type LargeUtf8, offset from expected alignment of 8 by 1) (expected Err)

Are there any user-facing changes?

no

@github-actions github-actions Bot added the arrow Changes to the arrow crate label Jul 21, 2026
@Rich-T-kid

Copy link
Copy Markdown
Contributor Author

🤔, it may also be worth to add a similar test at the arrow-flight level for FlightDataDecoder

Comment thread arrow-ipc/src/reader.rs Outdated
// Fixed-width: alignment == element byte width.
// Variable-width (e.g. StringArray): the offsets buffer drives alignment (Int32 → 4).
let cases: Vec<(ArrayRef, usize)> = vec![
(Arc::new(Int32Array::from_iter(0i32..20_000)) as _, 4),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we strictly need arrays this large? could reduce down to 1000 elements? (or even 500)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope, an array of size 2 would be testing the same thing.

@Jefffrey
Jefffrey merged commit 1233a18 into apache:main Jul 23, 2026
28 of 30 checks passed
@Jefffrey

Copy link
Copy Markdown
Contributor

thanks @Rich-T-kid

@alamb

alamb commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Epic -- thank you both

@Jefffrey Jefffrey added the development-process Related to development process of arrow-rs label Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate development-process Related to development process of arrow-rs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create more test to catch mis-aligned buffers in arrow-ipc

3 participants