Skip to content

Fix Flight LargeList schema encoding - #10298

Draft
yinli-systems wants to merge 3 commits into
apache:mainfrom
yinli-systems:kevin/fix-flight-large-list-schema-10291
Draft

Fix Flight LargeList schema encoding#10298
yinli-systems wants to merge 3 commits into
apache:mainfrom
yinli-systems:kevin/fix-flight-large-list-schema-10291

Conversation

@yinli-systems

@yinli-systems yinli-systems commented Jul 7, 2026

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

prepare_field_for_flight rebuilds DataType::LargeList with Field::new_list, silently changing the 64-bit offset type into a regular List in the Flight schema. The adjacent ListView / LargeListView path already preserves its large variant.

What changes are included in this PR?

  • Use Field::new_large_list for DataType::LargeList.
  • Exercise a real FlightDataEncoder to FlightDataDecoder IPC round trip covering List, nested LargeList<List>, FixedSizeList, parent and child nullability, and schema/field/child metadata.

Are these changes tested?

Yes, locally with Rust 1.96.1 on macOS:

  • Fault injection with the old new_list implementation: the round-trip regression test fails because decoded LargeList becomes List.
  • cargo test -p arrow-flight test_list_schema_round_trip
  • cargo test -p arrow-flight --lib (42 passed)
  • cargo test -p arrow-flight (unit, client, encode/decode, and doc tests passed)
  • cargo clippy -p arrow-flight --all-targets --all-features -- -D warnings
  • cargo fmt --all -- --check
  • git diff --check

I used AI assistance to inspect the nearby Flight schema-preparation paths and help generate the focused round-trip regression, then reviewed the resulting one-line production fix and validated it with the commands above.

Are there any user-facing changes?

Yes. Flight-encoded schemas now preserve LargeList instead of exposing it as List after schema preparation.

@github-actions github-actions Bot added arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate labels Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Schema for LargeList in Flight data encoder

1 participant