BUGV2-5495: Fix RowConverter panic with DictionaryArray in Struct/List - #77
Merged
Conversation
Cherry-picked from upstream arrow-rs 56.1.0 (PR apache#7627, issue apache#7165). Drop this commit when upgrading arrow-rs to >= 56.1.0. Original: Fix RowConverter panic when encoding DictionaryArrays in StructArray / ListArray. The RowConverter flattens Dictionary types during encoding but didn't account for this when decoding, causing List<Dictionary<K,V>> and Struct containing Dictionary fields to panic in GenericListArray::from() due to child type mismatch.
There was a problem hiding this comment.
Pull request overview
Fixes a panic in arrow-row row decoding when RowConverter encounters nested DictionaryArray values inside StructArray and ListArray by correcting the decoded nested schema to match the flattened child array types.
Changes:
- Update list decoding to build
ArrayDatausing a corrected(Large)ListelementDataTypederived from the decoded child array - Update struct decoding to build
ArrayDatausing correctedStructfieldDataTypes derived from decoded child arrays - Add regression tests covering Dictionary-in-Struct and List-of-Dictionary roundtrips (including empty struct)
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| arrow-row/src/list.rs | Corrects (Large)List decoded DataType to match flattened child array types |
| arrow-row/src/lib.rs | Corrects Struct decoded field types to match flattened child array types; adds regression tests |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
All 0.11.0-0.11.5 versions have been yanked from crates.io.
avantgardnerio
approved these changes
Mar 27, 2026
avantgardnerio
left a comment
avantgardnerio
left a comment
There was a problem hiding this comment.
I trust a cherry pick if the conflicts were minimal and the tests pass.
Just a reminder to set the PR message right before merge to include those upstream IDs
dispanser
added a commit
that referenced
this pull request
Mar 30, 2026
#77) * BUGV2-5495: Fix RowConverter panic with DictionaryArray in Struct/List Cherry-picked from upstream arrow-rs 56.1.0 (PR apache#7627, issue apache#7165). Drop this commit when upgrading arrow-rs to >= 56.1.0. Original: Fix RowConverter panic when encoding DictionaryArrays in StructArray / ListArray. The RowConverter flattens Dictionary types during encoding but didn't account for this when decoding, causing List<Dictionary<K,V>> and Struct containing Dictionary fields to panic in GenericListArray::from() due to child type mismatch. * Bump lz4_flex upper bound to 0.11.6 All 0.11.0-0.11.5 versions have been yanked from crates.io.
dispanser
added a commit
that referenced
this pull request
Mar 30, 2026
#77) (#78) * BUGV2-5495: Fix RowConverter panic with DictionaryArray in Struct/List Cherry-picked from upstream arrow-rs 56.1.0 (PR apache#7627, issue apache#7165). Drop this commit when upgrading arrow-rs to >= 56.1.0. Original: Fix RowConverter panic when encoding DictionaryArrays in StructArray / ListArray. The RowConverter flattens Dictionary types during encoding but didn't account for this when decoding, causing List<Dictionary<K,V>> and Struct containing Dictionary fields to panic in GenericListArray::from() due to child type mismatch.
avantgardnerio
pushed a commit
that referenced
this pull request
Apr 2, 2026
#77) (#78) * BUGV2-5495: Fix RowConverter panic with DictionaryArray in Struct/List Cherry-picked from upstream arrow-rs 56.1.0 (PR apache#7627, issue apache#7165). Drop this commit when upgrading arrow-rs to >= 56.1.0. Original: Fix RowConverter panic when encoding DictionaryArrays in StructArray / ListArray. The RowConverter flattens Dictionary types during encoding but didn't account for this when decoding, causing List<Dictionary<K,V>> and Struct containing Dictionary fields to panic in GenericListArray::from() due to child type mismatch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-picked from upstream arrow-rs 56.1.0 (PR apache#7627, issue apache#7165). Drop this commit when upgrading arrow-rs to >= 56.1.0.
Original: Fix RowConverter panic when encoding DictionaryArrays in StructArray / ListArray. The RowConverter flattens Dictionary types during encoding but didn't account for this when decoding, causing List<Dictionary<K,V>> and Struct containing Dictionary fields to panic in GenericListArray::from() due to child type mismatch.
Which issue does this PR close?gg