chore: formalize the default map field names to match default arrow spec - #10297
Merged
Conversation
In some places we write `keys` and in some `key`
yonipeleg33
approved these changes
Jul 7, 2026
yonipeleg33
left a comment
Contributor
There was a problem hiding this comment.
Nice! Some cleanups of hard-coded strings, otherwise LGTM
Member
Author
|
@Jefffrey can you please merge? |
Contributor
|
thanks @rluvaton |
9 tasks
Contributor
|
The actual Arrow spec uses https://github.com/apache/arrow/blob/35c5ffd12173284406e4a2c86405415444e596d7/format/Schema.fbs#L120 /// A Map is a logical nested type that is represented as
///
/// List<entries: Struct<key: K, value: V>>
|
Contributor
|
I hit an error due to this change when testing the upgrade to DataFusion: I was working on Here are the errors
I think this PR is a good one to align to the spec, but I think we should probably treat it as a "breaking API change" and hold it for the next breaking release to minimize downstream churn on a minor release I will prepare a PR |
alamb
added a commit
to alamb/datafusion
that referenced
this pull request
Jul 31, 2026
arrow-rs formalized MapBuilder's default field names from keys/values to key/value to match the Arrow spec (apache/arrow-rs#10297). Update the hardcoded schema in arrow_field()/arrow_metadata() and the nested_struct cast test to match, fixing a StructArray::new panic on mismatched field names.
This was referenced Jul 31, 2026
Contributor
|
Here is my proposed revert PR: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
N/A
Rationale for this change
In some places we write
keysandvaluesand in some caseskeyandvalue(without thes)so matching the names to match the default arrow spec names
this is also important as this is a source of common pitfalls when comparing data types from schema and actual data (where actual data might coming from another arrow impl like java which uses
keyandvalueand notkeysandvalues- https://github.com/apache/arrow-java/blob/a9f0086090e649b83c89ea864c82ed7fcc84ab2f/vector/src/main/java/org/apache/arrow/vector/complex/MapVector.java#L47-L49)This is similar to what was done for list:
item#6785What changes are included in this PR?
add constants for entries, key and value field name in
Field, replace manually every place, left out places that test different names specifically or made it more explicitI did it manually since I don't trust LLM that will do it correctly and might change tests that check that case specifically and did not want to use a script to change that
Are these changes tested?
Existing tests
Are there any user-facing changes?
the default
MapFieldNamesiskeyand notkeys,valueand notvalues