Skip to content

perf: Rework Parquet Thrift handling of boolean fields #9946

Description

@etseidl

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Boolean struct fields in the Thrift compact protocol are encoded solely as either FieldTypeFalse or FieldTypeTrue, with no encoded data. When parsing a Thrift field header, the current parser will save an Option<bool> in the FieldIdentifier struct, which is then used to fill in the value for the struct field. This leads to some complicated branching in read_field_begin, which is on the hot path.

Describe the solution you'd like
Remove the bool_val field from the FieldIdentifier and instead use the field_type to determine whether true or false was encoded.

Describe alternatives you've considered

Additional context
The current behavior is a carryover from the behavior of the old generated parser.

Metadata

Metadata

Assignees

Labels

enhancementAny new improvement worthy of a entry in the changelogparquetChanges to the parquet crateperformance

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions