Background
PR #23656 added FileWriteMetadata with a format_metadata: Option<Bytes> field designed to carry format-specific metadata (e.g., Thrift-serialized Parquet FileMetaData). The current implementation sets this to None because the typed ParquetMetaData is already accessible via ParquetSink::written() for Rust consumers.
Proposal
When a concrete FFI use case emerges (e.g., Python via PyO3, or C via datafusion-ffi), serialize the Parquet FileMetaData using Thrift Compact Protocol and populate format_metadata so that non-Rust consumers can access per-file column statistics without re-reading file footers.
Context
Background
PR #23656 added
FileWriteMetadatawith aformat_metadata: Option<Bytes>field designed to carry format-specific metadata (e.g., Thrift-serialized ParquetFileMetaData). The current implementation sets this toNonebecause the typedParquetMetaDatais already accessible viaParquetSink::written()for Rust consumers.Proposal
When a concrete FFI use case emerges (e.g., Python via PyO3, or C via
datafusion-ffi), serialize the ParquetFileMetaDatausing Thrift Compact Protocol and populateformat_metadataso that non-Rust consumers can access per-file column statistics without re-reading file footers.Context