Parquet v57.0.0 upgrade blocked by arrow crate version conflicts

[coderabbitai]

Problem

The upgrade of parquet from v56.2.0 to v57.0.0 is currently blocked by compilation errors caused by multiple versions of arrow-related crates in the dependency graph.

Context

• PR: Bump parquet from 56.2.0 to 57.0.0 #31
• Reported by: @leynos

Errors

1. Method Not Found (E0599)

ParquetRecordBatchReader cannot access schema() method due to version mismatch between arrow_array v56.2.0 and v58.0.0.

Location: chutoro-providers/dense/src/provider.rs:83

2. Trait Not Implemented (E0277)

DenseMatrixProviderError cannot convert from arrow_schema::error::ArrowError due to version conflicts.

Location: chutoro-providers/dense/src/provider.rs:95

3. Type Mismatch (E0308)

DataType mismatch between arrow_schema v56.2.0 and v58.0.0.

Location: chutoro-providers/dense/src/provider.rs:102

Full Error Output

Documenting chutoro-providers-dense v0.1.0 (/home/runner/work/chutoro/chutoro/chutoro-providers/dense)
error[E0599]: no method named `schema` found for struct `ParquetRecordBatchReader` in the current scope
Error:   --> chutoro-providers/dense/src/provider.rs:83:29
   |
83 |         let schema = reader.schema();
   |                             ^^^^^^ private field, not a method
   |
note: there are multiple different versions of crate `arrow_array` in the dependency graph
  --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-array-58.0.0/src/record_batch.rs:30:1
   |
30 | pub trait RecordBatchReader: Iterator<Item = Result<RecordBatch, ArrowError>> {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this is the trait that is needed
...
35 |     fn schema(&self) -> SchemaRef;
   |     ------------------------------ the method is available for `ParquetRecordBatchReader` here
   |
  ::: chutoro-providers/dense/src/provider.rs:4:46
   |
4  | use arrow_array::{Array, FixedSizeListArray, RecordBatchReader};
   |                                              ----------------- `RecordBatchReader` imported here doesn't correspond to the right version of crate `arrow_array`
   |
  ::: /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-array-56.2.0/src/record_batch.rs:30:1
   |
30 | pub trait RecordBatchReader: Iterator<Item = Result<RecordBatch, ArrowError>> {
   | ----------------------------------------------------------------------------- this is the trait that was imported

error[E0277]: `?` couldn't convert the error to `DenseMatrixProviderError`
Error:   --> chutoro-providers/dense/src/provider.rs:95:30
   |
95 |             let batch = batch?;
   |                         -----^ the trait `From<arrow_schema::error::ArrowError>` is not implemented for `DenseMatrixProviderError`
   |                         |
   |                         this can't be annotated with `?` because it has type `Result<_, arrow_schema::error::ArrowError>`
   |
note: `DenseMatrixProviderError` needs to implement `From<arrow_schema::error::ArrowError>`
  --> chutoro-providers/dense/src/errors.rs:8:1
   |
8  | pub enum DenseMatrixProviderError {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = note: the question mark operation (`?`) implicitly performs a conversion on the error value using the `From` trait
   = help: the following other types implement trait `From<T>`:
             `DenseMatrixProviderError` implements `From<arrow_schema::ArrowError>`
             `DenseMatrixProviderError` implements `From<parquet::errors::ParquetError>`
             `DenseMatrixProviderError` implements `From<std::io::Error>`

error[E0308]: mismatched types
Error:    --> chutoro-providers/dense/src/provider.rs:102:29
    |
102 |                     actual: column_array.data_type().clone(),
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `arrow_schema::DataType`, found `arrow_schema::datatype::DataType`
    |
note: two different versions of crate `arrow_schema` are being used; two types coming from two different versions of the same crate are different types even if they look the same
   --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-schema-56.2.0/src/datatype.rs:97:1
    |
97  | pub enum DataType {
    | ^^^^^^^^^^^^^^^^^ this is the expected type `arrow_schema::DataType`
    |
   ::: /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-schema-58.0.0/src/datatype.rs:96:1
    |
96  | pub enum DataType {
    | ^^^^^^^^^^^^^^^^^ this is the found type `arrow_schema::datatype::DataType`
    |
   ::: chutoro-providers/dense/src/errors.rs:2:5
    |
2   | use arrow_schema::{ArrowError, DataType};
    |     ------------ one version of crate `arrow_schema` used here, as a direct dependency of the current crate
    |
   ::: chutoro-providers/dense/src/provider.rs:7:5
    |
7   | use parquet::arrow::{ProjectionMask, arrow_reader::ParquetRecordBatchReaderBuilder};
    |     ------- one version of crate `arrow_schema` used here, as a dependency of crate `parquet`
    = help: you can use `cargo tree` to explore your dependency tree

Some errors have detailed explanations: E0277, E0308, E0599.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `chutoro-providers-dense` (lib) due to 3 previous errors
warning: build failed, waiting for other jobs to finish...
make: *** [Makefile:28: lint] Error 101

Next Steps

This issue needs to be resolved before the parquet upgrade can proceed. The root cause is dependency version conflicts that need to be addressed in the dependency tree.

[traycerai]

Warning

Insufficient credits. Please purchase more credits or upgrade your plan to continue using Traycer.