Support column and partition stats with Lance file format

[yihua]

Task Description

Currently, column stats and partition stats are not supported with Lance file format. Such support needs to be added.

Particularly, LanceUtils does not implement reading column stats metadata from the Lance files.

  @Override
  public List<HoodieColumnRangeMetadata<Comparable>> readColumnStatsFromMetadata(HoodieStorage storage,
                                                                                  StoragePath filePath,
                                                                                  List<String> columnList,
                                                                                  HoodieIndexVersion indexVersion) {
    throw new UnsupportedOperationException("readColumnStatsFromMetadata is not yet supported for Lance format");
  }

Task Type

Code improvement/refactoring

Related Issues

Parent feature issue: (if applicable )
Related issues:
NOTE: Use Relationships button to add parent/blocking issues after issue is created.

[puneetdixit200]

Hello maintainers, I would like to work on this task. I will inspect LanceUtils metadata handling, implement column and partition statistics extraction for Lance files, and add tests covering HoodieColumnRangeMetadata generation. Please assign this to me.

[rahil-c]

Hi @puneetdixit200, i have assigned to you for now. Feel free to give it a try!