Skip to content

Refine FileLike Protocol to be compatible with pandas ReadPickleBuffer #521

Description

@hiro-o918

Background

In #517, pd.read_pickle(file) requires cast(Any, file) to suppress a type error because the current FileLike Protocol defined in gokart/utils.py is not structurally compatible with pandas' ReadPickleBuffer Protocol.

# gokart/utils.py
return pd.read_pickle(cast(Any, file))

Current FileLike

class FileLike(Protocol):
    def read(self, n: int) -> bytes: ...
    def readline(self) -> bytes: ...
    def seek(self, offset: int) -> None: ...
    def seekable(self) -> bool: ...

pandas ReadPickleBuffer requirements

mode: str                                       # property
read(n: int = ..., /) -> bytes
readline() -> bytes
seek(offset, whence=..., /) -> int
seekable() -> bool
tell() -> int

Proposal

Refine FileLike so that it is a structural subtype of ReadPickleBuffer. This would let us drop the cast(Any, file) workaround.

Required changes:

  1. Update FileLike signatures in gokart/utils.py:
    • Add mode property
    • read(n: int = ..., /) -> bytes (default + positional-only)
    • seek(offset: int, whence: int = ..., /) -> int (return int, accept whence)
    • Add tell() -> int
  2. Update _ChunkedLargeFileReader in gokart/file_processor/base.py to match the new signatures and explicitly expose mode / tell (currently they only work at runtime via __getattr__).
  3. Simplify FileLike | BytesIO to just FileLike since BytesIO already satisfies ReadPickleBuffer.

Notes

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions