Skip to content

Broaden use of the reader slice/region fast path across reshape and xarray_dask_data call sites #198

Description

@BrianWhitneyAI

Summary

BioImage.get_image_data recently gained a slice fast path that defers a sub-region read directly to reader.get_image_data(...) instead of materializing the full array and slicing in memory (see bioio/bio_image.py ~L913–916, from #195). Right now that fast path only fires under a narrow condition. There are several other places in the repo where we either read the full image and then reshape/slice, or interact with xarray_dask_data directly, that could instead fall back to the reader — which often has a faster, format-native way to derive the result.

This issue tracks broadening use of the reader slice/region fast path (and reader fallbacks generally) across these call sites.

Existing fast path

# bioio/bio_image.py — get_image_data
if dimension_order_out is not None and kwargs and not stitching_mosaic:
    return self.reader.get_image_data(dimension_order_out, **kwargs)

# Otherwise read the full image: return it as-is, or reshape/slice it.
if dimension_order_out is None and not kwargs:
    return self.data
return biob.transforms.reshape_data(
    data=self.data,            # <-- full in-memory read, then slice
    given_dims=self.dims.order,
    return_dims=dimension_order_out,
    **kwargs,
)

Target areas

  1. get_image_data with slice kwargs but no dimension_order_out (bio_image.py ~L915–926). When a caller passes only slice kwargs (e.g. get_image_data(C=0)), the fast-path condition (dimension_order_out is not None) is not met, so we fall through to a full self.data read + reshape_data. We could default dimension_order_out to the natural order and still route through the reader slice fast path.

  2. get_image_dask_data (bio_image.py ~L834). Mirrors get_image_data but builds from xarray_dask_data. Evaluate an analogous reader fallback (reader.get_image_dask_data) so delayed slicing can use a reader-native region path where available.

  3. reshape_data on full arr.data in _transform_data_array_to_bioio_image_standard (bio_image.py ~L519–524) and the get_image_data fallback (~L921). Where the reshape is purely a slice/reorder, prefer pushing the selection down to the reader rather than reshaping a fully-materialized array.

  4. channel_names (bio_image.py ~L1056). Currently pulls self.xarray_dask_data[Channel].values, which forces task-graph construction just to read channel coordinate labels. Add a reader fallback that derives channel names from metadata.

  5. get_xarray_stack / generate_stack (bio_image.py ~L1016; bioio_base/transforms.py generate_stack). Stacking iterates scenes and concatenates .data / xarray_dask_data; check whether per-scene region reads can flow through the reader fast path.

Proposed approach

  • Add a reader fallback hook for these operations: when the reader exposes a faster region/metadata path, prefer it; otherwise fall back to the current xarray_dask_data / reshape_data behavior.
  • Preserve current behavior for mosaic stitching (_reconstruct_mosaic + MosaicTile), where results must reflect the stitched array, not raw reader output.
  • Keep changes behavior-preserving: same returned dims/shape/dtype/values, just cheaper derivation.

Related

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions