Skip to content

SuperLink construction crashes under pandas 3.0 (Copy-on-Write): numba rejects read-only boolean storage mask #74

@stoiver

Description

@stoiver

Under pandas 3.0 (where Copy-on-Write is the default), constructing a SuperLink fails with:

TypeError: No matching definition for argument type(s) array(float64, 1d, C), array(float64, 1d, C),
array(float64, 1d, C), array(float64, 1d, C), array(float64, 1d, C), readonly array(bool, 1d, C)
  pipedream_solver/nsuperlink.py: compute_storage_areas
    numba_compute_functional_storage_areas(...)

Cause

numba_compute_functional_storage_areas / numba_compute_functional_storage_volumes are decorated with an eager, writable-only signature whose last argument is boolean[:]:

@njit(float64[:](float64[:], float64[:], float64[:], float64[:], float64[:], boolean[:]), cache=True)
def numba_compute_functional_storage_areas(h, A, a, b, c, _functional):
    ...

But the mask passed in is built from a pandas comparison in __init__ (nsuperlink.py:431-432):

_functional = (_storage_type.str.lower() == "functional").values
_tabular    = (_storage_type.str.lower() == "tabular").values

In pandas 3.0, Copy-on-Write makes that .values a read-only array. numba treats boolean[:] as requiring a writable array and refuses to match a read-only one, so compute_storage_areas() (called from _setup_step during SuperLink.__init__) raises. On pandas < 3 the same .values is writable, so it works — which is why this only began appearing as pandas 3.0 rolled out on conda-forge/PyPI.

Minimal repro

import numpy as np
from pipedream_solver.nsuperlink import numba_compute_functional_storage_areas as f
a = np.array([True, False, True]); a.flags.writeable = False   # what pandas-3 .values yields
f(np.zeros(3), np.ones(3), np.ones(3), np.ones(3), np.ones(3), a)
# -> TypeError: No matching definition ... readonly array(bool, 1d, C)

(Equivalently, on pandas 2.x: pd.set_option("mode.copy_on_write", True) before building any SuperLink with functional storage.)

Suggested fix

Make the masks writable copies where they are created in nsuperlink.py (and the matching spot in superlink.py), so the kernels keep their fast eager signatures:

-        _functional = (_storage_type.str.lower() == "functional").values
-        _tabular    = (_storage_type.str.lower() == "tabular").values
+        _functional = np.array((_storage_type.str.lower() == "functional").values)
+        _tabular    = np.array((_storage_type.str.lower() == "tabular").values)

np.array(...) copies, yielding a writable contiguous array that numba accepts under both pandas 2.x and 3.x. (Alternatively, drop the eager boolean[:] from the two @njit signatures so numba lazily compiles a read-only-tolerant specialization.)

Same family as #73 (numpy 2.x np.bool8). Happy to open a PR if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions