BUG: tz-aware series with NaT raises exception on .to_numpy("datetime64") - #66529
Conversation
…dd note to v3.1.0
| if dtype is not None: | ||
| dtype = np.dtype(dtype) | ||
| if lib.is_np_dtype(dtype, "M") and is_unitless(dtype): | ||
| dtype = self._ndarray.dtype |
There was a problem hiding this comment.
wouldn't we usually raise here and ask the user to be more specific?
There was a problem hiding this comment.
@jbrockmendel, thank you for reviewing my PR.
Currently, on main, we raiseTypeError: 'float' object cannot be interpreted as an integer
Do you mean we should instead raise:
TypeError(
"Casting to unit-less dtype 'datetime64' is not supported. "
"Pass e.g. 'datetime64[ns]' instead."
)
?
As I understand it, we want pd.Series(pd.NaT).dt.tz_localize("UTC").to_numpy("datetime64") to give the same result as pd.Series(pd.NaT).to_numpy("datetime64") rather than raising an error.
I followed the comment in the issue and assumed that pd.Series(pd.NaT).dt.tz_localize("UTC").to_numpy("datetime64[ns]") should work both with and without [ns]. Maybe I am wrong and the comment is no longer relevant.
There was a problem hiding this comment.
id rather this raise and tell users to be more specific, but if it is allowed for some cases at the moment that would need a deprecation. if you feel strongly it should be allowed, im not inclined to argue the point
There was a problem hiding this comment.
Consistency in behavior is important. I think both pd.Series(pd.NaT).dt.tz_localize("UTC").to_numpy("datetime64") and pd.Series(pd.NaT).to_numpy("datetime64") should either raise or give ['NaT'].
Because we already disallow the unit-less "datetime64" dtype in .astype(), it seems reasonable to do the same here. Since the unit-less dtype is currently allowed for pd.Series(pd.NaT).to_numpy("datetime64"), this would be a deprecation.
|
|
||
| def test_to_numpy_tzaware_nat_unitless_dtype(): | ||
| # GH#59772 | ||
| result = Series(NaT).dt.tz_localize("UTC").to_numpy("datetime64") |
There was a problem hiding this comment.
we dont allow .astype("datetime64[ns]"); is it obvious we should be allowing to_numpy like this?
There was a problem hiding this comment.
Thanks, I see now that this is inconsistent with astype("datetime64[ns]") behaviour. I will correct to_numpy to raise here.
There was a problem hiding this comment.
I'm not saying its obvious this should raise, just that its something to discuss. What is your opinion?
There was a problem hiding this comment.
Personally, I prefer requiring users to specify the unit explicitly rather than setting it by default in the unit-less case. This can help avoid misunderstandings and unexpected results. Therefore, I am in favor of deprecating unit-less "datetime64" in to_numpy.
2 participants
doc/source/whatsnew/vX.X.X.rstfileGenerated with GPT-5.6 Thinking