GH-50464: [C++][Python] Simplify arrow_to_pandas DateOffset handling for nanoseconds/milliseconds

[raulcd]

Rationale for this change

Pandas 1.4 fixed the following:

• BUG: DateOffset with nanoseconds does not apply to Timestamp pandas-dev/pandas#43892

Which made us special case microseconds extracting those from our MonthDayNano and manually building microseconds.

DateOffset can be created with the following:

DateOffset(years=1, months=1,
                   days=1, seconds=1, microseconds=1,
                   minutes=1, hours=1, weeks=1, nanoseconds=1)

We currently only roundtrip Month, Day, Microseconds and Nano due to our MonthDayNano data type. We special cased microseconds but we were not special casing years, seconds, minutos, hours or weeks.

As the initial issue that made us special case has been solved upstream we can stop roundtripping microseconds and just return nanoseconds simplifying our codebase.

What changes are included in this PR?

Remove the unnecessary code to retrieve microseconds from nanoseconds.
Update test as microseconds won't be returned, only nanoseconds.

Are these changes tested?

Yes via CI

Are there any user-facing changes?

Yes, this is a change of the UX.

• GitHub Issue: [Python] Simplify arrow_to_pandas DateOffset handling for nanoseconds/milliseconds #50464

[github-actions]

⚠️ GitHub issue #50464 has been automatically assigned in GitHub to PR creator.

[raulcd]

@jorisvandenbossche would there be any reason to keep roundtriping microseconds on DateOffset(months=1, days=1, microseconds=1, nanoseconds=1) when we don't do it for years, seconds, minutes, hours, weeks?
We can simplify a little here with this PR.

[Copilot]

Pull request overview

This PR simplifies the Pandas DateOffset conversion path for month_day_nano_interval in arrow_to_pandas by no longer splitting nanoseconds into microseconds + remainder nanoseconds, and updates Python tests to expect nanoseconds-only roundtripping.

Changes:

• Simplify MonthDayNanoIntervalType → Pandas DateOffset conversion to pass interval.nanoseconds directly.
• Update interval/dateoffset-related tests to assert nanoseconds-only behavior.
• Adjust expected DateOffset construction in tests to reflect the new output shape.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
python/pyarrow/src/arrow/python/arrow_to_pandas.cc	Removes microseconds/remainder splitting and passes total nanoseconds into DateOffset.
python/pyarrow/tests/test_pandas.py	Updates roundtrip expectations for DateOffset values produced from month-day-nano intervals.
python/pyarrow/tests/test_array.py	Updates DateOffset expectations in interval array tests to use nanoseconds-only representation.

[pitrou]

I think this Copilot comment is right @raulcd , though obviously it already applied to the existing code.

[raulcd]

Thanks @pitrou, applied the fix. I am unsure whether all RETURN_IF_PYERROR checks are necessary or one after all PyDict_SetItemString is enough as I don't think PyErr_Clear is exercised but I suppose better safe than sorry. Is there a way to know which CPython methods from the C-API clear PyErr?

[pitrou]

Normally, C API functions shouldn't clear the error indicator. You should still call RETURN_IF_PYERROR after creating all the OwnedRefs, I think, lest one of them may be NULL.

[raulcd]

I think this is what I've done, have I missed any?

[pitrou]

No. But you need only one RETURN_IF_PYERROR after all the PyDict_SetItemString, not after each one of them.

[raulcd]

ah! I understand you now. Yes the single RETURN_IF_PYERROR after all PyDict_SetItemString is what I wasn't sure. Thanks for confirming.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[pitrou]

Thank you!

[raulcd]

I am going to merge this. This is currently not only simplifying part of the custom logic but fixing existing reference leaks.
If there are concerns about the UX change in the future we can discuss on a follow up.

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 6287272.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.