Skip to content

feat: add Spark-compatible atan2 function - #23962

Open
KarpagamKarthikeyan wants to merge 1 commit into
apache:mainfrom
KarpagamKarthikeyan:feat/spark-atan2
Open

feat: add Spark-compatible atan2 function#23962
KarpagamKarthikeyan wants to merge 1 commit into
apache:mainfrom
KarpagamKarthikeyan:feat/spark-atan2

Conversation

@KarpagamKarthikeyan

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Spark provides atan2(exprY, exprX), which returns the angle in radians between the positive x-axis and the point given by the coordinates (exprX, exprY). It was not yet implemented in datafusion-spark — only an auto-generated test stub existed at spark/math/atan2.slt with its query commented out.

What changes are included in this PR?

  • Add SparkAtan2 (implementing ScalarUDFImpl) in datafusion/spark/src/function/math/atan2.rs, computing y.atan2(x) element-wise via the Arrow binary kernel.
  • Register it in datafusion/spark/src/function/math/mod.rs.
  • Enable the atan2.slt sqllogictest.

The signature is exact(Float64, Float64) -> Float64, following the datafusion-spark convention of only accepting Spark-supported types. NULL in either argument propagates to NULL.

Note: DataFusion core already has an atan2, but this is implemented self-contained rather than delegating — matching the crate convention, where 14 of the 15 existing math functions reimplement rather than wrap core (including abs, ceil, floor, round, rint, all of which also exist in core; only pow delegates, as a special case). This keeps the Spark function library complete and consistent.

Are these changes tested?

Yes — datafusion/sqllogictest/test_files/spark/math/atan2.slt covers:

  • standard angles and all four quadrants (atan2 is quadrant-aware via the signs of both arguments),
  • both axes and the negative x-axis (atan2's range extends to π, unlike atan),
  • NULL propagation (either argument) and NULL-beats-Infinity,
  • NaN in every position, including atan2(NaN, Infinity) = NaN (for atan2, NaN propagates even over Infinity — unlike hypot, where Infinity dominates),
  • all four infinity quadrants and the one-infinite-argument cases,
  • signed zeros (atan2(-0, -1) = -π),
  • the array path (including a NULL row).

Are there any user-facing changes?

Yes — adds the Spark-compatible atan2 scalar function to datafusion-spark. No breaking changes to public APIs.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) spark labels Jul 29, 2026
@KarpagamKarthikeyan

Copy link
Copy Markdown
Contributor Author

cc @Jefffrey @comphead — a follow-up to my recently merged hypot PR, this adds a Spark-compatible atan2 function to datafusion-spark.

It follows the existing math conventions — exact(Float64, Float64) -> Float64, invoke via make_scalar_function, reimplemented self-contained (matching the 14/15 math functions that reimplement rather than delegate to core). Backed by Rust's f64::atan2.

Covered by sqllogictest cases: all four quadrants, both axes, the negative x-axis (range to π), NULL propagation, NaN in every position (incl. atan2(NaN, Infinity) = NaN — note NaN propagates even over Infinity here, unlike hypot), all four infinity quadrants, signed zeros, and the array path. Closes #23961.

Would appreciate a review when you have a moment. Thanks!

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.37500% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.70%. Comparing base (74eebbb) to head (5419c96).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/spark/src/function/math/atan2.rs 83.87% 3 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23962      +/-   ##
==========================================
- Coverage   80.70%   80.70%   -0.01%     
==========================================
  Files        1095     1096       +1     
  Lines      372554   372586      +32     
  Branches   372554   372586      +32     
==========================================
+ Hits       300661   300683      +22     
- Misses      53924    53929       +5     
- Partials    17969    17974       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Jefffrey

Copy link
Copy Markdown
Contributor

how is this different from the datafusion functions ver?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

spark sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Spark-compatible atan2 function

3 participants