Add FusedScan and nxtx annotations for FusedScan and SplitScan

[Matt711]

Description

Mainly a refactor: PR adds FusedScan (explicit is better than implicit) in addition to nvtx annotations for FusedScan and SplitScan.

Interesting enough, we're likely doing a lot of SplitScans so that is likely the path to optimize to get the most bang for buck for local and remote I/O work.

xref #22662

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Added fused (grouped) file scans for more efficient multi-file streaming reads.

• Performance & Efficiency

  • More granular split-aware scanning so each partition reads precise file subsets, improving parallel throughput.

• Improvements

  • NVTX-style runtime tracing added to per-split and fused scans for better profiling visibility.

• Tests

  • Updated streaming scan tests to validate fused and split behaviors.

Walkthrough

Adds a FusedScan IR node, records per-split paths on SplitScan, changes expand_scan_for_rank to emit SplitScan or FusedScan per rank, wraps delegated evaluations in NVTX ranges, and updates typing/wiring and tests to use the new node types.

Changes

Fused Scan IR Extension

Layer / File(s)	Summary
Actor-graph typing updates python/cudf_polars/cudf_polars/streaming/actor_graph/io.py	Import FusedScan and update scan_node producer_tasks typing to emit `SplitScan
expand_scan_for_rank: emit SplitScan | FusedScan python/cudf_polars/cudf_polars/streaming/io.py	Return type and local scans now `list[SplitScan
SplitScan storage and constructor python/cudf_polars/cudf_polars/streaming/io.py	SplitScan adds paths to __slots__, accepts paths: list[str] in __init__, includes paths in _non_child_args and in get_hashable.
NVTX instrumentation for evaluation python/cudf_polars/cudf_polars/streaming/io.py	SplitScan.do_evaluate and FusedScan.do_evaluate wrap Scan.do_evaluate in NVTX ranges that include filename(s) and split position/count.
FusedScan IR node definition python/cudf_polars/cudf_polars/streaming/io.py	New FusedScan(IR) stores grouped paths, base_scan, and parquet_options; provides get_hashable and do_evaluate delegating to Scan.do_evaluate inside an NVTX range.
StreamingScan typing alignment python/cudf_polars/cudf_polars/streaming/io.py	StreamingScan.scans and do_evaluate parameter typing updated to `list[SplitScan
Test import and assertion updates python/cudf_polars/tests/streaming/test_scan.py	Tests import FusedScan; asserts expect FusedScan for fused expansion and verify SplitScan.paths contains singleton input path; negative-path test passes a FusedScan into StreamingScan.do_evaluate.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• rapidsai/cudf#22758: Introduces rank-aware StreamingScan and related expansion/wiring that this PR extends to emit FusedScan nodes.

Suggested reviewers

• mroeschke
• pentschev

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main changes: adding FusedScan and NVTX annotations for FusedScan and SplitScan, matching the refactor work in the changeset.
Description check	✅ Passed	The description is clearly related to the changeset, explaining the refactor to add FusedScan and NVTX annotations with context about optimization targets.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Matt711]

/ok to test 9db0749

[rjzamora]

I'm noticing that we don't pass in the paths (or path maybe?) when we construct a SplitScan, even though each node is only mapped to a single file. I think this is because we pass in a paths argument to do_evaluate anyway.

I wonder if it would be clearer to use the same pattern for both FusedScan and SplitScan? More specifically, maybe we should pass in the paths subset to both or we shouldn't pass it in to either?

[rjzamora]

Thinking about this a bit more, I’d probably suggest the explicit-paths version for both wrappers: SplitScan(base_scan, paths=[path], ...) and FusedScan(base_scan, paths=local_paths, ...). Then base_scan consistently means “the original scan/options template,” and the wrapper’s paths consistently means “the files assigned to this task.”

[Matt711]

They now both take paths now

[rjzamora]

Cool - Sorry for adding more work for you here, but we probably need SplitScan.get_hashable() now too :)

[Matt711]

Thanks, I'm doing some more profiling. I think I like the idea of including the absolute paths in the annotations? I'll clean this PR up in a bit

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cudf_polars/cudf_polars/streaming/io.py`:
- Around line 330-332: The NVTX annotation currently embeds raw path strings in
the message (nvtx_annotate_cudf_polars call with message=f"SplitScan: {paths[0]}
[...]") which can leak filesystem or URL details; update both occurrences (the
SplitScan annotation and the similar annotation around line 431) to redact paths
by using a safe summary such as os.path.basename(paths[0]) or a compact form
like f"{len(paths)} files, first={os.path.basename(paths[0])}" (or "<redacted>"
when basename is not available), ensuring no raw absolute paths or query strings
are included in the NVTX message while preserving useful context (count and
first-file name).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7530b718-81b4-4cb1-9a70-da4c80b2afcb

📥 Commits

Reviewing files that changed from the base of the PR and between c30e488 and d3f4086.

📒 Files selected for processing (2)

• python/cudf_polars/cudf_polars/streaming/io.py
• python/cudf_polars/tests/streaming/test_scan.py

🚧 Files skipped from review as they are similar to previous changes (1)

• python/cudf_polars/tests/streaming/test_scan.py

[rjzamora]

Thanks @Matt711 !

[Matt711]

/merge