Skip to content

Support device buffer for the new libcurl multi poll-based backend (5/n)#966

Open
kingcrimsontianyu wants to merge 85 commits into
rapidsai:mainfrom
kingcrimsontianyu:multi-poll-device
Open

Support device buffer for the new libcurl multi poll-based backend (5/n)#966
kingcrimsontianyu wants to merge 85 commits into
rapidsai:mainfrom
kingcrimsontianyu:multi-poll-device

Conversation

@kingcrimsontianyu

@kingcrimsontianyu kingcrimsontianyu commented May 24, 2026

Copy link
Copy Markdown
Contributor

This PR supports device buffer for the multi poll-based remote I/O backend. In addition, this PR makes 3 improvements:

  • Refactor ConcurrentRequestLimiter with RAII Slot. try_acquire() now returns a move-only Slot that auto-releases on destruction. Acquire/release pairing is now structurally enforced.
  • Add a new CurlMultiAttachment RAII guard, that removes the easy handle from the multi handle on destruction.
  • Batch the transfer submission to reduce the number of curl_multi_wakeup calls from the initial "per sub-range" to the current "per reactor, per pread".
  • Copy _inbox (shared by the submission thread and reactor thread) to a reactor-private _pending deque, to decouple the submission latency from reactor's admission walk.
  • Use AI to significantly condense the comments and remove incorrect, unclear or redundant comments.

@kingcrimsontianyu kingcrimsontianyu moved this to Burndown in libcudf Jul 14, 2026
@kingcrimsontianyu kingcrimsontianyu self-assigned this Jul 14, 2026
@kingcrimsontianyu kingcrimsontianyu added feature request New feature or request non-breaking Introduces a non-breaking change c++ Affects the C++ API of KvikIO labels Jul 14, 2026
@kingcrimsontianyu kingcrimsontianyu changed the title (WIP) Support device buffer for the new libcurl multi poll-based backend (5/n) Support device buffer for the new libcurl multi poll-based backend (5/n) Jul 14, 2026
@kingcrimsontianyu
kingcrimsontianyu marked this pull request as ready for review July 20, 2026 03:33
@kingcrimsontianyu
kingcrimsontianyu requested review from a team as code owners July 20, 2026 03:33
@kingcrimsontianyu

Copy link
Copy Markdown
Contributor Author

Sending this PR to draft. The reactor thread logic is very messy and can potentially be improved by RAII. Refactoring to have a cleaner code now.

@kingcrimsontianyu
kingcrimsontianyu marked this pull request as draft July 20, 2026 19:09
@vuule vuule moved this from Burndown to Slip in libcudf Jul 20, 2026
@kingcrimsontianyu

kingcrimsontianyu commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

Performance results

  • 2026.07.24 cudf-polars TPC-H queries. "v2" is simply a rerun.
t: number of threads
r: number reactors
c: connections
+----------------------+----------+------------+--------------+
| Case                 | Date     | Total (s)  | Δ from base  |
+----------------------+----------+------------+--------------+
| easy_256t (baseline) | 07.25    |  278.6069  |     ——       |
| easy_256t (v2)       | 07.25    |  272.7131  |    -2.12%    | --> best result
| easy_256t            | 07.24    |  284.9486  |    +2.28%    |
| easy_256t (v2)       | 07.24    |  293.2409  |    +5.25%    |
+----------------------+----------+------------+--------------+
| mp_16r/512c          | 07.25    |  294.6391  |    +5.75%    |
| mp_16r/512c (v2)     | 07.25    |  291.7514  |    +4.72%    |
| mp_16r/512c          | 07.24    |  302.0218  |    +8.40%    |
| mp_16r/512c (v2)     | 07.24    |  300.1998  |    +7.75%    |
| mp_16r/256c          | 07.25    |  282.7292  |    +1.48%    |
| mp_16r/256c (v2)     | 07.25    |  280.0991  |    +0.54%    | --> best result
+----------------------+----------+------------+--------------+

The results above show that: (1) For EC2-S3, there exists certain level of network bandwidth variance. (2) Comparing the best results, the new multi-poll backend is ~3% slower than the existing easy thread pool backend, with the benefit of thread usage cut by a factor of 16.

@kingcrimsontianyu
kingcrimsontianyu marked this pull request as ready for review July 24, 2026 19:41
@kingcrimsontianyu

Copy link
Copy Markdown
Contributor Author

Event pool vs on-the-fly creation/destruction

On a temporary branch, the event pool was modified such that the get() method creates a CUDA event and put() method destroys it. Comparing use of event pool vs on-the-fly event creation/destruction on cudf-polars TPC-H SF-1k, no statistically significant difference in performance was observed.

  • Event pool runs: total time ranges 280~285 seconds.
  • On-the-fly creation/destruction: total time ranges 284~287 seconds.

cc @madsbk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Affects the C++ API of KvikIO feature request New feature or request non-breaking Introduces a non-breaking change

Projects

Status: Slip

Development

Successfully merging this pull request may close these issues.

2 participants