Implement per-(thread, CUDA context) bounce buffer cache singleton (libcurl multi poll-based backend 3/n)#965
Conversation
Co-authored-by: Lawrence Mitchell <wence@gmx.li>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
kingcrimsontianyu
added
breaking
| * @note noexcept: any failure during the underlying push or deallocation is caught and logged. | ||
| */ | ||
| void put(void* buffer, std::size_t size); | ||
| void put(void* buffer, std::size_t size) noexcept; |
There was a problem hiding this comment.
This PR is marked breaking since this function is made noexcept now.
| auto const n = defaults::remote_io_num_reactors(); | ||
| std::optional<std::size_t> const per_reactor_max = | ||
| (max_total == 0) ? std::nullopt : std::optional{std::max<std::size_t>(max_total / n, 1)}; | ||
| return new BounceBufferCachePerThreadAndContext(per_reactor_max); |
There was a problem hiding this comment.
I think this bounce buffer cache here will be useful for the io_uring implementation in the future (in place of the bounce buffer ring approach, which enforces usage ordering and likely hurts performance). But for now let it be specific to the MULTI_POLL backend. In the future perhaps we can make BounceBufferCachePerThreadAndContext templated on an internal enum to differentiate local I/O from remote I/O.
madsbk
left a comment
madsbk
left a comment
There was a problem hiding this comment.
Thanks @kingcrimsontianyu, looks very good!
|
/merge |
3 participants
This PR introduces a per-(thread, CUDA context) cache of bounce buffers with asynchronous, stream-ordered recycling. This establishes the foundation for the device-side concurrency control for the MULTI_POLL remote I/O backend. It bounds pinned-memory pressure per reactor while letting completed H2D transfers return their buffers to a free list without blocking the reactor thread.