Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cpp/include/kvikio/defaults.hpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION.
* SPDX-FileCopyrightText: Copyright (c) 2022-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
* SPDX-License-Identifier: Apache-2.0
*/

Expand Down Expand Up @@ -121,7 +121,7 @@ std::tuple<std::string_view, T, bool> getenv_or(
*/
class defaults {
private:
ThreadPool _thread_pool{get_num_threads_from_env()};
ThreadPool _thread_pool{get_num_threads_from_env(), make_thread_pool_init_task("kvikio")};
CompatMode _compat_mode;
std::size_t _task_size;
std::size_t _gds_threshold;
Expand Down
43 changes: 42 additions & 1 deletion cpp/include/kvikio/threadpool_wrapper.hpp
Original file line number Diff line number Diff line change
@@ -1,17 +1,58 @@
/*
* SPDX-FileCopyrightText: Copyright (c) 2025, NVIDIA CORPORATION.
* SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
* SPDX-License-Identifier: Apache-2.0
*/

#pragma once

#include <array>
#include <atomic>
#include <cstdio>
#include <functional>
#include <memory>
#include <string>

#include <BS_thread_pool.hpp>

#include <pthread.h>

namespace kvikio {

/**
* @brief Thread pool type used for parallel I/O operations.
*/
using ThreadPool = BS::thread_pool;

/**
* @brief Build a `BS::thread_pool` init task that names each worker thread.
*
* The returned functor is intended to be passed to `BS::thread_pool`'s
* constructor or `reset()` overloads that accept an `init_task`. It runs
* once per worker as the pool starts each thread, setting the OS-level
* thread name (`comm` on Linux) to `"<prefix>-<index>"` so profilers such
* as nsys, `top -H`, and `/proc/<pid>/task/<tid>/comm` show meaningful
* names instead of the parent process name.
*
* The per-thread index is assigned via an atomic counter captured by the
* returned functor, so different pools can share the same prefix without
* colliding.
*
* Linux caps thread names at 15 characters plus NUL, so keep @p prefix
* short (typically 10 characters or fewer).
*
* @param prefix Name prefix, e.g. `"kvikio"` or `"kvikio-dev"`.
* @return An init task suitable for `BS::thread_pool`.
*/
[[nodiscard]] inline std::function<void()> make_thread_pool_init_task(std::string prefix)
{
auto counter = std::make_shared<std::atomic<unsigned int>>(0);
return [counter = std::move(counter), prefix = std::move(prefix)]() {
unsigned int const idx = counter->fetch_add(1, std::memory_order_relaxed);
// Linux comm limit is 15 chars + NUL.
std::array<char, 16> name{};
std::snprintf(name.data(), name.size(), "%s-%u", prefix.c_str(), idx);
pthread_setname_np(pthread_self(), name.data());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NVTX has a function dedicated to thread renaming: nvtxNameOsThread . But it only worked with global NVTX domain, not with kvikio's custom domain ( https://nvidia.slack.com/archives/C07722DM4MA/p1739200643850559). What is interesting is that its documentation has the following statement:

Tools expect thread ID to be a number that uniquely identifies the thread at the time of the call. Note that a thread's ID can be reused after it is destroyed. Tools may choose how to handle aliasing of thread IDs.
POSIX pthread_t type returned by pthread_self() may not comply with these expectations. Please use OS-specific thread ID instead of pthread_t.

I was not sure if the "use OS-specific thread ID instead of pthread_t" recommendation applied to nvtxNameOsThread specifically, or to the profiling tools in general (including nsys) that want to consume NVTX.

But I think whatever works in nsys-UI is the one we should pick, so the method in this PR should be good.

};
}

} // namespace kvikio
5 changes: 3 additions & 2 deletions cpp/src/defaults.cpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
* SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
* SPDX-License-Identifier: Apache-2.0
*/

Expand All @@ -18,6 +18,7 @@
#include <kvikio/http_status_codes.hpp>
#include <kvikio/remote_handle.hpp>
#include <kvikio/shim/cufile.hpp>
#include <kvikio/threadpool_wrapper.hpp>
#include <string_view>

namespace kvikio {
Expand Down Expand Up @@ -220,7 +221,7 @@ void defaults::set_thread_pool_nthreads(unsigned int nthreads)
{
KVIKIO_EXPECT(
nthreads > 0, "number of threads must be a positive integer", std::invalid_argument);
thread_pool().reset(nthreads);
thread_pool().reset(nthreads, make_thread_pool_init_task("kvikio"));
}

unsigned int defaults::num_threads() { return thread_pool_nthreads(); }
Expand Down
5 changes: 3 additions & 2 deletions cpp/src/file_handle.cpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION.
* SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
* SPDX-License-Identifier: Apache-2.0
*/

Expand Down Expand Up @@ -80,7 +80,8 @@ ThreadPool* get_thread_pool_per_block_device(std::string const& file_path)
}

// First file on this block device: create a new dedicated thread pool
auto thread_pool = std::make_shared<ThreadPool>(defaults::num_threads());
auto thread_pool = std::make_shared<ThreadPool>(defaults::num_threads(),
make_thread_pool_init_task("kvikio-dev"));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we call it "kvikio-block-device" / "kvikio-block-dev" or just "kvikio", to avoid confusion with "development"?

dev_id_to_thread_pool_map.emplace(block_dev_info.id, thread_pool);
file_path_to_thread_pool_map.emplace(file_path, thread_pool);
return thread_pool.get();
Expand Down