Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Use hud to narrow down suspects, then dig deeper with instrumentation if needed.

**System:**
- Linux 5.8+
- x86_64 architecture
- x86_64 or aarch64 architecture
- Root privileges

**Your application needs debug symbols** (so hud can show function names):
Expand Down
9 changes: 5 additions & 4 deletions docs/TROUBLESHOOTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,12 @@ Default Tokio threads are named `tokio-runtime-w` (Tokio ≤ 1.x) or `tokio-rt-w
**Step 2: Understand the discovery chain**

hud tries four methods in order:
1. Default prefixes `tokio-runtime-w` / `tokio-rt-worker`
2. Stack-based classification (samples stack traces for 500ms, looks for Tokio scheduler frames)
3. Largest thread group heuristic (`{name}-{N}` patterns)
1. **Explicit prefix** (`--workers <prefix>`): match threads whose comm starts with the given prefix. No fallback.
2. **Default prefixes**: try `tokio-runtime-w` (Tokio ≤ 1.x) and `tokio-rt-worker` (Tokio 1.44+).
3. **Stack-based classification**: samples stack traces for 500ms, looks for Tokio scheduler frames. Catches custom-named runtimes automatically.
4. **Largest thread group heuristic**: picks the biggest group of threads sharing a `{name}-{N}` pattern.

If all three fail, you see `workers: 0`. Stack-based discovery handles most custom thread names automatically, but requires the target process to be actively running Tokio work during the 500ms sampling window.
If all four fail, you see `workers: 0`. Stack-based discovery handles most custom thread names automatically, but requires the target process to be actively running Tokio work during the 500ms sampling window.

**Step 3: Override with --workers**
```bash
Expand Down
119 changes: 119 additions & 0 deletions hud/src/profiling/event_processor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -429,3 +429,122 @@ fn is_blocking_pool_stack(call_stack: &[StackFrame]) -> bool {
// Genuine blocking pool: has Inner::run but NOT the worker scheduler
has_blocking_pool && !has_worker_scheduler
}

#[cfg(test)]
mod tests {
use super::*;
use crate::classification::FrameOrigin;

/// Build a `StackFrame` with just a function name. File, line, and origin
/// are irrelevant to `is_blocking_pool_stack` — it only inspects
/// `function`.
fn frame(function: &str) -> StackFrame {
StackFrame {
function: function.to_string(),
file: None,
line: None,
origin: FrameOrigin::Unknown,
is_user_code: false,
}
}

// ── is_blocking_pool_stack unit tests ─────────────────────────────

#[test]
fn blocking_pool_stack_is_detected() {
// A genuine spawn_blocking stack: Inner::run is present, but the
// multi-thread worker scheduler frame is NOT.
let stack = vec![
frame("std::thread::sleep"),
frame("myapp::do_blocking_work"),
frame("tokio::runtime::blocking::pool::Inner::run"),
];
assert!(is_blocking_pool_stack(&stack));
}

#[test]
fn worker_thread_stack_is_not_filtered() {
// A genuine async worker thread: both Inner::run AND the
// multi-thread scheduler worker frame are present. This must NOT
// be classified as blocking pool.
let stack = vec![
frame("myapp::handler::process_request"),
frame("tokio::runtime::scheduler::multi_thread::worker::Context::run"),
frame("tokio::runtime::blocking::pool::Inner::run"),
];
assert!(!is_blocking_pool_stack(&stack));
}

#[test]
fn unrelated_stack_is_not_filtered() {
// A stack with neither marker frame. User code, standard library,
// whatever — should not match.
let stack = vec![frame("myapp::main"), frame("std::io::Read::read")];
assert!(!is_blocking_pool_stack(&stack));
}

#[test]
fn empty_stack_is_not_filtered() {
assert!(!is_blocking_pool_stack(&[]));
}

#[test]
fn worker_scheduler_alone_is_not_filtered() {
// Has the worker scheduler frame but NOT Inner::run.
// Unusual in practice, but must not match.
let stack = vec![
frame("myapp::handler"),
frame("tokio::runtime::scheduler::multi_thread::worker::Context::run"),
];
assert!(!is_blocking_pool_stack(&stack));
}

#[test]
fn inner_run_prefix_variants_match() {
// The filter uses starts_with, so any suffix of Inner::run should
// match (e.g. closure wrappers, monomorphized generics).
let stack = vec![frame("tokio::runtime::blocking::pool::Inner::run::{{closure}}")];
assert!(is_blocking_pool_stack(&stack));
}

#[test]
fn worker_scheduler_substring_match() {
// The filter uses contains(), so the worker scheduler frame can
// appear anywhere in the function name (different Tokio versions
// may wrap it differently).
let stack = vec![
frame("tokio::runtime::blocking::pool::Inner::run"),
frame("tokio::runtime::scheduler::multi_thread::worker::run"),
];
assert!(!is_blocking_pool_stack(&stack));
}

#[test]
fn deep_blocking_pool_stack() {
// Realistic deep stack from a spawn_blocking task doing sync I/O.
let stack = vec![
frame("std::sys::pal::unix::fs::File::read"),
frame("std::fs::File::read"),
frame("myapp::data::load_config"),
frame("tokio::runtime::blocking::task::BlockingTask::poll"),
frame("tokio::runtime::blocking::pool::Inner::run"),
frame("std::thread::Builder::spawn::{{closure}}"),
];
assert!(is_blocking_pool_stack(&stack));
}

#[test]
fn deep_worker_stack_not_filtered() {
// Realistic deep stack from a Tokio worker running async code that
// happens to block. Both marker frames present.
let stack = vec![
frame("bcrypt::bcrypt::hash_with_result"),
frame("myapp::handler::hash_password"),
frame("hyper::proto::h1::dispatch::Dispatcher::poll_inner"),
frame("tokio::runtime::scheduler::multi_thread::worker::Context::run_task"),
frame("tokio::runtime::scheduler::multi_thread::worker::Context::run"),
frame("tokio::runtime::blocking::pool::Inner::run"),
];
assert!(!is_blocking_pool_stack(&stack));
}
}
Loading