[WIP] feat: add endpoint to dump heap allocation profile

[albertlockett]

Status: DRAFT

Outstanding:

• changelog
• documentation
• create a github issue for this
• test on linux, windows
• automated tests
• check performance overhead

Change Summary

Adds an endpoint (/api/v1/debug/pprof/heap) that can dump the a profile (in pprof format) heap allocations.

Further reading about this solution: https://www.polarsignals.com/blog/posts/2023/12/20/rust-memory-profiling

Run the app w/ profiling enabled. jemalloc must be the allocator (default on linux and macos):

_RJEM_MALLOC_CONF="prof:true,prof_active:true,lg_prof_sample:19" \
cargo run -- \
 --config ./configs/trafficgen-filter-debug-noop.yaml

scrape the endpoint, open in your favourite pprof viewer

curl -XGET http://localhost:8080/api/v1/debug/pprof/heap -o out.pprof
go tool pprof -http=:18080 ./target/debug/df_engine ./out.pprof

This endpoint will return 500 when:

• jemalloc is not the allocator, or jemalloc is used, but was compiled without the profiling feature
• if the jemalloc options prof:true,prof_active:true are not set

What issue does this PR close?

• Closes #NNN (TBD - I will create one)

How are these changes tested?

Manually

Are there any user-facing changes?

Yes, the user could call this endpoint

Changelog

• Added a .chloggen/*.yaml entry
• This PR is a chore (indicated in title)
• This is a documentation-only PR.

[codecov]

Codecov Report

❌ Patch coverage is 10.20408% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.23%. Comparing base (1c01d12) to head (b78222d).
⚠️ Report is 5 commits behind head on main.

❌ Your patch check has failed because the patch coverage (10.20%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3497      +/-   ##
==========================================
- Coverage   86.26%   86.23%   -0.03%     
==========================================
  Files         753      754       +1     
  Lines      295599   296460     +861     
==========================================
+ Hits       254985   255650     +665     
- Misses      40090    40286     +196     
  Partials      524      524              

Components	Coverage Δ
otap-dataflow	87.27% <10.20%> (-0.04%)	⬇️
query_engine	89.57% <ø> (ø)
otel-arrow-go	52.45% <ø> (ø)
quiver	92.19% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[opentelemetry-pr-dashboard]

Pull request dashboard status

Status last refreshed: 2026-07-22 19:57:37 UTC.

• Waiting on: Author
• Next step: Move out of draft to request review.

This automated status or its linked feedback items may be incorrect. If something looks wrong, please report it with the result you expected.