Exponential histogram support in ITS#3558
Draft
jmacd wants to merge 8 commits into
Draft
Conversation
Introduce the otap-df-expohisto crate: an allocation-free, zero-unsafe, zero-dependency OpenTelemetry exponential histogram. Bucket-index lookup tables are pre-generated and checked in (src/lookup_tables.rs, src/inverse_factors.rs) so the crate builds without a code-generation step or external build dependencies. Scoped to the core library (Histogram / HistogramNN / HistogramPN, mapping, lookup, boundary, quantile); benches, fuzz, and the exhaustive upstream test suite are omitted. This is the foundation for encoding ITS histogram instruments as OTLP exponential histogram points. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c95c1ba9-b197-45aa-9f7b-145982b3e4d6
Add the first ITS integration step for the vendored expohisto crate: encoding primitives that project aggregations onto OTLP ExponentialHistogramDataPoints, living alongside the MetricSet-to-OTLP code in the telemetry crate. - exponential_histogram_data_point: projects an expohisto HistogramView onto a positive-range exponential point, recovering OTLP zero_count as count - sum(positive bucket counts) and preserving sum/min/max. - mmsc_exponential_histogram_data_point: projects a min/max/sum/count summary onto the bucketless "basic" exponential form, so every histogram tier can share the exponential-histogram point type. These are covered by documented unit tests and are not yet wired into encode_metric; that follows once a histogram-carrying MetricValue variant and the Mmsc/Histogram<NORMAL>/Histogram<DETAILED> instrument enum land. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c95c1ba9-b197-45aa-9f7b-145982b3e4d6
Switch the internal Mmsc instrument's OTLP projection from an explicit-bounds Histogram point to a bucketless delta ExponentialHistogram point (scale 0, no positive/negative buckets), preserving exact min, max, sum, and count. This aligns the "basic" Mmsc tier with the forthcoming exponential-histogram instrument tiers so all tiers share a single OTLP point type. Wires the previously-added mmsc_exponential_histogram_data_point primitive into encode_metric, extends the metric-stream coalescing helpers (metric_data_compatible / append_metric_points) to handle ExponentialHistogram, updates the module documentation table, and adjusts the affected encoder tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c95c1ba9-b197-45aa-9f7b-145982b3e4d6
…n tiers Introduce the exponential-histogram instrument foundation in the telemetry crate, ahead of wiring it through the metric pipeline: - HISTOGRAM_NORMAL_WORDS (10) and HISTOGRAM_DETAILED_WORDS (26) name the expohisto word-pool sizes for the two histogram tiers. - Distribution is a delta instrument enum with a Basic tier (inline Mmsc, min/max/sum/count) and boxed Normal/Detailed exponential-histogram tiers. It offers basic()/normal()/detailed() constructors plus record, reset, count, is_empty, and same-tier merge, mirroring Mmsc's delta semantics and rejecting negative/NaN/infinite observations. This is additive and self-contained: Distribution is not yet carried by MetricValue or accepted by the #[metric_set] macro. Because it boxes expohisto histograms (which are Clone, not Copy), Distribution is not Copy; wiring it into MetricValue in a later step will make MetricValue non-Copy. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c95c1ba9-b197-45aa-9f7b-145982b3e4d6
…sizes Box the Basic(Mmsc) tier so all Distribution variants are pointer-sized, removing the variant_size_differences lint allowance instead of suppressing it. This keeps the enum uniformly small when carried by value. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c95c1ba9-b197-45aa-9f7b-145982b3e4d6
…ribution Prepare the internal telemetry aggregation for exponential-histogram tiers by adding a `MetricValue::Distribution(Box<Distribution>)` variant. Because the boxed distribution is not `Copy`, `MetricValue` is now `Clone`-only: - Rework `MetricValue` methods to be reference-based (`is_zero`, `zero_of_kind`, `add_in_place`, `reset`, `to_f64`, `to_u64_lossy`) with `Distribution` arms. - Keep serde `Serialize`/`Deserialize` for the existing scalar/Mmsc wire format; mark the new `Distribution` variant `#[serde(skip)]` since it is not yet produced by any instrument and has no stable wire representation. - Add `Distribution::tier_name` and `Distribution::summary` accessors and use them for validation formatting. - Fix the cross-crate ripple from dropping `Copy`: clone-on-yield in the metrics iterator, borrow/clone at merge sites in engine `pipeline_ctrl`, admin telemetry rendering (with deferred `unreachable!` seams for distribution rendering), and validation metric formatting. - Add a documented telemetry unit test exercising the Distribution variant's merge/reset/zero_of_kind behavior. Distribution values cannot yet be produced (no descriptor Instrument variant or macro support); those seams are intentionally deferred to a follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c95c1ba9-b197-45aa-9f7b-145982b3e4d6
…stogram point
Give MetricValue::Distribution a real, stable serde wire form instead of
skipping it: a distribution now serializes to the OTel exponential-histogram
point shape, consistent with the OTLP proto export.
- Add `ExponentialHistogramPoint` (scale, zero_count, positive offset/buckets,
count, and optional sum/min/max) as the serde wire form, plus
`Distribution::to_point` projecting each tier onto it (basic -> bucketless,
normal/detailed -> positive bucket range at the view scale).
- Introduce `DistributionValue { Live(Distribution), Point(...) }`. Both forms
serialize identically to the point shape; deserializing yields a read-only
`Point` because the vendored histogram cannot be rebuilt from buckets. This
is sufficient for the admin JSON endpoint and validation, which only read
distribution values.
- Change `MetricValue::Distribution` to `Box<DistributionValue>` and declare it
before `Mmsc` so an exponential-histogram point object is not mis-parsed as
an Mmsc summary under `#[serde(untagged)]`.
- Add tests: point projection with zero recovery, live->point serde
round-trip, MetricValue untagged disambiguation, and read-only point
reset/merge no-ops.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c95c1ba9-b197-45aa-9f7b-145982b3e4d6
github-actions
Bot
added
lang:rust
Pull request dashboard statusStatus last refreshed: 2026-07-23 03:12:52 UTC.
This automated status or its linked feedback items may be incorrect. If something looks wrong, please report it with the result you expected. |
…d-to-end Add `Instrument::ExponentialHistogram` and the `HistogramNormal`/ `HistogramDetailed` instruments so `#[metric_set]` fields can declare a distribution tier by field type. The macro maps these types to a delta F64 exponential-histogram instrument, and `encode_metric` projects the live `Distribution` (basic/normal/detailed) onto an OTLP exponential-histogram point via the tested `exphist` primitives, removing the last `dead_code` allow on the view primitive. Admin Prometheus/line-protocol rendering now expands distributions to their min/max/sum/count summary (full bucket rendering deferred) instead of panicking. Adds a macro-level integration test and an OTLP encode test (record -> snapshot -> merge -> encode). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: c95c1ba9-b197-45aa-9f7b-145982b3e4d6
github-actions
Bot
added
area:engine
Codecov Report❌ Patch coverage is ❌ Your patch check has failed because the patch coverage (56.26%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #3558 +/- ##
==========================================
- Coverage 86.80% 86.58% -0.22%
==========================================
Files 780 799 +19
Lines 318504 321683 +3179
==========================================
+ Hits 276464 278527 +2063
- Misses 41516 42632 +1116
Partials 524 524
🚀 New features to boost your workflow:
|
lquerel
reviewed
Jul 23, 2026
| @@ -0,0 +1,19 @@ | |||
| # Use this changelog template to create an entry for release notes. | |||
Contributor
There was a problem hiding this comment.
Should we combine these 3 chloggen yaml files into a single one?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change Summary
Experimental specification in open-telemetry/opentelemetry-specification#5230
What issue does this PR close?
Part of #2428
Fixes #2458
How are these changes tested?
See also https://github.com/jmacd/rust-expohisto.
Are there any user-facing changes?
Users get exponential histogram at normal and detailed level of internal metrics through ITS.
Changelog
.chloggen/*.yamlentrychore(indicated in title)