feat(engine)!: adopt ResourceDetectors for self-telemetry

[mwear]

Change Summary

• Replaces bespoke resource detection by adopting resource detectors from upstream Rust SDK and contrib projects.
• Default detectors were changed to env, service_instance which mirrors the Go collector's defaults for self-telemetry. host, os, process, container, and k8s are opt-in.
• Supporting work was done upstream in both opentelemetry-rust and opentelemetry-rust-contrib, but is not yet released. We'll update the deps in this PR to use the pending releases before merging.
  • feat(sdk): enable minimal-dependency SDK build opentelemetry-rust#3593 enables a slim build (the only dep is the API). The dep added in this PR is the current release which results in a heavier build.
  • opentelemetry-resource-detectors is pinned to open-telemetry/opentelemetry-rust-contrib@4f5296b as it contains new detectors and updates to existing ones.

What issue does this PR close?

• Closes Adopt ResourceDetectors for self-telemetry resource attributes #3177

How are these changes tested?

• New unit tests
• cargo xtask check

Are there any user-facing changes?

Yes, breaking. Default self-telemetry no longer emits host.id and container.id (now opt-in). When opted in, host.id derives from machine-id rather than hostname, and service.instance.id is a plain UUIDv7 instead of a base32 string.

Changelog

• Added a .chloggen/*.yaml entry
• This PR is a chore (indicated in title)
• This is a documentation-only PR.

[opentelemetry-pr-dashboard]

Pull request dashboard status

Status last refreshed: 2026-07-26 12:14:14 UTC.

• Waiting on: Author
• Next step: Address or respond to 2 review feedback items:
  • Inline threads: 1
  • Top-level feedback: 2
  • For each item, reply to move the discussion forward, e.g. link to the commit that addresses it, explain why no change is needed, or ask a follow-up question.

This automated status or its linked feedback items may be incorrect. If something looks wrong, please report it with the result you expected. If you believe this pull request is incorrectly routed as waiting on the author, comment /dashboard route:reviewers to route it from waiting on the author to waiting on reviewers. If the last refreshed time above predates your latest reply or push, the dashboard hasn't processed it yet.

[Copilot]

Pull request overview

This PR switches the engine’s self-telemetry resource attribution from bespoke, Linux-centric detection to upstream OpenTelemetry ResourceDetector implementations, making detection configurable via engine.telemetry.detectors and aligning defaults with the Go collector (env + service_instance).

Changes:

• Adds a detector registry (env, host, os, process, container, k8s, service_instance) and merges detected attributes as typed AttributeValues.
• Introduces engine.telemetry.detectors config (defaulting to ["env", "service_instance"]) and wires detection into controller startup with config taking precedence.
• Removes the old hardcoded host/container/service.instance detection from the engine and updates dependencies/lockfile accordingly.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
rust/otap-dataflow/crates/telemetry/src/resource_detectors.rs	New detector registry + typed conversion/merge logic; adds unit tests for mapping, precedence, and env/service instance behavior.
rust/otap-dataflow/crates/telemetry/src/lib.rs	Exposes the new resource_detectors module.
rust/otap-dataflow/crates/telemetry/src/collector.rs	Updates test config construction to include the new detectors field.
rust/otap-dataflow/crates/telemetry/Cargo.toml	Adds OTel SDK + contrib detector deps and temp-env for tests.
rust/otap-dataflow/crates/engine/src/context.rs	Removes bespoke host/container/service.instance detection and related tests.
rust/otap-dataflow/crates/engine/Cargo.toml	Drops uuid/data-encoding deps no longer needed after removing bespoke detection.
rust/otap-dataflow/crates/controller/src/lib.rs	Runs configured detectors and merges detected resource attributes into telemetry resource map (config wins).
rust/otap-dataflow/crates/config/src/pipeline/telemetry.rs	Adds TelemetryConfig.detectors with defaults and documentation.
rust/otap-dataflow/Cargo.toml	Adds workspace deps for opentelemetry_sdk and pins opentelemetry-resource-detectors to a contrib git rev.
rust/otap-dataflow/Cargo.lock	Records new deps and transitive updates (incl. opentelemetry-resource-detectors, temp-env, windows-sys bumps).
rust/otap-dataflow/.chloggen/resource-detectors-self-telemetry.yaml	Adds breaking-change changelog entry describing new defaults and semantic changes.

Comments suppressed due to low confidence (7)

rust/otap-dataflow/crates/telemetry/src/resource_detectors.rs:135

• Tests in rust/otap-dataflow are expected to be documented with Rust doc comments describing Scenario/Guarantees (see rust/otap-dataflow/AGENTS.md). This new test is missing those required doc comments.

    #[test]

rust/otap-dataflow/crates/telemetry/src/resource_detectors.rs:143

• Tests in rust/otap-dataflow are expected to be documented with Rust doc comments describing Scenario/Guarantees (see rust/otap-dataflow/AGENTS.md). This new test is missing those required doc comments.

    #[test]

rust/otap-dataflow/crates/telemetry/src/resource_detectors.rs:148

• Tests in rust/otap-dataflow are expected to be documented with Rust doc comments describing Scenario/Guarantees (see rust/otap-dataflow/AGENTS.md). This new test is missing those required doc comments.

    #[test]

rust/otap-dataflow/crates/telemetry/src/resource_detectors.rs:156

• Tests in rust/otap-dataflow are expected to be documented with Rust doc comments describing Scenario/Guarantees (see rust/otap-dataflow/AGENTS.md). This new test is missing those required doc comments.

    #[test]

rust/otap-dataflow/crates/telemetry/src/resource_detectors.rs:171

• Tests in rust/otap-dataflow are expected to be documented with Rust doc comments describing Scenario/Guarantees (see rust/otap-dataflow/AGENTS.md). This new test is missing those required doc comments.

    #[test]

rust/otap-dataflow/crates/telemetry/src/resource_detectors.rs:182

• Tests in rust/otap-dataflow are expected to be documented with Rust doc comments describing Scenario/Guarantees (see rust/otap-dataflow/AGENTS.md). This new test is missing those required doc comments.

    #[test]

rust/otap-dataflow/crates/telemetry/src/resource_detectors.rs:193

• Tests in rust/otap-dataflow are expected to be documented with Rust doc comments describing Scenario/Guarantees (see rust/otap-dataflow/AGENTS.md). This new test is missing those required doc comments.

    #[test]

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

❌ Patch coverage is 92.06349% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.79%. Comparing base (7987c8b) to head (0a2ba42).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3555      +/-   ##
==========================================
- Coverage   86.79%   86.79%   -0.01%     
==========================================
  Files         784      785       +1     
  Lines      319527   319556      +29     
==========================================
+ Hits       277347   277357      +10     
- Misses      41656    41675      +19     
  Partials      524      524              

Components	Coverage Δ
otap-dataflow	87.84% <92.06%> (-0.01%)	⬇️
query_engine	89.57% <ø> (ø)
otel-arrow-go	52.45% <ø> (ø)
quiver	92.20% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[drewrelmas]

This is great to see, thanks @mwear! Please take a look at the current merge conflicts.

[drewrelmas]

Could you please add a sample config at https://github.com/open-telemetry/otel-arrow/tree/main/rust/otap-dataflow/configs demonstrating resource detector usage?