Tech debt: revisit OTLP exporter (blocking reqwest) on OpenTelemetry SDK upgrade

[pedromvgomes]

Context

The cloud-side observability slice (WS-K, crates/common/src/telemetry.rs) exports logs/metrics/traces over OTLP using the blocking reqwest client:

# source/Cargo.toml
opentelemetry-otlp = { default-features = false, features = [
    "http-proto", "reqwest-blocking-client", "reqwest-rustls", "trace", "metrics", "logs",
] }

Why blocking (current OpenTelemetry 0.32)

We use the stable processors — TracerProvider::with_batch_exporter / MeterProvider::with_periodic_exporter. In 0.32 these export from a dedicated std::thread with no Tokio reactor attached, so an async reqwest client panics at runtime (there is no reactor running, must be called from the context of a Tokio 1.x runtime). The blocking client carries its own internal runtime and works from that thread. (We hit this panic during verification; switching to the blocking client fixed it.)

The alternative we deliberately did NOT take

0.32 does ship an async-runtime path, but it is experimental-feature-gated:

• rt-tokio
• experimental_trace_batch_span_processor_with_async_runtime
• experimental_metrics_periodicreader_with_async_runtime
• API: trace::span_processor_with_async_runtime::BatchSpanProcessor::builder(exporter, runtime::Tokio) + with_span_processor(...)

That path runs export as Tokio tasks on the app runtime and would allow the async reqwest client, but it's marked experimental (unstable API), needs more manual wiring per signal, and pulls export onto our request-handling runtime. We chose the stable blocking path on purpose.

Action (the trigger for this issue)

When we next bump the opentelemetry* crates (minor or major), re-evaluate this choice:

1. Has the async-runtime batch processor / periodic reader graduated from experimental to stable?
2. Does the stable processor still force the blocking client (i.e. is export still off-runtime)?
3. If async has become the stable/ergonomic path: switch the exporter to async reqwest (reqwest-client) + the async-runtime processors, and drop the blocking client.

If neither has changed, keep blocking and bump the version on this issue.

Pointers

• source/crates/common/src/telemetry.rs — build_enabled() (the three exporter builds + processors)
• source/Cargo.toml — opentelemetry-otlp feature set (reqwest-blocking-client)
• Branch where this landed: feature/ws-k-observability