Skip to content

beyla.ebpf: exclude_otel_instrumented_services = false is silently ignored (zero-value guard drops explicit false) #6884

Description

@tete17

What happened

Setting discovery { exclude_otel_instrumented_services = false } on beyla.ebpf has no effect. The vendored OBI keeps its default (true), so services detected as OTel-instrumented are still excluded — with no error or log line. It is currently impossible to disable this exclusion through the Alloy component.

Root cause

In internal/component/beyla/ebpf/beyla_linux.go (v1.18.1, tag v1.18.1, Discovery.Convert()):

// Common fields
d.SkipGoSpecificTracers = args.SkipGoSpecificTracers
if args.ExcludeOTelInstrumentedServices {
	d.ExcludeOTelInstrumentedServices = args.ExcludeOTelInstrumentedServices
}

The guard means the value is only copied when it is true. An explicit false is indistinguishable from unset, and d was initialized from beyla.DefaultConfig().Discovery, whose default is true — so false can never reach the Beyla config.

How we noticed (observed behavior)

Alloy v1.18.0 on AKS (kernel 6.8), beyla.ebpf with exclude_otel_instrumented_services = false, traces sampler always_on, instrumenting one namespace that mixes SDK-instrumented Node services and uninstrumented ones:

request Beyla span emitted?
ingress → service without OTel SDK yes, joins the incoming trace
kubelet probe → SDK-instrumented service (SDK does not sample it) yes (root)
ingress → SDK-instrumented service (SDK emits its span) no span at all

i.e. exactly the exclusion semantics of exclude_otel_instrumented_services: true, despite it being explicitly set to false.

Versions

  • Alloy v1.18.0 / v1.18.1 (github.com/grafana/beyla/v3 v3.28.0, OBI fork v1.328.0)
  • The bug appears to be as old as the flag itself in this component.

Note on main

The config-generation rewrite on main (internal/component/beyla/ebpf/internal/config/config_build.go) models the field as *bool:

if v := d.ExcludeOTelInstrumentedServices; v != nil {
	m["exclude_otel_instrumented_services"] = *v
}

which fixes this — presumably incidentally. Filing anyway so that:

  1. v1.18.x users have a findable explanation for the silent no-op (a backport of the one-line fix would be welcome if a 1.18.x patch release happens);
  2. the new codepath gets an explicit regression test for false — the same zero-value pattern still exists on main for the neighboring exclude_otel_instrumented_services_span_metrics field (if d.ExcludeOtelInstrumentedServicesSpanMetrics { ... }), which would hit the same problem if its OBI default ever becomes true.

Workaround

None found at the component level. For our use case (running Beyla alongside SDK-instrumented services on purpose, to compare the two) the only options are removing the SDK's exporter from the target service or patching Alloy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions