Bug: [SmartSwitch] DPU gNMI/gNOI service binds to loopback only after sonic-gnmi#712, breaking NPU proxy connectivity

[gpunathilell]

Is it platform specific

generic

Importance or Severity

Critical

Description of the bug

After sonic-gnmi#712 was merged, the gNMI/gNOI telemetry service on each DPU in a SmartSwitch now binds exclusively to 127.0.0.1:50052 (loopback), making every DPU unreachable from the NPU's internal gNOI proxy.

Root cause: In gnmi-native.sh, when no CERTS or X509 entry exists in ConfigDB (which is the DPU's default state), the script falls into the else branch and hardcodes:

TELEMETRY_ARGS+=" --noTLS --bind_address 127.0.0.1"

PR #712 added a strict enforcement in telemetry.go (setupFlags): when --noTLS is set, --bind_address must be a loopback address (net.IP.IsLoopback()). Since gnmi-native.sh already supplies 127.0.0.1, the service starts — but listens only on loopback.

The NPU's DPUProxy (pkg/interceptors/dpuproxy/proxy.go) dials each DPU directly at 169.254.200.x:{50052,8080} over the SmartSwitch midplane fabric using insecure gRPC. With the DPU now bound to 127.0.0.1 only, all those dials fail with connection refused.

Note: setting bind_address in the DPU's GNMI|gnmi ConfigDB entry would not help — the script never reads it, and even if it were passed to the binary, 169.254.x.x would fail the IsLoopback() check and prevent the service from starting entirely.

Steps to Reproduce

1. Deploy a SmartSwitch platform running a SONiC image built after sonic-gnmi#712, with no TLS certs configured on the DPUs.
2. From the NPU, verify DPU gNOI connectivity:

gnoi_client -target 169.254.200.1:50052 -module System -rpc Time

3. Repeat for 169.254.200.2, 169.254.200.3, 169.254.200.4. All fail with connection refused.
4. On any DPU, confirm the service is listening only on loopback:

ss -tlnp | grep 50052
# LISTEN  0  128  127.0.0.1:50052  0.0.0.0:*

Actual Behavior and Expected Behavior

Actual: The DPU telemetry service starts with --noTLS --bind_address 127.0.0.1, binding only to loopback. The NPU's DPUProxy cannot reach 169.254.200.x:50052 or 169.254.200.x:8080, causing all DPU gNOI operations (reboot, time, file) to fail. 100% reproduction rate.

Expected: The DPU gNMI/gNOI service should be reachable from the NPU over the 169.254.200.x midplane fabric. This network is internal and isolated — not exposed externally.

Relevant log output

ERROR:
  Code: Internal
  Message: failed to connect to DPU2: failed to connect to DPU2 on any port [50052 8080]:
  last error: rpc error: code = Unavailable desc = connection error:
  desc = "transport: Error while dialing: dial tcp 169.254.200.3:8080: connect: connection refused"

On the DPU (ss -tlnp | grep 50052):

LISTEN  0  128  127.0.0.1:50052  0.0.0.0:*  users:(("telemetry",pid=...,fd=...))

Output of show version, show techsupport

SONiC.master-<commit>

Attach files (if any)

N/A

[gpunathilell]

@vvolam fyi

[vvolam]

@hdwhdw FYI

[hdwhdw]

Thanks! Will take a look

[hdwhdw]

@qiluo-msft fyi

[hdwhdw]

Update: the issue is reproduced, fixed, and physically validated.

Root cause:

• sonic-gnmi#712 correctly restricted plaintext --noTLS listeners to loopback.
• SmartSwitch NPU-to-DPU proxy traffic uses the DPU's chassis-internal
  eth0-midplane IPv4 link-local address, so loopback-only DPU listeners reject
  the proxy connection.

Fix:

• telemetry: resolve no-TLS link-local bind by interface sonic-gnmi#721 adds an explicit
  --no_tls_link_local_interface mode. The binary waits for exactly one IPv4
  link-local address on the named interface and binds that concrete address.
  Direct --noTLS --bind_address remains loopback-only.
• [SmartSwitch] Restore DPU gNMI midplane connectivity #28564 classifies SmartSwitch DPUs through the
  launchers' existing sonic-cfggen render and passes only
  --noTLS --no_tls_link_local_interface eth0-midplane.
• The launch scripts do not query Redis directly, parse ip output, or poll.

Validation:

• Native arm64 gNMI and telemetry container build passed.
• On a physical SmartSwitch, both services listened only on the concrete DPU
  midplane address at ports 50052 and 8080.
• NPU TCP probes, direct gNOI System.Time, and the real UDS metadata-routed
  DPUProxy call passed.
• Negative probes confirmed no listener on the DPU management address or
  loopback.
• The DPU was restored to its original images, all DPUs remained reachable,
  the baseline RPC passed, and the testbed was released.

Status:

• All checks pass on telemetry: resolve no-TLS link-local bind by interface sonic-gnmi#721.
• Security, DCO/CLA, file protection, and internal conflict checks pass on
  [SmartSwitch] Restore DPU gNMI midplane connectivity #28564; its broad platform build matrix is still
  running.
• Review has been requested on both PRs.