[golden_config]: enable SWSS ZMQ on T2 and DRH topologies

[deepak-singhal0408]

Description of PR

Summary: Re-enable SWSS ZMQ on all T2 and DRH (disaggregated Regional Hub) topologies by injecting SYSTEM_DEFAULTS|swss_zmq status=enabled into the golden config generated by generate_golden_config_db.py.

ZMQ was globally disabled in the golden-config path by sonic-mgmt PR 25083, which removed the old update_zmq_config() that wrote the now-removed DEVICE_METADATA|localhost|orch_northbond_route_zmq_enabled leaf. The image-side knob has since been consolidated into a single SYSTEM_DEFAULTS|swss_zmq entry (sonic-buildimage 27674, sonic-swss 4630, sonic-sairedis 1922). This PR selectively re-enables ZMQ for the route-scaling roles (T2 family and disaggregated Regional Hub) using the new consolidated schema. The single swss_zmq knob drives BOTH the northbound (fpmsyncd ↔ orchagent ROUTE) and southbound (orchagent ↔ syncd SAI) route ZMQ channels.

Fixes # (N/A — internal test-infra follow-up; no GitHub issue)

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• New Test case
  • Skipped for non-supported platforms
• Test case improvement

Back port request

• 202311
• 202405
• 202411
• 202505
• 202511
• 202512
• 202605

Tracking issue/work item for backport/cherry-pick request: N/A — not backported, master only.

Approach

What is the motivation for this PR?

ZMQ is the route-performance datapath between fpmsyncd/orchagent/syncd and is required for T2-scale route programming. It was turned off in the golden-config path while the northbound and southbound knobs were consolidated into a single SYSTEM_DEFAULTS|swss_zmq entry. Now that the consolidated schema is in the image, this restores ZMQ for all T2 topologies (T2 chassis, pizzaboxes, UT2, LT2, FT2) and the disaggregated Regional Hub topologies (LRH, URH, DRH tgen route-convergence), covering both single-ASIC and multi-ASIC platforms.

How did you do it?

Added update_swss_zmq_config() to generate_golden_config_db.py, called from generate() right after update_zebra_nexthop_config(). Behavior:

• Gate on the topo-name substrings t2/lrh/urh/drh — covers the T2 family (t2, ft2, lt2, t2_single_node/UT2) and the DRH family (lrh_min, urh_min, drh_tgen_route_conv).
• Exclude chassis supervisors (device_info.is_supervisor()): they hold no BGP sessions and do no per-ASIC route programming, so the ZMQ knob does not apply.
• Select the injection model purely by multi_asic.is_multi_asic(), independent of topo family, so any current or future multi-ASIC member of either family is handled identically:
  • Multi-ASIC: enumerate the real ASIC namespaces (asic{N} for N in range(multi_asic.get_num_asics())) and set SYSTEM_DEFAULTS|swss_zmq|status=enabled under each namespace, because the ZMQ channels run per ASIC and the entry must land in each ASIC's CONFIG_DB.
  • Single-ASIC: set the same entry at the top level.
• ASIC count is autodetected via multi_asic.get_num_asics() rather than the module's num_asics argument, which is not passed on the general deploy-mg golden-config path.

How did you verify/test it?

Unit-level (offline, monkeypatched multi_asic):

• Multi-ASIC T2 → swss_zmq injected under asic0/asic1; sibling FEATURE and DEVICE_METADATA tables untouched.
• Single-ASIC T2 (FT2) → top-level injection; sibling tunnel_qos_remap preserved.
• Non-T2 (t1-lag) → no-op.
• py_compile and flake8 --max-line-length=120 clean.

Multi-ASIC CONFIG_DB injection on a KVM T2 chassis (vms-kvm-t2, topo t2_2lc_min_ports-masic; 2 line cards x 2 ASIC + supervisor):

• deploy-mg (config load_minigraph --override_config -y) applied with failed=0 on all three DUTs, so SYSTEM_DEFAULTS|swss_zmq passes YANG validation.
• Generated golden config: swss_zmq status=enabled under asic0 and asic1; DEVICE_METADATA/FEATURE not corrupted.
• Running CONFIG_DB per ASIC on both line cards (queried in each ASIC's DB container): SYSTEM_DEFAULTS|swss_zmq -> status=enabled.
• Chassis supervisor: no swss_zmq in its golden config or CONFIG_DB, confirming the supervisor guard.

End-to-end ZMQ datapath on a master nightly VS image (master.1160429, which carries the image-side consolidation), single-ASIC branch:

• Running CONFIG_DB: SYSTEM_DEFAULTS|swss_zmq -> status=enabled.
• orchagent: /usr/bin/orchagent -d /var/log/swss -b 1024 -z zmq_sync -m ...
• syncd: /usr/bin/syncd -u -z zmq_sync -B null -l -p .../sai.profile ...
• The swss_zmq entry emitted by this PR drives BOTH orchagent and syncd onto the ZMQ route datapath (-z zmq_sync). master orchagent.sh reads .swss_zmq from the per-namespace SWSS_VARS; the 202605 image predates this, which is why the -z proof is shown on master.

The multi-ASIC run proves the per-ASIC CONFIG_DB injection; the master run proves the consolidated knob drives the orchagent/syncd ZMQ datapath. orchagent.sh/syncd read swss_zmq per namespace identically, so multi-ASIC per-ASIC -z zmq_sync composes from the two.

Any platform specific information?

Applies to the T2 and DRH (disaggregated Regional Hub) route-scaling topologies, single- and multi-ASIC. Chassis supervisors are excluded. Not backported — master only.

Supported testbed topology if it's a new test case?

N/A — test-infrastructure change, not a new test case.

Documentation

N/A

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[deepak-singhal0408]

T2 Elastictest failure — root cause (fix-ordering)

The T2 check fails on teardown LogAnalyzer, all matches being the same fpmsyncd error:

ERR bgpN#fpmsyncd: sendMsg: zmq send failed, endpoint: tcp://localhost:810x, zmqerrno: 11 (EAGAIN)

Why: this PR enables the route-perf ZMQ producer (fpmsyncd) end-to-end via SYSTEM_DEFAULTS|swss_zmq. The orchagent route-ZMQ consumer is present in the baseline image (knob consolidation landed in sonic-swss#4630) but is still single-threaded and not burst-coalesced. When teardown runs a full config reload, the whole route table is reprogrammed in one burst; the consumer can't drain it, the ZMQ send buffer hits its high-water mark, and fpmsyncd's sendMsg retries (~40s) then throws. On that throw the update is neither queued nor persisted and is not replayed — so it is a real dropped route, not benign log noise. (The test cases pass because the drop occurs at teardown, after all assertions.)

The mechanism is platform-independent — same producer/consumer code path on real HW T2 — so this is not a KVM/vs-only artifact and shouldn't be masked with a LogAnalyzer ignore or a vs-only gate.

In-flight consumer redesign that resolves it (burst-coalesced route ingress):

• HLD: SONiC#2328 — orchagent route programming resiliency
• sonic-swss-common#1187 — ZmqRouteServer/ZmqRouteConsumerStateTable, burst-coalesced route ingress
• sonic-swss#4564 — orchagent ZmqRouteServer for concurrent route updates
• sonic-swss#4563 — orchagent per-Executor task-timing instrumentation
• sonic-utilities#4536 — show/clear orchagent tasks (per-Executor timing + scheduling latency)

Suggestion: sequence this enablement to land after the above consumer PRs are merged and present in the target T2/DRH baseline images.