MixedDbClient: chunk batched ZMQ Set/Del to stay under MQ_RESPONSE_MAX_COUNT (16 MB)

[hdwhdw]

Background

PR #675 introduces DbBatchSetTable / DbBatchDelTable on MixedDbClient, which collapse multi-key gNMI Set payloads into a single batched ZMQ message via ZmqProducerBatchedSet / ZmqProducerBatchedDel (added in sonic-swss-common#1188).

libswsscommon's ZMQ transport caps a single message at MQ_RESPONSE_MAX_COUNT = 16 MiB (see common/zmqserver.h and the size check in ZmqClient::sendMsg in common/zmqclient.cpp). When the cap is exceeded, the send fails outright.

Per @prabhataravind's review comment:

At 125K+ entries, this single ZMQ message will exceed MQ_RESPONSE_MAX_COUNT (16MB). We will revisit that later based on tests.

A realistic DASH route entry serializes to ~200 B, so a single batched message tops out at roughly 80k entries. Real-world DASH bulk updates can exceed that.

Problem

DbBatchSetTable / DbBatchDelTable currently send the whole batch as one ZMQ message regardless of size. Above the 16 MiB cap the send will fail and the entire gNMI Set is rejected.

Proposed work

1. Add size-aware chunking inside DbBatchSetTable and DbBatchDelTable:
   • Estimate per-entry serialized size (key + field/value pairs) as entries are appended to the SWIG vectors.
   • When the accumulated size approaches a configurable threshold (suggest 12-14 MiB to leave headroom for ZMQ framing), flush the current chunk via ZmqProducerBatchedSet/Del and start a new one.
   • Preserve ordering across chunks within a single DbBatchSetTable / DbBatchDelTable call.
2. Decide on failure semantics: today the operation is all-or-nothing per ZMQ message. With chunking, a mid-batch failure can leave a partial state. Options:
   • Best-effort: continue sending remaining chunks, return the first error.
   • Fail-fast: stop on first error.
   • Document either way in the helper docstrings.
3. Add a regression test exercising a payload that would exceed 16 MiB if sent as a single message.
4. (Optional) Make the chunk threshold tunable via env var or config for testbeds with non-default MQ_RESPONSE_MAX_COUNT.

Alternatives considered

Raising MQ_RESPONSE_MAX_COUNT in sonic-swss-common was discussed in PR #26679 review. It is not free — MQ_RESPONSE_MAX_COUNT drives m_sendbuffer.resize() on every ZMQ client and m_buffer.resize() on every ZMQ server, so every process grows by the new size whether it uses big messages or not. Client-side chunking in sonic-gnmi is the proper fix.

References

• Tracking issue: Enhancement: sonic-gnmi should use batched ZmqProducerStateTable APIs for gNMI operations sonic-buildimage#27250
• gNMI batching PR: MixedDbClient: batch ZMQ Set/Del for multi-key gNMI updates (#27250) #675
• SWIG glue PR: swsscommon: expose batched ZmqProducerStateTable APIs to SWIG sonic-swss-common#1188
• ZMQ cap discussion: [SmartSwitch] Support maximum dash configuration size sonic-buildimage#26679 (review thread r3101322039)

[sneelam20]

@hdwhdw Should this be assigned to you?

[hdwhdw]

Yes.