[master] Smart Counter Poll to allow counters to work properly on Broadcom platforms.

[justin-wong-ce]

Summary

This is change is to address #1753 and the master implementation of the HLD added in sonic-net/SONiC#2190.

FlexCounter.cpp assumes all ports support the same counter capabilities. This causes issues on most Broadcom platform switches as there are different types of ports on a switch that does not support the same set of counters.

Fix by dynamically discovering what each interface is capable of during initialization of syncd.

For more detials please refer to the above issue and HLD.

Testing

Master testing is also done, but due to frequent changes in master tests fail on other issues depending which commit is used for testing. However, none of the issues seem related to this change.

More comprehensive testing is done on the 202511 stable branch:
Testing is done on both a Arista-7060X6-16PE-384C-B-O128S2 and Arista-7260CX3-D108C8 on 202511 with the tests:

sonic-mgmt/tests/dhcp_relay/test_dhcp_counter_stress.py
sonic-mgmt/tests/drop_packets/test_drop_counters.py
sonic-mgmt/tests/drop_packets/test_configurable_drop_counters.py
sonic-mgmt/tests/gnmi/test_gnmi_countersdb.py
sonic-mgmt/tests/snmp/test_snmp_queue_counters.py

A full sonic-mgmt test suite run has also been ran (202511). There are no notable fallout compared to sonic-mgmt runs without this change. This is ran on both XGS and DNX Broadcom platforms.

EDIT: Jul 13, 2026
202605 Testing
Additional 202605 testing has been done.
Comparing full 202605 sonic-mgmt test results on TH5 (using Arista-7060X6-64PE) with and without these changes, there is no difference in the test results.

Performance Impact:

All logic change is only done in the counter initialisation stage of FlexCounters.cpp - there is no polling logic change at all.
Therefore, any performance impact is limited to any new execution of syncd - i.e. reboot / config reload / systemctl restart.

Tested on several topologies:

This impact seems reasonable for what this offers.

The fastest operation that would cause a syncd restart is a systemctl restart swss, that is a operation that spans minutes. The worst real life scenario on a HwSKU with older CPU and many interfaces takes additional ~17 seconds. New, high interface-count HwSKUs only takes an extra ~2 seconds.

As for memory usage - the impact is on the order of kilobytes, the impact is negligible as the system has GBs of RAM.

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[justin-wong-ce]

WIP - adding unit test

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[justin-wong-ce]

/azpw retry

[mssonicbld]

Retrying failed(or canceled) jobs...

[mssonicbld]

Retrying failed(or canceled) stages in build 1156144:

✅Stage Build:

• Job amd64: retried.

[justin-wong-ce]

Running the full sonic-mgmt test suite on 202511 to verify no surprise regressions at the moment, will update this thread when it is done.

Running the subset of counter related sonic-mgmt tests shows no issues.

@lolyu

Aside from an unrelated regression, there are no notable regressions from running sonic-mgmt with a syncd built with the changes from this PR.

[StormLiangMS]

@justin-wong-ce - I re-reviewed the latest head. The earlier cleanup comments look mostly addressed and CI is green, but I still have two correctness questions before approval.

1. In bulkAddObjectWithCounterGroups(), we build the bulk context from the largest supported counter group and add all VIDs. If a VID has no supported group, it can still be included in that bulk context, and the fallback path uses m_objectSupportedCountersGroupMap[vid], which default-inserts group 0. Could this still poll unsupported counters for management/no-capability ports? Should we build bulk contexts per actual group membership and skip VIDs without a group, using find() instead of operator[]?

2. For re-adding an existing VID with the same or a superset-compatible counter list, addObjectWithCounterGroups() updates group state but does not remove/overwrite existing single or bulk polling state in all cases (emplace() can leave the old entry, and bulk context add skips duplicate VIDs). Could we remove the VID from existing contexts before installing the new grouped state, matching legacy addObject() behavior?

[lolyu]

Review: #1774 (re-review at 7627bd9)

Thanks for the thorough revision — I verified every change against the full source at this head, not just the diff. All nine findings across my earlier rounds are resolved or consciously deferred with good reason. Approving.

The two real bugs and the latent one — all fixed:

• Tombstone leak → free-list. addGroup (1425) reuses m_freeGroupIndices before appending; cleanupCounterGroupMapping reclaims the index on last-reference removal (1499). The vector no longer accumulates permanent empty-set tombstones across churn. 👍
• m_failedPolls leak → fixed. removeObject now erase({rid, vid}) before dropping the object (1514), so re-added ports get an honest DEBUG→ERROR ramp again.
• m_failedPolls tier off-by-one → fixed. Now uint32_t n = ++m_failedPolls[{rid, vid}] — a single lookup with clean n==1 / n<=3 / n==4 tiers (the ERROR fires exactly once). The C++11 ++operator[] adaptation is functionally equivalent to the try_emplace I suggested.

Convention / duplication:

• addGroup helper now owns group creation at all five sites (index captured once, pre-push) — the four-spelling index inconsistency is now unwritable, and the sort-lambda is factored in.
• std::set copies → const& (2008/2067/2080/2082); unqualified find → std::find (2011).
• The 4× erase-then-emplace block collapsed to a single m_objectIdsMap[vid] = std::make_shared<...> at each site — cleaner than the helper I proposed.
• getSortedCGRef removed entirely, which resolves the mutable-interior-pointer concern more completely than constifying it would have.

On the SWSS_LOG_ENTER build constraint: fair point, and well handled. Resolving the dedup/encapsulation items by inlining (removing the accessor, using operator[] directly) rather than adding helper functions is the right synthesis — it achieves the cleanup without paying the per-call logging overhead the CI check imposes. Noted and agreed.

One optional follow-up (non-blocking): the bulk-vs-single dispatch duplication you're leaving in place pending removal of the fallback path is a reasonable call — consider a // TODO: remove fallback path once addObjectWithCounterGroups is proven stable marker so that intent is visible in-tree to the next reader.

Nice work — the feature was sound to begin with, and the new code is now tight. LGTM.

[StormLiangMS]

Hi @justin-wong-ce for the cherrypick, could you test with 202605 and update in the description?

[justin-wong-ce]

Hi @justin-wong-ce for the cherrypick, could you test with 202605 and update in the description?

I have ran the changes with 202511, 202605 sonic-mgmt is in progress - I expect more noise with that due to 202605 being newly branched.

I have manually created a cherry-pick here: #1956 - I will post further updates directly there.

[vaibhavhd]

202605 tests are in progress. Cherry pick will not be done until testing is complete with results.

[justin-wong-ce]

FYI @lolyu @vaibhavhd
The full sonic-mgmt testing on 202605 has completed. I have ran the tests on TH5 using Arista-7060X6-64PE.
Comparing the test results with and without these changes, I don't see any difference in the test results.

I have updated the PR description as well.

[mssonicbld]

Cherry-pick PR to 202605: #1995