feat(kafka): add MessageBroker/Kafka/Cluster/{id}/Topic/{topic} metrics

[shashank-reddy-nr]

Summary

Adds three new instrumentation modules that record per-cluster, per-topic Kafka metrics across all supported Kafka client versions. The cluster UUID is read from the client's own already-fetched Metadata — no extra broker connection is opened.

What's changed

• instrumentation/kafka-clients-cluster-metrics-0.11.0.0/ — covers Kafka producers [0.11.0.0, 2.0.0). ClusterIdHelper uses reflection to walk metadata.fetch().clusterResource().clusterId(). KafkaProducer_Instrumentation stores the resolved UUID in a @newfield volatile String nrClusterId and records MessageBroker/Kafka/Cluster/{id}/Topic/{topic}/Produce on each doSend().
• instrumentation/kafka-clients-cluster-metrics-2.0.0/ — covers producers and consumers [2.0.0, 3.7.0). Same reflection approach. KafkaConsumer_Instrumentation instruments both poll(Duration) and poll(long).
• instrumentation/kafka-clients-cluster-metrics-3.7.0/ — covers producers and consumers [3.7.0, ∞). Kafka 3.7 refactored KafkaConsumer into a thin facade; this module targets LegacyKafkaConsumer via the Weaver field-reference pattern (private final ConsumerMetadata metadata = Weaver.callOriginal()) to avoid reflection on the hot path.
• settings.gradle — registers all three new modules.

Metric shape

MessageBroker/Kafka/Cluster/{clusterId}/Topic/{topic}/Produce
MessageBroker/Kafka/Cluster/{clusterId}/Topic/{topic}/Consume

The cluster UUID is available once the Kafka client has completed its first metadata refresh. Until then, calls fall back to the existing topic-only metric path.

Before contributing, please read our contributing guidelines and code of conduct.

Overview

Describe the changes present in the pull request

Related Github Issue

Include a link to the related GitHub issue, if applicable

Testing

The agent includes a suite of tests which should be used to
verify your changes don't break existing functionality. These tests will run with
Github Actions when a pull request is made. More details on running the tests locally can be found
here,

Checks

• Your contributions are backwards compatible with relevant frameworks and APIs.
• Your code does not contain any breaking changes. Otherwise please describe.
• Your code does not introduce any new dependencies. Otherwise please describe.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 37.76%. Comparing base (1859c17) to head (b177dc7).
⚠️ Report is 91 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (1859c17) and HEAD (b177dc7). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (1859c17)	HEAD (b177dc7)
	3	2

Additional details and impacted files

@@              Coverage Diff              @@
##               main    #2914       +/-   ##
=============================================
- Coverage     70.71%   37.76%   -32.95%     
+ Complexity    10647     5466     -5181     
=============================================
  Files           881      879        -2     
  Lines         42947    43004       +57     
  Branches       6501     6512       +11     
=============================================
- Hits          30368    16240    -14128     
- Misses         9651    24501    +14850     
+ Partials       2928     2263      -665     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.