feat(stg): add OTel semconv messaging.kafka.cluster.id relationship rules#3088
Merged
Merged
Conversation
…ules Add EXT-SERVICE → KAFKATOPIC relationship synthesis rules for messaging.kafka.cluster.id (OTel incubating semconv) emitted by OTel instrumentation packages (Java #18978, Python #4727, JS #3577, .NET #4739). The existing rules in this file use kafka.cluster.id (a legacy attribute). The new rules cover the OTel-standard attribute name messaging.kafka.cluster.id so that spans from OTel-instrumented producers/consumers resolve to the correct KAFKATOPIC entity via the clusterId field in the KAFKATOPIC candidate lookup. Also add kafka.cluster.id tag to infra_kafkatopic_otel_jmxmetrics_stg, infra_kafkatopic_otel_jmxscraper_stg, and infra_kafkatopic_otel_prometheus_stg synthesis rules so that KAFKATOPIC entities from those receivers expose the cluster UUID needed by the candidate lookup. Assisted-by: Claude Sonnet 4.6
entityBot
previously approved these changes
Jul 23, 2026
entityBot
left a comment
entityBot
left a comment
Collaborator
There was a problem hiding this comment.
We don't think these changes require an API Review Board (ARB) review, so it is not required to merge this PR. Note that all API changes do require one: if you believe this PR needs an ARB review, please create the ticket and add the ARB:pending label. Reach #api-review-board in case of doubts.
The existing extServiceProducesSelfHostKafkaTopicClusterIdStg and extServiceConsumesSelfHostKafkaTopicClusterIdStg rules matched kafka.cluster.id on spans, which is not a real OTel span attribute. OTel instrumentation packages (Java, Python, JS, .NET) emit messaging.kafka.cluster.id per OTel incubating semconv. Update both rules to match messaging.kafka.cluster.id and remove the duplicate rules added in the previous commit. Assisted-by: Claude Sonnet 4.6
entityBot
previously approved these changes
Jul 23, 2026
entityBot
left a comment
entityBot
left a comment
Collaborator
There was a problem hiding this comment.
We don't think these changes require an API Review Board (ARB) review, so it is not required to merge this PR. Note that all API changes do require one: if you believe this PR needs an ARB review, please create the ticket and add the ARB:pending label. Reach #api-review-board in case of doubts.
…uster.id Add two STG relationship rules for Java kafka-clients-cluster-metrics metrics that carry messaging.kafka.cluster.id (OTel incubating semconv) as a data-point attribute: - extServiceProducesKafkaClientMetricsTopicClusterIdStg (kafka.producer.* metrics) - extServiceConsumesKafkaClientMetricsTopicClusterIdStg (kafka.consumer.* metrics) Both rules use lookupGuid(KAFKATOPIC, clusterId=messaging.kafka.cluster.id, topic=topic) matching the existing KAFKATOPIC candidates file which maps clusterId → kafka.cluster.id tag. This covers the app-side metrics path from opentelemetry-java-instrumentation PR #18978 which adds messaging.kafka.cluster.id to kafka-clients-cluster-metrics via ClusterResourceListener.onUpdate(). The collector-side (kafkametricsreceiver PR #48897) emits kafka.cluster.id as a resource attribute, handled by existing KAFKATOPIC synthesis tags added in the earlier commit. Assisted-by: Claude Sonnet 4.6
entityBot
approved these changes
Jul 23, 2026
entityBot
left a comment
entityBot
left a comment
Collaborator
There was a problem hiding this comment.
We don't think these changes require an API Review Board (ARB) review, so it is not required to merge this PR. Note that all API changes do require one: if you believe this PR needs an ARB review, please create the ticket and add the ARB:pending label. Reach #api-review-board in case of doubts.
shashank-reddy-nr
changed the title
mborroni
approved these changes
Jul 23, 2026
mborroni
approved these changes
Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant information
OTel instrumentation packages for Kafka emit the OTel incubating semantic convention attribute
messaging.kafka.cluster.idon producer and consumer spans.The existing STG relationship rules in
EXT-SERVICE-to-KAFKATOPIC.stg.yml(added by #2994) match onkafka.cluster.id— a legacy attribute not emitted by OTel instrumentation packages. This PR adds the parallel rules formessaging.kafka.cluster.idso that services instrumented with these OTel packages can have their spans resolved toKAFKATOPICentities.Changes:
relationships/synthesis/EXT-SERVICE-to-KAFKATOPIC.stg.yml— Two new rules:extServiceProducesOtelInstrKafkaTopicStg— producer spans withmessaging.kafka.cluster.idextServiceConsumesOtelInstrKafkaTopicStg— consumer spans withmessaging.kafka.cluster.idBoth rules look up
KAFKATOPICvia the existingclusterIdfield inKAFKATOPIC.stg.ymlcandidates, mappingmessaging.kafka.cluster.id→ entity tagkafka.cluster.id.entity-types/infra-kafkatopic/definition.stg.yml— Addkafka.cluster.id:tag toinfra_kafkatopic_otel_jmxmetrics_stg,infra_kafkatopic_otel_jmxscraper_stg, andinfra_kafkatopic_otel_prometheus_stgrules. Onlyinfra_kafkatopic_otel_kafkametrics_stghad this tag; the other three OTel receivers setkafka.cluster.idon their metrics but the tag was not being captured on the synthesized entity, which would have caused the candidate lookup to miss those entities.Flow:
ARB Jira ticket:
https://new-relic.atlassian.net/browse/NR-XXXX
Checklist
identifierwill be unique and valid.