Skip to content

fix: suppress OperationCanceledException APM errors during pod shutdown (5.7.4)#115

Merged
benspeth merged 1 commit into
masterfrom
teams/core/209/task/446605
Jun 11, 2026
Merged

fix: suppress OperationCanceledException APM errors during pod shutdown (5.7.4)#115
benspeth merged 1 commit into
masterfrom
teams/core/209/task/446605

Conversation

@ecofrankie

@ecofrankie ecofrankie commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Prevents OperationCanceledException raised during pod graceful shutdown from being recorded as APM errors.

Context

5.7.3 (PR #114) broadened the ReceiverWrapper guard to suppress LogError entries on shutdown. However, APM errors persisted in production. Root cause: Elastic APM's Azure Service Bus auto-instrumentation (via WithAllSubscribers()) captures transaction errors at the DiagnosticSource/Activity level -- upstream of ReceiverWrapper, so the guard has no effect on the APM channel.

Changes

  • Ev.ServiceBus.Abstractions/Listeners/ICancellationAwareTransactionManager.cs -- new optional interface that ITransactionManager implementations can implement to react to receive-loop cancellations
  • Ev.ServiceBus/Management/Wrappers/ReceiverWrapper.cs -- calls OnReceiveCancelled() via runtime cast in the cancellation guard
  • Ev.ServiceBus.Apm/ApmTransactionManager.cs -- implements ICancellationAwareTransactionManager; sets current APM transaction Outcome = Success to override the error state set by Azure SDK auto-instrumentation
  • Ev.ServiceBus.UnitTests/ReceiverWrapperTests.cs -- 2 new tests: OnReceiveCancelled called on cancellation, not called on other exceptions

…wn (5.7.4)

Added ICancellationAwareTransactionManager to Ev.ServiceBus.Abstractions —
an optional interface for ITransactionManager implementations that need to react
to receive-loop cancellations. ApmTransactionManager implements it by setting
the current Elastic APM transaction outcome to Success, overriding the error
state set by the Azure SDK auto-instrumentation. ReceiverWrapper calls
OnReceiveCancelled() via a runtime cast inside the existing cancellation guard.

The 5.7.3 guard already prevented LogError for shutdown cancellations; this
fix closes the remaining gap where APM errors were still recorded through
the Elastic APM agent's Azure Service Bus DiagnosticSource instrumentation.
@ecofrankie ecofrankie force-pushed the teams/core/209/task/446605 branch from 1c4ad2b to f791407 Compare June 10, 2026 10:23
@ecofrankie ecofrankie requested a review from benspeth June 10, 2026 10:26
@ecofrankie ecofrankie changed the title fix: suppress OperationCanceledException APM errors during pod shutdown (5.7.3 + 5.7.4) fix: suppress OperationCanceledException APM errors during pod shutdown (5.7.4) Jun 10, 2026
@ecofrankie ecofrankie self-assigned this Jun 10, 2026
@sokrates209 sokrates209 reopened this Jun 10, 2026
@sokrates209

Copy link
Copy Markdown

Looks good

@benspeth benspeth merged commit f0e645f into master Jun 11, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants