Implement observability and circuit breaker for identity service

[SurbhiAgarwal1]

Description

This PR implements observability (metrics) and a lightweight, in-memory circuit breaker for the identity service in token/services/identity. These enhancements improve the resilience and monitorability of the service under load and failure conditions.

Fixes: #1644

Changes

1. Metrics Instrumentation

• Defined IdentityMetrics struct in token/services/identity/metrics.go.
• Added support for the following metrics via metrics.Provider:
  • identity_requests_total (Counter): Total number of identity requests.
  • identity_errors_total (Counter): Total number of identity errors.
  • identity_request_latency_ms (Histogram): Request latency in milliseconds.
  • identity_inflight_requests (Gauge): Number of concurrent requests.
• Integrated metrics collection into all public entry points of the identity.Provider. Metrics are collected at the start of each call to ensure even rejected requests are tracked.

2. Circuit Breaker

• Implemented a lightweight CircuitBreaker in token/services/identity/cb.go.
• Supports configurable failure thresholds and cooldown periods.
• Automatically rejects requests with a "back-pressure" error when the circuit is open.
• Resets failure counts and closes the circuit after a successful operation or cooldown period.

3. Provider Integration

• Updated identity.Provider to support optional metrics and circuit breaker initialization via functional options (WithMetrics, WithCircuitBreaker).
• Wrapped core logic with the circuit breaker Allow() check and failure/success recording.
• Ensured zero impact on existing business logic and maintained full compatibility with the driver.IdentityProvider interface.

4. Testing

• Added token/services/identity/observability_test.go with unit tests covering:
  • Basic metrics increment sanity.
  • Circuit breaker opening after the failure threshold is reached.
  • Request rejection during the open state.
  • Circuit reset behavior after the cooldown period.

Verification Results

All tests in the identity service package passed successfully:

=== RUN   TestCircuitBreaker
--- PASS: TestCircuitBreaker (0.15s)
=== RUN   TestProviderObservability
--- PASS: TestProviderObservability (0.00s)
=== RUN   TestProvider_RegisterRecipientData
--- PASS: TestProvider_RegisterRecipientData (0.00s)
...
PASS
ok  github.com/hyperledger-labs/fabric-token-sdk/token/services/identity

[AkramBitar]

@SurbhiAgarwal1 thanks a lot for openning this PR.
Please like the GIT issue to this PR.
If we do not have one please create one for this PR with description about the problem.

[adecaro]

Hi @SurbhiAgarwal1 , I think you introduced in this PR changes concerning auditable identities that are not relevant here. Please, revert those changes. Thanks 🙏

[SurbhiAgarwal1]

Hey @adecaro @AkramBitar, I’ve cleaned up the history as you suggested. Would appreciate a look when you have a moment! 🙏

[AkramBitar]

@SurbhiAgarwal1

First I would like to thank you a lot for working on that PR.
Second, did you have the chance to look at the failed tests?

Regards,
Akramm

[AkramBitar]

Hello @SurbhiAgarwal1,

I would like to thank you for your efforts on this PR. I really appreciate your contribution.

Hello @adecaro,

I would like to ask your help with this PR.

Could you please review it once you will be back from your vacation?

Thanks a million,
Akram