Epic: production-grade diagnostics and observability for the DynamoDB EF Core provider

[j-d-ha]

We need to bring the provider’s diagnostics and observability surface closer to the maturity level expected from a production-ready EF Core provider.

Today the provider has useful logging, but it is still mostly logger-message based and does not yet offer the broader EF-style diagnostics surface seen in other providers. This epic is to capture the major diagnostics gaps and the comparison points we should use when designing follow-up work.

Current state

The provider currently exposes a relatively small event surface in:

• src/EntityFrameworkCore.DynamoDb/Diagnostics/DynamoEventId.cs
• src/EntityFrameworkCore.DynamoDb/Diagnostics/Internal/DynamoLoggerExtensions.cs

Current coverage includes:

• query execution start
• ExecuteStatement request metadata
• ExecuteStatement response metadata
• write execution start
• secondary-index selection diagnostics

There is also a useful Dynamo-specific response surface via:

• src/EntityFrameworkCore.DynamoDb/Extensions/DynamoEntityEntryExtensions.cs

That allows access to raw ExecuteStatementResponse, including ResponseMetadata.RequestId and optionally ConsumedCapacity, but this is currently more of a low-level escape hatch than a first-class diagnostics model.

Gaps to investigate and eventually slice into stories

1. EF-style diagnostics infrastructure, not just logger messages

The provider currently appears to log via LoggerMessage.Define(...) and IDiagnosticsLogger.Logger, but does not yet appear to use the richer EF diagnostics pattern based on:

• EventDefinition
• typed EventData
• NeedsEventData(...)
• DispatchEventData(...)
• DiagnosticSource integration

This is likely the biggest parity gap.

Relevant files:

• src/EntityFrameworkCore.DynamoDb/Diagnostics/Internal/DynamoLoggerExtensions.cs
• src/EntityFrameworkCore.DynamoDb/Diagnostics/Internal/DynamoLoggingDefinition.cs

2. Write-path executed/failure diagnostics

The current diagnostics surface includes ExecutingPartiQlWrite, but write observability is still incomplete.

We likely need to evaluate diagnostics for:

• executed single-statement writes
• failed single-statement writes
• executed transaction writes
• failed transaction writes
• executed batch writes
• failed batch writes

Relevant code paths:

• src/EntityFrameworkCore.DynamoDb/Storage/DynamoWriteExecutor.cs
• src/EntityFrameworkCore.DynamoDb/Storage/DynamoClientWrapper.cs

3. Query debug experience / ToQueryString()

Both query enumerables still throw NotImplementedException for ToQueryString().

This is a real usability and supportability gap for:

• debugging
• test baselines
• query inspection
• provider parity expectations

Relevant file:

• src/EntityFrameworkCore.DynamoDb/Query/Internal/DynamoShapedQueryCompilingExpressionVisitor.QueryingEnumerable.cs

4. Request correlation and request metadata

The provider does capture raw SDK response objects for tracked entities, but request correlation is not yet a first-class diagnostics feature.

We should evaluate a more explicit provider story for:

• ResponseMetadata.RequestId
• page-level request correlation
• surfacing request metadata through structured diagnostics payloads

Relevant files:

• src/EntityFrameworkCore.DynamoDb/Extensions/DynamoEntityEntryExtensions.cs
• src/EntityFrameworkCore.DynamoDb/Query/Internal/DynamoQueryContext.cs
• src/EntityFrameworkCore.DynamoDb/Metadata/Conventions/DynamoResponseShadowPropertyConvention.cs

5. Consumed capacity observability

The provider docs currently note that ConsumedCapacity requires low-level client usage and is not yet provider-configurable.

We should investigate:

• provider-level configuration for consumed capacity reporting
• structured diagnostics payloads/logging for consumed capacity
• whether request cost should be surfaced consistently across read paths and possibly write paths

Relevant files/docs:

• docs/diagnostics.md
• src/EntityFrameworkCore.DynamoDb/Storage/DynamoClientWrapper.cs

6. Diagnostics test coverage and parity harness

We have useful logger-capture integration tests today, but diagnostics parity is still thin relative to mature providers.

We should eventually add coverage for:

• event ID completeness
• structured payload dispatch
• diagnostic listener behavior
• ToQueryString() output
• write diagnostics success/failure coverage
• request/capacity metadata assertions

Relevant current test support:

• tests/EntityFrameworkCore.DynamoDb.IntegrationTests/SharedInfra/TestPartiQlLoggerFactory.cs

7. Documentation drift and diagnostics docs completeness

The diagnostics docs are helpful, but they need review for drift and completeness as the diagnostics surface evolves.

One current example is that the docs mention RowLimitingQueryWithoutPageSize, which does not appear to exist in the current event ID surface.

Relevant doc:

• docs/diagnostics.md

Comparison points to use

When slicing future work, compare against the diagnostics patterns used in:

EF Core Cosmos

• src/EFCore.Cosmos/Diagnostics/CosmosEventId.cs
• src/EFCore.Cosmos/Diagnostics/Internal/CosmosLoggerExtensions.cs
• src/EFCore.Cosmos/Diagnostics/CosmosQueryEventData.cs
• src/EFCore.Cosmos/Diagnostics/CosmosQueryExecutedEventData.cs

Cosmos is especially useful as a reference for:

• EF-style EventData
• structured command/query diagnostics
• executed events with richer payloads
• ToQueryString() behavior

Mongo EF Core provider

• src/MongoDB.EntityFrameworkCore/Diagnostics/MongoEventId.cs
• src/MongoDB.EntityFrameworkCore/Diagnostics/MongoLoggerExtensions.cs
• src/MongoDB.EntityFrameworkCore/Diagnostics/MongoQueryEventData.cs

Mongo is useful as a reference for:

• provider event surfaces beyond a minimal query log
• typed diagnostic payloads
• broader logging/test maturity

Desired end state

A production-ready diagnostics surface should likely include:

• structured EF-style diagnostics, not only plain logger messages
• meaningful executed and failed events for reads and writes
• first-class request correlation metadata
• first-class consumed-capacity observability
• useful query debug output such as ToQueryString()
• solid automated test coverage
• docs that reflect the actual provider behavior precisely

Notes

This epic is intentionally high level.
We are not creating the child stories yet; this issue exists to preserve the diagnostics context and the concrete code areas we need to revisit when we do the detailed slicing.