Skip to content

Implement outbox pattern for reliable event publishing in Products microservice#48

Closed
Copilot wants to merge 4 commits into
ai-test/202507-Baselinefrom
copilot/fix-45
Closed

Implement outbox pattern for reliable event publishing in Products microservice#48
Copilot wants to merge 4 commits into
ai-test/202507-Baselinefrom
copilot/fix-45

Conversation

Copilot AI commented Jun 30, 2025

Copy link
Copy Markdown

This PR implements the outbox pattern to ensure reliable event publishing when products are created or updated, addressing the critical requirement for guaranteed event propagation in the Products microservice.

Problem

Previously, product events were published directly to the message broker after database operations, creating a risk of lost events if the message broker was unavailable or if failures occurred between database commits and message publishing.

Solution

Implemented the outbox pattern with the following components:

🗃️ Database Changes

  • Added OutboxMessages table to store pending events atomically with product operations
  • Includes retry logic, error tracking, and processing timestamps
  • Optimized with indexes for efficient background processing

🔄 Core Components

  • OutboxProcessor: Processes pending messages and publishes to message broker
  • OutboxBackgroundService: Runs every 10 seconds to process pending events
  • Enhanced ProductsWriteStore: Atomic operations ensuring product changes and event storage happen in the same transaction

📝 Updated Use Cases

  • CreateProductDetails: Now stores events in outbox instead of direct publishing
  • UpdateProductDetails: Same transactional approach for update events

✅ Testing

  • 7 comprehensive unit tests covering all outbox functionality
  • Integration tests verifying end-to-end use case behavior
  • All existing tests continue to pass

Benefits

  • 🛡️ Guaranteed Delivery: Events are persisted before any external operations
  • 🔄 Retry Logic: Failed messages are retried with exponential backoff (max 3 attempts)
  • ⚡ Performance: Background processing doesn't block API responses
  • 🔍 Observability: Failed messages are logged with error details
  • 🔒 Consistency: Database and message broker state remain synchronized

Example Usage

The API contracts remain unchanged - the outbox pattern works transparently:

// Creating a product now automatically stores the event in outbox
var response = await createProductUseCase.Execute(new CreateProductRequest(
    new ProductDetails("New Product", "Description"),
    stock: 10,
    price: 99.99m
));

// Event will be processed and published by the background service
// within 10 seconds, with automatic retry on failures

Configuration

The background service is automatically registered in dependency injection:

builder.Services
    .AddScoped<IOutboxRepository, OutboxRepository>()
    .AddScoped<IOutboxProcessor, OutboxProcessor>()
    .AddHostedService<OutboxBackgroundService>();

Fixes #45.


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Copilot AI and others added 2 commits June 30, 2025 21:54
Co-authored-by: ElectNewt <1971861+ElectNewt@users.noreply.github.com>
Co-authored-by: ElectNewt <1971861+ElectNewt@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement outbox pattern in the Prodcuts microservice Implement outbox pattern for reliable event publishing in Products microservice Jun 30, 2025
Copilot AI requested a review from ElectNewt June 30, 2025 21:57

@ElectNewt ElectNewt left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please address the comments


private class ProductDetailEntity

public async Task AddOutboxMessage(string eventType, object eventData, string routingKey)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you dn't need this

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I've removed the unused AddOutboxMessage method. The use cases now only use CreateRecordWithOutboxMessage and UpdateProductWithOutboxMessage methods which handle the outbox pattern transactionally. Fixed in commit e4b89ec.

Task MarkAsFailed(int messageId, string errorMessage);
}

public class OutboxRepository : IOutboxRepository

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is an indirection layer that it is not needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I've removed the OutboxRepository indirection layer and moved the outbox operations directly to ProductsWriteStore. OutboxProcessor now depends on IProductsWriteStore directly, simplifying the architecture while maintaining the same functionality. Fixed in commit e4b89ec.

Co-authored-by: ElectNewt <1971861+ElectNewt@users.noreply.github.com>
Copilot AI requested a review from ElectNewt June 30, 2025 22:27
@ElectNewt ElectNewt changed the base branch from main to ai-test/202507-Baseline July 4, 2025 07:34
@ElectNewt ElectNewt closed this Jul 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement outbox pattern in the Prodcuts microservice

2 participants