Feat/contract activity heatmap#717
Open
TheCreatorNode wants to merge 5 commits into
Open
Conversation
- Create prune_events command to delete old ContractEvents - Support configurable retention period via --retention-days or EVENT_RETENTION_DAYS setting - Include --dry-run flag to preview deletions without executing - Add comprehensive test suite covering all functionality - Log number of deleted records with colored output Fixes SoroScan#366
## Description Implements comprehensive replication lag monitoring for multi-region database deployments. ## Changes ### Core Implementation - **soroscan/ingest/replication.py**: New ReplicationLagMonitor class with: - LSN-based lag measurement (fast method) - Write-test lag measurement (accurate method) - Replica health status checks - Configurable threshold-based alerting ### Prometheus Metrics - soroscan_replication_lag_seconds: Current lag in seconds (Gauge) - soroscan_replication_lag_checks_total: Total checks performed (Counter) - soroscan_replication_status: Health status 1=healthy, 0=unhealthy (Gauge) - soroscan_replication_alerts_total: Total alerts triggered (Counter) ### Celery Tasks - monitor_replication_lag(): Periodic lag measurement and alerting - check_replica_health(): Comprehensive replica status validation ### CLI Tool - check_replication_lag command with: - Single check or continuous monitoring modes - Configurable check intervals - LSN or write-test measurement methods ### Monitoring & Alerting - Grafana dashboard panels for lag visualization - Alert rules for warning/critical thresholds - Health status monitoring - Check failure detection ### Documentation - Comprehensive configuration guide - Usage instructions - Troubleshooting guide - Performance impact analysis ## Acceptance Criteria ✅ - [x] Replication lag measured - [x] Metrics exported to Prometheus - [x] Dashboard shows lag over time - [x] Alerts on lag > threshold ## Configuration Set environment variables: - REPLICA_DB_ALIAS: Database alias for replica connection - REPLICATION_LAG_THRESHOLD_SECONDS: Warning threshold (default: 5s) - REPLICATION_LAG_ALERT_THRESHOLD_SECONDS: Critical threshold (default: 10s) - REGION_NAME: Region identifier for multi-region deployments ## Testing - Run single check: python manage.py check_replication_lag - Run continuous: python manage.py check_replication_lag --continuous - View metrics: curl http://localhost:8000/metrics | grep replication
Implements Issue SoroScan#529: Webhook Replay with Event Filtering Features: - WebhookReplay model for tracking replay requests with status lifecycle (pending/processing/completed/failed) - Filter support: date range (start_date, end_date) and event types - REST API endpoints: create replay, list replays, get status, trigger from webhook detail - Rate limiting: max 5 replays per webhook per hour - Async processing via Celery: process_webhook_replay and cleanup_old_webhook_replays tasks - Batch processing: handles up to 100 events per batch - Real-time progress tracking: total_events, replayed_events, failed_events counters - Error handling with retry logic and dead-letter queue support - Prometheus metrics integration for operational observability - Comprehensive logging for debugging and auditing Acceptance Criteria Met: ✓ Replay endpoint with filters (date range, event types) ✓ Rate limiting on replay operations (5 per hour per webhook) ✓ Status tracking (pending → processing → completed/failed) Database Changes: - Added WebhookReplay model with proper indexes - Migration 0039_webhookreplay.py with CASCADE delete for data consistency API Changes: - POST /api/webhook-replays/ - Create replay request - GET /api/webhook-replays/ - List replays with filtering - GET /api/webhook-replays/{id}/ - Get replay status - POST /webhooks/{id}/replay/ - Trigger replay from webhook detail - GET /webhooks/{id}/replay-status/ - Get all replay statuses for a webhook
|
@TheCreatorNode Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #535