Skip to content

feat: implement rich logging, progress percentages, and quota handling for Gmail sync#80

Open
HyperVon wants to merge 1 commit into
thughari:devfrom
HyperVon:feature/gmail-sync-progress
Open

feat: implement rich logging, progress percentages, and quota handling for Gmail sync#80
HyperVon wants to merge 1 commit into
thughari:devfrom
HyperVon:feature/gmail-sync-progress

Conversation

@HyperVon

Copy link
Copy Markdown

Overview

This Pull Request implements rich progress feedback, percentage indicators, and resilient rate-limit handling for the Gmail active sync pipeline on both the backend and frontend.

Problem

Previously, triggering a manual Gmail sync was silent in the logs and showed static/uninformative status states (e.g. just a generic "Syncing...") in the UI. In addition:

  1. Long-running syncs (with hundreds of emails) could exceed the Gemini API daily free tier quota (20 requests per day), causing the backend to continuously throw 429 exceptions and freeze in a stuck state.
  2. A race condition existed where the backend would spawn the async task, but the frontend's immediate subsequent dashboard reload occurred before the database claimed the lock, requiring a manual refresh to see the active status.

Changes Made

1. Backend Service & Lock Management

  • Synchronous Sync Handshake: Refactored initiateManualSync in GmailIntegrationService.java to claim the lock and initialize the status synchronously on the HTTP thread, fully resolving the dashboard reload race condition.
  • Background Runner: Offloaded the heavy lifting (scanning, message downloading, and AI extraction) to the askExecutor asynchronously.
  • Startup Lock Clean: Created a StartupInitializer (CommandLineRunner) to automatically clear any stale database locks and sync statuses upon server boot.

2. Granular Progress & Quota Resilience

  • Step Counts & Percentages: Added real-time progress calculations for the email detail download stage (Downloading details (X/Y - P%)...) and the Gemini extraction stage (Extracting jobs with Gemini AI (batch X/Y - P%)...).
  • Cache Eviction: Added active cache eviction inside the sync status updates so that the frontend's dashboard queries bypass Spring's @Cacheable boundaries to fetch the live state.
  • Graceful Gemini Quota Failover: Monitored HttpClientErrorException (status 429) from Google AI Studio. Upon quota exhaustion, the sync gracefully terminates early, releases the lock, and displays a permanent user message: Sync paused: Gemini API daily quota exceeded. Please try again tomorrow.

3. Frontend UI & State Enhancements

  • Dynamic Polling Acceleration: Programmed a reactive effect in job.service.ts to temporarily accelerate polling to 2 seconds during active sync, reverting to 30 seconds on completion.
  • Type-safe UI Indicator: Leveraged Angular's control-flow @if (syncStatus(); as status) to narrow type-checking. Displayed solid, static status text next to the Sync button for legibility, with dynamic warning styling (in red) for error states.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant