feat(backend): implement background worker queue for processing AI feedback (fixes #317)

[Diwakar-odds]

feat(backend): implement background worker queue for processing AI feedback (fixes #317)

Summary

Migrates the synchronous AI resume analysis logic in resumeController.js to the BullMQ background worker queue (aiWorker.js), significantly improving API robustness and preventing HTTP request timeouts during long Gemini API calls.

Motivation

Closes #317

Previously, the analyzeResume endpoint invoked the Gemini model synchronously. Because parsing and analyzing PDF files can take a considerable amount of time, this approach was prone to HTTP timeouts and could block the Node.js event loop. By offloading this task to the established aiQueue, the endpoint now responds immediately with a jobId, allowing the frontend to poll for the result. This aligns the resume analysis flow with the other AI generation pipelines in the app.

Changes

• Worker (backend/workers/aiWorker.js): Extracted the Gemini PDF analysis logic, prompt, and JSON parsing from the controller into a new processResumeAnalysis worker function. Mapped this function to the "analyze-resume" job name.
• Controller (backend/controllers/resumeController.js): Removed the synchronous AI logic. The analyzeResume endpoint now enqueues an "analyze-resume" job with the target role and base64-encoded file buffer, returning a 202 Accepted status with the generated jobId.

Acceptance Criteria

• AI analysis task migrated to background worker queue.
• Endpoint returns a job identifier instead of blocking synchronously.
• Error handling is preserved via BullMQ's failed job mechanics.

Impact & Side Effects

The analyzeResume API response has changed from returning the final JSON synchronously (200 OK) to returning a job acceptance payload (202 Accepted). The frontend must use the jobId to poll the /api/jobs/:jobId endpoint for the final result.

How to Test

1. Upload a resume via POST /api/resume/analyze.
2. Observe the instant 202 response containing a jobId.
3. Hit GET /api/jobs/:jobId to check the status until it returns completed with the feedback JSON.

Quality Checklist

• AI queue integration follows existing BullMQ patterns.
• Base64 encoding used to safely pass binary buffers through Redis to the worker.

[Diwakar-odds]

Hi @Canopus-Labs! 👋 I've submitted this PR to resolve issue #317 by implementing the background worker queue for AI feedback processing.

🔍 Technical Analysis (L3 Core/Architecture Justification):

This PR directly tackles Core Architecture & Performance Reliability, which strongly satisfies the Level 3 criteria for the ECSoC '26 program:

1. Architectural Restructuring: Migrated a heavily-coupled, synchronous AI processing workflow (analyzeResume) into a decoupled, event-driven background job pipeline using BullMQ and ioredis.
2. Performance Optimization: Eliminated a severe HTTP bottleneck. Previously, processing PDF resumes synchronously via Gemini blocked the request thread and risked API gateway timeouts on large files. By offloading this to aiWorker.js, the API now returns an immediate 202 response, moving the heavy lifting entirely out of the main Node.js event loop.
3. Resilience: Implemented state persistence via the Redis-backed queue. If a worker crashes during AI generation, BullMQ preserves the job, enabling automatic retries without losing the user's uploaded file (which is safely serialized through Redis).

Because this fundamentally upgrades the core AI feedback pipeline and introduces a scalable message-queue architecture for the resume engine, I kindly request the maintainers to review this as an L3 Contribution and apply the ECSoC26, Level 3, good-pr, and good-backend labels if the code meets your standards!

Looking forward to your review! 🚀