Skip to content

feat(backend): add redis distributed caching layer (fixes #178)#187

Open
Diwakar-odds wants to merge 1 commit into
Mrinalray:mainfrom
Diwakar-odds:feat/issue-178-distributed-caching
Open

feat(backend): add redis distributed caching layer (fixes #178)#187
Diwakar-odds wants to merge 1 commit into
Mrinalray:mainfrom
Diwakar-odds:feat/issue-178-distributed-caching

Conversation

@Diwakar-odds

Copy link
Copy Markdown

Related Issue

Closes #178

Summary

Introduced a Redis distributed caching layer to cache Google Safe Browsing API responses, significantly reducing external API quotas and latency for redundant scans.

Motivation

Closes #178
Every URL check previously required a full round-trip to the Google Safe Browsing API. Adding a caching layer (Redis) saves processing time, reduces latency, and protects our API quota by directly serving recently scanned results.

Changes Made

  • Modified server.js:
    • Added Redis initialization logic utilizing the redis client.
    • Intercepts /check requests to fetch cached responses before pinging the external API.
    • Saves new responses to the cache with an 86400-second (24-hour) Time-To-Live (TTL).
    • Handles Redis connection errors gracefully by disabling the cache and proceeding normally without crashing the server.
  • Modified package.json & package-lock.json: Added redis (v4) dependency.
  • Modified README.md: Added setup instructions for the new optional REDIS_URL environment variable.

Acceptance Criteria

  • Introduce a caching layer (Memcached or Redis) to store the threat analysis results of commonly scanned URLs.
  • Save significant processing time and external API quotas.

Impact & Side Effects

  • Performance: Significantly faster response times for cached URLs.
  • Reliability: If the Redis instance is down, the server correctly falls back to bypassing the cache (no breaking changes or side effects).

Testing

  1. Configure REDIS_URL in .env (e.g., REDIS_URL=redis://localhost:6379).
  2. Start the local server with npm start.
  3. Perform a scan for a specific URL (e.g., example.com). Observe the [SCAN] log.
  4. Perform a scan for the same URL again. Observe the [CACHE HIT] example.com log instead of a new API call.
  5. Provide a fake/invalid Redis URL to verify the server starts and gracefully falls back to direct API requests without crashing.

Checklist

  • Code follows project style
  • Tested locally
  • No unrelated changes included
  • Documentation updated

@Diwakar-odds

Copy link
Copy Markdown
Author

Hi maintainers,

Thank you for assigning this issue to me. I have implemented the Redis caching layer as specified.

Technical Analysis

  • Integrated the redis package and built a resilient caching mechanism around the /check endpoint.
  • It intercepts Safe Browsing API calls, significantly reducing redundant external network requests and quota usage.
  • Added strict fallback mechanisms—if the Redis connection drops or is not configured, it fails gracefully and bypasses the cache without throwing 500 errors.

ECSoC26 XP Label Request

Based on the ECSoC '26 Guidelines, this PR falls squarely into the Level 3 (Core/Arch/Perf) category since it involves a core architectural upgrade (adding a distributed caching system) that directly optimizes performance and scalability.
Additionally, because this is an essential backend architecture improvement, it qualifies for the good-backend (+50 XP) bonus label.

Could you please add the ECSoC26, Level 3, and good-backend labels before review/merge? I am happy to make any changes if requested!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Core/Perf] Distributed caching system for recently scanned safe URLs

1 participant