Feature: Normalized Exact Match Caching for AI Chatbot (Redis)

[vivek0028]

📋 Description

This pull request introduces Normalized Exact Match Caching for the CropAssistant AI Chatbot using the project's existing Redis infrastructure.

Previously, every chatbot query—including repeated questions such as "What is a batch?"—triggered a new request to the Gemini/OpenAI API. This resulted in unnecessary API usage, increased costs, and response latencies of 2000ms+ for frequently repeated queries.

This implementation adds a Redis-based caching layer that normalizes user queries and stores successful AI responses for 24 hours. Repeated or minor formatting variations of the same question can now be served directly from Redis without making another LLM API request.

🚀 Key Changes

backend/services/aiService.js

• Added a getNormalizedCacheKey helper using crypto.createHash('sha256').

• Normalizes user queries by:

  • Converting text to lowercase.

  • Removing punctuation.

  • Normalizing whitespace.

  • Generating a deterministic SHA-256 cache key.

• Queries such as:

  • What is a batch?

  • what is a batch

  • What is a batch

  now resolve to the same cache key.

• Added Redis caching to:

  • chat

  • chatStream

  • chatWithBatchContext

• Successfully generated AI responses are stored in Redis with a 24-hour TTL:

  redis.set(cacheKey, response, "EX", 86400)
  

• Added graceful fallback behavior:

  • Redis available → use cache.

  • Redis cache hit → return cached response.

  • Redis cache miss → call LLM and cache the response.

  • Redis unavailable/error → continue using the live LLM API without breaking chatbot functionality.

🧠 Cache Safety

Dynamic queries that depend on real-time database information are not cached.

For example, queries involving tools such as search_batch bypass the cache to ensure users always receive the latest supply-chain and batch information from MongoDB.

This prevents stale operational data from being returned to users.

🌊 Streaming Support

Cached responses are also supported by the existing streaming implementation.

When a cached response is found, the response is written sequentially using the existing streamText helper so that the API maintains the expected streaming behavior.

⚡ Expected Benefits

Metric | Before | After -- | -- | -- Repeated query latency | ~2000ms+ | <50ms LLM API calls | Every request | Cache misses only API credit consumption | High | Reduced Repeated-query scalability | Limited | Improved

💰 Cost Optimization

Frequently repeated chatbot questions no longer require a new Gemini/OpenAI API request.

This can substantially reduce:

• Gemini/OpenAI API usage

• API credit consumption

• Backend processing overhead

• Network round trips

• Response latency for repeated questions

✅ Verification

• Added normalized SHA-256 cache key generation.

• Added Redis lookup before LLM requests.

• Added 24-hour cache expiration.

• Added caching to chat.

• Added caching to chatStream.

• Added caching to chatWithBatchContext.

• Verified cached streaming responses work with the existing streamText helper.

• Verified dynamic search_batch tool calls bypass caching.

• Verified Redis failures gracefully fall back to the live LLM API.

• Existing chatbot behavior remains unchanged for cache misses.

🔗 Related Issue

Closes #1221

🏷️ Labels

enhancement performance cost-optimization backend redis GSSoC'26

[vercel]

@vivek0028 is attempting to deploy a commit to the Nitya Gosain's projects Team on Vercel.

A member of the Team first needs to authorize it.