perf(core): implement multi-level caching mechanism for expensive computations and data retrieval

[Diwakar-odds]

Summary

The system currently performs expensive data retrieval and computational tasks without an intermediate caching layer. Implementing an LRU (Least Recently Used) memory cache or a multi-level caching strategy will significantly improve response times and reduce redundant processing.

Motivation

Performance profiling typically reveals that repeated data fetching or re-computation is a primary bottleneck. A configurable caching layer allows us to optimize these hot paths and reduce load on primary data stores.

Acceptance Criteria

• Implement a generic Caching Interface (with .get(), .set(), .invalidate()).
• Provide an in-memory LRU cache implementation of this interface.
• Integrate the cache into a known high-latency or high-frequency read path.
• Add tests to ensure cache hits/misses function correctly and TTL is respected.

How to Test

• Run tests to verify caching behavior (hit vs miss).
• Observe performance metrics or logging to confirm the underlying data source is not hit repeatedly for cached items.

---

Assignment Request (L3/Core Feature)

I would like to be assigned to this issue. Below is my proposed implementation plan:

Step-by-Step Implementation Plan:

1. Interface Design: Create a standardized cache interface to abstract the caching backend.
2. LRU Implementation: Implement an in-memory LRU cache with TTL (Time-To-Live) support to prevent memory leaks and stale data.
3. Integration: Wrap an existing expensive service method or data retrieval function with a cache check.
4. Testing: Write tests simulating repeated calls and verifying that the backend is only invoked on cache misses or after TTL expiry.

Could you also please add the ECSoC26 label?