Context Management for Session in Case of Context Length Breach

[ashish-spext]

Confirm this is a new feature request

• I've checked the current issues, and there's no record of this feature request

Describe the feature

Problem:
After a certain number of messages, the context length of the LLM may be exceeded. This scenario is currently unhandled.

Proposed Implementations:

1. Simple Sliding Window Approach:
   • Continuously remove the oldest messages to make room for new ones as the context approaches its limit.
2. Smart Context Management with Vector Matching:
   • Implement a vector-based retrieval system to inject only the most relevant older messages (beyond a certain threshold) into the context.

Additional Context

No response