Goal
Integrate Context Compiler into LiteLLM proxy using async_pre_call_hook to enforce deterministic conversational state before model execution.
Approach
- Intercept
/v1/chat/completions requests
- Extract latest user message
- Run compiler decision
Behavior:
- clarify → return string (short-circuit, no model call)
- update/state → inject compiled state as system message
- passthrough → forward request unchanged
Scope
- Minimal reference implementation
- No changes to core engine
- No grammar changes
- Optional use of cache for per-conversation state
Outcome
Demonstrate that Context Compiler can act as a drop-in reliability layer for LLM proxy infrastructure.
Goal
Integrate Context Compiler into LiteLLM proxy using
async_pre_call_hookto enforce deterministic conversational state before model execution.Approach
/v1/chat/completionsrequestsBehavior:
Scope
Outcome
Demonstrate that Context Compiler can act as a drop-in reliability layer for LLM proxy infrastructure.