Problem
The OpenAI Realtime token-streaming path is already in C++ (create_realtime_writer → RealtimeDeltaFormatter → NDJSON). But 6-10 control/setup events per realtime response go through Python model_dump() + stdlib json.dumps via vajra/entrypoints/common/base/base_transport.py:29.
Call sites in vajra/entrypoints/openai/handlers/realtime/realtime_processor.py:128, 183, 197, 210, 243, 277, 283 — each does await send_event_callback(event.model_dump(exclude_unset=False)) which hits the Python serialization loop.
Small absolute impact per call; large aggregate under realtime-heavy workloads.
Fix sketch
Extend RealtimeDeltaFormatter (or add a sibling) to emit the control-event sequence from C++. Either:
- Expose typed methods (
WriteSessionCreated, WriteResponseDone, …) that emit the right NDJSON lines from pre-baked templates, or
- Accept a small "control event" variant in
DeltaOutput and let the formatter branch.
Tie-in with VJ-191 if we're also restructuring the C++ entrypoints directory.
Non-goals
- Don't touch the token delta path — it already works.
Parent
Part of VJ-187.
Problem
The OpenAI Realtime token-streaming path is already in C++ (
create_realtime_writer→RealtimeDeltaFormatter→ NDJSON). But 6-10 control/setup events per realtime response go through Pythonmodel_dump()+ stdlibjson.dumpsviavajra/entrypoints/common/base/base_transport.py:29.Call sites in
vajra/entrypoints/openai/handlers/realtime/realtime_processor.py:128, 183, 197, 210, 243, 277, 283— each doesawait send_event_callback(event.model_dump(exclude_unset=False))which hits the Python serialization loop.Small absolute impact per call; large aggregate under realtime-heavy workloads.
Fix sketch
Extend
RealtimeDeltaFormatter(or add a sibling) to emit the control-event sequence from C++. Either:WriteSessionCreated,WriteResponseDone, …) that emit the right NDJSON lines from pre-baked templates, orDeltaOutputand let the formatter branch.Tie-in with VJ-191 if we're also restructuring the C++ entrypoints directory.
Non-goals
Parent
Part of VJ-187.