Skip to content

Move OpenAI Realtime control events to C++ writer #196

Description

@AgrawalAmey

Problem

The OpenAI Realtime token-streaming path is already in C++ (create_realtime_writerRealtimeDeltaFormatter → NDJSON). But 6-10 control/setup events per realtime response go through Python model_dump() + stdlib json.dumps via vajra/entrypoints/common/base/base_transport.py:29.

Call sites in vajra/entrypoints/openai/handlers/realtime/realtime_processor.py:128, 183, 197, 210, 243, 277, 283 — each does await send_event_callback(event.model_dump(exclude_unset=False)) which hits the Python serialization loop.

Small absolute impact per call; large aggregate under realtime-heavy workloads.

Fix sketch

Extend RealtimeDeltaFormatter (or add a sibling) to emit the control-event sequence from C++. Either:

  • Expose typed methods (WriteSessionCreated, WriteResponseDone, …) that emit the right NDJSON lines from pre-baked templates, or
  • Accept a small "control event" variant in DeltaOutput and let the formatter branch.

Tie-in with VJ-191 if we're also restructuring the C++ entrypoints directory.

Non-goals

  • Don't touch the token delta path — it already works.

Parent

Part of VJ-187.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions