Skip to content

Restructure entrypoints directory (both C++ and Python) #198

Description

@AgrawalAmey

Problem

The entrypoints codebase has two structural mistakes that keep getting reinforced by new code:

1. Anthropic code lives inside openai/ on the C++ side

csrc/vajra/native/entrypoints/openai/AnthropicFormatter.{h,cpp} is obvious blasphemy — an Anthropic-specific formatter under openai/. Same goes for PassthroughProtocol.h (vendor-neutral infra that has nothing OpenAI about it), and the factory wiring in csrc/vajra/native/entrypoints/openai/Pybind.cpp now registers create_sse_anthropic_writer from inside the openai subdirectory.

This directly contradicts the commit 2fe95a639 ("Restructure vendor entrypoints as equal citizens") that already fixed the same mistake on the Python side. The C++ side never got the equivalent refactor, and new contributions (including the one I just shipped) keep cementing the wrong layout.

2. The top-level vajra/entrypoints/ directory mixes concerns at one level

Today it looks like:

vajra/entrypoints/
├── server.py          # HTTP server launcher
├── common/            # shared base classes, async_engine, lifecycle
├── middleware/        # HTTP middleware
├── openai/            # OpenAI vendor
├── anthropic/         # Anthropic vendor
└── debug/             # Debug endpoints

HTTP server concerns (server.py, middleware/, the async engine wrapper) are at the same level as vendor adapters. Anything "not HTTP" (future gRPC, native in-process, ZMQ/Aeron wire protocols) has no natural home.

Desired structure

C++ side

Mirror the Python commit: promote vendor code to peer directories, and split vendor-neutral infrastructure out.

csrc/vajra/native/entrypoints/
├── common/                     # PipeTransport, PipeReadEnd, PassthroughProtocol,
│                               # SSEProtocol, NDJSONProtocol, PlainTextProtocol,
│                               # PlainTextFormatter, StreamingSessionContext (common
│                               # fields only), RealtimeSessionContext if it stays
│                               # OpenAI-flavored
├── openai/
│   ├── ChatCompletionFormatter.{h,cpp}
│   ├── ResponsesFormatter.{h,cpp}
│   ├── RealtimeDeltaFormatter.{h,cpp}
│   └── FinishReasonUtils.h
├── anthropic/
│   ├── AnthropicFormatter.{h,cpp}
│   └── FinishReasonUtils.h     # Anthropic stop_reason mapping
└── Pybind.cpp                  # factories and registrations for all of the above

Updating csrc/vajra/native/entrypoints/openai/Pybind.cpp to csrc/vajra/native/entrypoints/Pybind.cpp touches the module registration in csrc/vajra/native/Pybind.cpp (line 61-72). CMake globs pick up .cpp files recursively, so no CMakeLists changes needed.

Python side

Introduce an http/ (or http_server/) subdir that owns HTTP-specific infrastructure. Vendor adapters and debug endpoints stay at the top level (equal citizens) but move their routes under http/. Non-HTTP entrypoints get a home.

vajra/entrypoints/
├── http/
│   ├── server.py               # was entrypoints/server.py
│   ├── async_engine.py         # was entrypoints/common/async_engine.py
│   ├── lifecycle.py            # was entrypoints/common/lifecycle.py
│   ├── middleware/             # unchanged
│   ├── base/                   # was entrypoints/common/base/
│   ├── openai/                 # was entrypoints/openai/
│   ├── anthropic/              # was entrypoints/anthropic/
│   └── debug/                  # was entrypoints/debug/
└── __init__.py

Or flatter:

vajra/entrypoints/
├── server.py                   # still the HTTP server launcher
├── http_common/                # merged "common" + "middleware"
├── openai/
├── anthropic/
└── debug/

Pick whichever the team prefers — the important invariant is that HTTP-server internals are distinguishable from vendor adapters, and vendor code is not nested under another vendor on either side.

Scope of the refactor

  • Pure file moves + import fixups — no behavior changes.
  • Update vajra-serve console script entry point.
  • Update csrc/vajra/native/Pybind.cpp to call the relocated InitPybindSubmodule.
  • Run make build && make test and the full entrypoint test suite.
  • Single commit, zero functional diff.

Acceptance criteria

  • No vendor name appears as a parent directory of another vendor's code on either side.
  • HTTP-specific code is clearly separated from vendor adapters in Python.
  • All tests pass. grep -r "entrypoints.openai.Anthropic" returns nothing.

Notes

  • This should land before VJ-185 / VJ-188 / VJ-191, because those issues all touch files that are about to move. Doing the restructure first avoids a painful rebase.
  • Relates to VJ-187 (HTTP server perf umbrella) — restructure is the prerequisite for clean perf work.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions