No LangChain integration; ChatOpenAI fallback drops custom response fields & rejects video

[Khurdhula-Harshavardhan]

Severity: Medium (strategic) — no LangChain integration ships; the OpenAI-wrapper path can't carry custom response fields or video

The SDK is a wrapper over openai; it does not provide a LangChain chat model. LangChain users must fall back to ChatOpenAI(base_url=...), where (source-verified against langchain master):

• Request precontext: works via ChatOpenAI(extra_body={"precontext": ...}) (raw JSON, no validation). Do not use model_kwargs (spread as SDK kwargs → TypeError on unknown params).
• Custom response fields (precontext, reasoning, vcache) are silently dropped — _create_chat_result cherry-picks only documented fields. Recover only by overriding _create_chat_result (both non-stream and the streaming chunk path).
• Video is rejected: a standard {"type":"video",...} block hits convert_to_openai_data_block, which has no video branch → ValueError: Block of type video is not supported. Only provider-native (unrecognized-type) dicts pass through verbatim.
• Inline <precontext>/<think> tags are not stripped by LangChain.

Recommendation

Ship a dedicated langchain-interfaze package (LangChain no longer accepts integrations as PRs — publish independently, scaffold via langchain-cli integration new, pass the standard ChatModelUnitTests/ChatModelIntegrationTests):

• If only video diverges from OpenAI wire: subclass BaseChatOpenAI — expose precontext as a typed field injected in _get_request_payload, override _create_chat_result for custom response fields, and intercept video content before the standard converter.
• If video needs a materially different request/response shape: implement BaseChatModel directly (Fireworks/Groq precedent), using openai for transport but owning message/content mapping. Note with_structured_output then requires implementing bind_tools.