Skip to content

feat: PoC OpenAI Realtime API#870

Open
n-o-u-r-h-a-n wants to merge 53 commits into
mainfrom
feat/poc-openai-realtime
Open

feat: PoC OpenAI Realtime API#870
n-o-u-r-h-a-n wants to merge 53 commits into
mainfrom
feat/poc-openai-realtime

Conversation

@n-o-u-r-h-a-n
Copy link
Copy Markdown
Contributor

@n-o-u-r-h-a-n n-o-u-r-h-a-n commented May 20, 2026 •
edited
Loading

Context

AI/ai-sdk-java-backlog#386.

Supported the Realtime API using the official OpenAI client.

Feature scope:

  • Added and supported client secret key endpoints.
  • Added and supported 4 call service endpoints.
  • Added tests for newly supported endpoints.

Definition of Done

  • Functionality scope stated & covered
  • Tests cover the scope above
  • Error handling created / updated & covered by the tests above
  • Aligned changes with the JavaScript SDK
  • Documentation updated
  • Release notes updated

rpanackal and others added 30 commits April 2, 2026 11:52
@rpanackal
Minimal new module setup including spec
27f810c
@rpanackal
Generation partial-success
5325a87
@rpanackal
Remove examples
b78995b
@rpanackal
Successfully filter by path
a9bd960
@rpanackal
Attach spec filter command
6488b26
@rpanackal
Initial setup
9b92cfe
@rpanackal
Successful PoC with OpenAI Models
2cbaa4e
@rpanackal
Version 1
65f4131
@rpanackal
Stable api
5039f23
@rpanackal
Change class name
fd7ebc2
@rpanackal
Rebase with main
2441aad
@rpanackal
fix dependency analyse issues
ce8e94a
@rpanackal
Initial draft untested
e27fcfa
@rpanackal
Second draft
cda636f 
- Streaming no fully enabled
@rpanackal
Successful E2E
44ca574
@rpanackal
Streaming initial draft
db16d95
@rpanackal
Streaming E2E with chat completion
bdca14c
@rpanackal
isStreaming check simplified
0a9e25e
@rpanackal
Cleanup PoC and rename module
a829b3a
@rpanackal
Reduce Javadoc verbosity
042755c
@rpanackal
Restrict to /responses api
a5be638
@rpanackal
Cleanup comments
e13d605
@rpanackal
Charles review suggestions
1931f0f
@rpanackal
Charles review - round 2 suggestions
47a396c
@rpanackal
Add dependency
6e174f1
@rpanackal
Mark openai dependency optional and new client @Beta
4e684be
@rpanackal
Javadoc on sample code for store
941fae0
@rpanackal
Update allowed endpoints with method check
127bc45
@rpanackal
feat: [OpenAI] An AiCore wrapper for Responses API only (#819)
e4febc5 
* Minimal new module setup including spec

* Generation partial-success

* Remove examples

* Successfully filter by path

* Attach spec filter command

* Initial setup

* Successful PoC with OpenAI Models

* Version 1

* Stable api

* Change class name

* Add tests

* fix dependency analyse issues

* Initial draft untested

* Second draft
- Streaming no fully enabled

* Successful E2E

* Streaming initial draft

* Streaming E2E with chat completion

* isStreaming check simplified

* Cleanup PoC and rename module

* Reduce Javadoc verbosity

* Restrict to `/responses` api

* Cleanup comments

* Charles review suggestions

* Charles review - round 2 suggestions

* Add dependency

* Mark openai dependency optional and new client `@Beta`

* Cleanup and no throw on missing model

* pmd

* Responses API complete

* ChatCompletionCreateParams throws without model. Needs rethink client API creation forModel

* Cleanup and close with test documenting limitation

* First draft responses only

* jacoco limits

* Remove ResponseService wrapper since remote API behaviour changed
@rpanackal
Sync and Async nested class option
21e6014
rpanackal and others added 16 commits April 16, 2026 10:48
@n-o-u-r-h-a-n n-o-u-r-h-a-n changed the title feat: PoC: OpenAI Realtime API feat: PoC OpenAI Realtime API May 20, 2026
@n-o-u-r-h-a-n
Copy link
Copy Markdown
Contributor Author

n-o-u-r-h-a-n commented May 22, 2026 •
edited
Loading

A clear understanding of what I've reached so far and implemented code accordingly:

The OpenAI Java SDK exposes all Realtime endpoints including SIP call control (accept, hangup, refer, reject) and client_secrets as plain HTTP POST requests, with no SIP protocol on the client side.

For example, when a phone call comes in, OpenAI's infrastructure hold the incoming SIP call open without answering it and notifies the backend with a call_id via webhook; the backend then calls client.realtime().calls().accept(callId, params), which our SDK turns into a POST /v1/realtime/calls/{callId}/accept HTTP request forwarded through SAP Cloud SDK's Apache HTTP client (the same pipeline as the Responses API) and proxied by AI Core to OpenAI.

OpenAI's infrastructure then connects the answered call to a Realtime AI session, and the actual WebRTC/SIP media flows directly between the user's device (browser/phone) and OpenAI.

What's supported now:

  • New AiCoreOpenAiClient.realtime() factory method exposing OpenAI SDK's RealtimeService
  • 5 realtime endpoints added to AiCoreHttpClientImpl allowlist
  • api-version=preview query param auto-injected by making AiCoreOpenAiClient use a dedicated buildRealtimeClientOptions() that adds the query parameter to ClientOptions, making the OpenAI SDK append it transparently to every realtime call.
  • Added 5 service methods in AiCoreOpenAiService.java.
  • WireMock unit tests for all 5 endpoints based on hardcoded json files.

Problems:

  1. E2E test: testCreateRealtimeClientSecret() in AiCoreOpenAiTest.java and the /realtimeClientSecret mapping in the index page when running the application are not working because there is no gpt-realtime deployment in the ai-sdk-java-e2e resource group on AI Core.

  2. Tests in AiCoreOpenAiClientTest.java are depending on hardcoded json files regarding call id, new target uri of call transfer, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@n-o-u-r-h-a-n @rpanackal @CharlesDuboisSAP