Documentation
Documentation · OpenAI Realtime
OpenAI

Realtime API

Use OpenAI's Realtime API — speech-to-speech, text, image input, tool calling and transcription sessions — from your app with one SDK call to start the session. Everything after that is exactly what OpenAI documents. The gate meters every model response and enforces your usage limits for the length of the session.

Rate limiting guide Analytics guide All SDKs

Start a session

Follow OpenAI's WebRTC guide for your platform, with one change: where it tells you to fetch an ephemeral key and post your SDP offer to OpenAI yourself, call createRealtimeCall instead. It takes your offer and the same session configuration you would send to OpenAI (model, voice, instructions, tools) and returns the SDP answer to apply to your peer connection. Use whichever WebRTC library you already use.

Swift

let offer = try await peerConnection.offer(for: constraints)
try await peerConnection.setLocalDescription(offer)

let answer = try await gateAIClient.createRealtimeCall(
    offerSDP: offer.sdp,
    session: ["type": "realtime", "model": "gpt-realtime"],
    additionalHeaders: ["X-App-Feature": "voice-assistant"]
)
try await peerConnection.setRemoteDescription(
    RTCSessionDescription(type: .answer, sdp: answer.sdp)
)

Kotlin

val answer = client.createRealtimeCall(
    offerSdp = offer.description,
    sessionJson = """{"type":"realtime","model":"gpt-realtime"}""",
    additionalHeaders = mapOf("X-App-Feature" to "voice-assistant")
)
peerConnection.setRemoteDescription(
    observer,
    SessionDescription(SessionDescription.Type.ANSWER, answer.sdp)
)

TypeScript (React Native, Capacitor)

const offer = await peerConnection.createOffer();
await peerConnection.setLocalDescription(offer);

const answer = await client.createRealtimeCall({
  offerSdp: offer.sdp,
  session: { type: 'realtime', model: 'gpt-realtime' },
  context: { appFeature: 'voice-assistant' },
});
await peerConnection.setRemoteDescription({ type: 'answer', sdp: answer.sdp });

Dart

final answer = await client.createRealtimeCall(
  offerSdp: offer.sdp!,
  session: {'type': 'realtime', 'model': 'gpt-realtime'},
  context: const GateAIContext(appFeature: 'voice-assistant'),
);
await peerConnection.setRemoteDescription(
  RTCSessionDescription(answer.sdp, 'answer'),
);

From here on, use the session as OpenAI documents it: send and receive events on the data channel, add or remove audio tracks, call tools. Nothing about the session itself changes because it was started through a gate. The returned callId (rtc_…) is how the session appears in your Gate/AI logs, so keep it if you want to correlate a session with what the user experienced.

What is different from OpenAI's guide

  • No ephemeral keys. OpenAI's guide has your backend mint an ephemeral client secret that the app then uses to talk to OpenAI directly. Through a gate that step is unnecessary — the gate authenticates the device and holds your key — and POST /v1/realtime/client_secrets is refused with 403 (realtime_client_secrets_blocked). A session started with an ephemeral key would run outside your usage limits and never appear in your analytics.
  • The session can be ended by the gate. When a device's token budget runs out mid-session, or at the 60-minute maximum, the gate ends the session. Your app sees the peer connection disconnect and the data channel close; treat that as "the session ended". The next createRealtimeCall returns the structured 429 described in the rate limiting guide, with the window and reset time to show the user.
  • WebRTC only. The WebSocket transport (wss://api.openai.com/v1/realtime) and SIP are not available through a gate. WebRTC is OpenAI's recommended transport for client apps and handles audio far better than WebSocket on mobile networks.

Usage and cost

One request per model response

Each time the model finishes a response, Gate/AI records a request on /v1/realtime/calls with the tokens OpenAI reported for it, carrying the same device, app version, feature and user fields as the request that started the session. A five-minute conversation shows up as a handful of rows sharing one call ID, so Logs and Analytics work for realtime sessions exactly as they do for everything else.

Audio and text priced separately

Token counts keep the audio, text and cached split OpenAI reports, and cost uses the per-modality rates from the model price catalog — audio tokens cost substantially more than text. If a model has no audio rates yet, its realtime rows show a dash for cost until the rates are added.

Usage limits

Starting a session counts as one request against the device's request windows and the gate rate limit, like any other request. Token budgets apply for the whole session, not just at the start:

  • After every model response the device's token windows are re-checked. If one is exhausted the session is ended and a rate-limited request is logged with the window's code (for example device_daily_tokens_exceeded), so the cutoff is visible in Logs next to the responses that led to it.
  • Every session ends after 60 minutes, OpenAI's own maximum.
  • If the user's tier has no token limit, sessions run until the user ends them or the 60-minute cap. Set a token budget on the tier if a runaway session would be a problem for your bill.

Supported endpoints

Request Through a gate
POST /v1/realtime/calls Supported. This is what createRealtimeCall sends; you can also post the multipart body yourself with any HTTP client.
POST /v1/realtime/client_secrets Refused with 403 and the code realtime_client_secrets_blocked. Start sessions with createRealtimeCall instead.
WebSocket /v1/realtime Not available. Use WebRTC.
SIP Not available.
POST /v1/audio/transcriptions, /v1/audio/speech Supported as ordinary requests, unchanged.

Realtime is available on OpenAI gates.