Streaming

Set stream to true on a chat completion request. The gateway returns Server-Sent Events and forwards provider chunks as they arrive.

bash

curl --no-buffer http://127.0.0.1:4000/v1/chat/completions \
  -H 'Authorization: Bearer local-gateway-key' \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"Hello"}]}'

The gateway does not retry after a stream has begun. Client cancellation is propagated to the provider request.

When a stream stalls, check the stream idle timeout and cancellation counters on /status or /metrics. Local development can use the separate opt-in goroutine and goroutineleak profiles; profiling is never enabled on the gateway listener by default.

Response headers are flushed before the first provider data chunk so clients can observe the stream boundary promptly. The gateway does not buffer the complete response.

Streaming responses include:

http

Content-Type: text/event-stream
Cache-Control: no-cache, no-transform
X-Accel-Buffering: no
Server: Go-Feather-Route
X-Request-ID: <request-id>

The request ID is generated by the gateway when the client does not provide one, forwarded to the provider, and included in structured request logs. The anti-buffering header is understood by common reverse proxies; the deployment proxy must still be configured to flush SSE responses promptly.

Streaming ​

Streaming