Streaming

Set stream and the response arrives as a text/event-stream instead of one JSON body. Each API streams in its own native format, so the streaming code you already have keeps working: Chat Completions sends chat.completion.chunk frames, Responses sends typed response.* events, Messages sends Anthropic's message_start … message_stop sequence.

Works with

Every generation model, on all three APIs. Per-model differences in tool-call streaming and usage frames are listed below and in the capability matrix.

Request

Chat Completions

stream = client.chat.completions.create(
    model="mindshub_air",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Responses

stream = client.responses.create(
    model="mindshub_air",
    input="Count to five.",
    stream=True,
)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

Messages

with client.messages.stream(
    model="mindshub_air",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Count to five."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Chat Completions

const stream = await client.chat.completions.create({
  model: "mindshub_air",
  messages: [{ role: "user", content: "Count to five." }],
  stream: true,
  stream_options: { include_usage: true },
});
for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}

Responses

const stream = await client.responses.create({
  model: "mindshub_air",
  input: "Count to five.",
  stream: true,
});
for await (const event of stream) {
  if (event.type === "response.output_text.delta") process.stdout.write(event.delta);
}

Messages

const stream = client.messages.stream({
  model: "mindshub_air",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Count to five." }],
});
for await (const text of stream.textStream) process.stdout.write(text);

The samples assume a client built as shown in Getting started. Guard on chunk.choices being non-empty in the Chat Completions loop: the stream ends with a usage-bearing chunk that carries no choices.

Long-running requests

Stream any request that can take longer than about a minute and a half. A non-streaming request gets 110 seconds in total, counted from when it reaches the API, failover attempts and web-search turns included, and then returns our own assembly_timeout error. A stream is not held to that limit.

The limit comes from the edge in front of the API, which closes a connection that sends nothing for 125 seconds. A stream keeps the connection busy in two ways:

• While the model thinks, the stream sends keepalives. A reasoning model can go minutes without writing anything. After 15 seconds without a frame, the stream sends an SSE comment line, : keepalive, and repeats it until output resumes. The SSE format tells every parser to skip lines that start with :, and the OpenAI and Anthropic SDKs do. A hand-written parser has to skip them too. A client that times out when no event arrives, rather than no bytes, can still close a stream while the model is quiet: Codex does after five minutes unless you raise its timeout (see Codex).
• A slow start gets its 200 early. If the response hasn't started after 90 seconds, the 200 and SSE headers go out anyway, followed by keepalives until it does. This happens when the upstream is slow to answer, or while the server-side search loop runs on Fireworks-hosted models and Kimi. After that point a failure arrives inside the stream, as described for each API below, and the response carries none of the X-MindsHub-* headers. Your SDK does not retry a failure that arrives inside a stream, so retry it yourself where you would have retried the same failure's status (see Streaming failures).

A stream can still end early in two ways, and both cut it off without its final event, so treat a stream that ends that way as failed (see Streaming failures). Once its response has started, a stream that sends nothing but keepalives for 30 minutes is ended, whether the model is still thinking or its provider has stalled. And a stream that runs longer than ten minutes can be cut if the server holding it restarts, for example during a deploy.

Response

Chat Completions

A series of data: <json> lines, each carrying a chat.completion.chunk, terminated by data: [DONE]:

data: {"id":"chatcmpl-...","choices":[{"delta":{"content":"","role":"assistant"},"index":0}],"created":1785401212,"model":"haiku","object":"chat.completion.chunk"}

data: {"id":"chatcmpl-...","choices":[{"delta":{"content":"Hello"},"index":0}],"created":1785401212,"model":"haiku","object":"chat.completion.chunk"}

data: {"id":"chatcmpl-...","choices":[{"delta":{},"finish_reason":"stop","index":0}],"created":1785401212,"model":"haiku","object":"chat.completion.chunk"}

data: [DONE]

Differences from OpenAI's stream that your client has to handle:

• Keys with null values are omitted rather than sent as null. Non-terminal chunks have no finish_reason key at all.
• Usage arrives on request. Send stream_options: {"include_usage": true} and the stream ends with a usage-bearing chunk whose choices array is empty. kimi streams end with that chunk even without asking, and may carry extra provider fields (system_fingerprint, usage nested in the finish chunk); don't use a strict parser that rejects unknown fields.
• Errors before the stream opens are plain JSON. If the request fails before generation starts, you get an ordinary JSON error response with an error status, not an SSE stream, even though you asked for stream: true. The exception is a request whose response took more than 90 seconds to start: its 200 has already gone out, so the failure arrives as a data: frame whose object has an error key, and the OpenAI SDKs raise APIError on it.
• A mid-stream error closes the stream without an SSE error event. A stream that ends without a finish_reason chunk was a failed generation, except on Gemini models, which always send one; see Errors → Streaming failures.

Responses

Typed events, with event: names matching the payload type:

event: response.created
event: response.in_progress
event: response.output_item.added
event: response.content_part.added
event: response.output_text.delta      (repeated)
event: response.output_text.done
event: response.content_part.done
event: response.output_item.done
event: response.completed

Tool calls stream as response.function_call_arguments.delta events between an output_item.added and output_item.done pair for the function_call item. A call to a freeform custom tool arrives as a custom_tool_call item instead: its text comes in one response.custom_tool_call_input.delta, then response.custom_tool_call_input.done, once the call is complete.

Provider refusal explanations stream as response.refusal.delta events followed by response.refusal.done, inside a refusal content part. A refused turn terminates with response.incomplete and incomplete_details.reason: "content_filter". When the provider supplies no explanation, no refusal text events are emitted; the terminal reason still identifies the refusal.

Every event carries a monotonic sequence_number, so a client can detect a gap, and output items are strictly sequential: item N is closed before item N+1 opens. Text that resumes after a tool call opens a second message item rather than reopening the first.

The final response.completed event carries the complete response object, including real token usage. Three behaviors to code around:

• A request that fails before generation starts returns a JSON error, not an SSE stream, even with stream: true. Check the response Content-Type before parsing. If the response took more than 90 seconds to start, the 200 has already gone out, and the failure arrives as response.created followed by response.failed. That event's response.error.code is the string code the same failure has as a JSON error, so it can fall outside the five listed below: one of ours, such as param_not_supported, or the upstream provider's own. A failure whose JSON error has no string code gets one of the five instead.
• A mid-stream failure is reported in-band as a response.failed event, because the HTTP status is already 200 by then. It carries response.error.code (rate_limit_exceeded, context_length_exceeded, insufficient_quota, invalid_prompt or server_error) plus whatever output was produced before the failure, marked incomplete. response.failed and response.completed are mutually exclusive.
• A truncated turn terminates with response.incomplete, not response.completed, with incomplete_details.reason set to max_output_tokens or content_filter.

Two of those error.code values, context_length_exceeded and insufficient_quota, are ours rather than OpenAI's, which defines neither. They are there because they are actionable. Clients that switch on the OpenAI enum should treat an unrecognized code as a generic retryable error, which is what Codex does. Note that the SDK's stream.get_final_response() raises on anything but response.completed, so read the terminal event directly if you need to handle a truncated turn.

Events we don't emit

The Responses protocol defines 53 event types; this API emits 15, including the refusal events described above, response.incomplete, and response.failed. The remaining events are not emitted, so a client waiting on one of them waits forever. The absences, and why:

Events	Why
response.audio.*, response.image_gen_call.*	Audio and image output aren't supported yet.
response.file_search_call.*, response.code_interpreter_call.*	Hosted tools this API doesn't run. Those tool entries are dropped from tools and named in X-MindsHub-Dropped-Params.
response.mcp_*	Remote MCP runs, but a stream does not narrate it yet: a non-streamed turn reports mcp_list_tools / mcp_call items in output; a streamed one emits no mcp_* events, so a streaming client has nothing to replay and the server's tool list is fetched again next turn.
response.queued	Nothing is ever queued; background requests are served synchronously.
response.web_search_call.*	A non-streamed turn reports its searches as web_search_call items in output. The streaming events are not emitted yet, so a streamed search is still silent.
response.reasoning_summary_*, response.reasoning_text.*	Reasoning is delivered as a single reasoning item at the end of the stream, not as deltas.
response.output_text.annotation.added	Citations arrive on the finished output_text part, not as incremental events. On a stream they land on the terminal response.completed event.

Messages

The standard Anthropic event sequence:

event: message_start → event: ping → event: content_block_start
→ event: content_block_delta … → event: content_block_stop
→ event: message_delta → event: message_stop

Differences from Anthropic's native stream:

• ping is sent once, right after message_start, not periodically. Don't use pings as a liveness signal. What keeps a silent stream open is the : keepalive comment line described in Long-running requests, which the SDKs skip.
• On non-Claude models, message_start reports input_tokens: 0. The real counts arrive in the final message_delta. Token-tracking UIs will undercount mid-stream on non-Claude models.
• A failure after the body opens arrives as an error event, typed exactly as the same failure would have been before the body opened, and the SDKs raise it. A mid-generation failure can still surface as a normal-looking completion with truncated or empty content; if your application must detect failed generations, sanity-check the output as well.
• Thinking blocks stream too, in Anthropic's required order, with their signature, so a streaming client can replay them on its next request. See Reasoning.

Per-model differences

• Tool-call streaming varies by model family. OpenAI- and Claude-family models stream a tool call as an opening delta (with id, type, and the function name) followed by argument fragments. Gemini models deliver each tool call as a single chunk carrying the complete arguments.
• Web search on Fireworks-hosted models and Kimi is not streamed live. Those models search through a server-side loop; the finished answer is replayed to you as a stream once the loop completes, so the first token arrives late. A loop that runs past 90 seconds gets its 200 early, with keepalives until the answer arrives.
• Gemini models always send a terminal frame, even on failure, so the "no finish_reason means failure" rule above does not apply to them.

Errors you can hit

A request rejected before generation (400, 401, 402, 404, 429) is a plain JSON error in the lane's envelope, not an SSE stream. The exception is a response that took more than 90 seconds to start: its rejection arrives inside the stream, as described in Long-running requests. Mid-stream failures are lane-specific, as described above. The full catalog is in Errors.