Streaming

Set stream: true to receive a Server-Sent Events (SSE) stream. Each event contains a chunk of the response.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
from openai import OpenAI

client = OpenAI(
    base_url="https://api.bazaarlink.ai/v1",
    api_key="sk-bl-YOUR_API_KEY",
)

stream = client.chat.completions.create(
    model="anthropic/claude-sonnet-4.6",
    messages=[{"role": "user", "content": "Count to 10 slowly."}],
    stream=True,
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

SSE Format

1
2
3
4
5
6
7
data: {"id":"85e6f35c71a31f19-SJC","choices":[{"delta":{"content":"Hello"},"index":0}]}

data: {"id":"85e6f35c71a31f19-SJC","choices":[{"delta":{"content":" world"},"index":0}]}

data: {"id":"85e6f35c71a31f19-SJC","choices":[{"delta":{},"finish_reason":"stop","index":0}],"usage":{"prompt_tokens":10,"completion_tokens":4,"total_tokens":14}}

data: [DONE]

Usage in streaming

When streaming, usage data is returned in the final chunk before the [DONE] message, alongside a choices array with an empty delta and finish_reason: "stop".

Keep-alive & final chunk

Streams may include SSE comment lines and unknown or ping events as keep-alives; ignore them and parse data frames. A final data frame may include usage before data: [DONE].

1
2
3
4
5
6
7
: keepalive          <- SSE comment line — ignore, do NOT JSON.parse

data: {"id":"85e6f35c71a31f19-SJC","choices":[{"delta":{"content":"Hi"},"index":0}]}

data: {"id":"85e6f35c71a31f19-SJC","choices":[{"delta":{},"finish_reason":"stop","index":0}],"usage":{...}}

data: [DONE]

Stream Cancellation

Streaming requests can be cancelled by closing the client connection — e.g. calling AbortController.abort() or closing the stream object. BazaarLink stops relaying further chunks and cancels the outbound provider request as soon as the cancellation is received.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.bazaarlink.ai/v1",
  apiKey: "sk-bl-YOUR_API_KEY",
});

const controller = new AbortController();

const stream = await client.chat.completions.create(
  {
    model: "anthropic/claude-sonnet-4.6",
    messages: [{ role: "user", content: "Write a long story." }],
    stream: true,
  },
  { signal: controller.signal }
);

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);
}

// e.g. on a "Stop" button click:
controller.abort();

No charge for an interrupted stream

If a stream ends before the final usage chunk arrives — including a client-side cancellation — there is no token-count ground truth for that request, so BazaarLink refunds the full reservation. You are not billed for content already delivered to your client before the cancellation.

Not guaranteed to stop the provider instantly

Closing the connection stops BazaarLink from relaying and paying for further tokens right away, but whether the upstream provider halts generation on its own servers the instant the connection drops depends on that provider — some may keep computing briefly after disconnect.

Mid-Stream Errors

No choices field on error frames

If a failure happens after streaming has already started (e.g. the upstream connection drops), you get an SSE data frame shaped {error:{message,type,code}} instead of the usual {choices:[...]} — with no HTTP status change, since headers were already sent. Check for an error key before reading choices[0].delta, and bill only for tokens already streamed (partial billing applies).

1
2
3
4
5
data: {"id":"85e6f35c71a31f19-SJC","choices":[{"delta":{"content":"The capital of "},"index":0}]}

data: {"error":{"message":"The request could not be completed.","type":"upstream_error","code":502}}

data: [DONE]