Streaming & Reasoning Trace

Setu delivers streaming tokens via standard HTTP Server-Sent Events (SSE). When using reasoning models like quantum-reasoner, thought process traces are streamed prior to the final conversational answer.

Protocol Structure

When "stream": true is passed to POST /chat/completions:

1. The HTTP response headers include:
   httpCopy

   Content-Type: text/event-stream; charset=utf-8
   Cache-Control: no-cache, no-transform
   Connection: keep-alive

2. Each event begins with data: followed by a JSON chunk.
3. The final frame is always:
   textCopy

   data: [DONE]

Chunk Format

{
  "id": "chatcmpl-setu-1",
  "object": "chat.completion.chunk",
  "created": 1728259200,
  "model": "quantum-lite",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": "Hello"
      },
      "finish_reason": null
    }
  ]
}

Reasoning Trace Chunking

For quantum-reasoner, initial chunks contain reasoning logic inside delta.reasoning_content:

{
  "choices": [
    {
      "index": 0,
      "delta": {
        "reasoning_content": "To solve 3x + 12 = 45, subtract 12 from both sides..."
      }
    }
  ]
}

Once reasoning concludes, standard delta.content chunks stream the final response.