Why Streaming Changes How You Build LLM-Powered Interfaces

When I started building on the Anthropic API, the first thing I had to stop treating as a detail was streaming. It's easy to prototype an LLM feature with a standard request-response cycle — send a message, wait, render the result. It works fine until a real user is on the other end staring at a blank screen for three seconds.

Streaming is the difference between a chatbot that feels alive and one that feels like a form submission.

What Streaming Actually Is

Streaming generates and delivers text incrementally as the model produces it, rather than waiting for the full response to be ready. The transport mechanism is Server-Sent Events (SSE) — a one-way channel where the server pushes data to the client as it's available.

From a product perspective, this matters for two reasons:

Perceived latency drops dramatically. Users see the first token in under a second instead of waiting for the full response. Even if the total generation time is the same, the experience feels faster because something is happening immediately.

You can cancel early. If the model starts going in the wrong direction, the user (or your system) can cut the stream and save tokens. In agentic workflows where model calls chain together, this adds up.

When should you not use streaming? Backend pipelines where a machine is consuming the output, data extraction tasks where you need the complete structured response before you can do anything, and automated agent workflows where partial state is worse than no state. Streaming is a mechanism for incrementally delivering content — it's not always the right one.

The Bridge Problem

Here's where it gets interesting at the implementation level. The Anthropic SDK exposes streaming as an async iterator — it gives you chunks as they arrive through a for await loop. That's great for server-side code. But a Response object in a Next.js API route expects a ReadableStream, not an async iterator. They're not the same thing.

The solution is a ReadableStream wrapper that bridges the two:

const readable = new ReadableStream({
  async start(controller) {
    for await (const chunk of stream) {
      if (
        chunk.type === 'content_block_delta' &&
        chunk.delta.type === 'text_delta'
      ) {
        controller.enqueue(chunk.delta.text);
      }
    }
    controller.close();
  },
});

What this does: ReadableStream maintains an internal queue. As chunks arrive from the Anthropic stream, we filter for the ones we care about (content_block_delta events with text_delta type) and push them into that queue via controller.enqueue(). The client reads from the queue via reader.read(), which suspends when the queue is empty and resumes when a new chunk arrives. When controller.close() is called, read() returns done: true on the next call.

It's a producer/consumer pattern — the Anthropic stream is the fast producer, the client is the slower consumer, and ReadableStream is the buffer between them.

Understanding the Stream Structure

Every Anthropic streaming response follows the same envelope pattern:

message_start
  content_block_start
    content_block_delta
    content_block_delta
    ...
  content_block_stop
  content_block_start
    content_block_delta
    content_block_delta
  content_block_stop
message_stop

message_start carries the outer metadata — model, token usage, stop reason. The actual content lives in content_block_delta events. For basic text responses, you only need to track content_block_delta with type: 'text_delta'. Once you add tool use to the mix (covered in the next post), content_block_start becomes critical because it tells you what type of block is opening — text or a tool call.

Why This Matters for Production Systems

The ReadableStream wrapping pattern isn't just a TypeScript quirk — it's a concrete example of a problem that comes up constantly in applied AI work: the model's output format doesn't match what your application layer expects, and you need an adapter layer between them.

That adapter layer — whether it's a stream bridge, a JSON parser, or a response transformer — is often where the real engineering work happens in LLM-powered products. The model call itself is straightforward. Making its output integrate cleanly with the rest of your stack is not.