Documentation | ThinkMate

Implement real-time streaming AI responses for a better user experience.

Overview

This tutorial shows you how to:

Set up Server-Sent Events (SSE) streaming
Display streaming responses in real-time
Handle streaming errors
Show typing indicators

Why Streaming?

Streaming provides:

Instant feedback - Users see responses immediately
Better UX - Feels faster and more responsive
Progressive loading - Can show partial results

Step 1: Update Your Chat Component

Modify your chat component to use streaming:

"use client";

import { useState } from "react";

export default function StreamingChat() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState("");
  const [streamingContent, setStreamingContent] = useState("");
  const [isStreaming, setIsStreaming] = useState(false);

  const handleStream = async () => {
    if (!input.trim() || isStreaming) return;

    const userMessage: Message = { role: "user", content: input };
    setMessages((prev) => [...prev, userMessage]);
    setInput("");
    setIsStreaming(true);
    setStreamingContent("");

    try {
      const response = await fetch("/api/ai/stream", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          message: input,
          messages: messages,
        }),
      });

      if (!response.ok) {
        throw new Error("Stream failed");
      }

      const reader = response.body?.getReader();
      const decoder = new TextDecoder();

      if (!reader) {
        throw new Error("No reader available");
      }

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        const chunk = decoder.decode(value);
        const lines = chunk.split("\n");

        for (const line of lines) {
          if (line.startsWith("data: ")) {
            try {
              const data = JSON.parse(line.slice(6));
              
              if (data.done) {
                // Stream complete
                setMessages((prev) => [
                  ...prev,
                  { role: "assistant", content: streamingContent },
                ]);
                setStreamingContent("");
                setIsStreaming(false);
              } else {
                // Append chunk
                setStreamingContent((prev) => prev + data.content);
              }
            } catch (e) {
              // Invalid JSON, skip
            }
          }
        }
      }
    } catch (error) {
      console.error("Stream error:", error);
      setIsStreaming(false);
      setStreamingContent("");
    }
  };

  return (
    <div className="min-h-screen bg-base-200 p-4">
      <div className="max-w-4xl mx-auto">
        {/* Messages */}
        <div className="bg-base-100 rounded-lg p-4 mb-4 min-h-[400px] max-h-[600px] overflow-y-auto">
          {messages.map((msg, idx) => (
            <div
              key={idx}
              className={`flex ${
                msg.role === "user" ? "justify-end" : "justify-start"
              } mb-4`}
            >
              <div
                className={`max-w-[80%] rounded-lg p-3 ${
                  msg.role === "user"
                    ? "bg-primary text-primary-content"
                    : "bg-base-300 text-base-content"
                }`}
              >
                <p className="whitespace-pre-wrap">{msg.content}</p>
              </div>
            </div>
          ))}

          {/* Streaming content */}
          {isStreaming && streamingContent && (
            <div className="flex justify-start mb-4">
              <div className="bg-base-300 rounded-lg p-3 max-w-[80%]">
                <p className="whitespace-pre-wrap">{streamingContent}</p>
                <span className="inline-block w-2 h-4 bg-base-content ml-1 animate-pulse">
                  |
                </span>
              </div>
            </div>
          )}

          {/* Typing indicator */}
          {isStreaming && !streamingContent && (
            <div className="flex justify-start mb-4">
              <div className="bg-base-300 rounded-lg p-3">
                <span className="loading loading-dots"></span>
              </div>
            </div>
          )}
        </div>

        {/* Input */}
        <div className="flex gap-2">
          <input
            type="text"
            value={input}
            onChange={(e) => setInput(e.target.value)}
            onKeyPress={(e) => e.key === "Enter" && handleStream()}
            placeholder="Type your message..."
            className="flex-1 input input-bordered"
            disabled={isStreaming}
          />
          <button
            onClick={handleStream}
            disabled={isStreaming || !input.trim()}
            className="btn btn-primary"
          >
            {isStreaming ? "Streaming..." : "Send"}
          </button>
        </div>
      </div>
    </div>
  );
}

Step 2: Handle SSE Format

The streaming endpoint sends data in SSE format:

data: {"content":"Hello","done":false}
data: {"content":" world","done":false}
data: {"content":"","done":true}

Parse each line:

for (const line of lines) {
  if (line.startsWith("data: ")) {
    const data = JSON.parse(line.slice(6));
    // Process data.content and data.done
  }
}

Step 3: Add Error Handling

Handle streaming errors gracefully:

try {
  const response = await fetch("/api/ai/stream", { ... });
  
  if (!response.ok) {
    if (response.status === 403) {
      alert("Subscription required. Please upgrade your plan.");
    } else if (response.status === 400) {
      alert("Invalid request");
    } else {
      alert("Stream failed");
    }
    setIsStreaming(false);
    return;
  }
  
  // Continue with streaming...
} catch (error) {
  console.error("Stream error:", error);
  setIsStreaming(false);
  alert("Connection error. Please try again.");
}

Step 4: Add Cancellation

Allow users to cancel in-progress streams:

const [abortController, setAbortController] = useState<AbortController | null>(null);

const handleStream = async () => {
  const controller = new AbortController();
  setAbortController(controller);
  
  try {
    const response = await fetch("/api/ai/stream", {
      signal: controller.signal,
      // ... other options
    });
    // ... rest of streaming logic
  } catch (error) {
    if (error.name === "AbortError") {
      console.log("Stream cancelled");
    }
  }
};

const handleCancel = () => {
  if (abortController) {
    abortController.abort();
    setIsStreaming(false);
    setStreamingContent("");
  }
};

// Add cancel button
{isStreaming && (
  <button onClick={handleCancel} className="btn btn-error">
    Cancel
  </button>
)}

Step 5: Optimize Performance

Debounce Rapid Updates

import { useMemo } from "react";

// Only update UI every 100ms
const debouncedContent = useMemo(() => {
  return streamingContent;
}, [streamingContent]);

Auto-scroll to Bottom

import { useEffect, useRef } from "react";

const messagesEndRef = useRef<HTMLDivElement>(null);

const scrollToBottom = () => {
  messagesEndRef.current?.scrollIntoView({ behavior: "smooth" });
};

useEffect(() => {
  scrollToBottom();
}, [messages, streamingContent]);

Best Practices

Show typing indicator while waiting for first chunk
Handle connection errors gracefully
Allow cancellation of long streams
Auto-scroll to show latest content
Debounce updates for performance
Show progress for very long responses

Common Issues

Stream Stops Mid-Response

Solution: Check for network timeouts and handle reconnection

Chunks Arrive Out of Order

Solution: The SSE format ensures order, but verify your parsing logic

Memory Issues with Long Streams

Solution: Limit message history and clear old messages

Next Steps

Learn about Custom Prompts
Explore AI Components
Check out Subscription Access Control for access management