Building a Streaming AI Chat UI in React (with a Node.js Backend)

If you’ve ever wired up a normal request/response UI — call an API, show a spinner, render the result — you know that pattern cold. Streaming breaks it. Instead of one response landing all at once, you get dozens or hundreds of small chunks trickling in over a few seconds, and your UI has to look alive the entire time: text appearing character by character, the scroll position tracking the newest content, a “typing” indicator that has to disappear at exactly the right moment. Get it wrong and you end up with a chat window that jumps, flickers, or silently stalls when the connection drops mid-sentence.

I’ve shipped a couple of these now — most recently a support-bot UI backed by an LLM streaming endpoint — and the UI problems are almost never about the model. They’re about state management: partial tokens, scroll ownership, optimistic bubbles, and recovering gracefully when a stream just stops. This is the same territory I covered in my Socket.IO chat app post, and a lot of those patterns (message list virtualization, typing indicators, connection state) carry over directly. This time we’re doing it with Server-Sent Events instead of WebSockets, because SSE is the simpler, more common transport for one-directional AI token streams.

By the end of this article you’ll have a fully working streaming chat UI: a custom hook that consumes SSE from a Node backend, a message list with correct auto-scroll behavior, optimistic pending bubbles, retry logic for dropped streams, and accessibility wiring so screen reader users aren’t left out of the loop.

UI Architecture for a Streaming Chat Interface

Before writing code, let’s define the shape of the system. A streaming chat UI has three moving parts:

1. A minimal Node.js backend that proxies to an LLM (or any token-generating source) and re-emits tokens as an SSE stream.
2. A useChatStream hook on the client that owns the EventSource connection, buffers incoming tokens, and exposes clean state to components.
3. Presentational components — MessageList, MessageBubble, Composer — that are dumb by design. They render state; they don't fetch it.

Here’s the mental model I use for message state:

// types.ts
export type MessageRole = 'user' | 'assistant';

export type MessageStatus =
  | 'complete'   // done streaming, final content
  | 'streaming'  // actively receiving tokens
  | 'pending'    // optimistic bubble, no tokens yet
  | 'error';     // stream failed or was aborted

export interface ChatMessage {
  id: string;
  role: MessageRole;
  content: string;
  status: MessageStatus;
  createdAt: number;
}

Every assistant message goes through pending -> streaming -> complete (or -> error). That status field is what almost every UI decision hangs off of: whether to show a cursor blink, whether to auto-scroll, whether to render a retry button.

On the backend, keep it boring. Here’s a minimal Express + SSE endpoint that streams tokens from an upstream LLM call (I’m mocking the token source here — swap in your OpenAI/Anthropic streaming call and it plugs in the same way):

// server/chatStream.ts
import { Request, Response } from 'express';

// Simulates an LLM token stream. In production this would be
// `for await (const chunk of openai.chat.completions.create({ stream: true, ... }))`
async function* generateTokens(prompt: string): AsyncGenerator<string> {
  const words = `Sure — here's what I'd suggest for "${prompt}": start small, ship often, and measure before you optimize.`.split(' ');
  for (const word of words) {
    await new Promise((r) => setTimeout(r, 60 + Math.random() * 90));
    yield word + ' ';
  }
}

export async function chatStreamHandler(req: Request, res: Response) {
  const { prompt } = req.body ?? {};

  // Required SSE headers. Disable buffering on proxies (nginx) via X-Accel-Buffering.
  res.writeHead(200, {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache, no-transform',
    Connection: 'keep-alive',
    'X-Accel-Buffering': 'no',
  });

  const send = (event: string, data: unknown) => {
    res.write(`event: ${event}\n`);
    res.write(`data: ${JSON.stringify(data)}\n\n`);
  };

  const heartbeat = setInterval(() => res.write(': ping\n\n'), 15000);

  req.on('close', () => {
    clearInterval(heartbeat);
  });

  try {
    for await (const token of generateTokens(prompt)) {
      send('token', { token });
    }
    send('done', {});
  } catch (err) {
    send('error', { message: 'Stream failed upstream' });
  } finally {
    clearInterval(heartbeat);
    res.end();
  }
}

Two details worth calling out: the heartbeat comment (: ping) keeps intermediary proxies from killing an idle connection, and X-Accel-Buffering: no matters if you ever sit this behind nginx — without it, nginx buffers the whole response and your "streaming" UI just waits and dumps everything at once.

Note that native EventSource only supports GET requests with no custom body, so if your endpoint needs a POST body (like prompt above), you have two options: switch to fetch with a ReadableStream reader (shown as an alternative later), or encode the prompt as a query param for GET. I'll show the fetch-based approach in the hook below since it's more flexible for real AI chat use cases — it lets you send POST bodies while still parsing SSE-formatted output.

Consuming Server-Sent Events in React

This is the core piece. I’m using fetch + a ReadableStream reader instead of the EventSource API, because in practice almost every AI chat backend needs a POST body (the prompt, conversation history, auth headers), and EventSource can't send one.

// hooks/useChatStream.ts
import { useCallback, useRef, useState } from 'react';
import type { ChatMessage } from '../types';

interface UseChatStreamOptions {
  endpoint: string;
  onError?: (error: Error) => void;
}

interface UseChatStreamResult {
  messages: ChatMessage[];
  isStreaming: boolean;
  sendMessage: (content: string) => void;
  retryLast: () => void;
}

function createId() {
  return `${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
}

export function useChatStream({ endpoint, onError }: UseChatStreamOptions): UseChatStreamResult {
  const [messages, setMessages] = useState<ChatMessage[]>([]);
  const [isStreaming, setIsStreaming] = useState(false);
  const abortRef = useRef<AbortController | null>(null);
  const lastPromptRef = useRef<string>('');

  const updateMessage = useCallback((id: string, patch: Partial<ChatMessage>) => {
    setMessages((prev) => prev.map((m) => (m.id === id ? { ...m, ...patch } : m)));
  }, []);

  const streamFrom = useCallback(
    async (prompt: string, assistantId: string) => {
      const controller = new AbortController();
      abortRef.current = controller;
      setIsStreaming(true);

      try {
        const res = await fetch(endpoint, {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ prompt }),
          signal: controller.signal,
        });

        if (!res.ok || !res.body) {
          throw new Error(`Stream request failed: ${res.status}`);
        }

        const reader = res.body.getReader();
        const decoder = new TextDecoder();
        let buffer = '';
        let accumulated = '';
        let sawFirstToken = false;

        while (true) {
          const { value, done } = await reader.read();
          if (done) break;

          buffer += decoder.decode(value, { stream: true });

          // SSE frames are separated by a blank line ("\n\n").
          // A single read() can contain zero, one, or many complete frames,
          // and can also end mid-frame — hence the buffer.
          const frames = buffer.split('\n\n');
          buffer = frames.pop() ?? ''; // last chunk may be incomplete, keep it

          for (const frame of frames) {
            const lines = frame.split('\n');
            let event = 'message';
            let data = '';
            for (const line of lines) {
              if (line.startsWith('event:')) event = line.slice(6).trim();
              if (line.startsWith('data:')) data += line.slice(5).trim();
            }
            if (!data) continue;

            if (event === 'token') {
              const { token } = JSON.parse(data) as { token: string };
              accumulated += token;
              sawFirstToken = true;
              updateMessage(assistantId, { content: accumulated, status: 'streaming' });
            } else if (event === 'done') {
              updateMessage(assistantId, { status: 'complete' });
            } else if (event === 'error') {
              throw new Error('Upstream stream error');
            }
          }
        }

        if (!sawFirstToken) {
          // Stream closed with no tokens at all — treat as failure.
          throw new Error('Stream ended before any content arrived');
        }
      } catch (err) {
        if ((err as Error).name === 'AbortError') return; // user-initiated cancel
        updateMessage(assistantId, { status: 'error' });
        onError?.(err as Error);
      } finally {
        setIsStreaming(false);
        abortRef.current = null;
      }
    },
    [endpoint, onError, updateMessage],
  );

  const sendMessage = useCallback(
    (content: string) => {
      lastPromptRef.current = content;

      const userMsg: ChatMessage = {
        id: createId(),
        role: 'user',
        content,
        status: 'complete',
        createdAt: Date.now(),
      };

      // Optimistic assistant bubble — appears instantly, before any token arrives.
      const assistantMsg: ChatMessage = {
        id: createId(),
        role: 'assistant',
        content: '',
        status: 'pending',
        createdAt: Date.now(),
      };

      setMessages((prev) => [...prev, userMsg, assistantMsg]);
      void streamFrom(content, assistantMsg.id);
    },
    [streamFrom],
  );

  const retryLast = useCallback(() => {
    if (!lastPromptRef.current) return;
    setMessages((prev) => {
      const withoutFailed = [...prev];
      const lastAssistantIdx = [...withoutFailed]
        .reverse()
        .findIndex((m) => m.role === 'assistant' && m.status === 'error');
      if (lastAssistantIdx === -1) return prev;
      const realIdx = withoutFailed.length - 1 - lastAssistantIdx;
      withoutFailed[realIdx] = { ...withoutFailed[realIdx], status: 'pending', content: '' };
      void streamFrom(lastPromptRef.current, withoutFailed[realIdx].id);
      return withoutFailed;
    });
  }, [streamFrom]);

  return { messages, isStreaming, sendMessage, retryLast };
}

The buffering logic (buffer.split('\n\n'), popping the last incomplete chunk) is the part people get wrong most often. TCP doesn't care about your SSE frame boundaries — a single reader.read() call might hand you half a token event, or three full events glued together. Always treat the stream as raw bytes and reassemble frames yourself.

Handling Partial Tokens and Auto-Scrolling Message Lists

Now for the part that makes or breaks the feel of the UI: rendering the growing message and keeping the viewport pinned to the bottom — but only when the user hasn’t scrolled up to read history.

// components/MessageList.tsx
import { useEffect, useRef, useState } from 'react';
import type { ChatMessage } from '../types';
import { MessageBubble } from './MessageBubble';

interface MessageListProps {
  messages: ChatMessage[];
  onRetry: () => void;
}

const SCROLL_BOTTOM_THRESHOLD_PX = 80;

export function MessageList({ messages, onRetry }: MessageListProps) {
  const containerRef = useRef<HTMLDivElement>(null);
  const [autoScroll, setAutoScroll] = useState(true);

  // Track whether the user is anchored to the bottom. If they scroll up
  // to reread something, we stop yanking them back down on every token.
  const handleScroll = () => {
    const el = containerRef.current;
    if (!el) return;
    const distanceFromBottom = el.scrollHeight - el.scrollTop - el.clientHeight;
    setAutoScroll(distanceFromBottom < SCROLL_BOTTOM_THRESHOLD_PX);
  };

  useEffect(() => {
    if (!autoScroll) return;
    const el = containerRef.current;
    if (!el) return;
    // Runs on every render where content changed (i.e. every token),
    // so this needs to be cheap — direct scrollTop assignment, no smooth
    // scrolling, which would fight itself at high token rates.
    el.scrollTop = el.scrollHeight;
  }, [messages, autoScroll]);

  return (
    <div
      ref={containerRef}
      onScroll={handleScroll}
      className="message-list"
      role="log"
      aria-label="Conversation"
    >
      {messages.map((msg) => (
        <MessageBubble key={msg.id} message={msg} onRetry={onRetry} />
      ))}
      {!autoScroll && <ScrollToBottomButton onClick={() => setAutoScroll(true)} />}
    </div>
  );
}

function ScrollToBottomButton({ onClick }: { onClick: () => void }) {
  return (
    <button type="button" className="scroll-to-bottom" onClick={onClick}>
      ↓ New messages
    </button>
  );
}

A few things I learned the hard way:

• Don’t use scrollIntoView({ behavior: 'smooth' }) on every token. At 20+ tokens per second, queued smooth-scroll animations stack up and the list visibly stutters. Direct scrollTop assignment is instant and correct.
• Respect user scroll intent. The SCROLL_BOTTOM_THRESHOLD_PX check is what stops the classic "I scrolled up to read something and the UI yanked me back down" bug. This is directly lifted from the same pattern I used for the Socket.IO chat's "new message" indicator — same problem, same fix, different transport.
• Key on message id, never index. Since messages mutate in place (content grows on the same message object), an unstable key causes React to occasionally remount the bubble and lose any local animation state.

Optimistic UI While Waiting for the First Token

The gap between hitting “send” and the first token arriving is where a chat UI feels laggy if you don’t handle it. The sendMessage function above already pushes a pending assistant bubble immediately — here's how that renders:

// components/MessageBubble.tsx
import type { ChatMessage } from '../types';

interface MessageBubbleProps {
  message: ChatMessage;
  onRetry: () => void;
}

export function MessageBubble({ message, onRetry }: MessageBubbleProps) {
  const { role, content, status } = message;

  return (
    <div className={`bubble bubble--${role} bubble--${status}`}>
      <div className="bubble__avatar" aria-hidden="true">
        {role === 'user' ? '🧑' : '🤖'}
      </div>

      <div className="bubble__content">
        {status === 'pending' && <TypingIndicator />}

        {(status === 'streaming' || status === 'complete') && (
          <p>
            {content}
            {status === 'streaming' && <span className="cursor" aria-hidden="true" />}
          </p>
        )}

        {status === 'error' && (
          <div className="bubble__error">
            <p>Something went wrong generating this response.</p>
            <button type="button" onClick={onRetry}>
              Retry
            </button>
          </div>
        )}
      </div>
    </div>
  );
}

function TypingIndicator() {
  return (
    <div className="typing-indicator" aria-hidden="true">
      <span />
      <span />
      <span />
    </div>
  );
}

The pending status is what gives the app its "instant" feel — the bubble, avatar, and typing dots render on the same tick as the user's message, well before the network round trip completes. The blinking .cursor span during streaming is a cheap CSS trick (animation: blink 1s step-end infinite) that reinforces "this is still live" without any JS-driven interval.

One subtlety: don’t conflate pending (no tokens yet) with streaming (tokens arriving). They need different visuals — a typing indicator vs. a growing text block with a cursor — and collapsing them into one state is a common source of the awkward "text pops in all at once" bug when developers only branch on isLoading.

Reusing Real-Time Chat UI Patterns from a Socket.IO-Style App

If you’ve built the Socket.IO chat from my earlier post, most of this will look familiar, and that’s intentional — a streaming AI response and a live human message have the same UI requirements: append-only lists, presence/typing states, and connection status. The differences are narrower than they first appear:

Concern Socket.IO chat Streaming AI chat Transport Persistent bidirectional WebSocket One-shot SSE/fetch stream per request “Typing” signal Explicit typing event from peer Implicit — pending/streaming status Message arrival Whole message at once Same message, token by token Connection lifecycle Long-lived, reconnect on drop Short-lived per request, retry on failure

The MessageList and MessageBubble components above are close to drop-in replacements for their Socket.IO equivalents — same auto-scroll logic, same bubble layout. What changes is the connection-status hook. In the Socket.IO app I had a useSocketConnection hook exposing connected/reconnecting; here's the streaming equivalent, useful for showing a small status pill in the header:

// hooks/useConnectionStatus.ts
import { useEffect, useState } from 'react';

type ConnectionStatus = 'idle' | 'streaming' | 'error';

export function useConnectionStatus(isStreaming: boolean, hasError: boolean): ConnectionStatus {
  const [status, setStatus] = useState<ConnectionStatus>('idle');

  useEffect(() => {
    if (hasError) setStatus('error');
    else if (isStreaming) setStatus('streaming');
    else setStatus('idle');
  }, [isStreaming, hasError]);

  return status;
}

It’s simpler than the WebSocket version because there’s no persistent connection to babysit — each message is its own request/stream lifecycle, so “connection status” is really just “is this specific request still in flight.”

Error and Retry States for Dropped Streams

Streams die for mundane reasons: a flaky mobile connection, a proxy timeout, the upstream LLM API rate-limiting you mid-generation. Your UI needs to treat this as a first-class case, not an afterthought.

The useChatStream hook already sets status: 'error' on the assistant message when streamFrom throws (except on user-initiated abort), and retryLast re-sends the original prompt into the same message slot rather than creating a new bubble — that matters for UX continuity, since replacing the failed bubble in place feels far less jarring than appending a whole new attempt below it.

Let’s wire it up in the composer, including a way to actually cancel an in-flight stream (useful for a “Stop generating” button):

// components/Composer.tsx
import { FormEvent, useState } from 'react';

interface ComposerProps {
  onSend: (text: string) => void;
  onStop: () => void;
  isStreaming: boolean;
}

export function Composer({ onSend, onStop, isStreaming }: ComposerProps) {
  const [value, setValue] = useState('');

  const handleSubmit = (e: FormEvent) => {
    e.preventDefault();
    const trimmed = value.trim();
    if (!trimmed || isStreaming) return;
    onSend(trimmed);
    setValue('');
  };

  return (
    <form className="composer" onSubmit={handleSubmit}>
      <input
        value={value}
        onChange={(e) => setValue(e.target.value)}
        placeholder="Ask something…"
        disabled={isStreaming}
        aria-label="Message"
      />
      {isStreaming ? (
        <button type="button" onClick={onStop}>
          Stop
        </button>
      ) : (
        <button type="submit" disabled={!value.trim()}>
          Send
        </button>
      )}
    </form>
  );
}

To support “Stop”, expose the abort controller from the hook:

// addition to useChatStream.ts
const stopStreaming = useCallback(() => {
  abortRef.current?.abort();
}, []);

return { messages, isStreaming, sendMessage, retryLast, stopStreaming };

For network-level resilience beyond manual retry, add exponential backoff for transient failures (timeouts, 5xx) rather than surfacing an error immediately — this is worth doing on the server side too, but a light client-side retry saves a visible failure state for genuinely flaky connections:

// utils/retryWithBackoff.ts
export async function retryWithBackoff<T>(
  fn: () => Promise<T>,
  { retries = 2, baseDelayMs = 500 }: { retries?: number; baseDelayMs?: number } = {},
): Promise<T> {
  let lastError: unknown;
  for (let attempt = 0; attempt <= retries; attempt++) {
    try {
      return await fn();
    } catch (err) {
      lastError = err;
      if (attempt < retries) {
        await new Promise((r) => setTimeout(r, baseDelayMs * 2 ** attempt));
      }
    }
  }
  throw lastError;
}

Wrap the initial fetch call in streamFrom with this if your upstream is prone to transient 502s — but don't retry automatically once tokens have already started streaming. Partial content plus a silent restart is more confusing than just showing the error state and letting the user click Retry.

Accessibility Considerations for Live-Updating Content

This is the section most streaming chat tutorials skip, and it shows — screen reader users routinely get either dead silence or a firehose of every single token being announced. Neither is usable.

The core tool is an aria-live region, but it needs careful throttling. Announcing every token is unusable noise; announcing only on completion means screen reader users get no sense that something is happening at all. My approach: a visually-hidden live region that updates on a debounce, not on every token.

// components/LiveAnnouncer.tsx
import { useEffect, useRef, useState } from 'react';
import type { ChatMessage } from '../types';

interface LiveAnnouncerProps {
  messages: ChatMessage[];
}

const ANNOUNCE_DEBOUNCE_MS = 1200;

export function LiveAnnouncer({ messages }: LiveAnnouncerProps) {
  const [announcement, setAnnouncement] = useState('');
  const timerRef = useRef<ReturnType<typeof setTimeout> | null>(null);

  const last = messages[messages.length - 1];

  useEffect(() => {
    if (!last || last.role !== 'assistant') return;

    if (last.status === 'pending') {
      setAnnouncement('Assistant is responding…');
      return;
    }

    if (last.status === 'complete') {
      if (timerRef.current) clearTimeout(timerRef.current);
      setAnnouncement(last.content); // announce the full final answer immediately
      return;
    }

    if (last.status === 'error') {
      setAnnouncement('Response failed. A retry option is available.');
      return;
    }

    // While streaming: debounce so we announce a settled chunk of text,
    // not every individual token.
    if (timerRef.current) clearTimeout(timerRef.current);
    timerRef.current = setTimeout(() => {
      setAnnouncement(last.content);
    }, ANNOUNCE_DEBOUNCE_MS);

    return () => {
      if (timerRef.current) clearTimeout(timerRef.current);
    };
  }, [last?.content, last?.status]);

  return (
    <div
      aria-live="polite"
      aria-atomic="true"
      role="status"
      style={{
        position: 'absolute',
        width: 1,
        height: 1,
        overflow: 'hidden',
        clip: 'rect(0 0 0 0)',
        whiteSpace: 'nowrap',
      }}
    >
      {announcement}
    </div>
  );
}

Mount <LiveAnnouncer messages={messages} /> once, near the root of your chat view — not inside MessageList, so it doesn't get remounted when the list re-renders.

A few other accessibility details that matter in practice:

• role="log" on the scrollable message container (used in MessageList above) tells assistive tech this is a stream of sequential, appended content — more appropriate than a generic region.
• aria-live="polite", not "assertive". Assertive interrupts whatever the screen reader is currently doing, which is jarring for a background response. Polite waits for a pause.
• Disable the input during streaming (disabled={isStreaming} in Composer) and expose it via aria-label, so keyboard/screen-reader users get clear feedback that submission is unavailable, not just a silently ignored Enter key.
• Respect prefers-reduced-motion for the blinking cursor and typing-dot animations — wrap them in a media query and fall back to a static indicator.

@media (prefers-reduced-motion: reduce) {
  .cursor,
  .typing-indicator span {
    animation: none;
  }
}

Wrapping Up

You now have the full loop: a Node SSE endpoint that streams tokens with proper buffering headers, a useChatStream hook that parses raw SSE frames off a fetch ReadableStream, a message list that auto-scrolls without fighting the user, optimistic pending bubbles, retry/stop controls for dropped streams, and a debounced live region so the experience doesn't exclude screen reader users.

A few directions worth taking this next:

• Swap the mock token generator for a real LLM streaming call (OpenAI’s stream: true, Anthropic's streaming Messages API, or a self-hosted model) — the SSE plumbing on both ends stays identical, you're just changing what feeds generateTokens.
• Persist conversation history to a database keyed by session, and hydrate messages on mount instead of starting empty every load.
• Add markdown rendering to the assistant bubble content (react-markdown works well) — just make sure you're not re-parsing the full markdown tree on every single token; debounce the render the same way we debounced the live region.
• Rate-limit and authenticate the streaming endpoint — an open SSE route that proxies to a paid LLM API is an expensive thing to leave unprotected.

If you built the Socket.IO chat app from my earlier post, try running both side by side — you’ll notice how much of the UI layer (scroll handling, bubble rendering, typing states) is genuinely transport-agnostic. That’s the real takeaway here: streaming AI chat isn’t a new UI paradigm, it’s the same real-time chat patterns applied to a different kind of “typing” event.