Real-time updates

  • Lesson 23 of 40
  • 10 min
  • Beginner
  • 10 steps

In a chat, a live score or a notification, the news starts on the server. The challenge is getting it to the client without the client asking over and over.

Message delay
30 ms
Requests per user / min
0
Empty responses
0%
Open connections
1M
AliceBobServer 1chatServer 1Server 2chatServer 2Pub/subchannelsPub/submessagespresence1 new message for Aliceanything new?Bob sendsthe server cannot call15%GET /messages · every 2 sBob sends94%500,000 req/s · 90% emptyone of 1,000,000GET /messages · every 2 sThe billusers ÷ interval1,000,000 ÷ 2 srequests500,000 / sempty (90%)450,000 / sexample values20%waits until there is newsGET /messages · open up to 30 s15%new message from Bobevent stream (SSE)Bob sends with POST15%new message from BobWebSocket · both waysWebSocketAlice's socketBob's socketnew message from Bobpublish to "alice"no socket for Aliceoffline · subwaysaved: 1042, 1043saved firstalice · seen 4 s agoonlineheartbeat every 30 sHow the news arrivesPollingup to the intervalclient asksLong pollinglowserver waitsSSElowserver → clientWebSocketlowestboth waysdelay and direction of each technique
  • request
  • response
  • slow request
  • health check

Step 1 of 10

Problem

Bob writes. How does Alice find out?

Bob's message is already on the chat server. Ideally it shows up on Alice's screen right away.

But HTTP works by request and response: the client always goes first. The server has no way to open a conversation with Alice's app to say "you have a message".

Step 2 of 10

Technique

Polling: ask again and again

The simplest fix: Alice's app calls GET /messages every 2 seconds. Most answers come back empty; now and then one carries the message.

It is built from ordinary requests, but a message can take up to 2 s to show. If a message arrives every 20 s (example value), only 3 of the 30 requests per minute bring something: 90% come back empty.

What changed

  • Alice asks every 2 s

Step 3 of 10

Problem

A million phones asking

Now multiply. 1,000,000 users asking every 2 s make 500,000 requests per second, and 450,000 of them (90%) answer "nothing new".

You would pay for a fleet of servers whose main job is saying no. And if you shorten the interval so messages arrive sooner, the bill gets worse.

What changed

  • Users: 1 → 1,000,000 (example)
  • 500,000 requests/s, 90% empty

Step 4 of 10

Solution

Long polling: the server holds the answer

Instead of answering "nothing" at once, the server keeps the request open until there is a message or about 30 s pass. Then it responds and the app asks again straight away.

The message arrives as soon as it exists (about 100 ms in the example) and, with one message every 20 s, every request ends with something: 3 per minute and none empty. In exchange, each user keeps a connection open nearly all the time.

What changed

  • The server holds each request until there is news
  • Delay: up to 2 s → about 100 ms
  • Empty responses: 90% → 0%

Step 5 of 10

Technique

Server-Sent Events: a response that never ends

With SSE the app makes a single request and the server never closes it: it answers with Content-Type: text/event-stream and writes each event into that stream as it happens.

It goes one way (server → client), it is plain HTTP and the browser reconnects on its own with EventSource. To send, Alice still uses ordinary requests.

What changed

  • One open event stream to Alice
  • Delay: about 100 ms → about 50 ms (example)

Step 6 of 10

Technique

WebSockets: talk in both directions

A WebSocket starts as an HTTP request with Upgrade: websocket; the server accepts with 101 Switching Protocols and the connection stays open both ways.

Either side can send at any moment, with very few extra bytes per message. Alice and Bob each hold a socket and messages cross the server in milliseconds.

What changed

  • Connections open in both directions
  • Sending no longer needs separate requests

Step 7 of 10

Concept

At scale: which server has Alice?

Every open connection takes memory, so a million users are spread across many servers: for example, 20 with 50,000 connections each. Now Bob's socket is on server 2 and Alice's on server 1: server 2 cannot hand her anything.

The way out is pub/sub: server 2 publishes to Alice's channel, and the server holding her socket is subscribed to that channel and forwards it. Watch the message go the long way round.

What changed

  • Server 2
  • Pub/sub between servers
  • Each message reaches the server holding the recipient's socket

Step 8 of 10

Problem

Alice goes offline

Alice walks into the subway, her socket drops and Bob's messages have nobody to reach.

That is why every message is saved first and only then pushed if the recipient is connected. When she is back, Alice's app says "the last one I saw is 1041" and asks for everything after it. If she stays offline, the phone platform's push notification service alerts her.

What changed

  • Alice offline
  • Messages are saved first
  • On reconnect she asks for everything after 1041 · push alert

Step 9 of 10

Technique

Presence: who is online

The green dot next to a name is presence: each app sends a small heartbeat over its socket every 30 s and the server records "last seen". No heartbeat for a minute and the user turns offline.

With a million users that is about 33,000 heartbeats per second (1,000,000 ÷ 30 s). Keep presence in a fast store with keys that expire on their own, and send changes only to people looking at that contact.

What changed

  • Heartbeat every 30 s
  • Presence store with expiring keys

Step 10 of 10

Summary

Recap

Polling asks every so often; long polling leaves the question open; SSE pushes events from server to client; WebSocket opens a channel both ways.

Open connections force you to spread users across servers, joined by pub/sub. Save every message before delivering it so whoever was offline can catch up.

You will learn

  • Why plain HTTP gives the server no way to speak first
  • How polling and long polling differ
  • When to pick Server-Sent Events and when WebSockets
  • How to spread millions of open connections
  • What to do with offline users and how to know who is online