Design Live Comments

A cup final is streaming live and 5 million people are watching the same video. A goal goes in, and thousands of them type "GOAL" in the same second. Each viewer should see a lively stream of comments scroll past within a second or two, and someone who joins at half time should see what people were just saying.

The write side is small: a comment is a short line of text. The hard part is the read side: one write may have to reach millions of open connections, spread over a thousand servers, fast. A live video platform with a comment panel is the kind of product this question means. The steps follow A Framework for Any Design; try each "Your turn" box before you read on.

#Clarify the problem

• Functional: post a comment on a live video; see new comments as they arrive; see the most recent comments on joining; scroll back through older comments.
• Non-functional: new comments shown within 2 seconds; every posted comment stored durably; up to 5 million viewers on one video; viewers never see duplicates; the comment panel failing must not affect the video.
• Out of scope: replies, likes, recordings, the moderation console.

#Estimate the scale

• Comments at peak: 50 million viewers ÷ 600 seconds ≈ 83,000 comments a second.
• On the biggest video: 5 million ÷ 600 ≈ 8,300 a second.
• Daily average: 2 billion ÷ 86,400 ≈ 23,000 a second, so the peak is between three and four times the average.
• Storage: 2 billion × 200 bytes = 400 GB a day, about 146 TB a year before replication.
• Connections: 50 million open streams at an assumed 50,000 per server (load test it) is 1,000 realtime servers.

Suppose you show at most 10 comments a second on any video and each server sends every viewer one batch every 500 ms. The biggest video then costs 5 million × 2 = 10 million small messages a second spread over 1,000 servers: 10,000 each. That is a lot, but it no longer depends on how fast people type.

#Define the API

POST /videos/{id}/comments          body: { "text": "what a goal",
                                            "request_id": "r-51f" }
  201 { "comment_id": "7193...", "created_at": "..." }
  400 empty or too long   403 blocked or muted
  404 not live            429 posting too fast
 
GET /videos/{id}/live               Accept: text/event-stream
                                    Last-Event-ID: 4121 (on reconnect)
  id: 4122
  data: { "comments": [ { "comment_id": "7193...", "user": "ana",
                          "text": "what a goal" }, ... ] }
 
GET /videos/{id}/comments?before=7193...&limit=50
  200 { "comments": [...], "next_before": "7101..." }

The decision worth arguing is the transport. Viewers almost only listen, so use server-sent events rather than WebSockets: plain HTTP, friendly to proxies and load balancers, and browsers reconnect on their own, sending the last event id they saw. Posting is rare enough to be an ordinary POST, which gets authentication and rate limiting from the normal HTTP stack. A retry with the same request_id returns the first comment instead of posting it twice. The realtime communication lesson compares the options.

The history read pages with a cursor (before a comment id), not an offset. With 8,300 new comments a second, "page 2" would mean something different every time you asked.

#Model the data

comments                       -- wide-column store, about 200 B a row
  video_id     BIGINT
  bucket       INT             -- time bucket, sized per video
  comment_id   BIGINT          -- time-ordered 64-bit id
  user_id      BIGINT
  text         VARCHAR(200)
  status       TINYINT         -- visible | removed
  PRIMARY KEY ((video_id, bucket), comment_id DESC)
 
recent:{video_id}              -- in-memory list, last 200 shown
  [ { seq, comment_id, user, text }, ... ]

Writes are appends, reads are "newest N for a video, before an id", and nothing joins. A wide-column store such as Cassandra fits: partition by video, sorted by id, built for heavy writes. The time-ordered id gives newest-first order for free.

The bucket keeps partitions bounded. A quiet stream uses one-hour buckets. The biggest video at 8,300 a second would put 8,300 × 3,600 ≈ 30 million rows, 6 GB, in an hour's partition, so busy videos use one-minute buckets (500,000 rows, 100 MB). Store the bucket size on the video, so readers know which buckets to walk back through.

#High-level design

Writes and fan-out are split. The comment service makes a comment durable; the fan-out path decides what is shown and gets it to the right servers. The realtime servers only hold connections, so they can be restarted without touching storage.

#Deep dives

Interviewers push on getting one comment to millions of connections, very hot videos, and what "in order" and "no duplicates" really mean here.

#Fan-out: which server holds which viewer

A viewer's SSE connection lands on whichever of the 1,000 realtime servers the load balancer picks. A new comment on video 77 must reach exactly the servers that hold viewers of 77. Something has to know that mapping.

With pub/sub, the pub/sub layer is the registry: each channel lives on one pub/sub node (chosen by hashing the video id), and that node knows which servers subscribed. A dead realtime server's subscriptions vanish with its connection, so stale entries clean themselves up. The biggest video ends up subscribed by all 1,000 servers, which is fine at 10 comments a second: 10 × 1,000 = 10,000 messages a second from one pub/sub node. Unsampled, the same video would be 8,300 × 1,000 ≈ 8.3 million a second from that one node.

Each realtime server then batches. It collects what arrived for each video over 500 ms and writes one SSE event per viewer. Batching turns "10 comments a second × 5,000 local viewers" into 2 writes a second per viewer, and the 500 ms wait fits easily inside the 2-second goal.

#Very hot videos: sample, do not drop

When a video gets more comments than anyone can read, the panel needs a readable sample. The fan-out workers are the right place: the stream is partitioned by video id, so exactly one worker sees all of a video's comments and can count its rate.

How the worker samples: below the cap of 10 a second, everything passes. Above it, the worker always keeps comments from the creator and moderators, drops exact repeats ("GOAL" ×3,000 is noise), and fills the rest with a random pick from each 100 ms window. The poster always sees their own comment, because their client shows it as soon as the POST succeeds. The cost: on the biggest stream, almost everything people write is stored but never broadcast.

Joins spike too. If 1 million people join in the first minute, that is 1,000,000 ÷ 60 ≈ 17,000 recent-history reads a second, all for one key. Each realtime server reads the recent list for a video at most once a second and serves every joining viewer from that copy, so the cache sees at most 1,000 reads a second for the video.

#Ordering, duplicates and history

Be precise about what you promise. Comment ids come from many comment servers whose clocks differ by a few milliseconds, so ids are only roughly in posting order. Live order is set elsewhere: all of video 77's comments pass through one stream partition and one worker, which numbers them 4,121, 4,122, and so on. Pick a pub/sub that delivers one publisher's messages on one channel in order (most do over a single connection), and every viewer sees the same sequence. History is sorted by comment id, so it can differ from the live order by a few milliseconds' worth of comments; nobody notices.

Pub/sub stores nothing, so a server that is restarting misses messages. Reconnects fill the gap from the recent list using Last-Event-ID; if the gap is longer than the 200 comments kept (20 seconds at 10 a second), the client skips ahead, which is right for a live panel. A comment service that dies between the insert and the stream append leaves a comment stored but never shown live, which "a sample is fine" allows. A restarted worker replays from its last committed position, so a comment can arrive twice; clients drop any comment id they have already shown. "Never twice on screen" comes from that check, not from the network.

Older history comes from the store: before an id, newest first, walking back through buckets. Paging is rare compared with live reading, so it can go to the database.

#Bottlenecks and failures

At 10x (500 million viewers, 50 million on one stream), the realtime tier grows to 10,000 servers. Sampling keeps pub/sub load per video at 10 comments a second times the number of subscribed servers, so the hottest channel becomes 100,000 messages a second; add a relay layer so one pub/sub node feeds a few dozen relays that feed the servers. Storage grows to 4 TB a day, which a wide-column cluster handles by adding nodes, but the hottest video now sends 83,000 writes a second to one partition. Add a shard number from 0 to 15 to its partition key and have history reads merge the 16. The first thing to strain is the realtime servers' outbound bandwidth, because delivery cost grows with viewers, not with comments.

#What a strong answer sounds like

#Follow-up questions

#How would comments replay with the recording?

Store each comment's offset into the video when it was posted. During playback the client fetches comments for the next minute of video by offset and shows each at its moment. That is a cacheable read, not a realtime problem.

#A moderator removes a comment that was already shown. What happens?

Mark it removed in the store and publish a small "remove 7193..." event on the video's channel. Clients delete it from the panel, and the recent list drops it.

#How would you add live heart reactions?

Reactions arrive far faster than comments and nobody needs each one. Count them per video per second at the fan-out worker and broadcast only the counts.

#How would you run this across regions?

Give each region its own realtime servers and pub/sub. The worker owning a video publishes once to each region, so a comment crosses an ocean once, not once per viewer.

#How do you stop spam and bots?

Rate-limit posting per account and per IP, require a verified account to comment on big streams, and filter text before publishing. Sampling also mutes floods: 1,000 copies of one link are one repeat.