Design a Chat App

Ana sends Ben a message and his phone shows it a moment later, with one tick for sent, two for delivered and a colour change when he reads it. If Ben is in a tunnel, the message waits and arrives when he comes out, in the right place in the conversation.

Interviewers like it because it tests what request-and-response designs never touch: millions of open connections, routing to whichever server holds the recipient, ordering, and what "delivered" really promises. It builds on Realtime Communication, so read that first if WebSockets and connection servers are new. The steps follow A Framework for Any Design.

#Clarify the problem

"Chat app" can mean team chat or a phone messenger. Pin it down first.

• Functional: send and receive one-to-one and group messages (up to 200 members); get messages sent while offline; see history; sent, delivered and read receipts; online and last-seen status.
• Non-functional: an acknowledged message is never lost (durability); one agreed order per conversation; online delivery under 500 ms; 99.99% availability for sending; presence may be a few seconds stale.
• Out of scope: end-to-end encryption, search, media storage, multiple devices (see the follow-ups).

#Estimate the scale

• Messages: 50 million × 40 = 2 billion a day. 2 × 10⁹ ÷ 86,400 s ≈ 23,000 a second on average. Assume an evening peak of 3x: about 70,000 a second.
• Deliveries: one-to-one messages go to one person; group messages go to every other member. Assume 80% of messages are one-to-one and the rest reach 20 other group members on average: 0.8 × 1 + 0.2 × 20 = 4.8 deliveries per message. That is 23,000 × 4.8 ≈ 110,000 deliveries a second on average and about 330,000 at peak.
• Connections: assume 20% of daily users are online at the peak: 10 million open WebSockets. If one connection server holds 100,000 (an assumption: it depends on memory per connection, here about 10 KB, so 1 GB per server), that is 100 servers; plan for 200 so a failure or a deploy has room.
• Heartbeats: each client pings every 30 seconds so dead connections are noticed. 10 million ÷ 30 ≈ 330,000 pings a second: as many as peak deliveries. They must stay on the connection servers and never touch a database.

#Define the API

Client to server, over the WebSocket
  send     { client_msg_id: "c-7f3a", conv_id: "c_123", body: "lunch?" }
  ack      { conv_id: "c_123", up_to_seq: 1044 }       "I have everything up to 1044"
  read     { conv_id: "c_123", up_to_seq: 1044 }       "I have seen everything up to 1044"
  ping     every 30 s
 
Server to client
  sent     { client_msg_id: "c-7f3a", conv_id: "c_123", seq: 1044, ts: ... }
  message  { conv_id: "c_123", seq: 1044, sender: "ana", body: "lunch?", ts: ... }
  receipt  { conv_id: "c_123", user: "ben", kind: "delivered" | "read", up_to_seq: 1044 }
  presence { user: "ben", online: false, last_seen: ... }
  error    { client_msg_id: "c-7f3a", code: "not_member" | "too_large" | "rate_limited" }
 
HTTP
  GET  /v1/conversations/{conv_id}/messages?before_seq=1000&limit=50   history, newest first
  GET  /v1/sync?since={user_cursor}                                    what changed while offline
  POST /v1/conversations   { members: [...] }                          create a group
       403 not allowed, 404 no such conversation, 429 rate limited

The decision worth arguing is who names a message. The client names it with client_msg_id, an idempotency key, so a retried send is stored once. The server gives it a seq, its position in the conversation, and clients sort by that, never by their own clock. Acks and receipts say "up to seq N", so one small message confirms a whole batch.

#Model the data

messages                       -- about 200 bytes a row
  conv_id        BIGINT        -- partition key (with bucket)
  bucket         INT           -- month number, keeps partitions bounded
  seq            BIGINT        -- clustering key: order inside the conversation
  sender_id      BIGINT
  client_msg_id  VARCHAR(36)
  body           TEXT          -- up to 4 KB
  created_at     TIMESTAMP
  PRIMARY KEY ((conv_id, bucket), seq)
 
dedupe                         -- PRIMARY KEY (conv_id, client_msg_id) -> seq
                               --   kept 1 day; a retried send gets the stored seq back
conversations                  -- conv_id, kind (direct | group), members, last_seq
members                        -- PRIMARY KEY (user_id, conv_id):
                               --   last_delivered_seq, last_read_seq, last_activity

The access patterns are narrow: append to one conversation, 23,000 times a second, and read "the newest 50" or "everything after seq N" in one conversation. Nothing joins. That is the shape a wide-column store (such as Cassandra or HBase) is built for: rows sharing a partition key sit together on disk, sorted by the clustering key, so both reads are one sequential scan of one partition. A relational database works too, but at 146 TB a year you would shard it by hand.

The members table is keyed by user: it answers "which conversations does Ana have, and what has she read?" for the inbox screen and for sync.

#High-level design

Split the system by what each part remembers. Connection servers hold sockets, so they run no business logic and are deployed rarely. The chat service holds nothing and deploys freely. The session cache holds only who is connected where.

#Deep dives

#Connection servers and routing between them

With 10 million sockets on 100 to 200 servers, Ana and Ben are almost never on the same one.

Keep each registry entry honest with a TTL: the connection server refreshes all its entries every 30 seconds in one batch, and an entry expires after 90 seconds. If a server dies, its entries vanish within 90 seconds; anything routed there meanwhile fails over to the push path, and nothing is lost because the message is already stored.

Deploys are the quiet danger. Restarting a server drops 100,000 sockets at once. Drain servers one at a time, and make clients reconnect after a random delay: spread over 30 seconds, 100,000 reconnects is about 3,300 a second instead of a spike.

#Delivery guarantees and ordering

Weak answers say "exactly once" here. Delivery to a phone is at least once: the server resends anything not acknowledged, so a message may arrive twice when an ack is lost. The client drops any (conv_id, seq) it already has. At-least-once delivery plus dedupe on the receiver looks like exactly once on screen; client_msg_id does the same for retried sends.

The sequence must come from one place per conversation. Either route every send for c_123 to the same chat server (by consistent hashing on the conversation id) and let it keep the counter, or keep the counter in a store that supports an atomic increment or compare-and-set for a single key. Plain quorum reads and writes are not enough: two servers can both read 1043 and both write 1044. If a number is handed out and the write then fails, the gap is harmless; clients treat gaps as "fetch from the store".

#Offline delivery and sync

The store is the delivery mechanism; sockets and push notifications only say "there is something new". On reconnect the client calls /v1/sync. The server reads the user's conversation ids from members, looks up each one's last_seq in one batched read, returns those ahead of what the client holds, and the client pulls each missing range. After a week away, the client fetches the conversation list first and each chat's messages when it is opened.

Phone push notifications go through the platform's push service, which is best effort: it can delay or drop them, and it can collapse several into one. Nothing depends on them, so never treat "push sent" as "delivered". The same holds for a socket that died without anyone noticing yet: if Ben does not ack within about 10 seconds, send the push anyway. The notification system case study covers that pipeline.

#Receipts and presence

Receipts are cursors, not rows per message. Ben's delivered and read positions are two numbers in his members row. "Read up to 1045" marks every earlier message read with one write. In a one-to-one chat, push each cursor change to the other person. In a group of 200, pushing every member's read cursor to every other member is 200 × 199 ≈ 40,000 messages when everyone reads one message. Instead, show "read by 37" computed when someone opens the message info.

Presence quietly costs the most. Online status lives only in the session cache, with the same 90-second TTL, never in the database. The trap is fan-out. If each user goes online or offline 20 times a day, that is 50 million × 20 ÷ 86,400 ≈ 11,600 changes a second; pushing each one to 100 contacts would be 1.16 million presence messages a second, ten times the message deliveries. So push presence only to people who have that chat on screen; everyone else fetches it when they open the chat.

#Storage and the partition key

Without the month bucket, a busy conversation's partition grows forever on the same few nodes. A full group of 200 sending 40 messages each a day writes 200 × 40 × 200 B = 1.6 MB a day; over five years that one partition would hold about 3 GB. Bucketing by month caps it near 48 MB. Reading across a month boundary costs a second query.

#Bottlenecks and failures

At 10x (500 million daily users), peak deliveries reach about 3.3 million a second and stored writes about 230,000 a second on average. Connection servers grow to 1,000 or more. The store adds nodes; conversation ids hash evenly. The first thing to strain is the session cache, with 100 million entries refreshed every 30 seconds: shard it by user id. Presence fan-out is next, which is why it is limited to open chats.

#What a strong answer sounds like

#Follow-up questions

#How do you support several devices per user?

The session registry maps a user to a list of (device, server) pairs, and each device keeps its own delivered cursor. A message goes to every connected device; "delivered" for the sender means at least one device acked. Read state syncs across devices as the read cursor moves.

#Where do photos and videos go?

The client uploads the file to blob storage with a short-lived signed URL, then sends a normal message holding the file's id. Large files never touch the chat path.

#What changes with end-to-end encryption?

The server stores and forwards ciphertext it cannot read, so server-side search and moderation go away, and groups need key distribution among members. Routing, ordering and storage stay the same.

#How do edits and deletes work?

An edit or delete is a new message with its own seq that points at the original seq. Clients apply it in order like any other message, so everyone ends with the same history, and offline users catch up through the same sync.

#How would you run this across regions?

Users connect to their nearest region. Each conversation has a home region that owns its sequence, so senders elsewhere pay one extra cross-region round trip. Stored messages replicate to other regions for history.