Design an Email Service

Someone in another company presses Send. Their mail server looks up your domain, connects, and hands over a message. A few seconds later it appears on your phone and your laptop, filed in the inbox, with spam already moved aside, and you can find it next year by typing one word. A web mail service with labels and search is the kind of product this question means.

Interviewers like it because a familiar screen hides several classic problems: a receive path that must never lose an accepted message, storage growing by petabytes, a search index per person, retries against servers you do not control, and devices that must agree. The steps follow A Framework for Any Design. Try each "Your turn" box before you read on.

#Clarify the problem

• Functional: receive mail over SMTP; filter spam and viruses; list mail by label; open messages and attachments; send, with retries and bounces; label, mark read, archive, delete; search your own mail; sync devices.
• Non-functional: no accepted message is ever lost (durability above everything); new mail visible within 5 seconds; inbox list in under 200 ms; search in under 500 ms; receiving stays up even when reading is degraded.
• Out of scope: calendar, contacts, chat, ads, end-to-end encryption, IMAP (discussed as a follow-up).

#Estimate the scale

Assume an average email is 30 KB stored: 10 KB of headers and body, and one in 20 emails carries attachments averaging 400 KB, which adds 400 ÷ 20 = 20 KB on average. Peak is twice the average, since mail is spread over the world's time zones.

• Received: 100 million × 40 = 4 billion emails a day. 4,000,000,000 ÷ 86,400 ≈ 46,000 a second, about 90,000 at peak.
• Sent: 100 million × 5 = 500 million a day, about 5,800 a second.
• Reads: if each user opens a list or a message 20 times a day, that is 2 billion reads a day, about 23,000 a second.
• Inbound bandwidth: 4 billion × 30 KB = 120 TB a day, and 120 TB ÷ 86,400 s ≈ 1.4 GB/s.
• Per user: 40 × 365 × 30 KB ≈ 440 MB a year.

#Define the API

Other mail servers reach you over SMTP on port 25. Your apps use your own API:

GET  /messages?label=INBOX&limit=50&cursor=...
  200 { "messages": [ { "id": "m_9f2", "from": "...", "subject": "...",
        "snippet": "...", "labels": ["INBOX", "UNREAD"] }, ... ], "next_cursor": "..." }
 
GET  /messages/{id}              headers, body, attachment list
GET  /attachments/{id}           302 to a short-lived signed URL in object storage
 
POST /attachments                upload first, get an id back
POST /messages/send    Idempotency-Key: draft_771
  body: { "to": [...], "subject": "...", "body": "...", "attachment_ids": [...] }
  202 Accepted   { "id": "m_a01", "status": "queued" }
  400 Bad Request         an address is malformed
  413 Payload Too Large   over the size limit
  429 Too Many Requests   over the daily sending limit
 
POST /messages/{id}/labels   { "add": ["Receipts"], "remove": ["INBOX"] }
GET  /search?q=from:bob invoice has:attachment&cursor=...
 
GET  /changes?since=81237
  200 { "changes": [ ... ], "latest": 81240 }
  410 Gone        too far behind; do a full resync

The API decision worth arguing is sync. Re-downloading the inbox on every wake fetches 50 rows when maybe one changed, and misses label changes on older mail. Instead, a device asks a per-user change log for "everything after 81,237". It catches every kind of change, and a device offline for a week just gets more.

Label changes are operations ("add Receipts, remove INBOX"), not a new label set, so two devices editing different labels both win.

#Model the data

messages
  user_id       BIGINT
  message_id    BIGINT
  thread_id     BIGINT
  dedupe_key    BINARY(32)     -- hash of Message-ID header + content
  from_addr     VARCHAR(320)
  subject       VARCHAR(1000)
  snippet       VARCHAR(200)
  received_at   TIMESTAMP
  labels        BIGINT[]       -- INBOX, UNREAD, SPAM, user labels
  body_key      VARCHAR(64)    -- object storage key
  attachments   JSON           -- [{ name, size, blob_key }]
  PRIMARY KEY (user_id, message_id),  UNIQUE (user_id, dedupe_key)
 
label_index                      -- one row per (message, label)
  user_id, label_id, received_at, message_id
  PRIMARY KEY (user_id, label_id, received_at DESC, message_id)
 
labels    (user_id, label_id, name, total, unread)
changes   (user_id, seq, message_id, kind)   PRIMARY KEY (user_id, seq)

A label is a tag, and a message can carry many. A folder is just a label with the rule "one per message", and Inbox, Sent, Spam and Trash are system labels. Archiving removes INBOX; nothing moves.

Almost every query starts with "for this user", so partition everything by user_id. A sharded relational database fits: on one shard, a delivery writes the message, label rows, counters and change-log entry in one transaction, so counts and the sync log never disagree with the mailbox. Bodies and attachments never change once written, which suits object storage.

#High-level design

A durable queue splits the receive path: the edge only stores the message safely, and everything slow happens after. The send path is in the deep dives.

#Deep dives

#The receive path: when to say 250

SMTP has a clear contract. 4xx means "try again later", and the sender will. 5xx means "permanently refused", and the sender's server tells its user. 250 after the body means you now own the message. So store it durably before replying, and decide where each check runs.

The pipeline is at-least-once on purpose: a sender may retry after a lost 250, and the queue may hand a message to two workers. Instead of chasing exactly-once, make the mailbox write idempotent with a dedupe key per recipient (see idempotency and retries). A message to three of your users is stored once and queued as three deliveries.

Back-pressure comes free: if the queue nears its limit, SMTP in answers 451, a temporary failure, and senders retry later. Mail is delayed, never dropped.

#Mailbox storage, partitioned by user

Map users to shards with a directory, not user_id modulo the shard count, so you can move one user without rehashing everyone (see sharding and partitioning). Metadata grows 2 TB a day, so new shards keep arriving.

Attachments are parsed out at delivery and stored by the SHA-256 hash of their bytes, so a 5 MB file sent to 200 of your users takes 5 MB, not 1,000 MB. Access always goes through a message the user owns, never by hash. Since many messages can share one object, a background sweep deletes objects nothing has referenced for a few days; the grace period protects a delivery that is reusing the object right now.

#Search, one user at a time

Every search is "my mail only". That decides how to split the index.

Index headers, body text and attachment names; a slower job adds text pulled out of documents. from:, has:attachment and label: become filters. Results are checked against the mailbox before showing, so a message deleted a second ago never appears. Search systems covers the inverted index.

#The send path: retries and bounces

Before returning 202, one transaction on the user's shard writes the message to Sent and an outbox row; a relay moves outbox rows to the queue, so a crash cannot leave mail in Sent that never leaves. The draft id is the idempotency key, so a double tap sends once. Recipients on your own service skip SMTP and go straight to the delivery workers.

• Delivery: group recipients by domain, look up its MX record, connect over TLS, and sign with DKIM so the receiver can verify the message came from you.
• Temporary failure (4xx, timeout, no answer): retry with growing gaps, say 5 minutes, then 15, then hourly. The SMTP standard suggests senders keep trying for at least four to five days before giving up.
• Permanent failure (5xx, such as "no such user"), or retries used up: put a bounce naming the address and reason into the sender's inbox. Only this path makes bounces.
• Reputation: receivers judge your sending IP addresses, so rate-limit accounts and send bulk and personal mail from separate address pools.

#Sync across devices

Each user has a counter. Every mailbox change, in its own transaction, takes the next number and writes a row to changes. Taking the number locks the counter row until commit, so changes commit in number order and a reader never skips one that commits late. A device remembers the last number it saw. IMAP has the same idea, a per-mailbox modification sequence.

• Telling devices: an open connection for web, a mobile push for phones (see realtime communication). The push only says "something changed" and the device pulls, so a lost push only delays sync.
• Offline edits: the device queues operations such as "mark m_9f2 read" and replays them on reconnect. Operations on different labels do not conflict; if two devices set the same flag differently, the last write wins.
• Far behind: keep 30 days of change log. A device that asks for something older gets 410 and resyncs the recent window, such as the newest 500 messages per label.

Because a user lives on one shard, the counter is a plain column, not a distributed sequence. Design File Sync and Design a Chat App use similar cursors.

#Bottlenecks and failures

At 10x, about 460,000 messages a second and 1.2 PB a day, storage cost is the first wall. Compress bodies, store attachments once, tier old mail and enforce quotas. SMTP in and the workers are stateless and scale out behind several MX records; mailbox and search shards grow by adding more.

#What a strong answer sounds like

#Follow-up questions

#How does "undo send" work?

Put the message on the outbound queue with a delay, say 10 seconds. Undo deletes it from the queue before it is released. Once SMTP out has handed it to another provider, it cannot be recalled.

#A mailing list sends one message to 1 million of your users. What happens?

Store the body once and queue 1 million deliveries: about 22 seconds of the system's average 46,000 a second. Give bulk senders a lower-priority queue so personal mail is not stuck behind it.

#How do you stop your own users from sending spam?

Per-account rate limits, stricter for new accounts, and a spam score on outgoing mail. A hijacked account shows up as a sudden burst to strangers; suspend sending and ask the owner to verify.

#Could you add end-to-end encryption?

Yes, but the server can no longer read bodies, so server-side search and content filtering stop working. Search moves to the device and filtering relies on sender reputation.

#How would you support IMAP clients?

Add an IMAP front end that maps labels to folders and the change log to IMAP's modification sequences. The catch: a message with two labels appears in two folders, and deleting it from one should only remove that label.