Design a News Feed

You open a social app and see posts from the people you follow, newest or most interesting first. Scroll down and older ones load. Post something and your followers see it within seconds. Behind that screen is one hard question: when someone posts, do you do the work then, copying the post into every follower's feed, or later, gathering posts when each follower opens the app?

Interviewers like it because the two obvious answers fail at scale in opposite directions, and the fix is a hybrid you must justify with numbers. It also tests caching and pagination. It follows A Framework for Any Design.

#Clarify the problem

Pin down what the feed contains, how it is ordered, and the shape of the follow graph. The last one decides the design.

• Functional: publish a post; see a feed of posts from followed accounts, newest first at launch and ranked later; scroll back through older posts.
• Non-functional: the feed loads in under 200 ms at p99 on our side; it stays available when parts fail; a new post reaches followers' feeds within about 5 seconds; feeds may be eventually consistent.
• Out of scope: media upload, search, notifications, ads, recommendations from accounts you do not follow.

#Estimate the scale

• Feed reads: 100 million × 10 = 1 billion a day. 10⁹ ÷ 86,400 ≈ 11,600 a second; call it 12,000. Assume a 3x peak: 36,000 a second.
• Posts: 10 million a day ÷ 86,400 ≈ 116 a second; about 350 at peak. Reads outnumber posts 100 to 1.
• Post storage: about 1 KB of text and metadata per post (media lives in a blob store), so 10⁷ × 1 KB = 10 GB a day, about 3.7 TB a year.
• Read bandwidth: a page of 20 posts at 1 KB is 20 KB; 36,000 × 20 KB = 720 MB/s at peak.

#Define the API

POST /v1/posts
  body: { "text": "...", "media_ids": ["m_91"]? }
  201 Created   { "post_id": "7311045529", "created_at": "..." }
  400 Bad Request          empty or too long
  429 Too Many Requests    posting too fast
 
GET /v1/feed?limit=20&cursor=opaque-string
  200 OK  { "posts": [ { "post_id", "author", "text", "media_urls", "created_at", "like_count" } ],
            "next_cursor": "opaque-string" or null }
  400 Bad Request          cursor expired or invalid: start again from the top
 
POST   /v1/users/{id}/follow
DELETE /v1/users/{id}/follow

The decision worth arguing is cursor pagination. With ?page=2, the server skips the first 20 items, but if 3 new posts arrived since page 1, items 18 to 20 of the old page 1 show up again. A cursor says "continue after this exact item", so new posts at the top cannot shift the next page. The cursor is opaque to the client, so its contents can change when ranking arrives. The deep dive below shows both.

#Model the data

posts           post_id    BIGINT PRIMARY KEY   -- time-sortable id
                author_id  BIGINT
                text       VARCHAR(2000)
                media_ids  LIST
                created_at TIMESTAMP
                deleted    BOOLEAN
 
followers       (author_id, follower_id)        -- "who follows X": fan-out
following       (follower_id, author_id)        -- "whom does Y follow": reads
 
author_posts    author_id -> newest post ids    -- for pull and rebuilds
 
feed            user_id -> set of (post_id, author_id), sorted by post_id, max 500

Posts are written once, read by id, and partitioned by post_id: a wide-column or key-value store fits, with a cache in front for hot posts. A post id that sorts by time (see Unique IDs) lets you order feeds by id alone, no timestamp lookups.

The follow graph is stored twice, once per direction, each partitioned by its first column, so both questions are single-partition reads. Keeping two copies in step is the cost; write both in the follow request and repair with a background job.

The feed lists live in an in-memory store with replicas, such as Redis. They hold ids, not posts, so an edit or delete never has to touch 50 million feeds. And they are derived data: if a list is lost, rebuild it from following and author_posts.

#High-level design

Writes and reads are separate services. The post service stores the post and publishes an event; fan-out workers do the slow copying into feeds. The feed service reads one list, adds posts from the few huge accounts the user follows, and fills in post bodies from the post cache.

#Deep dives

#Fan-out on write, on read, or both

This is the question interviewers care about most. Fan-out on write (push) copies each new post's id into every follower's feed when it is posted. Fan-out on read (pull) stores nothing per reader: when you open the app, it fetches the newest posts of everyone you follow and merges them.

Put numbers on both. Pull for everyone means each feed open reads about 200 author lists: 36,000 × 200 = 7.2 million list reads a second at peak. Push for everyone means 69,000 inserts a second at peak, which is fine, until a 50-million-follower account posts. If the fan-out fleet can do 200,000 inserts a second, that one post takes 5 × 10⁷ ÷ 2 × 10⁵ = 250 seconds and delays every other post behind it.

Pick a threshold, say 100,000 followers, and say what decides it: below it, a post costs at most 100,000 inserts, half a second of the fleet. Above it, the account's posts are pulled at read time. If a typical user follows a handful of such accounts, a feed read merges one pushed list with a few pulled lists, all from cache. What would change my mind: if users followed hundreds of huge accounts each, the pulled side would dominate, and I would lower the threshold or precompute merged feeds for the most active readers.

Also skip inactive users when pushing. Someone who has not opened the app in 30 days gets no inserts; when they return, build their feed by pull once, then switch them back to push.

#Feed storage and pagination

Each feed holds up to 500 ids, kept as a set sorted by post id, not a list you prepend to. Fan-out for two authors can finish out of order, and a queue delivers an event again after a worker crash; a sorted set puts late posts in place and ignores repeats (see Idempotency and Retries). Readers rarely scroll past a few pages; anyone who goes past 500 falls back to pulling from author_posts. Watch why page numbers break when posts keep arriving.

That works because ids sort by time. A ranked feed has no such order: scores change as posts get likes. So when ranking arrives, the feed service ranks once when the session starts, caches that ordered list for about 10 minutes, and the cursor becomes "snapshot id + position". If the snapshot has expired, the API returns 400 and the app starts from the top. That is why the cursor is opaque.

#Ranking, in outline

The interview needs ranking's shape, not its model: a stage the feed service calls, with a time budget.

1. Candidates: the ~500 ids from the pushed list plus the pulled lists.
2. Filter: drop deleted posts, blocked authors, posts already seen.
3. Score: a model predicts how likely the reader is to engage, from features like how often they interact with the author, the post's age and its early engagement.
4. Re-rank for variety: no more than 2 posts in a row from one author.

Give ranking about 50 ms of the 200 ms budget. If the ranking service is slow or down, return the chronological feed instead: a worse feed is better than no feed.

#Cache strategy

Four caches do the work, each for a different reason:

• Feed lists (800 GB): the pushed feeds, replicated. Losing a node means rebuilding those users' feeds by pull, so warm them gradually.
• Post objects: hydration reads by id. Posts from the last two days are about 2 × 10 GB = 20 GB and take most reads, so they fit easily. Use cache-aside and delete the entry on edit or delete.
• Recent posts of huge accounts: read by millions of feeds. Copy these hot keys into each feed server's memory for a few seconds so one cache node does not take all those reads.
• Follow lists, both directions: followers for the fan-out workers, following for the feed service, so neither hammers the graph store.

Like counts change constantly; keep them out of the post object, in a separate counter that is cached briefly and updated in batches. Otherwise every like invalidates a hot post.

#Bottlenecks and failures

At 10x the traffic (360,000 feed reads and about 690,000 fan-out inserts a second at peak), the raw feed data grows to about 8 TB, which means a larger cluster partitioned by user id with consistent hashing. The fan-out queue is partitioned by author so workers scale out. The first things to hurt are the hot keys of huge accounts, which the local in-memory copies handle, and the fan-out fleet, which you can relieve by lowering the threshold.

#What a strong answer sounds like

#Follow-up questions

#What happens when I unfollow someone?

Remove that author's ids from your feed list; it holds at most 500 entries, so the scan is cheap. A fan-out already in flight may add one more post afterwards, so the read path also drops ids whose author is not in your following list, which it reads anyway.

#How do you handle a deleted post?

Mark it deleted and delete its cache entry. Feed lists still hold the id, and hydration drops it. Nothing touches millions of feeds.

#How does the app show "5 new posts"?

Every minute or so, the app asks for the count of ids above its newest one. That is a cheap read of the feed list. Pushing over an open connection is possible but costs a connection per active user.

#How would you run this in two regions?

Serve feeds from each region's caches and stores. Replicate posts and follows between regions asynchronously, and run fan-out in each region for the users it serves. A post may reach the other region's feeds a second or two later.

#An account crosses the 100,000 threshold. What changes?

New posts switch from push to pull. Its recent posts already sit in its followers' lists, so nothing has to move; the read path merges both and removes duplicates by id.

#What to practise next

The fan-out queue here is a stream with partitions and consumer groups; see Message Queues and Streams. For a design where delivery must be instant rather than pulled, try Design a Chat App.