Design a News Feed

Facebook / Twitter Timeline System Design

Overview

A news feed shows each user a personalized stream of posts from the people and pages they follow. You scroll Facebook, X, or Instagram and see a mix of fresh content, often re-ranked by relevance. The product sounds simple — "show me my friends' posts" — but it hides one of the most important system design tradeoffs: do you assemble each feed when a user requests it, or do you assemble it gradually as posts are written?

This is labeled "Hard" because no single strategy works well for both a normal user with a few hundred followers and a celebrity with 50 million followers. The important parts are the fan-out decision (write-time vs read-time), the hot-key / celebrity problem that breaks naive fan-out-on-write, the hybrid approach that production systems usually run, and ranking the assembled candidates.

In this walkthrough you will scope the feed, do rough estimation math that motivates the design, lay out the data model and API, draw the read and write paths, and then examine fan-out, the celebrity problem, ranking, and cache management.

Functional requirements

Keep the list tight and confirm it before designing.

• Post. A user creates a post (text, optionally media) that should reach their followers' feeds.
• View feed. A user opens the app and gets a personalized feed of recent posts from accounts they follow, paginated as they scroll.
• Follow / unfollow. A user follows other users; the feed reflects the current follow graph.

Explicitly out of scope (say so — scoping is graded): comments and likes (treat as a separate engagement service), direct messaging, notifications, ads insertion, and media transcoding/storage internals (assume a blob store + CDN). We keep ranking in scope because "feed" without ranking is just a join, and the ranking signal shapes the storage.

Non-functional requirements

• Read-heavy and latency-sensitive. Feed views vastly outnumber posts, and a feed open sits in the user's critical path. Target p99 feed-load latency under ~200 ms.
• High availability over strong consistency. A user not seeing a post for a few seconds is fine; a feed that fails to load is not. Eventual consistency is acceptable.
• Scalable to skewed graphs. The follower-count distribution is extremely skewed (most users have hundreds of followers, a handful have tens of millions). The design must not fall over on the heavy tail.
• Freshness. New posts should appear within seconds, not minutes.

Estimations

State assumptions; the reasoning matters more than the digits.

Rounded assumptions

• Daily active users (DAU): ~500M.
• New posts/sec. 500M DAU × 0.4 posts each = 200M posts/day; ÷ 86,400 s/day ≈ ~2,300 → ~2K posts/sec.
• Feed reads/sec. 500M DAU × 10 opens each = 5B feed reads/day; ÷ 86,400 ≈ ~58K → ~60K reads/sec (call it ~2–3× that at peak). Each open returns ~20 posts.
• Fan-out writes/sec. At ~500 followers average, pure fan-out-on-write does ~2K posts/sec × 500 ≈ ~1M feed-cache writes/sec — the number that drives the whole design (deep dive 1).

How the numbers affect the design

Storage

• Per post: id (8 bytes), author id (8 bytes), text (~300 bytes), media pointer (~100 bytes), timestamp (8 bytes), metadata — round to ~500 bytes.
• New post data is around ~100 GB/day, or ~180 TB over 5 years.
• Timeline cache: if you precompute feeds, store post id references, not full posts. Keeping the last ~800 post ids per user works out to 500M users × ~800 entries × ~16 bytes ≈ ~6.4 TB raw — and Redis sorted-set overhead multiplies that several ×, so realistically tens of TB (after evicting dormant users' feeds), partitioned across a Redis cluster.

Core entities / data model

Two access patterns dominate: "give me a user's followers" (for write fan-out) and "give me a user's feed" (for read). The follow graph is queried in both directions, so you typically store the edge twice or use a graph/wide-column store indexed both ways.

• Posts live in a sharded wide-column / KV store partitioned by post_id (point lookups by id).
• Follow graph lives in a store indexed by follower_id and followee_id so you can resolve "who do I follow" (read fan-out) and "who follows me" (write fan-out) cheaply.
• FeedCache is an in-memory store (Redis or similar): each user's feed is an ordered list kept sorted by a score (in Redis, a sorted set), holding post-id references — not the post bodies, so a post edit or delete is applied once in the Post Store instead of being rewritten across every follower's feed.

API design

Three RESTful endpoints carry the product — publish a post, read the feed, and manage follows.

Create a post

POST /api/posts
Body: { "text": "...", "media_url": "..." }
201 Created
Returns: { "post_id": "...", "created_at": "..." }

Publish a post. The body is stored once in the Post Store, then fanned out to followers' feeds asynchronously (the write path, deep-dived below).

Read the feed

GET /api/feed?cursor=<opaque>&limit=20
200 OK
Returns: { "posts": [ {post...}, ... ], "next_cursor": "<opaque>" }

Read the caller's home feed, one page at a time — the hot, read-heavy path served from the precomputed feed cache. Pagination uses an opaque cursor, not a numeric offset — offset pagination breaks when new posts shift positions and is O(n) deep into the list.

Follow / unfollow

POST /api/follow      Body: { "followee_id": "..." }   204 No Content
DELETE /api/follow    Body: { "followee_id": "..." }   204 No Content

Follow or unfollow a user. These edit the follow graph that fan-out reads to decide whose feeds a new post lands in.

High-level architecture

Two paths matter: a write path that delivers a new post into followers' feeds, and a read path that assembles and returns a feed page. We'll walk each, then combine them.

1. Write path (publish a post)

A new post is stored once, then fanned out to followers' precomputed feeds asynchronously.

flowchart LR
    Client[Client] -->|"POST post"| PostSvc[Post Service]
    PostSvc --> PostDB[(Post Store)]
    PostSvc --> FanoutQ[[Fan-out Queue]]
    FanoutQ --> FanoutW[Fan-out Workers]
    Graph[(Follow Graph)] --> FanoutW
    FanoutW -->|"push post_id to followers' feeds"| FeedCache[(Feed Cache / Redis)]

1. The Post Service stores the post once in the Post Store.
2. It enqueues a fan-out job rather than pushing inline, so the slow part happens off the request path.
3. Fan-out workers look up the author's followers in the Follow Graph.
4. Workers push the post_id (a reference, not the body) into each follower's feed-cache sorted set — except for celebrities, who are handled at read time (deep dive 1).

2. Read path (open the feed)

A feed read serves the precomputed feed, merges in fresh celebrity posts, ranks, and hydrates bodies for the page.

flowchart LR
    Client[Client] -->|"GET feed"| FeedSvc[Feed Service]
    FeedSvc -->|"read precomputed ids"| FeedCache[(Feed Cache / Redis)]
    FeedSvc -->|"pull recent celebrity posts"| PostDB[(Post Store)]
    FeedSvc -->|"hydrate top N bodies"| PostDB
    FeedSvc -->|"feed page"| Client

1. The Feed Service reads the precomputed post-id list from the Feed Cache — the cheap, common case.
2. It pulls recent posts from the small set of celebrities the user follows (those were not precomputed).
3. It merges and ranks the candidates, then hydrates the top N post bodies from the Post Store.
4. It returns one page, with an opaque cursor for the next.

3. Combined architecture

Putting them together: writes precompute most feeds asynchronously, while reads serve from the precomputed cache and merge in only the celebrity slice live.

flowchart LR
    Client[Client] --> GW[API Gateway]
    GW -->|"POST post"| PostSvc[Post Service]
    PostSvc --> PostDB[(Post Store)]
    PostSvc --> FanoutQ[[Fan-out Queue]]
    FanoutQ --> FanoutW[Fan-out Workers]
    Graph[(Follow Graph)] --> FanoutW
    FanoutW -->|"push post_id to followers"| FeedCache[(Feed Cache / Redis)]
    GW -->|"GET feed"| FeedSvc[Feed Service]
    FeedSvc -->|"read precomputed ids"| FeedCache
    FeedSvc -->|"pull celebrity posts"| PostDB
    FeedSvc -->|"rank + hydrate"| PostDB
    FeedSvc -->|"feed page"| Client
    GW -->|"follow / unfollow"| FollowSvc[Follow Service]
    FollowSvc -->|"add / remove edge"| Graph

Follow / unfollow is a simple write to the Follow Graph — add or remove the directed edge (stored both directions, per the data model). Two subtleties worth naming: apply the edge before fan-out reads it, so "I follow someone, then see their next post" holds; and, as a deliberate simplification, a follow does not backfill their old posts into your feed cache — you start receiving their future posts (their existing posts surface only via the celebrity/pull path, if at all). If that leaves a new follower staring at an empty-looking feed, the common mitigation is to backfill the last k recent posts of the followee at follow time.

Deep dives

1. Fan-out: on write vs on read

This is the central decision. "Fan-out" means delivering a new post to the feeds of everyone who should see it.

Alice has 500 followers; she posts P:
  → append P's id to all 500 follower feed-caches   (500 small writes, async)
  → each follower's next feed read is O(1): just read the ready-made list

A celebrity with 50M followers posts once:
  → 50M cache writes from one action — a write storm that takes minutes to drain

Bob follows 1,000 accounts; he opens his feed (page size 20):
  → fetch the top ~20 recent posts from each followee   (1,000 lookups, ~20K candidates)
  → merge those sorted lists → take the global top 20
  → nothing was precomputed, so every feed open repeats this work

2. The celebrity / hot-key problem and the hybrid

Pure push breaks on celebrities (50M-write storms); pure pull breaks on the read-heavy common case. The production answer is hybrid:

• Fan-out-on-write for normal authors. When a user with a follower count below a threshold (say ~10K–100K) posts, push to followers' feed caches as usual.
• Fan-out-on-read for celebrities. Authors above the threshold are flagged "celebrity" and do not fan out on write. Their posts are stored once. At read time, the Feed Service additionally pulls recent posts from the (small) set of celebrities the requesting user follows and merges them into the precomputed feed.

This bounds the worst case on both sides: write amplification is capped at the threshold, and read-time pull is bounded by how many celebrities a user follows (usually a handful, not thousands). The merge is cheap because the precomputed part is already assembled and only the celebrity slice is pulled live. The threshold is a tunable knob — lower it if write storms appear, raise it if read latency creeps up.

flowchart LR
    Normal[Normal author - below threshold] -->|"post: push to followers"| FeedCache[(Feed Cache - precomputed)]
    Celeb[Celebrity - above threshold] -->|"post: store once, no fan-out"| CelebCache[(Celebrity posts - cached + replicated)]
    Reader[Reader] -->|"open feed"| FeedSvc[Feed Service]
    FeedSvc -->|"read precomputed ids"| FeedCache
    FeedSvc -->|"pull recent posts of followed celebrities"| CelebCache
    FeedSvc -->|"merge + rank, top 20"| Reader

Normal authors still fan out on write into each follower's precomputed cache; celebrities are stored once and not pushed — so one celebrity post never triggers a 50M-write storm (the hot-key write problem disappears). At read time the Feed Service unions the precomputed feed with a live pull of the handful of celebrities you follow, then re-scores the cached ids and the pulled celebrity ids together over that unified candidate set before taking the top 20 — a plain positional merge would rank the two sets on different scales (see deep dive 3). Because those celebrity entries — both a celebrity's recent-posts list and the post bodies — are read by millions, they live in a replicated cache so no single shard becomes the hot key (the read side of the problem).

3. Ranking the feed

Reverse-chronological is the baseline, but engagement-driven feeds re-rank candidates. The pipeline is candidate generation → scoring → ordering:

• Candidate generation: the merged set of precomputed + pulled celebrity post ids (a few hundred candidates).
• Scoring: a model (or weighted heuristic) scores each candidate on signals like recency, author affinity, predicted engagement, and content type. Store the score alongside the post id in the sorted set so the cache stays ordered.
• Ordering & pagination: take the top N by score; the opaque cursor encodes the boundary so the next page continues deterministically.

Compute or refresh scores at read time over the small candidate set, then cache the ranked page briefly (a few seconds) to absorb rapid re-opens without recomputing.

4. Cache management: TTL, refresh, and misses

Two derived caches carry the read path — the precomputed feed cache (per user) and the celebrity posts cache. Both are rebuildable from the durable Post Store + Follow Graph, so losing an entry costs latency, not data — but each still needs an eviction and refresh policy.

Precomputed feed cache (per user)

• Bounded length. Keep only the most recent ~800 post ids per user; appending past the cap drops the oldest. This caps total memory and matches how people read — almost nobody scrolls past a few hundred items.
• TTL on dormant users. Evict (and stop writing to) the feed of anyone who hasn't opened the app in weeks, and skip fan-out to them. This reclaims memory and avoids wasted writes to followers who never read; rebuild their feed lazily on next login.
• Refresh. Fan-out workers append new posts as they arrive (write-time). Ranking scores are refreshed at read time (deep dive 3), so the entry stays "ids + current-ish scores," not a frozen ranking.

Celebrity posts cache

• Recent window + TTL. Store each celebrity's recent posts (the last few hundred, or a rolling time window); older ones age out. Replicated so the hot reads spread across replicas (deep dive 2).
• Refresh (cache-aside). Updated when the celebrity posts; on a miss, the Feed Service reads from the Post Store and repopulates.

Scrolling past the cache — the cold tail

The feed cache only holds the recent ~800 ids, so scrolling deeper exhausts it. Two standard answers:

1. Cap the feed depth. Most products stop the home feed after a few hundred items ("You're all caught up"); older content is reached via profiles and search, not infinite home-feed scroll. Simplest, and what users expect.
2. Read-time generation for the tail. If deep scroll must work, past the cached boundary the cursor switches to building older pages on demand from the Follow Graph + Post Store — a fan-out-on-read for the tail only. Slower, but deep scrolls are rare, so it's an acceptable cold path.

Tradeoffs & bottlenecks

• Write amplification vs read latency. The fan-out choice is a direct trade: push pays at write time for cheap reads; pull pays at read time for cheap writes. Hybrid pays a bounded amount on each side. State the threshold knob explicitly.
• Feed staleness vs cost. Inactive users still get pushed-to feed caches under pure push — wasted work. Some systems skip fan-out to long-dormant users and lazily backfill on next login, trading a slower first load for cheaper writes.
• Hot keys on celebrities. Even with pull, a celebrity's posts are read by millions; without dedicated caching and replication that shard melts.
• Consistency. Eventual consistency is fine for feeds — a post appearing a few seconds late is acceptable, which is what makes asynchronous fan-out workable. But "follow then immediately see their next post" should hold, so apply follow-graph changes before fan-out reads them.
• Storage cost of precomputed feeds. Storing references (post ids + scores), not bodies, keeps the feed cache in the tens-of-TB range (see estimations) rather than storing full post bodies per follower; hydrate bodies only for the page being returned.
• Fan-out worker backlog. A burst of posting (a major event) can back up the fan-out queue; size workers for peak and let the queue absorb spikes, accepting slightly higher freshness latency under load.

Extensions if asked

If the interviewer wants to push beyond the core feed, add only the extension that changes the design discussion:

• Notifications for important posts. Send selected post events through a notification queue.
• Rate limits and abuse. Limit posting, follow/unfollow, and feed-refresh abuse.
• Media delivery. Images and videos need blob storage, transcoding, and CDN delivery.

What interviewers look for & common mistakes

What interviewers usually reward:

• Driving to the fan-out decision from the numbers — showing ~1M cache writes/sec and the 50M-follower write storm to motivate why pure push fails.
• Naming the celebrity / hot-key problem and reaching the hybrid (push for normal, pull for celebrities) with a tunable threshold.
• Storing references, not bodies, in the feed cache, and hydrating only the returned page.
• Treating ranking as candidate-gen → score → order, and not freezing scores at write time.
• Cursor pagination over offset, with a clear reason.

Before you finish, do a quick mistake check:

• Did you justify the fan-out strategy from the read/write numbers?
• Did you handle celebrities separately from normal authors?
• Did you store references, not full post bodies, in the feed cache?
• Did you use cursor pagination instead of offset pagination?
• Did you treat ranking as candidate generation, scoring, and ordering?
• Did you avoid freezing ranking scores at write time?