Design Pastebin

Text Paste & Sharing System Design

Overview

Pastebin lets a user drop in a block of text — a log dump, a config file, a code snippet — and get back a short URL like https://pst.bin/aZ4kPq9 that anyone can open to read that exact text. Open the link and you see the paste; optionally it disappears after an hour, a day, or a week. GitHub Gist, Hastebin, and Pastebin.com are all variations on this idea.

It is labeled "Easy" because the feature set is tiny — write a blob, read it back by key — but it is a sharp interview question because one design decision separates a good answer from a weak one: where the large text body lives. The metadata (key, owner, timestamps, size, expiry) is small and structured and belongs in a database. The paste body can be megabytes of text, and stuffing that into a relational row is the classic mistake this question is designed to catch. The other levers are the same read-heavy machinery a URL shortener uses: unique keys, aggressive caching, a CDN, and expiration handling.

In this walkthrough you will scope the problem, do lightweight estimation math, design the split data model and the API, draw the architecture, and then dig into the three things that make or break the answer: the metadata/blob storage split, unique key generation, and read scaling with expiration.

Functional requirements

These are the things the system must do. Keep the list short and get the interviewer to agree before building anything.

• Create a paste. Given a block of text (and optional TTL / privacy setting), store it and return a unique short URL.
• Read a paste. Given the key from the URL, return the original text.
• Expiration. A paste may carry a TTL after which it stops resolving and is eventually deleted.
• Unlisted / private pastes. An unlisted paste is reachable only by its (hard-to-guess) key; it is not listed or indexed. This is the common default.

Explicitly out of scope (say this — scoping is graded): editing pastes after creation, syntax highlighting and rendering, full user accounts, comments, and search over paste contents. Pastes are effectively write-once, read-many immutable objects, which simplifies caching enormously.

Non-functional requirements

These are the qualities the system must have. For Pastebin the dominant ones are:

• Read-heavy, low latency. A paste is written once and read many times (a shared link, a Stack Overflow answer). Reads vastly outnumber writes, and the read sits in the user's critical path. Target p99 read latency in the tens of milliseconds.
• High availability for reads. A dead reader breaks every shared paste. Availability of the read path matters more than strong consistency — a just-created paste being readable a second late is fine; an outage is not.
• Durability. A stored paste must not be silently lost before its expiry.
• Unpredictable keys. Keys should not be trivially enumerable, or a scraper can walk the keyspace and scrape unlisted pastes.
• Scalable storage. Millions of pastes, some large, retained for their lifetime — the store must grow horizontally.

Estimations

State assumptions out loud, but keep the math light. The interviewer cares how the numbers change the design, not perfect arithmetic.

Rounded assumptions

• New pastes created: ~1M / day → about ~10 writes/sec.
• Reads dominate. With a 100:1 read:write ratio, expect about ~1,000 reads/sec (~1K/sec).
• Average paste size: ~10 KB of text; cap a single paste at, say, ~1–5 MB.
• Storage growth: 1M × 10 KB ≈ 10 GB/day → ~3.5 TB/year of paste bodies (before expiry reclaims space).
• A 7-character base62 key gives ~3.5 trillion possibilities — far more than we will ever mint.

How the numbers affect the design

Core entities / data model

The single most important modeling decision: split metadata from the body. There are two stores, joined only by the key.

Metadata (relational or key-value DB) — small, structured, one row per paste:

Body (blob / object store) — the large text itself, stored as an object at the path recorded in blob_ref (e.g. s3://pastes-bucket/<key>; storing blob_ref explicitly lets you repath or content-address objects later without migrating every row). Object storage such as S3 or GCS is durable, cheap per GB, and effectively unbounded.

The access pattern is a single point lookup: given key, load the metadata row, then fetch the body. That pattern — plus the size split — drives everything else.

API design

Two endpoints carry the whole product. Keep them RESTful.

Create

POST /api/pastes
Body: { "content": "…text…", "expires_in": "1d", "unlisted": true }
201 Created
Returns: { "key": "aZ4kPq9", "url": "https://pst.bin/aZ4kPq9", "expires_at": "2026-07-09T00:00:00Z" }

expires_in and unlisted are optional. Reject bodies over the size cap with 413 Payload Too Large.

Read

GET /{key}            → 200 with the paste text (or an HTML view)
GET /api/pastes/{key} → 200 { "content": "…", "created_at": "…", "expires_at": "…" }

If the key does not exist or the paste has expired, return 404 Not Found (treat expired-but-not-yet-swept the same as gone). The human-facing read lives at the root of the short domain (pst.bin/{key}) so links stay short.

High-level architecture

Build the architecture in layers: understand the write path and the read path separately, then merge them. The recurring theme is that metadata and body travel to two different stores.

1. Create paste flow

The user submits text; we mint a key, write the body to the blob store, then write the metadata row.

flowchart LR
    User[User] -->|"POST /api/pastes\ncontent"| App[App Server]
    App -->|"get unique key"| Keys[Key Generator]
    Keys --> App
    App -->|"1. PUT body (key -> text)"| Blob[(Blob Store)]
    App -->|"2. INSERT key, blob_ref, expiry"| DB[(Metadata DB)]
    App -->|"return key + URL"| User

Order matters: write the body first, then the metadata row. If the body write fails, no metadata row points at a missing blob. If the process dies after the body write but before the metadata insert, you have an orphaned blob (harmless — a background sweep or lifecycle rule reclaims blobs with no metadata). Writing metadata first would risk a row that resolves to a non-existent body — a broken paste.

2. Read paste flow

The reader has a key and wants the text as fast as possible.

flowchart LR
    Browser[Browser] -->|"GET /aZ4kPq9"| App[App Server]
    App -->|"check key"| Cache[(Cache)]
    Cache -->|"hit: text"| App
    Cache -.->|"miss"| DB[(Metadata DB)]
    DB -.->|"blob_ref + expiry"| App
    App -.->|"fetch body"| Blob[(Blob Store)]
    Blob -.->|"text"| App
    App -.->|"fill cache"| Cache
    App -->|"200 text"| Browser

On a cache hit the app returns the text immediately. On a miss it reads the metadata row (checking expires_at), fetches the body from the blob store, writes the result back into the cache, and returns it. This is the cache-aside pattern. Because pastes are immutable, cached values never go stale (only expire), which makes caching unusually safe here.

3. Combined architecture

A load balancer sends creates and reads to stateless app servers. Creates hit the key generator, blob store, and metadata DB. Reads go cache-first, fall through to the metadata DB and blob store, and popular public pastes are served from a CDN before they ever reach the app.

flowchart LR
    Client[Client / Browser] --> CDN[CDN / edge cache]
    CDN -->|"hit: cached paste"| Client
    CDN -.->|"miss"| LB[Load Balancer]
    LB -->|"POST /api/pastes"| App[App Servers]
    LB -->|"GET /key"| App
    App -->|"create: get key"| Keys[Key Generator]
    App -->|"read: check cache"| Cache[(Redis Cache)]
    App -->|"metadata"| DB[(Metadata DB)]
    App -->|"body"| Blob[(Blob Store)]
    App -->|"paste or URL"| Client

Because reads dominate by 100:1, the read path gets the most optimization (cache + CDN). The write path mainly needs a unique key and the correct two-store write order.

Deep dives

1. Metadata / blob storage split — the heart of this question

The defining decision is what goes in the metadata DB versus the blob store, and how reads and writes flow across both.

What goes where.

2. Unique key generation

Each paste needs a short, unique, hard-to-guess key. This is the same problem a URL shortener solves, so lean on that rather than re-deriving it. The two workable families:

• Counter + base62. Keep a monotonic counter; base62-encode the next integer into a fixed-width key. Zero collisions by construction, so no read-before-write. Sequential integers produce enumerable keys, though — bad for unlisted pastes — so scramble the ID with a reversible permutation (a Feistel network or multiply-by-a-coprime) before encoding. The full counter-and-range-allocation mechanics are worked through in the URL shortener guide's key-generation section — cross-reference it instead of repeating it.
• Random + collision check. Generate a random 7-char base62 key and INSERT with a uniqueness constraint; on the rare conflict, retry with a new key. At ~10 writes/sec across a mostly-empty 3.5-trillion keyspace, collisions are astronomically rare, so the retry cost is negligible. Keys are unpredictable by default — exactly what unlisted pastes want.

3. Read scaling and expiration

At ~1K reads/sec concentrated on a hot minority of pastes, the read path must avoid touching the metadata DB and blob store on every hit — and expired pastes must actually get cleaned up.

Layered read path: CDN → cache → stores.

flowchart LR
    Browser[Browser] -->|"GET /aZ4kPq9"| CDN[CDN / edge cache]
    CDN -->|"hit: cached paste"| Browser
    CDN -.->|"miss"| App[App Server]
    App -->|"check key"| Cache[(Redis cache)]
    Cache -->|"hit: text"| App
    Cache -.->|"miss"| DB[(Metadata DB)]
    DB -.->|"blob_ref + expiry"| App
    App -.->|"fetch body"| Blob[(Blob Store)]
    App -.->|"fill cache"| Cache
    App -->|"200 text"| CDN
    CDN --> Browser

Because pastes are immutable, a cached body is never stale — it can only expire. That removes the invalidation-on-update headache a mutable system has. Two rules keep the cache honest with expiry:

• Cache popular paste bodies in Redis under LRU eviction, so hot pastes stay resident.
• Set the cache/CDN entry's TTL at or below the paste's expires_at, so an expired paste can never keep resolving from a stale edge copy. A public paste can be CDN-cached at the edge; an unlisted paste should not be broadly edge-cached — serve it with Cache-Control: private (or no-store) so the CDN passes each request through to the origin rather than storing it at the edge.

Expiration: lazy delete + a sweeper / lifecycle policy. Combine two mechanisms:

• Lazy delete on read. On any read, compare now to expires_at. If expired, return 404 immediately (and delete the row + blob, or mark them dead). This guarantees an expired paste is never served, even if the background cleanup is behind.
• Background sweeper + object-store lifecycle. A periodic job scans for rows past expires_at and deletes the row and its blob, reclaiming space that lazy delete alone would leave dangling for never-read pastes. Simpler still: set an object-store lifecycle policy that auto-expires the body, and have the sweeper (or the same policy on the metadata store, e.g. a DynamoDB TTL) drop the metadata. The store does the deletion for you, no cron to babysit.

Tradeoffs & bottlenecks

• Blob store vs. DB for the body. Object storage is cheap, durable, and scales without bound, but adds a second store and a hop. A DB column is simpler but bloats the table and burns costly IOPS on large values. For anything past toy scale, split them.
• Random vs. counter keys. Random keys are unpredictable for free but need a uniqueness constraint and (rarely) a retry; counter keys never collide but are enumerable unless you permute the ID space. Unlisted-by-default nudges Pastebin toward random.
• Eager vs. lazy expiry. Eager deletion (delete exactly at expires_at) is precise but needs tight scheduling; lazy-plus-sweeper is simpler and robust but can leave a paste physically present a little past its expiry (still never served, thanks to the read-time check). Lazy + sweeper is the pragmatic default.
• Cache stampede on a viral paste. If a hot paste falls out of cache, many concurrent misses can hammer the metadata DB and blob store for the same key. Mitigate with request coalescing (single-flight) or serving a slightly stale copy while one request refills. Immutability makes this safe.
• Single points of failure. The metadata DB, cache, and key generator all need replication; the read path should degrade gracefully — a cache outage should fall back to the stores, which must be provisioned to absorb the surge.

Extensions if asked

Add only the extension that changes the design discussion:

• Analytics / view counts. Emit a view event asynchronously to a queue and aggregate offline; do not synchronously write a counter before every read, or you reintroduce a per-read DB write on the hot path.
• Syntax highlighting / rendering. Render on the client, or render once and cache the HTML alongside the raw body. Keep the raw text as the source of truth in the blob store.
• Compression. Text compresses well (often 3–5×); gzip bodies before storing to cut blob storage and transfer cost, decompress on read (or let the CDN serve compressed).
• Abuse prevention. A public paste service attracts spam, malware, and secret-leak dumps. Add create rate limits, size caps, content scanning, and takedown tooling.
• Accounts and dashboards. If users can own and list their pastes, add owner_id indexing and auth — but keep unlisted pastes reachable only by key.

What interviewers look for & common mistakes

What interviewers usually reward:

• The storage split, stated plainly — small metadata in a DB, the large body in object storage, the row holding only a reference, with the reasoning (row bloat, IOPS, backups).
• Tight scoping — cutting editing, highlighting, accounts, and content search up front, and noting pastes are immutable write-once objects (which makes caching safe).
• Read-first thinking — a 100:1 read-heavy system fronted by cache-aside + CDN, with hit-rate and immutability reasoning.
• Sensible key generation — reusing the shortener's counter/random approaches and naming the enumeration risk for unlisted pastes.
• Real expiration — lazy delete on read for correctness plus a sweeper/lifecycle policy for space reclamation.

Before you finish, do a quick mistake check:

• Did you put the paste body in a blob store and keep only a reference in the DB?
• Did you make the read path cache-first (and CDN-fronted for public pastes)?
• Did you choose keys that are not trivially enumerable for unlisted pastes?
• Did you handle expiry with both lazy delete and a background sweeper / lifecycle policy?
• Did you size storage from the blob bodies (TBs/year), not from the metadata rows?
• Did you order the two-store write as body-first, then metadata?