Design a Unique ID Generator

Snowflake Distributed ID System Design

Overview

A unique ID generator hands out identifiers that are guaranteed never to repeat, even when many machines are generating them at the same time. Almost every large system needs one: a row in a database needs a primary key, an event in a log needs an ID, a message in a chat needs an ID so replies can point at it. When you have one database, the database can hand out IDs for you (an auto-increment column). The interesting problem starts when you have many machines, spread across many data centers, all creating records at once, and you still need every ID to be unique — without the machines constantly talking to each other to agree.

It is labeled "Easy" because the goal is narrow: produce unique numbers, fast. But it is a favorite interview question because it forces you to reason about coordination — how do independent machines avoid stepping on each other without a central bottleneck? The well-known answer, popularized by Twitter, is called Snowflake: pack a timestamp, a machine id, and a small counter into a single 64-bit number, and generate it locally on each machine with no coordination on the hot path.

In this walkthrough you will scope the problem, size the ID, design the 64-bit layout and the tiny bit of per-machine state it needs, draw how a machine generates an ID by itself and how it gets its machine id at startup, and then go deep on the three things that separate a shallow answer from a strong one: the ladder of approaches (and why Snowflake wins), the bit layout plus the clock-skew problem, and how machine ids are assigned without collisions.

Functional requirements

These are the things the system must do. Keep the list short and get the interviewer to agree before you build anything.

• Generate a unique ID. A single call — next_id() — returns an identifier that has never been returned before and never will be again, across the whole fleet.
• IDs are numeric and fit in 64 bits. They should be storable as a standard 64-bit integer (a BIGINT column, a long), because that is small, fast to index, and cheap to compare.
• IDs are roughly time-ordered. An ID generated later should almost always be numerically larger than one generated earlier. This is the property that makes IDs useful as sort keys and cursors, not just as unique tags.

Explicitly out of scope (state this — scoping is graded): a global total order (a strict ranking where every ID everywhere is perfectly ordered — Snowflake does not promise this, and the deep dive explains why), human-readable IDs, and hiding information leaked by the ID. We will still discuss the leakage, because it is a real tradeoff.

Non-functional requirements

These are the qualities the system must have. For an ID generator the dominant ones are:

• Uniqueness (absolute). This is non-negotiable. Two calls anywhere in the fleet must never return the same ID. A duplicate primary key can corrupt data or silently drop rows.
• Roughly time-sortable. IDs generated later should sort after earlier ones. The standard term is k-sortable — ordered by time except within a tiny window where near-simultaneous IDs from different machines may swap. This is not the same as a strict total order.
• High throughput, low latency. A busy service may need millions of IDs per second across the fleet, and each next_id() sits in the write path of every insert, so it must return fast — sub-microsecond (nanoseconds) as an in-process library call, or microseconds when reached over the local network as a sidecar.
• No single point of failure. If getting an ID required a round trip to one central server, that server would be both a bottleneck and a way for the whole system to go down. IDs must be generated locally.
• Highly available. If the ID service is down, nothing can be created. It needs to keep working even when parts of the infrastructure fail.

Estimations

State your assumptions out loud, but keep the math lightweight. The point is to justify why 64 bits and why the layout looks the way it does.

Rounded assumptions

• Peak generation: ~1M–10M IDs/sec across the whole fleet.
• Fleet size: up to ~1,000 machines generating IDs concurrently.
• Per-machine peak: a busy machine might need thousands of IDs per millisecond during a burst.
• Lifetime: the scheme should keep producing unique IDs for decades, not years.

That is enough. Now say what the numbers imply for the design.

How the numbers affect the design

The single most important arithmetic is the bit budget. A 64-bit integer has 64 bits to spend. Spend them on time (so IDs sort), on machine identity (so machines don't collide), and on a counter (so one machine can mint many IDs in the same millisecond). The exact split is the second deep dive.

Core entities / data model

There is no big database here. The "data model" is really two things: the shape of the ID itself, and the tiny bit of state each machine keeps to generate the next one.

The 64-bit ID layout. The ID is a single 64-bit integer, split into fields. A common split (the one Twitter's Snowflake used) is:

 bit 1             bit 2..42            bit 43..52    bit 53..64
 |  sign (1 bit)  |  timestamp (41 bits) | machine (10) | sequence (12) |
 |       0        |   ms since epoch     |   0..1023    |   0..4095     |

 total = 1 + 41 + 10 + 12 = 64 bits

• Sign bit (1 bit): always 0, so the number is always positive. Many languages only have signed 64-bit integers, and a negative ID sorts and reads badly, so this bit is left unused.
• Timestamp (41 bits): milliseconds since a chosen start date (the epoch). In the high bits, so bigger timestamp → bigger ID → time-sortable.
• Machine id (10 bits): which machine generated this ID, 0 to 1023. This is what keeps two machines from colliding in the same millisecond.
• Sequence (12 bits): a per-millisecond counter, 0 to 4095, that increments for each ID a single machine makes within the same millisecond, then resets when the millisecond rolls over.

Per-machine state. To generate the next ID, each machine only needs to remember two small values in memory:

That is the entire state. No shared table, no per-request database read. Generating an ID is just reading the clock, comparing it to last_timestamp, updating sequence, and packing the three fields together.

Interfaces

This system exposes a function, not a REST resource, so we describe the call contract rather than HTTP endpoints. It can be embedded as a library inside each application server, or run as a sidecar / small service that the app calls over the local network.

Generate one ID

next_id() -> int64
Returns: 1382054186741039104   (a 64-bit integer)

The core call. It reads the local clock, combines it with the machine id and the sequence counter, and returns a packed 64-bit integer. It never blocks on the network. The only time it waits is the rare case where the per-millisecond sequence overflows and it must wait a fraction of a millisecond for the clock to advance (covered in the deep dive).

Generate a batch (optional)

next_ids(count) -> int64[]
Body: { "count": 100 }
Returns: [ 1382054186741039104, 1382054186741039105, ... ]  (100 IDs)

A convenience for callers that need many IDs at once (e.g. a bulk insert). When the generator runs as a service rather than an in-process library, batching amortizes the network round trip — one request yields many IDs — so the network cost per ID drops toward zero. The service still generates each ID by the same local rule; batching is only about reducing calls.

High-level architecture

There are two flows worth separating for a beginner: how a running machine generates an ID all by itself (the hot path, which must never coordinate), and how a machine gets its machine id when it starts up (the one moment coordination is allowed).

1. Local ID generation (the hot path)

Every machine generates IDs on its own using only in-memory state. No network call, no shared database.

flowchart LR
    Caller["App code<br/>calls next_id()"] --> Gen["ID Generator<br/>(in-process)"]
    Gen -->|"1. read current time (ms)"| Clock["Local clock"]
    Gen -->|"2. compare to last_timestamp<br/>3. bump or reset sequence"| State["In-memory state<br/>machine_id, last_timestamp, sequence"]
    Gen -->|"4. pack timestamp + machine_id + sequence"| ID["64-bit ID"]
    ID --> Caller

    style Gen fill:#dcfce7,stroke:#16a34a,color:#14532d

Read the diagram in this order:

1. The app calls next_id().
2. The generator reads the local clock in milliseconds.
3. It compares that time to last_timestamp. If it is a new millisecond, it resets sequence to 0. If it is the same millisecond, it increments sequence.
4. It packs the timestamp, machine id, and sequence into one 64-bit integer and returns it.

The key point: steps 1–4 are all local memory operations. Even at millions of IDs per second, no machine ever contacts another machine to mint an ID. This is why the design has no hot-path bottleneck.

2. Machine-id assignment at startup

The one thing machines must not collide on is their machine id — if two machines both think they are machine 7, they can produce identical IDs in the same millisecond. So each machine gets a unique machine id exactly once, when it boots, from a coordinator.

flowchart LR
    Node["New machine<br/>booting up"] -->|"1. request a machine id"| Coord["Coordinator<br/>(leases unique ids)"]
    Coord -->|"2. lease id = 7 (unique)"| Node
    Node -->|"3. store id in memory,<br/>start generating"| Gen["ID Generator<br/>ready"]
    Coord -.->|"tracks which ids are in use"| Registry[("id registry")]

    style Coord fill:#e0f2fe,stroke:#0284c7,color:#0c4a6e

Read the diagram in this order:

1. A machine starts up and asks the coordinator for a machine id.
2. The coordinator hands back an id that no other live machine holds (e.g. 7).
3. The machine keeps that id in memory and begins generating IDs locally.

The coordinator (a service like ZooKeeper or etcd) is only touched at startup — never on the hot path. This is the crucial split: coordinate once, generate locally forever after. If the coordinator is briefly down, running machines keep minting IDs; only new machines have to wait for an id, and that is rare.

3. Combined architecture

Putting it together: a fleet of machines each generate IDs locally from in-memory state; a coordinator hands out unique machine ids at startup only.

flowchart TB
    subgraph Fleet["Fleet of machines (each generates locally)"]
      N1["Machine 7<br/>next_id() → local"]
      N2["Machine 42<br/>next_id() → local"]
      N3["Machine 108<br/>next_id() → local"]
    end
    Coord["Coordinator<br/>(machine-id leases, startup only)"]
    Coord -.->|"lease id at boot"| N1
    Coord -.->|"lease id at boot"| N2
    Coord -.->|"lease id at boot"| N3

    N1 -->|"unique 64-bit IDs"| Apps["Databases / logs / messages"]
    N2 -->|"unique 64-bit IDs"| Apps
    N3 -->|"unique 64-bit IDs"| Apps

    style Coord fill:#e0f2fe,stroke:#0284c7,color:#0c4a6e

The dashed lines (coordination) happen once per machine, at boot. The solid lines (ID generation) happen millions of times per second, entirely locally. That contrast is the heart of the design.

Deep dives

1. Approaches to generating unique IDs

This is the headline. Below are three strategies in increasing order of robustness for a large, distributed system. Each is a valid answer somewhere; the goal is to understand why the naive ones break at scale and why the last one is the standard.

Machine A: 9f1c2d4e-7a8b-4c3d-9e0f-1a2b3c4d5e6f
Machine B: 2b7e1516-28ae-4c2a-9f1d-3c4d5e6f7a8b
(different, with overwhelming probability — no coordination)

Machine A → ticket server → 1001
Machine B → ticket server → 1002
Machine A → ticket server → 1003
(perfectly ordered, small, unique — but every ID is a network round trip)

Same millisecond T, machines 7 and 42 both generate:

Machine 7:  [ T | machine=7  | seq=0 ]   → unique
Machine 7:  [ T | machine=7  | seq=1 ]   → unique (same ms, seq bumped)
Machine 42: [ T | machine=42 | seq=0 ]   → unique (different machine id)

No two IDs collide: same-machine IDs differ by sequence,
cross-machine IDs differ by machine id. No coordination needed.

2. Snowflake anatomy, capacity, and the clock problem

Recall the 64-bit split: 1 sign + 41 timestamp + 10 machine + 12 sequence. Let's make the capacity concrete, then face the one thing that can break it: the clock.

Capacity math (this justifies the layout):

sequence: 2^12 = 4,096 IDs per millisecond, per machine
          → 4,096 × 1,000 ms ≈ 4.1M IDs/sec per machine
machine:  2^10 = 1,024 machines
          → 4.1M × 1,024 ≈ 4.2B IDs/sec fleet-wide (ceiling)
time:     2^41 ms ≈ 69.7 years of unique timestamps from the epoch

That comfortably covers our "millions/sec, ~1,000 machines, decades" estimate, with room to spare. If you needed more machines you could steal bits from the sequence (fewer IDs/ms but more machines), and vice versa — the split is a tunable budget, not a fixed law.

Why the timestamp goes in the high bits. Because the timestamp occupies the most significant bits, comparing two IDs numerically compares their timestamps first. So a later ID is almost always a larger number — the IDs are k-sortable. This matters for two reasons:

• Index locality: new IDs are slightly larger than recent ones, so they append near the end of a B-tree index instead of scattering — the opposite of random UUIDs, and much friendlier to insert performance. The flip side: because every insert lands near the right-most index page, that page becomes a write hotspot (contention on one page under heavy concurrent inserts) — the cost of good locality, and the one place random keys actually win.
• Cursor pagination: because IDs increase with time, "give me the next page after ID X" is a cheap range scan — the ID doubles as a time-ordered cursor. Because the ordering is only k-sortable, this is approximate: good enough for feeds and infinite scroll, but not exact strict creation order (near-simultaneous IDs from different machines can be paged out of true time order).

They are only roughly sorted (k-sortable, not a strict total order): two IDs made in the same millisecond on different machines can be ordered by machine id rather than true time, so near-simultaneous IDs may swap. That is fine for indexing and paging; it is not a global clock.

The clock-skew problem. The timestamp comes from the machine's own clock. Clocks are kept in sync by NTP. For small drift NTP usually slews — nudging the clock faster or slower so it never actually runs backward — and only steps it (a sudden jump, which can be backward) for a large offset. Reading a monotonic clock avoids most back-steps, but you must still guard against the ones that slip through — a VM pause or migration, a host reboot, or an operator setting the clock by hand. If the clock does jump back, the machine could produce a timestamp it has already used — and with the same machine id and sequence, that means a duplicate ID. This is clock skew, and it is the classic Snowflake failure.

Here is the danger, step by step:

t = 1000 ms  → generate IDs with timestamp 1000, last_timestamp = 1000
t = 1001 ms  → generate IDs with timestamp 1001, last_timestamp = 1001
[NTP steps the clock BACKWARD by 2 ms]
t = 999 ms   → clock now reads 999, which is < last_timestamp (1001)
             → if we blindly used 999, we would REISSUE timestamps
               1000 and 1001 → DUPLICATE IDs. Not allowed.

The fix is to remember last_timestamp and never trust the clock to run backward. The robust default is to advance logical time monotonically: issue from max(now, last_timestamp) rather than from the raw clock, so the timestamp you pack never decreases even if the wall clock briefly does.

• On each next_id(), read the clock. If now >= last_timestamp, use now — the normal case.
• If now < last_timestamp by a large margin (say, seconds), something is genuinely wrong — refuse to issue IDs (throw) and raise an alarm. This is what the original Twitter Snowflake did on now < last_timestamp. Better to stop than to risk duplicates; some systems take the machine out of rotation until its clock is trustworthy again.
• For a tiny back-step you may instead wait until the clock catches back up to last_timestamp, then proceed — a weaker fallback that's fine when the jump is sub-millisecond. When you wait, yield or sleep (release the CPU) rather than hot-spin in a tight loop, so you don't peg a core while stalling.

Sequence overflow. Within a single millisecond, the sequence counter can only reach 4,095. If a machine is asked for a 4,097th ID in the same millisecond, the counter has no more values. The fix mirrors the clock fix: wait (yielding, not hot-spinning) for the next millisecond, reset sequence to 0, and continue. Because a millisecond is a tiny wait and 4,096 IDs/ms is already a huge rate, this is rare and cheap. This leans on the same monotonic-timestamp invariant: the sequence resets to 0 only when logical time genuinely advances to a greater millisecond, so a clock that dips backward and then forward can't reset the sequence early and re-collide within a millisecond it already used.

Clock rewind across a restart. The max(now, last_timestamp) guard only protects a continuously running process, because last_timestamp lives in memory. If the machine crashes and restarts while the clock has been rewound (an NTP step-back, a VM pause or migration, a host reboot with a bad hardware clock), the in-memory last_timestamp is gone — the fresh process reads the rewound clock, believes it is earlier than it really is, and happily reissues timestamps it already used before the crash → duplicate IDs. The fix is to persist the last-issued timestamp durably (a small high-watermark written periodically to disk or the coordinator) and, on startup, refuse to generate until the wall clock passes that persisted value (or block for a bounded delay until it does). This is the restart-time counterpart of the in-memory guard, and it is the failure mode most people miss.

3. Assigning machine ids, and the sortability/coordination tradeoffs

The whole scheme rests on one guarantee: no two live machines share a machine id. If they do, they can produce identical IDs in the same millisecond. So how does each machine get a unique id — without collisions, and without coordinating on every request?

The key idea is that assignment happens once, at startup, and never on the hot path. A few ways to do it, concept first:

• Leased from a coordinator (the robust default). A coordination service (ZooKeeper or etcd) hands each booting machine a machine id no other live machine holds, and reclaims it when the machine dies (via a lease/heartbeat). This is the safest because the coordinator actively prevents two machines from getting the same id.
• Static configuration. Bake the machine id into each machine's config or deployment. Simple, but you must never accidentally deploy the same id twice — human error becomes a duplicate-ID risk.
• Derive from the host. Compute a machine id from something already unique per host (e.g. a hash of the private IP or hostname). Zero coordination, but you must fit the derived value into 10 bits without collisions, which gets fragile as the fleet grows.

Whatever the mechanism, the win is the same: coordination happens once at startup, never per ID. That is precisely why Snowflake scales — the one moment of agreement is amortized over the millions of IDs the machine then mints alone.

flowchart LR
    Boot["Machine boots"] -->|"lease a machine id (once)"| Coord["Coordinator"]
    Coord -->|"id = 7, held via lease"| Boot
    Boot --> Run["Generate millions of IDs<br/>locally, no coordination"]
    Coord -.->|"reclaim id 7 if machine dies"| Coord

    style Run fill:#dcfce7,stroke:#16a34a,color:#14532d

Now the tradeoffs worth naming in the interview:

• Roughly ordered, not strictly ordered. Snowflake gives per-machine monotonic IDs (a single machine's IDs strictly increase) but only roughly global order across machines — near-simultaneous IDs from different machines can be ranked by machine id rather than true time. Claiming a strict global total order is a common trap; if you need one, you need a central sequencer (Approach B) and its bottleneck. Say plainly that k-sortable is what Snowflake offers.
• Information leakage. Because the ID embeds a timestamp and a counter, anyone who sees IDs can infer when a record was created and roughly how many you create (watch the sequence climb, or diff two IDs' timestamps). A random UUID leaks none of this. If hiding creation rate/volume matters, that is a real con of time-based IDs — note it as the tradeoff against random IDs.
• Modern coordination-free alternatives. If you want time-ordering without running a coordinator for machine ids, mention UUIDv7 and ULID: 128-bit IDs with a timestamp in the leading bits plus random tail bits. They keep the time-sortability that makes indexing nice, need no machine-id coordination at all (the random tail makes collisions astronomically unlikely, the same probabilistic guarantee as UUIDv4 — not an absolute one), but cost you the compact 64-bit size — the same size-vs-simplicity tradeoff as Approach A, with sortability added back.

Tradeoffs & bottlenecks

• Local generation vs. strict ordering. Generating locally (Snowflake) removes the central bottleneck but gives up a strict global total order. A central sequencer gives strict order but reintroduces the bottleneck. Pick based on whether you truly need total order (usually you don't — k-sortable is enough).
• Clock dependence. The design trusts the machine clock. That buys coordination-free generation but exposes you to clock skew; advancing logical time via max(now, last_timestamp) (and refusing on a large backward jump) is mandatory, not optional — plus a persisted high-watermark so a crash/restart with a rewound clock can't reissue used timestamps.
• Bit budget is fixed at 64. More machine-id bits means fewer sequence bits (lower per-machine throughput) and vice versa. There is no way to add capacity without either shrinking another field or going to 128 bits. Choose the split for your fleet size and per-machine rate.
• Sequence exhaustion under bursts. A single machine is capped at 4,096 IDs/ms with a 12-bit sequence. A hotter machine must wait for the next millisecond, or you rebalance bits. Know your per-machine ceiling.
• Coordinator at startup. The machine-id coordinator is a dependency, but only at boot. It should be replicated so new machines can still start during a failure — but a brief coordinator outage does not stop running machines from minting IDs.
• Information leakage. Time- and counter-based IDs reveal creation time and volume. If that is sensitive, use random IDs and accept their downsides, or don't expose raw IDs externally.

Extensions if asked

If the interviewer wants to push beyond the core generator, add only the extension that changes the design discussion:

• Coordination-free machine ids. Replace the coordinator entirely with UUIDv7 or ULID (128-bit, time-sortable, no machine id needed), trading the compact 64-bit size for zero coordination. Good when you can't run ZooKeeper/etcd.
• Multi-data-center IDs. Split the machine-id field into a data-center id + a machine id (e.g. 5 bits each) so IDs are unique across regions without a global coordinator. This is the original Twitter layout.
• Hiding the timestamp. If leakage matters, apply a reversible scramble to the ID before exposing it externally, or keep internal Snowflake IDs private and expose separate opaque tokens. There's no dedicated guide yet; this is a natural follow-up if the interviewer asks about privacy.
• Handling clock skew without waiting. Some designs add a bit or borrow from the sequence as a "generation" counter that bumps on a detected back-step, letting the machine keep issuing IDs after a small clock jump without pausing. A good discussion point once the basic clock guard is covered.

What interviewers look for & common mistakes

What interviewers usually reward:

• Local, coordination-free generation. Recognizing that the central requirement is minting IDs without a hot-path round trip, and that a central counter is the bottleneck to avoid.
• A clear 64-bit layout with justified fields. Timestamp in the high bits for sortability, machine id for uniqueness, sequence for per-millisecond throughput — with capacity math (4,096 IDs/ms, 1,024 machines, ~69 years).
• Handling the clock going backward. Advancing logical time via max(now, last_timestamp), refusing on a large backward jump, and persisting a high-watermark so a restart with a rewound clock can't reissue used timestamps — plus the sequence-overflow "wait for next millisecond" fix.
• Honest ordering claim. Saying IDs are k-sortable / roughly ordered, not a strict global total order.
• Startup-only machine-id assignment. Explaining how machine ids stay unique via a coordinator (or config/derivation) touched only at boot.

Before you finish, do a quick mistake check:

• Did you keep IDs at 64 bits and justify the field split with capacity math?
• Did you generate IDs locally with no coordination on the hot path?
• Did you handle the clock moving backward (advance via max(now, last_timestamp), refuse on a large jump, persist a high-watermark across restarts) to prevent duplicates?
• Did you handle sequence overflow within a millisecond (wait for the next ms)?
• Did you say the ordering is roughly by time (k-sortable), not a strict global total order?
• Did you explain how each machine gets a unique machine id without colliding, at startup only?
• Did you note the information-leakage tradeoff of time-based IDs vs. random UUIDs?