Scale Interview
System DesignHard

Design a Stock Exchange

Order Matching Engine System Design

Overview

A stock exchange takes a stream of buy and sell orders for a symbol and continuously matches them against each other, fairly and deterministically, at a rate of hundreds of thousands to millions of orders per second — and it must do so in microseconds, never lose an accepted order, and publish every resulting trade to thousands of subscribers almost instantly. The heart of it is a order book per symbol and a matching engine that applies one rule above all others: price-time priority.

It is labeled "Hard" because the usual system-design toolkit actively works against you here. The reflex to "put it in a database and scale horizontally" is exactly wrong: a disk round-trip is milliseconds, and matching must happen in microseconds, so there is no database in the hot path. The reflex to "parallelize with locks and threads" is also wrong: matching must be deterministic — given the same input orders in the same order, the engine must always produce the same trades, so the same symbol is processed by a single thread, not a lock-contended pool. And yet the system must never lose an accepted order and must survive a crash by recovering to the exact pre-crash state. Reconciling "in-memory and single-threaded for speed and determinism" with "durable and recoverable" is the real puzzle, and the answer is event sourcing over a sequenced append-only log.

The architecture that falls out of these constraints: an order gateway validates and risk-checks incoming orders; a sequencer stamps each accepted order with a strictly increasing number (defining the one true order of events); the matching engine consumes that sequenced stream in memory and emits trades; and a market-data publisher fans the resulting book updates out to the world.

In this walkthrough you'll scope the system, size it, model the book, design the API, draw the hot path, then go deep on the matching engine and order-book data structure, on durability and crash recovery via an event-sourced log, and on low-latency market-data dissemination.

Out of scope (say so): clearing and settlement (T+1 transfer of cash and shares), the regulatory reporting pipeline, market surveillance for manipulation, the auction mechanics of the opening/closing cross, and margin/lending. We keep the core: accept an order, match it against a limit-order book with price-time priority, guarantee durability, and publish market data.

Functional requirements

  • Place a limit order. A trader submits a buy or sell for a symbol at a specified price and quantity. It matches immediately against the opposite side if the price crosses; any unfilled remainder rests in the book.
  • Place a market order. A trader submits a buy or sell for a quantity with no price limit; it matches against the best available prices until filled (or the book is exhausted).
  • Cancel an order. A trader cancels a resting order by id; if it is still in the book (not yet fully filled), it is removed.
  • Match orders. The engine continuously matches incoming orders against the book by price-time priority, producing full fills, partial fills, and resting remainders.
  • Publish market data. Every book change (new resting order, trade, cancel) is published so subscribers can reconstruct the book and see the last trade.

Out of scope (state it): clearing/settlement, auctions and the open/close cross, stop and iceberg order types (mention in extensions), short-sale locates, and post-trade regulatory reporting.

Non-functional requirements

  • Ultra-low latency. End-to-end order-to-ack and order-to-trade must be microseconds to low milliseconds, not "fast enough." This single constraint forces in-memory matching with no database on the hot path.
  • Determinism. Given the same sequence of orders, the engine must always produce the exact same trades. This is required both for fairness (no reordering advantage) and for recovery (replaying the log must reproduce the state exactly).
  • High throughput. Hundreds of thousands to millions of orders/sec across the market, with per-symbol hot spots for liquid names.
  • Durability of every accepted order. Once the gateway acks an order, it must survive a crash. An accepted order that vanishes is worse than a rejected one — it breaks the audit trail and the trader's position.
  • Fairness. Strict price-time priority. No order may jump the queue at its price level.
  • Correctness over availability at the extreme. If durability cannot be guaranteed for a moment, the engine must stop accepting rather than accept an order it might lose. A halted market is recoverable; a lost order is not.

Estimations

State assumptions first; the goal is to justify the "in-memory, single-threaded, log-backed" design, not to produce exact numbers.

Rounded assumptions

  • Symbols: ~10,000 actively traded.
  • Order rate: peak ~1M orders/sec market-wide, but heavily skewed — the top ~5 names each do ~50–100K orders/sec (≈250–500K/sec combined), with the remaining ~500K/sec spread across the long tail of thousands of near-idle symbols. The market-wide total and the hot-symbol rate compose because only a handful of names run hot at once.
  • Match rate: many orders don't immediately trade (they rest or cancel). Assume ~20% cross and produce trades~200K trades/sec market-wide at peak.
  • Latency budget: target p99 < 100 microseconds inside the matching engine (order in → trade/ack out), and low-single-digit milliseconds end-to-end including the gateway and network.
  • Book size in memory: a liquid symbol may hold 100K–1M resting orders across a few thousand price levels. At ~100 bytes per order that is ~10–100 MB per hot symbol — trivially memory-resident.
  • Durability write rate: every accepted order is one append to the sequenced log → ~1M appends/sec market-wide, sharded per symbol/sequencer. Each sequencer (and thus each symbol) writes its own log partition, so recovery and standby replay filter by partition cleanly — no single global log to demultiplex.
  • Market-data fan-out: thousands of subscribers × 200K updates/sec → the publisher's job is fan-out, not computation.

How the numbers affect the design

Signal from the numbers Design decision
p99 < 100 µs; disk I/O is ~1–10 ms No database in the hot path. The book lives in RAM; matching is pure in-memory work.
50–100K orders/sec on one symbol, must be deterministic One symbol = one single-threaded matching engine. No locks, no reordering.
Different symbols are independent Shard by symbol across many engines; the market scales horizontally by symbol, not by threading one book.
~1M accepted orders/sec must be durable Append every accepted order to a sequenced write-ahead log before it hits the engine; replay reconstructs state.
10–100 MB per hot book Memory is not the constraint; determinism and latency are. Keep everything resident.
200K trades/sec × thousands of subscribers Market data is a fan-out problem — use multicast / a fan-out tier, not per-subscriber recomputation.

Caution

Do not reach for a database in the matching path. A single disk or network round-trip is 1–10 ms — 10,000× your entire latency budget. The order book is an in-memory data structure; durability comes from an append-only log written outside the match loop, not from the database being the book.

Core entities / data model

Four things matter: the order, the price level (a FIFO queue of orders), the order book (two sides of price levels), and the trade produced by a match. Prices are represented as integer ticks, never floats.

Entity Key fields
Order order_id, symbol, side (buy/sell), type (limit/market), price_ticks (integer), quantity, remaining_qty, sequence_no, trader_id, timestamp, status
PriceLevel price_ticks → FIFO queue of resting order_ids (arrival order preserved)
OrderBook symbol → { bids: sorted price levels (descending), asks: sorted price levels (ascending) }
Trade trade_id, symbol, price_ticks, quantity, buy_order_id, sell_order_id, sequence_no, timestamp

Prices are integers, always. A price of $100.25 with a $0.01 tick is stored as the integer 10025 ticks. Floating-point prices are a correctness bug waiting to happen: 0.1 + 0.2 != 0.3 in IEEE-754, and an exchange cannot have two orders at "the same price" that compare unequal. Integer ticks make price comparison exact and price-level bucketing trivial.

Ticks also define the price-level buckets. The book's bid side is a set of price levels sorted descending (best bid = highest buy price first); the ask side is sorted ascending (best ask = lowest sell price first). Within each price level, orders sit in a FIFO queue ordered by arrival — this is what encodes the time half of price-time priority.

The sequence_no is the linchpin. It is assigned by the sequencer at acceptance and defines the single, total order of every event for a symbol. It is what makes the log replayable and the engine deterministic — the book is a pure function of the ordered sequence of orders and cancels.

About timestamp. The timestamp on an Order/Trade is assigned once, by the sequencer at ingest, and stored in the log alongside the sequence_no. It is data, not a wall-clock read the engine performs. The matching engine never reads the clock — it consumes the timestamp already recorded in the event — so replay reproduces every timestamp identically and determinism holds.

Caution

Do not model prices as floating-point numbers. Use integer ticks. Float rounding produces orders that should be at the same price level but aren't, silently breaking price-time priority and producing non-reproducible matches — fatal for both fairness and replay-based recovery.

Where each lives. The OrderBook, PriceLevels, and Orders all live in memory inside the matching engine process for that symbol — this is the whole point. The durable copy is the sequenced event log (the write-ahead log of accepted orders and cancels); Trades produced by matching are also appended to a log. A relational store or data warehouse is fed asynchronously off the log for reporting, positions, and end-of-day reconciliation — never read during matching.

API design

The interface is a small set of order operations plus a market-data subscription. Traders connect over a low-latency binary protocol (in practice FIX or a proprietary binary protocol over a persistent TCP session), but we describe the logical operations.

Place an order

POST /orders
Body: { "symbol": "AAPL", "side": "buy", "type": "limit",
        "price_ticks": 19025, "quantity": 100, "client_order_id": "c-123" }
200 OK
Returns: { "order_id": "...", "sequence_no": 88123401, "status": "accepted" }

The gateway validates and risk-checks the order (deep dive 1), the sequencer assigns sequence_no, and the order is durably logged before this ack is returned. status: accepted means "durable and admitted to the engine," not "matched." Fills arrive asynchronously on the trader's execution stream. The client_order_id lets the client dedupe on reconnect — the gateway rejects a duplicate as idempotent. This dedup state lives in the gateway, over a bounded window (recent ids per trader/session), not forever — long enough to cover a reconnect, not a permanent index.

Cancel an order

POST /orders/{order_id}/cancel
Body: { "client_order_id": "c-124" }
200 OK
Returns: { "status": "cancel_accepted", "sequence_no": 88123402 }

Cancel is itself a sequenced event — it enters the same ordered stream as orders. Whether the cancel actually removes anything depends on a race: if the order fully filled before the cancel's sequence number, the cancel is a no-op ("too late to cancel"). This is resolved deterministically by sequence order (deep dive 1).

Order / execution stream (per trader)

GET /executions  (streamed: WebSocket / TCP session)
Events: order_accepted, partial_fill, fill, cancel_accepted, cancel_rejected, order_rejected

The trader's private stream of what happened to their orders — fills, partial fills, cancels. Delivered asynchronously because matching is decoupled from the request/response. A cancel_rejected carries a reason: unknown (no such order id), already-filled (fully filled before the cancel's sequence number), or already-canceled (a prior cancel already removed it).

Market-data subscription (public)

SUBSCRIBE market-data/{symbol}   (multicast or streamed)
Feeds:
  L1: best bid / best ask / last trade         (top of book)
  L2: full depth — every price level and its aggregate size
Each message carries a strictly increasing seq_no for gap detection.

Public market data: L1 is the top of book; L2 is full depth. Every message carries a sequence number so subscribers can detect and recover gaps (deep dive 3).

High-level architecture

The system is a pipeline, not a request/response service. An order flows through stages, each doing one job, and the hot stages hold no database:

gateway (validate + risk) → sequencer (assign total order + durable log) → matching engine (in-memory match) → market-data publisher (fan-out)

1. The order hot path

flowchart LR
    Trader["Trader\n(FIX/binary)"] -->|"place/cancel"| GW["Order Gateway\n(validate + risk check)"]
    GW -->|"accepted order"| SEQ["Sequencer\n(assign seq_no)"]
    SEQ -->|"append"| LOG[("Sequenced Log\n(WAL, append-only)")]
    SEQ -->|"sequenced stream"| ME["Matching Engine\n(in-memory book,\nsingle thread/symbol)"]
    ME -->|"trades + book updates"| MD["Market-Data\nPublisher"]
    ME -->|"private fills"| GW
    GW -->|"execution stream"| Trader
    MD -->|"L1/L2 feed"| Subs["Subscribers"]
  1. The gateway authenticates the session, validates the order (symbol exists, price is a valid tick, quantity > 0), and runs risk checks (does the trader have buying power / position limits?). Rejected orders never reach the engine.
  2. The sequencer assigns a strictly increasing sequence_no and appends the accepted order to the durable log before letting it proceed. This is the durability boundary: past this point the order cannot be lost.
  3. The matching engine consumes the sequenced stream and matches against its in-memory book, emitting trades and book updates. It touches no database.
  4. The market-data publisher fans book updates out to public subscribers; private fills go back to the originating trader via the gateway.

Tip

The durability boundary is the sequencer's log append, and it sits before the matching engine — not after. The engine can then be a pure in-memory function of the log. If it crashes, you replay the log; you never needed the engine to persist anything itself.

2. Sharding by symbol

flowchart TD
    In["Incoming orders"] --> Router["Symbol Router"]
    Router -->|"AAPL, MSFT…"| E1["Engine Shard 1\n(single thread each symbol)"]
    Router -->|"GOOG, AMZN…"| E2["Engine Shard 2"]
    Router -->|"TSLA, NVDA…"| E3["Engine Shard 3"]
    E1 --> MD["Market-Data Publisher"]
    E2 --> MD
    E3 --> MD

Different symbols are completely independent — an AAPL order never interacts with an MSFT order. So the market scales horizontally by symbol: route each symbol to an engine shard, and within a shard, each symbol's book is processed by a single thread. This is how you get millions of orders/sec across the market while every individual book stays single-threaded and deterministic.

3. Durability and recovery topology

flowchart LR
    SEQ["Sequencer"] -->|"append seq_no + order"| LOG[("Append-only\nEvent Log")]
    LOG -->|"replay on startup"| PRIMARY["Primary Engine\n(in-memory book)"]
    LOG -->|"same stream, live"| STANDBY["Hot Standby Engine\n(replays same log)"]
    PRIMARY -->|"heartbeat"| STANDBY
    STANDBY -.->|"promote on\nprimary failure"| PRIMARY

The primary and a hot standby both consume the identical sequenced log, so the standby's book is always a near-live mirror of the primary's. If the primary dies, the standby — already at (or a few events behind) the same state — is promoted. Because both are deterministic functions of the same ordered log, they cannot disagree (deep dive 2).

4. Combined architecture

flowchart LR
    Trader["Traders"] --> GW["Order Gateways\n(validate + risk)"]
    GW --> SEQ["Sequencer\n(total order)"]
    SEQ --> LOG[("Sequenced Event Log\n(WAL)")]
    SEQ --> ME["Matching Engines\n(sharded by symbol,\nsingle thread/book)"]
    ME --> TLOG[("Trade Log")]
    ME --> MD["Market-Data Publisher"]
    ME --> GW
    LOG -.->|"replay"| STANDBY["Hot Standbys"]
    LOG -.->|"async feed"| WH[("Reporting / Positions\n(warehouse, off hot path)")]
    MD --> Subs["Market-Data Subscribers"]

The hot path (gateway → sequencer → engine → publisher) holds no database. The reporting store, position keeper, and analytics all read off the log asynchronously, never blocking a match. This is the core discipline: the fast path is in memory and log-backed; everything durable-but-slow hangs off the log downstream.

Deep dives

1. The matching engine and the order-book data structure

This is the centerpiece. Two things must be exactly right: the data structure that gives O(1)-ish access to the best price with correct FIFO ordering, and the concurrency model that keeps matching deterministic.

The order-book data structure. The book has two sides. Each side is a collection of price levels, and each price level is a FIFO queue of resting orders:

  • Best-price access must be fast. Every incoming order matches against the best opposite price first. So the price levels are kept sorted: bids descending (highest buy first), asks ascending (lowest sell first). A common implementation is a sorted structure (balanced tree / skip list) keyed by price for O(log P) access, or — since ticks are integers over a bounded range — a flat array indexed by tick giving O(1) best-price lookup with a pointer to the current best level.
  • Within a price level, FIFO. Orders at the same price are a queue ordered by arrival (sequence number). The head of the queue is the next to match. This is the time in price-time priority: the earliest order at a price fills first.
  • Cancel needs O(1) lookup. A hash map order_id → node lets a cancel find and unlink a resting order from its price-level queue in O(1), rather than scanning.
flowchart TD
    subgraph Book["AAPL Order Book"]
        subgraph Asks["Asks (ascending)"]
            A1["190.27 → [o9(50), o12(100)]"]
            A2["190.26 → [o4(200)]"]
            A3["190.25 → [o1(100), o7(300)]  ← best ask"]
        end
        subgraph Bids["Bids (descending)"]
            B1["190.24 → [o2(100), o8(150)]  ← best bid"]
            B2["190.23 → [o5(400)]"]
            B3["190.22 → [o3(100)]"]
        end
    end

The matching algorithm for an incoming buy limit order at price P, quantity Q:

while Q > 0 and best_ask_price <= P:            # price crosses
    resting = head of best_ask price level      # FIFO: earliest first
    fill = min(Q, resting.remaining_qty)
    emit Trade(price = resting.price, qty = fill)   # trade at RESTING price
    Q -= fill; resting.remaining_qty -= fill
    if resting.remaining_qty == 0:
        remove resting from queue (it fully filled)
    # advance to next resting order / next price level as needed
if Q > 0:                                        # unfilled remainder
    insert new resting order at price P (tail of that level's FIFO queue)

Two subtleties interviewers probe:

  • Trades execute at the resting (maker) order's price, not the incoming (taker) price. A buy limit at 190.30 hitting a resting ask at 190.25 trades at 190.25 — the incoming order gets price improvement. This is standard and follows from matching against the best resting price.
  • A market order is the same loop with P = +∞ (buy) or P = 0 (sell) — it walks the book taking the best prices until Q is filled or the book side is empty. The pseudocode above shows the buy side (best_ask_price <= P); the sell side mirrors it, matching against bids while best_bid_price >= P. Any unfilled remainder of a market order is not rested (it's canceled or rejected, per exchange rules) — a market order never becomes a resting limit. In practice a bare market order is dangerous: on a thin book the P = +∞ loop can sweep to absurd prices, so real exchanges bound it with price bands / collars (reject fills beyond a percent from the reference price) or convert it to a marketable-limit order capped at the collar.

The cancel race. A cancel is a sequenced event. If order o1's fill (sequence 100) precedes the cancel of o1 (sequence 105) in the stream, o1 is already gone when the cancel is processed → cancel is a no-op. If the cancel (sequence 100) precedes any further fill, o1 is unlinked and future orders never see it. Because everything is one ordered stream on one thread, this race has exactly one deterministic outcome — there is no ambiguity, no lock, no lost update.

Why single-threaded per symbol beats locking.

Multi-threaded book with locks — parallelize matching across cores (avoid — nondeterministic and slow under contention)

Let a pool of threads process incoming orders for a symbol concurrently, guarding the shared book with locks (or fine-grained per-price-level locks).

  • Nondeterminism. Two orders arriving nearly simultaneously can be matched in either order depending on thread scheduling — the trades produced are not reproducible. This breaks fairness (whoever's thread grabbed the lock first "wins" arbitrarily) and, worse, breaks replay: replaying the log won't reproduce the same trades, so recovery is impossible.
  • Contention. The best-price levels are the hottest data in the entire system — every order touches the top of book. Threads serialize on that lock anyway, so you pay lock overhead for essentially no parallelism.
  • Verdict: avoid. Locks buy you nothing here (the hot path serializes on the top of book regardless) and cost you determinism, which is non-negotiable.
Single thread per symbol — one deterministic sequence of operations (recommended)

One symbol's book is owned by exactly one thread, which processes the sequenced stream in order, one event at a time. No locks. No shared mutable state across threads for that book.

  • Deterministic by construction. The book is a pure function of the ordered event stream. Same log → same trades, every time. This is what makes replay-based recovery and hot-standby mirroring work.
  • Cache-friendly and fast. A single thread pinned to a core, working a memory-resident book with no lock instructions and no cache-line ping-pong between cores, is astonishingly fast — this is how real engines hit sub-microsecond per-order processing.
  • Scales by symbol, not by thread. Parallelism comes from running thousands of independent symbol engines across cores/machines. Each is single-threaded; the market as a whole is massively parallel.
  • Verdict: the production design. LMAX Disruptor, most modern exchanges, and every serious matching engine use single-writer-per-book. You get determinism and speed simultaneously — the locking approach gives you neither.

Warning

The most common architectural mistake here is proposing a lock-protected, multi-threaded matching engine to "scale." It destroys determinism (and therefore recoverability) and gains no real parallelism because the top of book is a single hot contention point. The right answer is single-threaded per symbol; scale by sharding symbols across engines.

Pre-trade risk checks at the gateway. Before an order is sequenced, the gateway must confirm the trader can actually afford it — but this check sits on the hot path, so it cannot be a synchronous DB lookup.

  • What's checked. Available buying power = the cash / buying-power limit − open buy exposure (the notional already committed by this trader's resting buy orders); a buy is rejected if it would exceed it. Position limits — max long/short quantity per symbol and aggregate exposure caps — are checked the same way. Sells are checked against inventory (or short-sale allowance, out of scope here).
  • Where the state lives. The gateway keeps an in-memory position/exposure cache per trader. It is updated asynchronously from the trade log, never by a synchronous DB call on the hot path — the gateway reads local memory to decide accept/reject in nanoseconds.
  • The accept-then-fill race. At accept time the gateway reserves the order's exposure against the trader's cash / buying-power limit (optimistically, before any fill). When the fill later arrives on the trade log, the async updater converts reserved open exposure into a settled position and frees the difference for partial fills or cancels. Because the reservation happens synchronously at accept but the position update is async, the cache holds both resting-order exposure and confirmed positions so it never double-counts or lets a trader over-commit while a fill is in flight.

2. Durability and crash recovery via an event-sourced log

The tension: the engine is in-memory (for speed) and single-threaded (for determinism), but an accepted order must survive a crash and the engine must recover to the exact pre-crash state. The resolution is event sourcing over a write-ahead log.

The core idea. Do not persist the order book. Persist the ordered stream of events that produced it — every accepted order and cancel, in sequence-number order, appended to a durable log before the matching engine applies them. The book is then a derived structure: book = replay(events). If the engine crashes, you rebuild the exact book by replaying the log from the start.

sequenceDiagram
    participant GW as Gateway
    participant SEQ as Sequencer
    participant LOG as Durable Log
    participant ME as Matching Engine
    GW->>SEQ: accepted order
    SEQ->>SEQ: assign seq_no
    SEQ->>LOG: append(seq_no, order)  [fsync / replicated]
    LOG-->>SEQ: durable ack
    SEQ->>ME: deliver sequenced order
    ME->>ME: match in memory, emit trades
    Note over SEQ,ME: order is durable BEFORE the engine touches it

Why the append happens before matching. The durability boundary must be before the state changes, not after. If the engine matched first and logged after, a crash between the two would lose the trade — the book would move but the log wouldn't show why, and replay would diverge. Logging the input first means the input stream is the source of truth and the engine is a replaceable, deterministic consumer of it.

Determinism is what makes replay correct. Replay only reconstructs the exact book if matching is a pure, deterministic function of the ordered input — the engine reads no wall clock (any timestamp is pre-assigned by the sequencer and lives in the log), does no random tie-breaking, and has no thread races. This is why determinism (deep dive 1) is a hard requirement and not just a nicety: without it, replay(events) would not equal the pre-crash book, and recovery would be impossible.

Snapshots — bounding replay time. Replaying a full trading day's log at startup is too slow. Periodically write a snapshot of the in-memory book tagged with the last-applied sequence_no. Recovery becomes: load the latest snapshot, then replay only the log entries after its sequence number.

recover():
    snapshot = load_latest_snapshot()      # book state @ seq_no = S
    replay log entries with seq_no > S      # only the tail since the snapshot
    resume consuming the live stream
Snapshot on the matching thread — pause matching to serialize the book (avoid stalling the hot path)

Taking the snapshot on the matching thread stalls matching for the duration of serialization — a latency spike for every order in flight. For a large book this pause is visible and unacceptable on the hot path.

Better: snapshot from the hot standby (which mirrors the same log) or from a fork/copy-on-write so the primary keeps matching. The snapshot only needs a consistent sequence_no boundary, not the primary's attention.

Snapshot from the hot standby — serialize the mirror, not the primary (recommended)

The hot standby already replays the same log to the same state, so let it do the serialization work while the primary keeps matching untouched.

  • Zero hot-path impact. The primary never pauses; the standby serializes its book at a chosen sequence_no boundary, which is all recovery needs.
  • Consistent boundary. The standby freezes at seq S, writes the snapshot tagged with S, then resumes replaying — recovery loads that snapshot and replays only seq_no > S.
  • Verdict: the standard approach. Snapshotting is I/O-heavy and latency-sensitive to interrupt, so move it off the writer that's on the hot path.

The log itself must be durable and ordered. In practice the sequenced log is written to a replicated, append-only store (a low-latency messaging system such as Aeron/Kafka-style logs, or a purpose-built replicated journal). "Durable" means the append is acknowledged only after it is on stable storage / replicated to enough nodes to survive a node loss — that's what the gateway waits on before acking the trader.

Hot-standby failover. Because the standby replays the identical sequenced log, its book is always in lockstep (or a few events behind) the primary. On primary failure:

flowchart LR
    LOG[("Sequenced Log")] --> P["Primary\n(seq_no = N)"]
    LOG --> S["Standby\n(seq_no = N-k)"]
    P -->|"heartbeat lost"| DET["Failure Detector"]
    DET -->|"promote"| S
    S -->|"finish replaying\nto log head, then\naccept new orders"| NEWP["New Primary"]

The standby finishes replaying to the head of the log (catching up the last k events), then takes over accepting new orders. No state is lost because the log — not the engine's memory — is the source of truth. The key correctness property: both engines are deterministic replays of the same ordered log, so they cannot diverge as long as matching is strictly deterministic (same ordered inputs, no wall-clock reads, no random tie-breaking).

Caution

Do not treat the in-memory order book as the durable state. It is derived. The durable state is the sequenced, append-only event log written before matching. If you snapshot the book but log matching outputs after applying them, a crash between apply and log corrupts recovery. Log the ordered inputs first; rebuild the book by deterministic replay.

3. Market-data dissemination — low-latency fan-out with gap recovery

Every book change must reach subscribers fast and in order. This is a fan-out problem, not a computation problem — the engine already produced the update; the job is to deliver it to thousands of consumers with microsecond-class latency and a way to recover dropped messages.

L1 vs L2 feeds. Split the feed by weight:

  • L1 (top of book): best bid, best ask, last trade price/size. Tiny and by far the most subscribed. Publish on every top-of-book change.
  • L2 (full depth): every price level and its aggregate resting size on both sides. Heavier; publish incremental updates (this level's size changed) rather than full snapshots.

Sending aggregate size per price level (not individual orders) keeps L2 compact and hides other traders' individual order sizes.

Fan-out transport — multicast vs TCP.

Per-subscriber TCP unicast — send each update to each subscriber separately (avoid for the core low-latency feed)

The publisher opens a TCP connection to each subscriber and sends every update N times, once per subscriber.

  • Write amplification. 200K updates/sec × thousands of subscribers = the publisher serializes and sends the same bytes millions of times/sec. It becomes the bottleneck and adds latency that grows with subscriber count.
  • Unfair latency. Subscriber #1 gets the update microseconds before subscriber #3000 — an information-timing advantage that violates fair-access expectations.
  • Verdict: avoid for the primary real-time feed. Fine for slow, out-of-band clients; wrong for the hot fan-out.
UDP multicast for the real-time feed + a TCP recovery channel (recommended)

Publish each update once to a multicast group; the network fabric replicates it to every subscriber. The publisher does O(1) work per update regardless of subscriber count, and all subscribers receive it at essentially the same instant (fair timing).

  • Sequence numbers on every message. Each feed message carries a strictly increasing seq_no. A subscriber detects a gap the instant it sees seq_no jump (received 5010 after 5008 → 5009 is missing).
  • Gap recovery via a separate channel. Multicast/UDP is lossy and unordered by nature, so pair it with a retransmission service: on a detected gap, the subscriber requests the missing sequence range over a reliable (TCP) side channel, or reads from a snapshot/recovery feed that periodically publishes a full book image so a subscriber can resync without replaying from the beginning.
  • Verdict: the industry-standard design. The fast path is fire-once multicast with sequence numbers; correctness (gap recovery) is handled out-of-band so it never slows the common case. This is how real exchange feeds (e.g. ITCH-style protocols) work.
flowchart LR
    ME["Matching Engine"] -->|"book updates\n(+ seq_no)"| PUB["MD Publisher"]
    PUB -->|"multicast once\n(seq_no on every msg)"| Sub1["Subscriber A"]
    PUB -->|"multicast once"| Sub2["Subscriber B"]
    Sub1 -.->|"gap detected:\nrequest seq 5009"| RETX["Retransmit /\nSnapshot Service"]
    RETX -.->|"resend range\n(TCP side channel)"| Sub1

Reconstructing the book (mid-day join). A subscriber that connects after the open can't replay the whole day, so it resyncs from a snapshot — the same snapshot-plus-replay pattern used for engine recovery, applied to the client's view of the book:

  1. Request a book snapshot at some seq_no = S (full L2 image from the snapshot/recovery service).
  2. Meanwhile buffer live incremental multicast updates; on load, apply only those with seq_no > S (discard the ones already reflected in the snapshot).
  3. From then on, detect gaps by sequence number — any jump (e.g. 5010 after 5008) means a missing message.
  4. On a gap, request retransmit of the missing range over the TCP side channel, or pull a fresh snapshot if the client has fallen too far behind.

Tip

Sequence numbers are the connective tissue of the whole system. The sequencer stamps them, the log orders by them, the engine replays by them, hot standbys stay in lockstep by them, and market-data subscribers detect gaps by them. One monotonic counter provides ordering, determinism, recovery, and gap detection all at once.

Tradeoffs & bottlenecks

  • In-memory determinism vs. durability. Matching in memory with no database is what buys microsecond latency, but memory isn't durable. The resolution is event sourcing: log the ordered inputs durably before matching, and treat the book as a derived, replayable structure. You get both — at the cost of a recovery/snapshot mechanism you must build carefully.
  • Single-threaded matching vs. parallelism. One thread per symbol gives determinism and (surprisingly) top speed, but caps a single symbol at one core's throughput. Mitigation: shard symbols across engines. A single ultra-hot symbol that exceeds one core is a genuine ceiling — you cannot split one book across cores without losing determinism.
  • Multicast vs. TCP for market data. Multicast is O(1)-per-update fan-out with fair timing but is lossy and needs a gap-recovery channel. TCP is reliable but has per-subscriber write amplification and unfair timing. Production uses multicast for the fast feed + TCP recovery for gaps.
  • Snapshot frequency. Frequent snapshots bound recovery time but cost CPU/IO (take them off the hot path, from a standby). Infrequent snapshots make replay long. Tune to a target recovery-time objective.
  • Risk checks in the hot path. Pre-trade risk checks (buying power, position limits) add latency at the gateway but are non-negotiable for correctness. Keep them fast (cached limits, in-memory position state); never a synchronous DB call.
  • Sequencer as a single point. The sequencer defines the one true order and is inherently a single writer per shard — a potential bottleneck and SPOF. Mitigate with per-symbol/shard sequencers and a replicated log. On failover, ordering is recovered from that replicated log: the new sequencer resumes from the last durably-replicated sequence number, so it never reuses or skips a seq_no (the gap between "assigned" and "durably replicated" is exactly what a fresh sequencer must not re-enter). In-flight orders that were never durably acked are simply re-driven by clients on reconnect — idempotent via client_order_id, so a re-send is deduped rather than double-booked.

Extensions if asked

If the interviewer wants more than the core engine:

  • More order types. Stop / stop-limit (activate when the market reaches a trigger price — held in a separate trigger structure, injected as a live order when triggered); iceberg / reserve (only a slice is visible in the book, replenished as it fills); IOC/FOK (immediate-or-cancel, fill-or-kill — match then discard remainder). Each is a variation on the matching loop, not a new architecture.
  • Auctions / opening & closing cross. A different matching mode that collects orders over a window and computes a single clearing price that maximizes executed volume, rather than continuous matching. Mention it; don't design it here.
  • Clearing and settlement. Post-match, trades flow to a clearinghouse that nets positions and settles cash/shares (T+1). Entirely downstream of the engine — fed off the trade log.
  • Market surveillance. Detecting spoofing / layering / wash trades by analyzing the order-and-cancel stream. A separate analytics pipeline reading the log, not part of the hot path.
  • Circuit breakers / halts. Rules that pause trading on extreme moves. The engine checks a halt flag and rejects/queues orders while halted — a control-plane concern layered on the gateway.

What interviewers look for & common mistakes

What interviewers usually reward:

  • No database in the hot path. Stating up front that matching is in-memory and that a disk round-trip blows the latency budget by 10,000× — and knowing durability comes from a log written outside the match loop.
  • Price-time priority, correctly. Best price first, then FIFO by arrival within a price level, and matching at the resting (maker) price. Knowing that the FIFO queue is what encodes "time."
  • Single-threaded per symbol, sharded across symbols. Recognizing that determinism forbids a lock-contended multi-threaded book, and that parallelism comes from sharding symbols — not from threading one book.
  • Determinism as a first-class requirement — and connecting it to recovery: the book is a deterministic replay of the ordered log, which is why determinism matters (not just fairness).
  • Event sourcing for durability. Logging ordered inputs before matching, snapshots to bound replay, and hot-standby failover that stays in lockstep because both replay the same log.
  • Integer ticks, not floats. Knowing that float prices silently break price-level equality and reproducibility.
  • Multicast + sequence numbers + gap recovery for market data, and understanding fan-out (not computation) is the problem.

Before you finish, do a quick mistake check:

  • Did you keep the database out of the matching hot path, with durability from a sequenced log written before matching?
  • Did you specify price-time priority precisely — best price, then FIFO — and match at the resting order's price?
  • Did you argue for single-threaded-per-symbol matching and reject the lock-based multi-threaded book, then scale by sharding symbols?
  • Did you make determinism explicit and tie it to replay-based recovery?
  • Did you use event sourcing (log the ordered inputs) plus snapshots and a hot standby for durability and failover?
  • Did you represent prices as integer ticks rather than floating-point?
  • Did you design market data as multicast fan-out with sequence numbers and a gap-recovery/snapshot channel?

Practice this live

Run this exact question with our AI voice interviewer and get feedback.

Start the interview →