Scale Interview
System DesignHard

Design Robinhood

Stock Trading App System Design

Overview

Open Robinhood, tap a stock, watch the price tick live, and hit Buy — and it feels instant. Behind that tap sits a subtle truth that decides the entire design: Robinhood is a broker, not an exchange. It does not match your buy against someone else's sell. It validates that you can afford the trade, then routes your order out to a venue — a public exchange or a market maker — and waits for the fill to come back. The matching engine, the limit order book, price-time priority — all of that lives at the exchange (see stock-exchange). Robinhood's job is the client side: take the order, route it, book the result, and keep millions of users' portfolios correct.

It is labeled "Hard" because three unforgiving problems stack on top of each other. First, order routing under async fills: you submit an order to a venue and the acks, partial fills, cancels, and rejects trickle back over time — you must track a state machine, never submit the same order twice, and never lose a fill. Second, real-time market-data fan-out: on a volatile morning, millions of users all stare at the live quote for the same handful of hot symbols, and pushing every tick to every user melts your servers. Third, portfolio consistency: a user's cash and share positions are real money, so a fill must update them atomically and exactly once, and your books must reconcile against the clearing firm's.

In this walkthrough you'll scope the broker, size it, model the ledger, design the API, draw the order and market-data paths, then go deep on the three things that decide correctness: the order lifecycle and routing, the market-data fan-out, and portfolio/positions consistency.

Out of scope (say so): the exchange's internal matching engine and limit order book (that's stock-exchange), the funding rail that moves cash between the user's bank and the brokerage (that's the payment-system / ACH problem), options and crypto mechanics, tax-lot reporting, and the regulatory reporting pipeline. We keep the broker core: place an order, route it, book the fills, fan out quotes, and keep the portfolio correct.

Functional requirements

  • Place an order. A user submits a buy or sell for a symbol and quantity (market or limit). The broker validates buying power (or share availability for a sell), routes it to a venue, and reports back.
  • Cancel an order. A user cancels a still-open (unfilled or partially filled) order; the broker sends a cancel request to the venue, which may or may not win the race against a fill.
  • Receive and book fills. Fills, partial fills, acks, cancels, and rejects arrive asynchronously from the venue; each updates the order's state and, on a fill, the user's portfolio.
  • Stream real-time market data. Millions of users watch live quotes for the symbols they hold or watch; the broker ingests the venue feed and fans it out over WebSockets.
  • View portfolio, positions, and history. A user sees current cash, share positions, total value, and an ordered history of orders and fills.

Out of scope (state it): the exchange's matching engine, the bank-funding rail, order types beyond market/limit (stop/trailing — mention in extensions), options/crypto, and tax reporting.

Non-functional requirements

  • Exactly-once effect on orders and fills. A double-tapped Buy must not place two orders, and a re-delivered fill must not credit shares twice. This forces idempotency on order submission and on fill booking.
  • Strong consistency on the portfolio. Cash and positions are the CP choice in CAP — we'd rather reject an order than let a user spend buying power they don't have.
  • Durability of accepted orders and booked fills. Once we ack an order or book a fill, it must survive a crash. A lost fill means a user's shares vanish — unacceptable.
  • Low-latency, high-fan-out market data. Millions of concurrent WebSocket subscribers, hot symbols with tens of thousands of watchers each. Quotes must feel live (sub-second) but need not be tick-perfect.
  • Bursty write load. Order volume spikes ~10× at the 9:30am open and on volatile "meme-stock" days. The order path must shed and queue load without dropping orders.
  • Auditability. Every order transition and every fill is traceable and immutable — regulators, disputes, and reconciliation all read these records. This pushes us to an append-only ledger.

Estimations

State assumptions; the goal is to justify the routing pipeline, the pub/sub fan-out, and the ledger — not to hit exact numbers.

Rounded assumptions

  • Users: ~25M accounts, ~5M daily active, ~1M concurrent at the open.
  • Orders: ~10M orders/day ⇒ ~115/sec average, but bursty: the 9:30am open and volatile days push ~5–10K orders/sec peak. Order volume is low by web standards — the difficulty is correctness under bursts, not raw QPS.
  • Fills: each order produces 1–N fill reports (partials) ⇒ assume ~2 fills/order ⇒ ~20M fill events/day to book idempotently.
  • Market data (the real firehose): ~10K tradable symbols; the venue feed emits ~1–5M quote updates/sec market-wide at peak. Users don't need every tick — they need a smooth quote.
  • Fan-out: ~1M concurrent WebSocket connections; a hot symbol (say the meme stock of the day) may have tens of thousands to hundreds of thousands of simultaneous watchers. This is the scaling problem: one symbol's update × 200K watchers = huge write amplification if done naively.
  • Ledger writes: ~20M fills/day × 2 entries (cash + position) ⇒ ~40M ledger rows/day — modest to store, the hard part is booking each exactly once.

How the numbers affect the design

Signal from the numbers Design decision
Orders are low-QPS but every one is money Optimize the order path for correctness/idempotency, not throughput.
Fills arrive async, may re-deliver Book fills idempotently, keyed by venue fill id.
1–5M quote updates/sec, users want smooth not tick-perfect Conflate quotes (drop intermediate ticks); don't fan out every tick.
A hot symbol has 200K watchers Pub/sub by symbol topic + a fan-out tier; publisher does O(1) work per update, not O(subscribers).
Order load spikes 10× at the open Queue order submissions; never drop an accepted order.
Positions and cash are real money Strongly consistent ledger; derive buying power and positions from it.

Caution

Do not size this system like an exchange. The order rate is tiny (~thousands/sec at peak). The two things that are actually large are the market-data fan-out (millions of concurrent watchers) and the correctness burden (never double-order, never lose a fill, never let the portfolio drift). Design for those, not for QPS.

Core entities / data model

The model separates the order (intent + lifecycle), the fill (an immutable execution report from the venue), the position/cash ledger (the source of truth for what the user owns), and the idempotency key (dedup for retried submissions). Money is stored as integer minor units (cents), never floats — 0.1 + 0.2 != 0.3 in IEEE-754, and a rounding error on money silently loses or conjures value.

Entity Key fields
Order order_id (PK), user_id, symbol, side (buy/sell), type (market/limit), limit_price, quantity, filled_qty, avg_fill_price, status, client_order_id, venue_ref, created_at
Fill fill_id (PK — venue-assigned, unique), order_id, quantity, price, venue, executed_atimmutable
LedgerEntry entry_id (PK), user_id, account (cash / position:AAPL), direction (debit/credit), amount (integer minor units, or share count), fill_id, posted_at
Position user_id + symbol (PK), quantity, avg_cost, version — a derived cache of the ledger, not the truth
Wallet/Cash user_id (PK), cash_balance, buying_power (a derived cachesettled_cash − Σ open reservations), version — derived cache of the cash ledger
IdempotencyKey client_order_id (PK — unique per user), request_fingerprint, order_id, stored response, status, expires_at

Order.status is a forward-only state machine — the heart of routing correctness (deep dive 1):

stateDiagram-v2
    [*] --> PENDING
    PENDING --> REJECTED: "buying-power / validation fail"
    PENDING --> ROUTED: "sent to venue"
    ROUTED --> ACKED: "venue accepted"
    ACKED --> PARTIAL: "partial fill"
    PARTIAL --> PARTIAL: "another partial"
    ACKED --> FILLED
    PARTIAL --> FILLED
    ACKED --> PENDING_CANCEL: "cancel requested"
    PARTIAL --> PENDING_CANCEL: "cancel requested"
    PENDING_CANCEL --> CANCELLED: "cancel won"
    PENDING_CANCEL --> FILLED: "fill won the race"
    PENDING_CANCEL --> PARTIAL: "fill won the race"
    ROUTED --> REJECTED: "venue rejected"

The LedgerEntry rows are an append-only, double-entry record: a buy fill of 10 shares at $150 writes a credit of 10 shares to position:AAPL and a matching debit of $1,500 from cash. The two legs are in different units (shares vs cents), so they don't "sum to zero" — the invariant is that they're consistent: share_qty × fill_price == cash_delta at the reported fill price, and the cash ledger and share ledger are each independently append-only. A dropped or duplicated leg breaks that equality. Positions and cash balances are derived from summing the ledger; the Position and Wallet rows are performance caches updated in the same transaction (deep dive 3). You never overwrite a position in place — a bare mutable shares column has no audit trail and turns every concurrent fill into a lost-update race.

Important

The broker's ledger is your claim about what each user owns. The clearing firm / exchange is the source of truth for whether the shares and cash actually moved. Reconciliation (deep dive 3) is how you confirm your books match reality — a self-consistent ledger can still disagree with the outside world.

API design

A small REST API for orders plus a WebSocket subscription for market data. Every order-placing call carries a client-generated client_order_id for idempotency; every amount is an integer in minor units.

Place an order

POST /api/orders
Header: Idempotency-Key: <client-generated client_order_id>
Body: { "symbol": "AAPL", "side": "buy", "type": "limit",
        "limit_price": 15000, "quantity": 10 }
201 Created
Returns: { "order_id": "ord_123", "status": "PENDING" }   (replay returns the same body)

The broker validates buying power (buy) or share availability (sell), then routes to a venue (deep dive 1). status: PENDING means "accepted and about to route," not "filled" — fills arrive asynchronously on the WebSocket. The Idempotency-Key is mandatory: a double-tapped Buy or a network retry with the same key returns the original order instead of placing a second one.

Cancel an order

POST /api/orders/{order_id}/cancel
200 OK
Returns: { "order_id": "ord_123", "status": "PENDING_CANCEL" }

Requests cancellation of a still-open order. The broker forwards a cancel to the venue; whether it actually cancels depends on a race with an in-flight fill — the venue may report CANCELLED or may report a FILLED that beat the cancel. The final state is resolved by whichever report the venue sends (deep dive 1).

Get an order / order history

GET /api/orders/{order_id}
GET /api/orders?status=open&cursor=<opaque>&limit=50
200 OK
Returns: { "order_id": "ord_123", "status": "PARTIAL", "filled_qty": 4,
           "avg_fill_price": 14998, "fills": [ ... ] }

Read order state and its fills. Read-heavy path; served from a replica.

Get portfolio / positions

GET /api/portfolio
200 OK
Returns: { "cash": 250000, "buying_power": 180000,
           "positions": [ { "symbol": "AAPL", "quantity": 10, "avg_cost": 14990, "market_value": 152000 } ],
           "total_value": 402000 }

Current cash, buying power, positions, and total value — read from the derived caches, kept consistent with the ledger (deep dive 3). market_value folds in the live quote from the market-data service.

Subscribe to market data (WebSocket)

WS /api/marketdata
→ { "action": "subscribe", "symbols": ["AAPL", "TSLA"] }
← { "symbol": "AAPL", "bid": 14998, "ask": 15002, "last": 15000, "ts": 1720000000 }   (conflated stream)

The client subscribes to the symbols on screen; the server streams conflated quotes (deep dive 2). Subscriptions are dynamic — the client updates them as the user scrolls between screens, so the server only pushes what the user is actually looking at.

Fill receiver (called by the venue's drop-copy / execution feed, not the client)

Internal: venue execution report → Fill Booking Service
Body: { "fill_id": "vx_evt_998", "order_id": "ord_123", "qty": 4, "price": 14998, "ts": ... }

The venue reports executions asynchronously over a dedicated session (FIX drop copy or a proprietary feed). Each carries a venue-assigned fill_id stored under a unique constraint so a re-delivered report is booked at most once (deep dive 3).

High-level architecture

Three flows matter: the order write path (validate buying power → route to venue → ack), the fill path (async execution reports book into the portfolio), and the market-data path (ingest the venue feed → conflate → fan out over WebSockets). We'll draw each, then combine.

1. Order write path

An order claims its idempotency key, gets its buying power checked against the ledger, and is routed out to a venue — the broker never matches it.

flowchart LR
    App["Mobile App"] -->|"place order + Idempotency-Key"| OS["Order Service"]
    OS -->|"insert client_order_id unique or return stored result"| DB[("Orders + Ledger DB")]
    OS -->|"reserve buying power in ledger one tx"| DB
    OS -->|"routed order"| SOR["Smart Order Router"]
    SOR -->|"route to best venue"| Venue["Exchange / Market Maker"]
    OS -->|"return PENDING"| App
  1. The request arrives with a client-generated client_order_id. The Order Service inserts it under a unique constraint; if it already exists, this is a retry — return the stored result and stop.
  2. In one transaction, validate and reserve buying power (a buy reserves cash against the ledger; a sell reserves shares) so a second concurrent order can't spend the same funds (deep dive 3).
  3. Hand the accepted order to the Smart Order Router, which picks a venue (an exchange or a PFOF market maker) and submits it.
  4. Return PENDING immediately. Acks and fills arrive later on the fill path.

Caution

Do not build a matching engine here. The broker validates and routes; the exchange matches. A common and fatal interview mistake is to design a limit order book inside Robinhood — that's the venue's job, not the broker's.

2. Fill path (async execution reports)

The venue reports executions on its own schedule, over a dedicated session. Each report books into the portfolio idempotently.

flowchart LR
    Venue["Exchange / Market Maker"] -->|"execution report fill_id"| FB["Fill Booking Service"]
    FB -->|"insert fill_id unique dedup"| DB[("Orders + Ledger DB")]
    FB -->|"advance order state + post ledger entries one tx"| DB
    FB -->|"push order update"| Push["User Push / WebSocket"]
    Push -->|"filled 4 at 149.98"| App["Mobile App"]
  1. The venue POSTs / streams an execution report with a venue-assigned fill_id.
  2. The Fill Booking Service inserts the fill_id under a unique constraint. Already there? Duplicate delivery — ack and ignore (an idempotent consumer).
  3. For a new fill, in one transaction: advance the order state machine (ACKED → PARTIAL → FILLED), post the double-entry ledger rows (cash ↔ position), and release any over-reserved buying power.
  4. Push the update to the user's app.

3. Market-data path (ingest → conflate → fan out)

The venue feed is a firehose; the broker ingests it once, conflates it, and fans it out over pub/sub to WebSocket servers.

flowchart LR
    Feed["Venue Market-Data Feed (firehose)"] --> Ingest["MD Ingestor (one consumer)"]
    Ingest -->|"conflate per symbol"| PS["Pub/Sub (topic per symbol)"]
    PS --> WS1["WS Server 1"]
    PS --> WS2["WS Server 2"]
    WS1 -->|"quotes for subscribed symbols"| U1["Users A..N"]
    WS2 -->|"quotes"| U2["Users ..M"]

The ingestor consumes the venue feed once (not once per user), maintains the latest quote per symbol, and publishes conflated updates to a topic per symbol. WebSocket servers subscribe only to the topics their connected users care about, and push to those users. The publisher does O(1) work per symbol update regardless of how many users watch it (deep dive 2).

4. Combined architecture

flowchart LR
    App["Mobile App"] --> GW["API Gateway"]
    GW -->|"place / cancel order"| OS["Order Service"]
    OS -->|"idempotency + reserve buying power one tx"| DB[("Orders + Ledger DB (strongly consistent)")]
    OS --> SOR["Smart Order Router"]
    SOR --> Venue["Exchange / Market Maker"]
    Venue -->|"execution reports"| FB["Fill Booking Service"]
    FB --> DB
    FB --> Push["Push / WS"]
    Venue -->|"market-data feed"| Ingest["MD Ingestor"]
    Ingest --> PS["Pub/Sub (per-symbol topics)"]
    PS --> WSFleet["WebSocket Fleet"]
    WSFleet --> App
    GW -->|"portfolio reads"| Replica[("Read Replica")]
    Recon["Reconciliation Job"] -->|"ledger vs clearing firm"| DB
    Recon -.->|"flag mismatch"| Alert["Exceptions Queue"]

The Orders + Ledger DB is the strongly consistent hub: orders, the append-only ledger, idempotency keys. The order path reserves buying power and routes; the fill path books executions idempotently; the market-data path is a completely separate pipeline (a fan-out problem, not a money problem). A daily reconciliation job checks the ledger against the clearing firm's records — the safety net.

Tip

Keep the three paths separate. Orders and fills are a money-correctness problem (idempotency, ledger, consistency). Market data is a fan-out problem (pub/sub, conflation, WebSockets). Conflating them — e.g. trying to push fills through the same firehose as quotes — muddies both.

Deep dives

1. Order lifecycle and routing

This is where the "broker, not exchange" distinction becomes concrete. An order's life is: validate → reserve buying power → route to a venue → receive async acks and fills → finalize. The broker never matches; it orchestrates a lifecycle around an external venue that does.

Step 1 — validate buying power, atomically. Before routing a buy, confirm the user can afford it: buying_power >= limit_price × quantity (for a market order, use a conservative estimate from the current quote plus a buffer). The trap is the same check-then-act race as any money system — two orders reading the same buying power both pass, both route, and the user over-commits. The fix is to make the check and the reservation one atomic operation against the ledger:

BEGIN
  -- book the reservation as a ledger fact: available -> reserved, keyed by order_id
  INSERT LedgerEntry(user_id=:uid, type=reserve, amount=:cost, order_id=:oid)  -- UNIQUE(order_id, type)
  UPDATE wallets
     SET buying_power = buying_power - :cost, version = version + 1
   WHERE user_id = :uid AND version = :read_version AND buying_power >= :cost
  -- 0 rows affected  -> ROLLBACK: insufficient buying power, reject the order
  -- 1 row affected    -> we won; continue in the SAME tx:
  INSERT Order(..., status = PENDING)
  INSERT IdempotencyKey(client_order_id, order_id, ...)
COMMIT

The reservation debits buying power (not settled cash) so the funds committed to this open buy can't be double-spent by another order. Crucially the reservation is itself an append-only ledger entry (available → reserved, keyed by order_id), not just a decrement of a bare column — so buying power stays fully derivable (settled_cash − Σ open reservations) and the wallets.buying_power value is only a cache the reconciliation job can rebuild. When the fill lands, the reservation converts to a real cash debit; if the order is rejected or cancelled, the reservation is released (a reversing entry). This is the broker analog of authorize-then-capture.

Step 2 — idempotent submission. The client generates one client_order_id per intended order and reuses it on every retry. The Order Service claims it with an atomic insert under a unique constraint — whoever inserts wins and owns the order; a concurrent retry hits a duplicate-key error and replays the stored response. This is the only race-free way to stop a double-tapped Buy from placing two orders.

Server-generated order id, check-then-insert (avoid — double-order race)

Let the server mint the order id and "check if we've seen this request, insert if not."

  T1: no matching order found -> route order to venue
  T2: no matching order found -> route order to venue    (both passed)
  -> two orders on the venue, user bought 20 shares meaning to buy 10
  • Con: the server can't tell a retry from a new order without a client-supplied key, and the check-then-insert gap is a race window. On a flaky mobile network (which auto-retries), this double-orders constantly.
  • Verdict: avoid. The client must generate one key per intended order; the claim must be an atomic unique-constraint insert.
Server-side idempotency-key table without a client-generated key (works, but fragile)

Keep a server-side "requests seen" table and a check-then-insert, but derive the key from request fields (user, symbol, qty, a coarse timestamp) rather than a client-supplied one.

  • Con: the derived key can't distinguish a genuine second order (the user really does want another 10 shares) from a retry, and concurrent retries still race the check-then-insert unless every path locks the same row. It leans on a reaper to expire stale claims.
  • Verdict: better than a bare check-then-insert, but the honest fix is a client-generated key — the client is the only party that knows "this is the same intended order."
Client-generated client_order_id, atomic insert-or-return (recommended)

The client generates a client_order_id per intended order and reuses it on retries. The server inserts it under a unique constraint before routing; the winner routes, the loser replays the stored result. Pass the same id through to the venue too, so even a stray re-submission that reaches the venue is deduped there — defense in depth.

  • Pro: exactly-once order placement regardless of retries or concurrency; replays are cheap.
  • Con: you store and expire keys (a TTL longer than any realistic retry) and must handle an IN_PROGRESS claim whose owner crashed mid-route via a lease/timeout.
  • Verdict: recommended — the canonical dedup, identical in spirit to idempotent charges in payment-system.

Step 3 — route out (this is the part unique to a broker). The Smart Order Router decides where to send the order. Retail marketable orders typically go to a market maker under PFOF (which pays for the flow and fills from inventory, often with price improvement); other orders route to a public exchange. The router picks the venue on execution quality and submits over a session (FIX or a proprietary API). Crucially, the broker holds the order's identity (order_id) and maps it to the venue's reference (venue_ref), because every subsequent report references the venue's id. That mapping must be durably committed — if the broker crashes between sending the order and persisting venue_ref, a re-delivered fill can't be matched back to its order_id; on restart, reconcile any order stuck in ROUTED with no venue_ref by re-querying the venue before accepting new reports.

Step 4 — async reports drive the state machine. Acks, fills, partials, cancels, and rejects arrive over time. Each is applied to the state machine, and the transition rules must be strict:

sequenceDiagram
    participant App as Mobile App
    participant OS as Order Service
    participant V as Venue
    participant FB as Fill Booking
    App->>OS: place buy 10 AAPL
    OS->>OS: reserve buying power, status PENDING
    OS->>V: route order
    V-->>OS: ack, status ACKED
    V-->>FB: fill 4 at 149.98
    FB->>FB: book 4 shares, status PARTIAL
    V-->>FB: fill 6 at 150.01
    FB->>FB: book 6 shares, status FILLED
    Note over OS,FB: fills are async and may be partial or re-delivered

The cancel race. A user taps Cancel on a partially filled order. The broker sends a cancel to the venue, but a fill for the remaining shares may already be in flight. Two legal outcomes: the venue reports CANCELLED (cancel won) or reports a final FILLED (fill won). The broker cannot decide this itself — the venue is authoritative — so the order sits in PENDING_CANCEL until the venue's report resolves it. Booking must handle a fill that arrives after a cancel request: it's still a real execution and must be booked.

Warning

Fills are asynchronous, can be partial, can arrive out of order, and can be re-delivered. Never assume one order = one fill, and never book a fill without deduping on the venue fill_id. A re-delivered fill booked twice credits phantom shares and corrupts the portfolio.

2. Real-time market-data fan-out

On a volatile morning, a million users open the app and most of them are staring at the same handful of hot symbols. The venue feed emits millions of quote updates per second. The naive design — every user's connection subscribes to the raw feed, or the server pushes every tick to every watcher — collapses instantly. This is a fan-out problem, and the two levers are conflation (send less) and pub/sub topics (send once, fan out cheaply).

Lever 1 — conflate: you don't need every tick. A human looking at a price cannot perceive 500 updates/sec, and a quote that updates ~2–10× per second looks perfectly live. So the ingestor keeps only the latest quote per symbol and publishes on an interval, collapsing a burst of intermediate ticks into one update. This is the single biggest lever: it turns a 1–5M-update/sec firehose into a bounded, human-scale stream.

Push every venue tick to every subscriber (avoid — write amplification explosion)

Forward each raw tick straight through to every WebSocket watching that symbol.

  • Con: a hot symbol at 5K ticks/sec × 200K watchers = 1 billion messages/sec for one symbol. The WebSocket fleet drowns, latency balloons, and phones burn battery rendering updates no human can see. You're spending enormous resources delivering imperceptible detail.
  • Verdict: avoid. Retail quote display does not need tick-by-tick fidelity; conflate first.
Conflate globally at a fixed interval only (works, but wasteful on cold symbols and laggy on hot ones)

Publish the latest quote for every symbol every N ms regardless of whether anyone is watching or whether it changed.

  • Con: you publish updates for thousands of idle symbols nobody is watching (wasted work), and a fixed interval is a blunt instrument — too slow for a fast-moving hot symbol, too fast for a dormant one.
  • Better: conflate per symbol, on change, and only publish topics that have active subscribers. Combine with the pub/sub tier below so effort scales with attention, not with the symbol universe.
Conflate per symbol + pub/sub topic per symbol + WebSocket fan-out tier (recommended)

Ingest the feed once, maintain the latest quote per symbol, and publish conflated updates to a topic per symbol. WebSocket servers subscribe to a topic only when at least one connected user watches that symbol, and unsubscribe when the last one leaves. Each server receives one update per symbol per interval and fans it out to its locally connected watchers.

  • Publisher does O(1) per update. One conflated update per symbol goes to the topic; the pub/sub layer and the WS fleet handle replication. The publisher's cost is independent of subscriber count — the whole point.
  • Effort tracks attention. No subscribers for a symbol ⇒ no fan-out work for it. Hot symbols get their own topic that many WS servers subscribe to; cold symbols cost nothing.
  • Hot-symbol handling. If one topic's fan-out is still huge (200K watchers of the meme stock), the WS servers — not the ingestor — absorb it, and you scale the WS fleet horizontally. Each WS server pushes to its own connections only.
  • Verdict: recommended. Conflation bounds the rate; per-symbol pub/sub bounds the fan-out cost. Together they turn an impossible firehose into a tractable stream. (This mirrors the exchange's own market-data dissemination — see stock-exchange — but tuned for retail smoothness over microsecond fidelity.)

Connection management at a million WebSockets. Connections are sticky and stateful, so the WS fleet is separate from the stateless order API. Each WS server tracks which symbols its connections subscribe to and maintains the union as its topic subscription set. When a user scrolls from AAPL to TSLA, the client sends a subscribe/unsubscribe so the server only pushes what's on screen. On reconnect, the client re-sends its subscriptions and the server sends a snapshot (current quote) before resuming the live stream — the same snapshot-then-incremental pattern the exchange uses for mid-day joins.

Tip

The market-data path is read-only and eventually consistent — a quote a few hundred milliseconds stale is fine. Keep it completely separate from the order/ledger path, which is strongly consistent. Never let quote fan-out share a database or a code path with money movement.

3. Portfolio and positions consistency

A user's portfolio is real money: cash plus share positions. The source of truth is an append-only ledger of cash and share movements; positions and buying power are derived from it. The three hard requirements: a fill updates the portfolio atomically and exactly once, and the broker's books reconcile against the clearing firm's.

The ledger is the truth; positions are a cache. The tempting shortcut — a shares column you UPDATE on every fill — is the one to avoid. It has no audit trail (you can't prove how a position got there), it makes every bug silent, and every concurrent fill is a lost-update race. Instead, every fill posts balanced double-entry rows, and the position is derived:

Mutable position column: UPDATE positions SET shares = shares + qty (avoid — no history, silent corruption)

Keep one authoritative shares number per (user, symbol) and mutate it on each fill.

  • Con: no audit trail — you can't reconstruct or prove a position, so a missed or double-applied fill is undetectable. If a re-delivered fill runs the UPDATE twice, the user gains phantom shares and nothing alerts. Reconciliation against the clearing firm has nothing to compare against.
  • Verdict: avoid. For anything touching real shares/cash, a bare mutable column throws away the only thing that makes it correct — provability.
Mutable position column plus a separate audit log (better, but two things that can disagree)

Keep the mutable shares column for fast reads, but also append every fill to a separate audit log "for the record."

  • Con: the position and the audit log are updated by two writes that can diverge — a crash between them, or a bug in one path, and the "authoritative" number no longer matches its own history. You get the appearance of an audit trail without the guarantee that the balance is a pure function of it. Reconciliation now has to decide which of your own two records to trust.
  • Verdict: halfway there. The point of a ledger is that the balance is derived from the log, not stored alongside it — so make the log the truth and the position a cache of it, not a parallel record.
Append-only ledger, position + cash derived and cached (recommended)

Book each fill as balanced ledger entries, keyed by the venue fill_id, in one transaction:

buy 10 AAPL at $150, fill_id vx_998
  BEGIN
    INSERT Fill(fill_id = vx_998, ...)         -- UNIQUE on fill_id: dedup
    INSERT LedgerEntry(account = position:AAPL, direction = credit, amount = 10 shares, fill_id = vx_998)
    INSERT LedgerEntry(account = cash,          direction = debit,  amount = 150000,     fill_id = vx_998)
    UPDATE positions SET quantity = quantity + 10, avg_cost = ..., version = version + 1 WHERE user_id = :uid AND symbol = 'AAPL'
    UPDATE wallets   SET cash_balance = cash_balance - 150000, buying_power = buying_power + :released_reservation
    -- :released_reservation is the exact proportional release for this fill; the reconciliation job can rebuild buying_power = settled_cash − Σ(remaining open reservations) from the ledger at any time
  COMMIT
  • Idempotent by fill id. The Fill insert under a unique constraint on fill_id is the dedup: a re-delivered execution report hits a duplicate-key error and the whole transaction is a no-op. This is why the venue's fill_id (not your own generated id) is the dedup key — the venue defines what "the same fill" means.
  • Atomic. The fill record, both ledger entries, and both cache updates commit together. There is no state where shares landed but cash didn't.
  • Derived + cached. The Position and Wallet rows are caches updated in the same transaction as the ledger. A reconciliation job re-sums the ledger and rebuilds a cache if it ever drifts — the ledger wins.
  • Verdict: recommended. Append-only ledger as truth, cached position/cash for fast reads, fill_id uniqueness for exactly-once booking.

Buying power is derived, not a free-floating number. buying_power = settled_cash − Σ(remaining open reservations). When an order is placed, buying power drops by the reservation (deep dive 1); when the fill books, the reservation converts to a real cash debit; when an order cancels or rejects, the reservation is released. On a partial fill, release the reservation proportionally to the filled quantity (filled_qty × reserved_unit_price) and leave the rest held for the still-open remainder — then recompute buying power as settled_cash − Σ(remaining open reservations) rather than nudging the cache by an ad-hoc delta. Release the whole reservation on the first partial and a concurrent order can double-spend the buying power that the unfilled shares still need. Deriving it from the ledger + open reservations — rather than mutating a standalone balance — is what keeps it from drifting.

flowchart LR
    F["Fill vx_998: buy 10 AAPL @ 150"] -->|credit 10 shares| PE["Position entry AAPL"]
    F -->|debit 150000 cents| CE["Cash entry"]
    PE --> L[("Append-only ledger")]
    CE --> L
    L -->|sum per account| Der["Derived position + cash"]
    L -->|same tx| Cache["Cached position + wallet for fast reads"]

Reconciliation against the clearing firm. Your ledger is what you believe the user owns; the clearing firm (which handles the actual settlement of cash and shares) is the source of truth for what they actually own. These can drift: a fill report is lost, a corporate action (split, dividend) adjusts shares outside the order flow, or a booking bug creeps in. A daily reconciliation job compares the broker's positions and cash against the clearing firm's records, symbol by symbol and account by account. A mismatch the live path missed — usually a lost fill — is repaired using the same idempotent booking logic; a true discrepancy goes to an exceptions queue for a human.

There's also a cheaper internal check the double-entry ledger gives for free: a fill's two legs are in different units (shares and cents), so they don't sum to zero — but they must stay consistent, share_qty × fill_price == cash_delta at the reported fill price, with the cash ledger and the share ledger each independently append-only. If that equality fails, a leg was dropped or duplicated — a bug caught by pure arithmetic, no external system needed.

Settled vs unsettled — T+2. A fill doesn't instantly transfer cash and shares. Equity trades settle on T+2 (trade date plus two business days). So the app shows the position the instant it fills, but the ledger distinguishes executed (shares credited, spendable for display) from settled (cash actually moved). This matters for cash availability — proceeds from a sale aren't fully withdrawable until settlement — so the ledger tracks pending vs settled cash separately, and reconciliation against the clearing firm keys off the settlement records.

Caution

Do not mutate a shares or cash column in place, and do not book fills without deduping on the venue fill_id. A re-delivered fill applied twice, or a position overwritten with no ledger, silently corrupts what a user owns — and without a ledger you can't even detect it, let alone reconcile it against the clearing firm.

Tradeoffs & bottlenecks

  • Broker-routes vs internal-matching. Robinhood routes orders to venues rather than matching them internally. Routing is far simpler (no order book, no matching engine, no microsecond latency budget) and is the actual regulatory/business model (best execution, PFOF). Building a matching engine would be wrong and pointless — the liquidity lives at the exchanges. The cost is dependence on external venues and async fills you don't control.
  • Every-tick vs conflated quotes. Conflation drops intermediate ticks to make fan-out tractable and screens smooth, at the cost of tick-perfect fidelity that retail users can't perceive anyway. A pro-trading product might expose a raw feed as a premium tier; the default is conflated.
  • Strong consistency (ledger) vs availability. Orders, fills, and the portfolio are CP — reject an order rather than over-commit buying power, and never book a fill twice. Market data and portfolio reads are AP, served from replicas and a conflated stream where a little staleness is fine. The split lets each side pick the right point on the spectrum.
  • Real-time (executed) vs settled (T+2) positions. Showing executed positions immediately is what users expect, but cash isn't truly available until settlement. Tracking pending vs settled separately is the reconciliation with reality — skip it and you let users withdraw money that hasn't settled.
  • Bursty order load. The 9:30 open and meme-stock days spike order volume ~10×. Queue accepted orders and process them durably rather than dropping any; scale the routing tier horizontally. The market-data fleet spikes independently (everyone watching the same hot symbol) and scales separately.
  • Venue as an external dependency. The broker's correctness depends on the venue's execution reports, which can be delayed, re-delivered, or lost. Idempotent fill booking plus daily reconciliation against the clearing firm is what makes the broker trustworthy despite an unreliable feed.

Extensions if asked

Add only the extension that changes the design discussion.

  • More order types. Stop / stop-limit / trailing-stop orders trigger when the market reaches a price. The broker can hold them server-side and inject a live order when the trigger hits (watching its own market-data feed), or route them to a venue that supports them natively. Each is a variation on the routing flow, not a new architecture.
  • Fractional shares. Buying $5 of a $150 stock. The broker aggregates fractional demand and trades whole shares at the venue, then allocates fractions to users on its own books — the ledger already supports fractional share amounts (store shares as a scaled integer), so it's a booking/allocation concern.
  • Options and crypto. Different instruments, venues, and settlement rules, but the same broker skeleton — validate, route, book fills, reconcile. Crypto often settles instantly (no T+2) and trades 24/7, which changes the market-data and settlement assumptions.
  • Margin. Buying power becomes cash + borrowable margin, with maintenance requirements and margin calls. It plugs into the buying-power derivation (deep dive 1/3) as a larger, risk-adjusted limit, plus a risk engine that can force-liquidate.
  • Circuit breakers / trading halts. The exchange halts a symbol on extreme moves. The broker checks a halt flag and rejects or queues orders for halted symbols — a control-plane flag layered on the order path.
  • The meme-stock load spike. A single symbol drawing millions of watchers and a flood of orders stresses both paths at once: the market-data fleet (fan-out of one hot topic) and the order path (burst of orders on one symbol). Scale the WS fleet and the routing tier independently; the per-symbol pub/sub topic and the queued order path are exactly what absorb it.

What interviewers look for & common mistakes

What interviewers usually reward:

  • Knowing Robinhood is a broker, not an exchange — it validates buying power and routes orders to venues (PFOF market makers / exchanges) and books the async fills, rather than matching internally.
  • Idempotent order submission and fill booking — a client-generated client_order_id claimed under a unique constraint for orders, and the venue fill_id under a unique constraint for fills, so a double-tap places one order and a re-delivered fill books once.
  • The order state machine — pending → routed → acked → partial/filled/cancelled/rejected — with fills async, partial, out-of-order, and the cancel-vs-fill race resolved by the venue.
  • Market data as a fan-out problem — conflate quotes and fan out via per-symbol pub/sub over WebSockets, so the publisher does O(1) work per update and effort tracks attention, not the symbol universe.
  • An append-only double-entry ledger as the source of truth for cash and positions, with buying power derived from it, atomic exactly-once fill booking, and reconciliation against the clearing firm.
  • Integer minor units for money, pending vs settled (T+2), and strong consistency for the portfolio while reads and quotes stay eventually consistent.

Before you finish, do a quick mistake check:

  • Did you avoid designing a matching engine — the broker routes, the exchange matches?
  • Did you make order submission idempotent (client-generated key, atomic unique-constraint claim) so a double-tap doesn't double-order?
  • Did you dedup fills on the venue fill_id so a re-delivered execution report doesn't credit phantom shares?
  • Did you model the order state machine with async, partial, out-of-order fills and the cancel race resolved by the venue?
  • Did you conflate quotes and fan out via per-symbol pub/sub over WebSockets, instead of pushing every tick to every user?
  • Did you back positions and cash with an append-only ledger (not a mutable column), derive buying power from it, and reconcile against the clearing firm?
  • Did you use integer minor units and distinguish executed from settled (T+2)?

Practice this live

Run this exact question with our AI voice interviewer and get feedback.

Start the interview →