Design a Digital Wallet

Alice owes Bob $20 for dinner. She opens a wallet app, types Bob's name and taps Send. Her balance drops by $20, Bob's rises by $20, and both see the transfer in their history. The kind of product this question means is a payments-app wallet such as Venmo. Behind that one tap: two balances that must change together, a double tap that must not send $40, and two phones spending the same $30 at the same moment.

It looks like Design a Payment System, and it reuses its idempotency keys and ledger rules. The difference is that here both sides of every transfer are accounts you hold, possibly on different database shards, so you have to move money between two of your own databases without ever creating or losing a cent. The steps follow A Framework for Any Design.

#Clarify the problem

• Functional: top up a wallet from a card; send money to another user; view the balance and a paged history.
• Non-functional: no negative balances, even under concurrent requests; a transfer retried any number of times moves money once; no acknowledged transfer is lost; every balance is explained by an append-only ledger kept for 7 years; a send answers within about 500 ms; the recipient is credited within seconds.
• Out of scope: withdrawals, currencies, fraud, spending limits, card processing (the payment system's job).

#Estimate the scale

• Transfers: 50 million ÷ 86,400 ≈ 580 a second on average. Payday and holidays run hotter; plan for 5x, about 3,000 a second.
• Top-ups: 5 million ÷ 86,400 ≈ 60 a second, small next to transfers.
• Balance reads: if users open the app 300 million times a day, 300 million ÷ 86,400 ≈ 3,500 single-row reads a second.
• Database writes: assume one relational primary safely takes about 5,000 short write transactions a second. Today's peak of 3,000 transfers fits on one. At 10x, 30,000 a second, it does not.

#Define the API

POST /v1/topups            Idempotency-Key: tp-6f1c
  body: { "amount_cents": 5000, "payment_method": "pm_..." }
  201 { "topup_id": "tp-6f1c", "status": "processing" }
 
POST /v1/transfers
  body: { "transfer_id": "t-81", "to_user": "bob", "amount_cents": 2000, "note": "dinner" }
  201 { "transfer_id": "t-81", "status": "sent" }
  200 { "transfer_id": "t-81", "status": "sent" }       same id seen before
  409 { "error": "insufficient_funds", "balance_cents": 1000 }
  422 same transfer_id with a different body
  404 no such recipient
 
GET /v1/wallet
  200 { "balance_cents": 1000, "as_of_entry": 99812 }
 
GET /v1/wallet/history?before=99812&limit=50
  200 { "entries": [ { "transfer_id": "t-81", "amount_cents": -2000,
                       "balance_after": 1000, "at": "..." }, ... ], "next": "99761" }

The decision worth arguing is when "sent" is true. The API answers once the money has left Alice's balance, not once it reaches Bob's. From that moment the transfer cannot be lost: it either completes or is reversed back to Alice, automatically. Waiting for Bob's side would put a second database on the latency path for no gain in safety. Money is always whole cents in integers, never floating point.

#Model the data

wallets
  wallet_id      VARCHAR(32) PRIMARY KEY
  balance_cents  BIGINT        -- cache of SUM(ledger_entries), same transaction
  status         VARCHAR(8)    -- active | frozen
 
ledger_entries                 -- append only
  entry_id       BIGINT PRIMARY KEY
  account_id     VARCHAR(32)   -- "alice", or "transit:shard-A"
  transfer_id    VARCHAR(36)
  amount_cents   BIGINT        -- change to that account: -2000, +2000
  balance_after  BIGINT        -- cheap history and audits; NULL for transit
  INDEX (account_id, entry_id DESC)
 
transfers                      -- on the sender's shard
  transfer_id    VARCHAR(36) PRIMARY KEY
  from_wallet, to_wallet, amount_cents, request_hash
  status         VARCHAR(10)   -- sent | completed | reversed
 
applied_credits                -- on the receiver's shard
  transfer_id    VARCHAR(36) PRIMARY KEY
  outcome        VARCHAR(8)    -- credited | refused
 
outbox                         -- on the sender's shard
  transfer_id, payload, created_at

Every operation is a short transaction over a few rows that must all commit or none: a conditional debit, two ledger entries, a transfer row. That calls for a relational database with ACID transactions, with synchronous replication so an acknowledged transfer survives losing the primary. When it outgrows one primary, shard it by wallet_id: a wallet's balance, entries and history then always sit together, so balance reads and history pages stay single-shard.

#High-level design

This is the design for the 10x scale, sharded. At today's scale the same service talks to one database, and both legs below become one transaction. A top-up is simpler: the payment system charges the card, and when it confirms, the wallet service credits Alice from a card-funding account, keyed by the top-up id so a repeated confirmation credits once.

#Deep dives

#Moving money across shards

With 16 shards and wallets spread evenly, Bob is on Alice's shard only 1 time in 16, so about 94% of transfers cross shards. Sharding by wallet does not avoid the problem; it makes it the normal case. Four ways to handle it:

My pick: one database while it fits, and a saga once it does not. Two details make the saga safe. First, the order: debit first, credit second. The debit is the only step that can fail for a business reason (not enough money), so it runs first. Once it commits, the credit is nearly certain, and the rare refusal (a frozen or closed wallet) has a simple reversal: give the money back. The opposite order could credit Bob and then fail to debit Alice, creating money from nothing.

Second, each leg balances on its own through a transit account per shard. Alice's leg moves $20 from Alice to transit:A; Bob's moves $20 from transit:B to Bob. While a transfer is in flight, transit:A is +2000 and transit:B is −2000, so the books across all shards still sum to zero. The sum of the transit accounts always equals money in flight, which gives reconciliation a precise check.

Two traps. Every transfer on a shard touches its transit account, so a cached balance row for it would be the hottest lock in the system: transit accounts get entries only, and the reconciler sums them. And a refusal is final. Shard B stores it in applied_credits, and the reversal runs only if the transfer is still sent; otherwise a redelivery after Dan is unfrozen would pay him after Alice got her money back.

#Preventing overdraft under concurrency

Alice has $30 and two devices each send $20 within a millisecond. If the service reads the balance, checks it in code, then writes, both reads see 3000 and both transfers go through: −$10. The check and the write must be one atomic step.

BEGIN;
INSERT INTO transfers (transfer_id, from_wallet, to_wallet, amount_cents, status)
     VALUES ('t-81', 'alice', 'bob', 2000, 'sent');      -- duplicate id: stop here
UPDATE wallets SET balance_cents = balance_cents - 2000
 WHERE wallet_id = 'alice' AND balance_cents >= 2000;    -- 0 rows: ROLLBACK, 409
INSERT INTO ledger_entries ... ('alice', -2000), ('transit:shard-A', +2000);
INSERT INTO outbox (transfer_id, payload) VALUES ('t-81', '...');
COMMIT;

The conditional UPDATE takes a row lock on Alice's wallet. The second debit waits for it. In PostgreSQL at read committed, it then rechecks its WHERE against the new balance of 1000 and changes nothing. At repeatable read or serializable it fails with a serialization error instead; retry it, and the retry sees 1000. Either way, no overdraft. Transactions and isolation covers the levels; name the one you rely on.

The lock lasts only for this short transaction, and only the sender's row is locked. Credits need no balance check, which matters for hot accounts (see the follow-up questions).

#Ledger, audit and reconciliation

The ledger rules are the ones from the payment system lesson: entries sum to zero, are never edited, and mistakes are fixed with reversing entries. What a wallet adds is that the ledger is the balance. The balance_cents column is a cache written in the same transaction as the entries, and balance_after on each entry lets anyone check any point in history without summing years of rows.

Run these checks continuously, and page someone when one fails:

• Per wallet: the cached balance equals the latest balance_after, and the sum of entries. Sample wallets all day; check every wallet nightly.
• Per transfer: any transfer still sent after a minute is stuck. A sweeper re-drives it from the outbox.
• Across shards: all accounts together sum to zero, and the transit accounts sum to the amount of transfers still in flight.
• Against the outside world: total of all wallets equals the money in the company's pooled bank account, and top-ups match the payment system's settlement records.

#Bottlenecks and failures

At 10x, 30,000 transfers a second at peak, each cross-shard transfer is two local transactions: 60,000 a second. The worker marks transfers completed in batches, which adds little. At about 5,000 per primary, that is 12 primaries; use 16 for headroom and growth. Balance reads, 35,000 a second at 10x, are one-row lookups on the owner's shard. History pages can come from replicas a second behind, with the app showing its own just-sent transfer as pending.

#What a strong answer sounds like

#Follow-up questions

#A merchant wallet receives 1,000 credits a second. What breaks?

Its balance row: every credit waits on its lock. Credits never need a balance check, so append the entries without updating the cached balance, and fold them into it every second. Or split the account into 10 sub-accounts and sum them on read.

#How would you add withdrawals to a bank?

Debit the wallet into a "payouts pending" account first, then call the bank with an idempotency key. An unknown outcome stays pending until the bank's records settle it, the same care as in the payment system.

#How would you support several currencies?

One wallet per currency. A conversion is a transfer through the company's account for each currency, at a recorded rate, so every currency's ledger still sums to zero.

#How do you prove the ledger was never edited?

Give the service insert-only permission on the ledger table, store a hash of the previous entry in each entry so any edit breaks the chain, and publish daily totals somewhere the database cannot change.

#Why not compute the balance from entries on every read?

An active wallet has thousands of entries, and 3,500 reads a second would sum them all. The cached column is exact because it changes in the same transaction as the entries, and the nightly check proves it.