CDNs

A shop serves product images from one region. At peak, pages pull 5,000 images a second at about 200 KB each: 5,000 × 200 KB = 1 GB/s, or 8 Gbit/s, all leaving one set of servers. Users 8,000 km away wait at least 80 ms per round trip for every image, though the images are the same for every shopper.

#The idea

A content delivery network (CDN) is a set of caching servers, called edges, in many cities. Users reach a nearby edge. If it has the file, it answers; if not, it fetches the file once from your servers (the origin) and keeps a copy for everyone after.

Latency: an edge 500 km away is 2 × 500 ÷ 200 = 5 ms per round trip instead of 80 (from Networking Essentials, light in fibre covers about 200 km per ms). Load: if edges answer 95% of requests, the origin sends 1 GB/s × 0.05 = 50 MB/s instead of 1 GB/s.

The cost is the cost of any cache: copies in hundreds of places that you cannot update instantly.

#A file's life on a pull CDN

Step 5 shows why hit rates are lower than you expect: every edge fills its own cache. Many CDNs add an origin shield, one middle cache that all edges ask before the origin, so the origin sees one miss per file instead of one per edge.

#Push or pull

#Cache keys and invalidation

The cache key decides which requests share a copy. By default it is roughly the host, path and query string. Too much in the key and the hit rate collapses: if ?utm_source=mail is part of the key, every campaign link stores its own copy of the same image. Too little and users see the wrong content: if the key ignores the ?size=small that changes the response, everyone gets whichever size was cached first. Keep only the parts that change the bytes.

Three ways to change a cached file, best first:

• Versioned names. Put a version or a content hash in the file name (app.3f9a1c.js) and cache it for a year. A new version is a new URL, so nothing needs purging. Keep the HTML that points to it on a short TTL.
• TTL. Let copies expire. Simple, but users may see old content for the whole TTL.
• Purge. Ask the CDN to drop a path from every edge. It takes time to reach every edge, and then every edge misses at once.

#Signed URLs for private files

Paid videos and private documents still belong on a CDN, but only the right people should fetch them. A signed URL carries an expiry time and a signature over the path and expiry, often a keyed hash (HMAC) made with a secret that your app and the CDN share. The edge checks the signature and the time on its own, without calling your servers.

https://cdn.example.com/videos/v881/seg-0042.ts
    ?expires=1767225600
    &sig=HMAC(secret, "/videos/v881/seg-0042.ts" + "1767225600")

Anyone holding the URL can use it until it expires, so keep expiry short: minutes, not days. For a video made of hundreds of segments, a signed cookie covering the folder saves signing each one.

#What to put on a CDN

• Yes: images, scripts, styles, fonts, video segments, downloads, and public responses that are the same for everyone, even with a TTL of a few seconds (a live score).
• With signed URLs: paid or private media.
• No: pages that differ per user, and anything that must be up to the second for every reader.

Uncached requests still gain from a CDN: users handshake with a nearby edge, which keeps warm connections to your origin.

#Numbers worth remembering

• Origin load = total requests × (1 − hit rate). Going from 90% to 99% hits cuts it tenfold.
• Video bandwidth: viewers × bitrate. Remember 8 bits per byte.

#In the interview

Bring a CDN in whenever there are static files, media, or users spread across regions. Say what goes on it and how it is invalidated: "Images and video segments are identical for every user and most of our bytes, so they go on a CDN. File names carry a content hash, so I cache them for a year and never purge. HTML gets a short TTL."

Follow-ups to expect:

• "How does a user reach the nearest edge?" DNS that answers by the resolver's location, or anycast: one IP announced from many places, so the network routes to the closest.
• "What if the CDN goes down?" Fail over to a second CDN by DNS, or to the origin, which must then survive 100% of the traffic. Usually it cannot, so a second CDN is the real plan.
• "Why is the hit rate low?" Too many cache keys (tracking parameters, cookies), short TTLs, or a long tail of files spread over many edges. An origin shield helps the last one.

#Common mistakes

#What to practise next

CDNs handle data that flows one way and changes rarely. For data that must reach users the moment it changes, read Realtime Communication. To see a CDN carry most of a system's bytes, work through Design Video Streaming.