Technologies referenced in this case study: API Gateways & Edge Proxies · Redis

Related case studies: Distributed Caching · Blob Storage · Load Balancer · API Gateway · Rate Limiting · Caching Fundamentals · Scaling Reads · Large Blobs

How to Use This Case Study#

Organized for interview use first, reference second.

Without a well-designed edge:
t=0:      Launch email to 20M users; product page and images served from us-east origin
t=+30s:   Traffic 40x normal; users in Sydney see 1.8s TTFB (4 RTTs at ~220ms each)
t=+1min:  Origin egress saturates at 20 Gbps; image requests queue
t=+2min:  Origin app servers at 100% CPU rendering the same page 50,000 times per second
t=+5min:  Errors spike; launch trends on social media for the wrong reason

With a well-designed edge:
t=0:      Same email; page cached at edge with max-age=60, stale-while-revalidate=300
t=+30s:   97% of requests served from 300+ PoPs; Sydney TTFB ~40ms
t=+30s:   Shield tier collapses misses: origin sees ~1 request per object per shield per minute
t=+1min:  Origin at 15% CPU; egress ~0.5 Gbps
t=+10min: Price change published → surrogate-key purge → global in < 1s
Result:   A launch, not an incident.

Executive Summary

If you only read one section, read this.

What This Interview Actually Tests#

A CDN is not a "put CloudFront in front of it" question. Everyone knows edges cache things.

This is a freshness, correctness and blast-radius question that tests:

• Whether you decide what is cacheable and for how long based on who is harmed by staleness
• Whether you design the cache key — the single most consequential and most error-prone decision in edge caching
• Whether you protect the origin from misses, purges, and cold caches — not just from users
• Whether you treat the CDN as a third-party dependency that can fail, leak, and lock you in

The key insight: The hit ratio is a business metric and the cache key is a security boundary. A 95% → 99% hit ratio cuts origin load 5×; a cache key missing one header can serve a logged-in user's account page to strangers. Staff engineers optimize the first without ever risking the second.

The L5 vs L6 vs L7 Contrast — Start Here#

The Staff Positions#

The Three Intents#

🎯 Staff Move: "I'll design for semi-dynamic content — catalog pages and public API responses — because that's where freshness, cache keys and origin protection collide. Static assets are nearly solved by versioned URLs, and video is a bandwidth-economics problem I'd treat separately."

The Five Fault Lines#

In the Wild: Real Production Systems#

Netflix — Open Connect#

Netflix built its own CDN, Open Connect, and gives caching appliances (OCAs) to ISPs to install inside their networks at no charge, plus deploying them at internet exchange points. Content is pre-positioned during off-peak hours based on predicted popularity rather than pulled on demand, so the overwhelming majority of Netflix video traffic is served from an appliance close to — often inside — the viewer's ISP. The control plane (steering which OCA serves a client) runs in AWS; the bytes never touch it.

Staff insight: For large media with predictable demand, push beats pull. And it's a build-vs-buy decision driven by scale: at a double-digit share of peak internet traffic, owning the edge is cheaper and better than renting it. Almost nobody else is at that scale.

Fastly — Instant Purge and Surrogate Keys#

Fastly built its platform around fast, tag-based invalidation: responses carry a Surrogate-Key header with tags (e.g., product-123 category-9), and a purge by tag propagates globally in roughly 150ms by their public description. That makes it practical to cache semi-dynamic content — news articles, product pages, API responses — with long TTLs and purge on write, rather than short TTLs. On June 8, 2021, a customer's valid configuration change triggered a latent software bug and caused most of Fastly's network to return errors for under an hour, taking many major sites down simultaneously.

Staff insight: Instant purge changes the freshness tradeoff — "cache long, purge on change" becomes viable. The 2021 outage is the canonical example that a CDN is a correlated dependency across a large share of the web.

Cloudflare — Anycast, Tiered Cache, and a Global Regex#

Cloudflare routes users via anycast to 300+ cities, offers Tiered Cache (lower-tier PoPs fetch from upper-tier PoPs before origin), and runs WAF and code (Workers, on V8 isolates with very low startup cost) at every PoP. On July 2, 2019, a new WAF managed rule containing a regular expression with catastrophic backtracking was deployed globally at once and drove CPU to 100% across the network for about 27 minutes; the public post-mortem led to staged rollouts for WAF rules.

Staff insight: Every piece of logic you run at the edge runs everywhere at once. Edge config and code need staged, canaried rollouts even more than origin code does, because the blast radius is your entire internet presence.

What Interviewers Probe#

System Architecture Overview#

Reading the diagram: Requests reach the nearest PoP via anycast/DNS. The PoP terminates TLS (saving ~2 round trips to a distant origin), normalizes the cache key, and serves hits. Misses go to a regional shield, which collapses concurrent misses so origin sees roughly one fetch per object per shield. The origin tells the edge what to do through headers — Cache-Control for freshness and privacy, Surrogate-Key for purge targeting — so cache policy lives with the team that owns the content. A publish event triggers a tag purge. The origin only accepts traffic from the CDN, so attackers can't bypass the edge. The DNS layer is the escape hatch if the CDN itself fails.

Quick-Reference: The 30-Second Cheat Sheet#

Key Numbers Worth Memorizing#

Interview Walkthrough

Common prompts: "Design a CDN", "Design the edge layer for a global e-commerce site", "Our origin can't handle launch traffic — design caching." If the prompt is literally "design a CDN" (i.e., you are the CDN provider), spend proportionally more time on PoP architecture, routing, and purge propagation; if it's "use a CDN for our product", spend it on cacheability, keys, freshness and failure. Ask which.

Phase 1: Requirements & Framing (2–3 min)#

"Two framings are possible — building a CDN as a provider, or designing our product's edge on top of one. I'll assume the second unless you'd prefer the first. Then: what content are we serving — static assets, HTML pages, API responses, video? How fresh must each be — can a price be 60 seconds stale? Is any of it personalized? And where are users — global or one region? I'll assume a global e-commerce site: 50M daily users, 70% outside the origin's region, static assets plus catalog pages plus a public product API, with prices that must update within 60 seconds and nothing personalized in the shared cache."

Established:

• Content classes drive strategy
• Freshness SLO is a product decision you'll get signed off
• Personalization boundary is explicit
• Geography justifies the edge

Phase 2: Core Entities & API (1–2 min)#

The contract between origin and edge is HTTP headers:

Static asset:   Cache-Control: public, max-age=31536000, immutable
Catalog page:   Cache-Control: public, s-maxage=60, stale-while-revalidate=300, stale-if-error=86400
                Surrogate-Key: product-123 category-9 page-template-v4
                Vary: Accept-Encoding
Account page:   Cache-Control: private, no-store
Public API:     Cache-Control: public, s-maxage=10, stale-while-revalidate=30
Purge API:      POST /purge  { "tags": ["product-123"], "soft": true }

🎯 Staff Move: "The origin owns cache policy through headers, so the team that knows the content decides its freshness. The platform owns guardrails — like refusing to cache any response that sets a cookie, regardless of what the headers say."

Phase 3: High-Level Architecture (≤5 min)#

Draw the overview: users → DNS/anycast → PoPs → shields → origin, with a purge path and a config path. Name owners:

1. Routing (DNS/anycast, failover) — platform/networking
2. PoPs (TLS, WAF, cache, collapsing) — CDN vendor + platform config
3. Shield tier — CDN feature, platform-configured
4. Origin (emits headers, accepts only CDN traffic) — product teams + platform
5. Purge pipeline (publish events → tag purge) — platform, fed by product systems

Phase 4: Transition to Depth#

"The architecture is standard. Where designs actually fail is in three places: the cache key — which determines both hit ratio and whether we ever leak content — freshness and purge at scale, and what happens when the cache is cold or the CDN itself fails. I'll go in that order."

Phase 5: Deep Dives (25–30 min)#

Deep dive 1 — Cache key (7 min).

Incoming:  GET /p/red-shoes?utm_source=email&color=red&size=9&fbclid=abc
           Cookie: session=...; ab_test=B     Accept-Language: de-DE,de;q=0.9
Normalize: path lowercase, strip trailing slash
           query: keep allowlist {color, size}, sorted → color=red&size=9
           headers: lang = first supported of Accept-Language → 'de'
           cookies: none in key; stripped from origin request for cacheable routes
Key:       /p/red-shoes|color=red&size=9|lang=de|enc=br
Variants:  ~8 languages × 2 encodings = 16 per URL (not thousands)

Deep dive 2 — Freshness (7 min). Prices must be visible within 60s. Options: TTL 60s (simple, origin load = 1 fetch per object per shield per minute), or TTL 1 day + tag purge on change (better hit ratio, depends on purge reliability). Staff default: TTL 60s + SWR 300s as the floor guarantee, plus tag purge for faster updates — purge makes it faster, TTL makes it bounded even if purge fails.

Deep dive 3 — Origin protection (6 min).

Worst case: global purge of homepage at 50K RPS
  No shield, no collapsing:  ~300 PoPs × concurrent misses → thousands of origin requests in 1s
  Shield (6) + collapsing:    ~6 origin requests
  Soft purge (mark stale):    0 blocking origin requests — served stale while 6 revalidate
Bulk purge (2M products):     rate-limit purges to ~5K tags/s; prefer soft purge; stagger

Deep dive 4 — Failure (5 min). Origin down → stale-if-error for 24h keeps catalog up. CDN down → DNS failover (edge hostname TTL 60s) to CDN B or to origin direct with origin capacity for ~20–30% of traffic plus aggressive shedding.

Deep dive 5 — Security (3 min). Never cache Set-Cookie responses; private, no-store on personalized routes; origin accepts only CDN traffic (IP allowlist + secret header or mTLS); unkeyed-header poisoning defenses.

Phase 6: Wrap-Up (2–3 min)#

"To summarize: versioned immutable assets; semi-dynamic pages with short TTL, stale-while-revalidate and tag purge; an allowlist cache key with personalization excluded by default; a shield tier and collapsing so the origin sees a handful of requests per object; stale-if-error for origin outages; and DNS failover for CDN outages. Next I'd add hit-ratio and freshness SLOs per route, an automated leakage test in CI, and evaluate multi-CDN once revenue per minute justifies it. Biggest risk: a cache-key or header mistake that caches private content — that's where I'd put the automated guardrails first."

Common Timing Mistakes#

1. The Staff Lens#

1.1 Why This Problem Exists in Staff Interviews#

Edge caching is deceptively cross-cutting. The CDN configuration is owned by one team; the headers that control it are emitted by dozens; the content's freshness requirements come from product; the costs land in finance; and the failures — stale prices, leaked pages, global outages — land on everyone. A Senior engineer can configure a CDN correctly for one application. A Staff engineer designs the contract that lets fifty teams cache safely without each one learning the hard way.

1.2 The L5 vs L6 vs L7 Contrast — Visual#

1.3 The Staff Question That Cuts Through Everything#

"If this response were served to a different user, or two minutes late, who would be harmed — and how badly?"

Ask it per route. The answers sort every response into a strategy:

• Nobody, ever → immutable, cache forever with versioned URLs
• Nobody, if within N seconds → shared cache with TTL N, SWR, purge
• The user, if another user sees it → private, no-store; never in the shared cache
• The business, if late (prices, inventory, legal notices) → short TTL + purge + product-owned freshness SLO

2. Problem Framing & Intent#

2.1 The Three Intents — Explained#

Intent 1: Static asset delivery. Solved problem if done right: content-hashed filenames, 1-year TTL, immutable, served from edge with Brotli/gzip and image format negotiation (AVIF/WebP). The remaining risks are deploy ordering (HTML references a new asset before it's uploaded, and the 404 gets cached) and cache pollution. Hit ratio should be 98–99%+.

Intent 2: Semi-dynamic content and API acceleration. The hard case. Catalog pages, search result pages for anonymous users, public API responses, article pages. They change — sometimes often — and the business cares how fast changes appear. The toolkit: short TTLs, SWR, micro-caching for extremely hot URLs (1–10s TTLs flatten spikes), tag purge on change, request collapsing, and a strict personalization boundary. Even uncacheable dynamic requests benefit from the edge: TLS termination near the user and warm, reused connections from edge to origin save 100–300ms for distant users.

Intent 3: Large media and video. Different economics. Objects are large (MB–GB), the catalog has a long tail, and egress cost dominates. Segment-level caching (2–6s chunks), tiered caches with large disk tiers, byte-range support, admission policies so one-time views don't evict popular content, and — at very large scale — pre-positioning and ISP-embedded caches (Netflix Open Connect). Metrics are rebuffer ratio, startup time and origin egress.

🎯 Staff Move: "These three share a CDN but not a strategy. Static assets want versioning, semi-dynamic wants purge and freshness contracts, video wants bandwidth economics. I'll focus on semi-dynamic, and I'll give static a one-line answer so we don't spend time on a solved problem."

2.2 When NOT to Use (Shared) Edge Caching#

2.3 What the Interviewer Leaves Underspecified#

2.4 Precise Terminology#

3. The Five Fault Lines#

3.1 Fault Line 1: Freshness vs Hit Ratio#

Origin load for one hot page, 6 shields, TTL 60s:           6 requests/min
Same page, TTL 1s (micro-cache):                             6 requests/s
Same page uncached at 50K RPS:                               50,000 requests/s
Staleness bound with TTL 60s + SWR 300s:                    ≤ 60s fresh; up to 360s if origin slow

🎯 Staff Move: "Purge makes updates fast; TTL makes them bounded. I want both, because a purge pipeline that silently drops messages will happen, and I'd rather be 60 seconds stale than 3 days stale."

When to deviate: News and live events: purge-first with instant purge and long TTLs, because a headline update matters in seconds and traffic is enormous. Legal/compliance content: purge + verification (fetch from several PoPs after purge and assert new content).

3.2 Fault Line 2: Cache Key Precision vs Fragmentation#

The asymmetric risk: an under-specified key serves wrong content; an over-specified key serves slow content. Wrong is worse — especially when "wrong" means another user's data.

Leakage guardrails (non-negotiable):

• Never cache responses containing Set-Cookie
• Default all routes to non-cacheable unless the route is explicitly cacheable
• Strip Cookie and Authorization from origin requests on cacheable routes, so the origin cannot personalize them
• CI test: request each cacheable route as two different users; assert identical bodies

When to deviate: A/B tests on cacheable pages — include the experiment bucket (not the user ID) in the key, with a small number of buckets.

3.3 Fault Line 3: Flat vs Tiered Caching#

The Staff default: Regional shields with collapsing at both edge and shield. The shield also improves hit ratio for long-tail content, because it aggregates demand from many PoPs.

When to deviate: Latency-critical dynamic API calls that aren't cacheable — bypass the shield and go edge → origin directly over warm connections.

3.4 Fault Line 4: Single CDN vs Multi-CDN#

The Staff default: Single CDN with a tested DNS failover path to origin (or to a warm secondary for revenue-critical hostnames), low TTL on the edge hostname, and config expressed in a vendor-neutral format where possible. Move to active multi-CDN when revenue per minute of outage justifies roughly doubling edge operational effort.

🎯 Staff Move: "A CDN is a single global dependency, even though it's physically distributed. The question isn't whether it'll fail, it's whether we can move traffic in minutes when it does — and whether our origin survives the cold cache."

3.5 Fault Line 5: Logic at the Edge vs at the Origin#

The Staff default: Keep the edge stateless and small: token validation (not issuance), bucket assignment, redirects, header normalization, image transforms. Anything that needs strong state stays at origin. All edge logic ships through staged rollout by PoP group with automatic rollback — Cloudflare's 2019 incident is the reason.

When to deviate: Latency-critical global products (ads, bidding) where 100ms matters — push more to the edge, with the operational investment that implies.

4. Failure Modes & Operational Reality#

The lifecycle of a cached object — most failures happen at a transition:

4.1 Purge Stampede#

Setup: homepage + 40K category pages tagged 'layout-v7'. 300 PoPs, no shield,
       hard purge. Normal origin load 2K RPS (98% hit ratio at 100K RPS).

t=0:      Design team ships a header change; publisher hard-purges tag 'layout-v7'
t=+0.2s:  40K objects evicted in every PoP
t=+0.5s:  100K RPS of requests → ~all miss; collapsing per PoP helps, but 300 PoPs ×
          40K hot objects = up to 12M distinct origin fetches over the next minute
t=+5s:    Origin at 60K RPS (30× normal); render latency 8s; timeouts
t=+10s:   Edges receive 5xx; nothing stale to serve (hard purge) → users see errors
t=+4min:  Caches refill slowly; origin recovers

Detection: origin_rps spike correlated with purge_events_total; edge_hit_ratio collapse; origin_5xx_rate.

Mitigation: Stop further purges; enable/extend stale serving at the edge if the vendor allows; shed at origin.

Prevention: Shield tier (origin fetches ÷ ~50); soft purge (mark stale, serve stale while revalidating) as the default; purge rate limits per publisher; broad tags (layout, template) require approval; staggered purges for bulk changes.

Owner: Platform owns the purge pipeline and its limits; the publishing team owns what they purge.

4.2 Cache Leakage — Private Content in the Shared Cache#

t=0:      A refactor moves the account page behind a new route; the framework's
          default Cache-Control is 'public, max-age=300' for GET routes
t=+1min:  User A loads /account; response (with A's name, address, last orders) cached
t=+1–6min: Every user requesting /account in that PoP sees User A's page
t=+20min: Support tickets; security incident declared
Scale:    Every PoP caches its own first visitor's page. Hundreds of users exposed.

This is not hypothetical: on December 25, 2015, a caching configuration change at Valve caused Steam store pages containing other users' account information to be served to some users for a period, per Valve's public statement.

Detection: private_marker_in_cached_response_total (origin injects a response header like X-Private: 1 on personalized responses; the edge alarms if it ever caches one); synthetic two-user checks every minute.

Mitigation: Purge everything under the route; disable caching for the route via emergency config; assess exposure from logs.

Prevention: Default non-cacheable (opt in per route); strip cookies/authorization on cacheable routes; never cache Set-Cookie; CI two-user test; edge rule that refuses to cache responses carrying the private marker.

Owner: Security + platform (guardrails); product team (route headers). Legal/privacy for disclosure decisions.

4.3 Cache Poisoning via Unkeyed Inputs#

An attacker sends GET / with X-Forwarded-Host: evil.example. The origin uses that header to build absolute URLs for script tags. The response — now loading scripts from the attacker's domain — is cached under the key for /, because X-Forwarded-Host isn't in the key. Every visitor to that PoP gets the poisoned page until TTL.

Detection: Content integrity monitors (fetch key pages from many PoPs, diff against expected); CSP violation reports spike.

Prevention: Edge strips all non-allowlisted request headers before forwarding to origin; origin never trusts forwarding headers for URL generation; everything the origin uses to vary the response must be in the key (or stripped).

Owner: Platform (header normalization); security (testing).

4.4 CDN Provider Outage#

t=0:      Provider pushes a bad config/software update; 85% of PoPs return 503
t=+1min:  Synthetic monitors fail from all regions; revenue drops to near zero
t=+3min:  Incident declared; decision: fail over?
t=+5min:  DNS change: edge hostname → secondary CDN (or origin LB); TTL 60s
t=+6–10min: Traffic shifts as resolvers honor TTL (some take longer)
t=+10min: Secondary CDN has a cold cache → origin sees 20–40% of total traffic
t=+15min: Origin sheds low-priority routes; catalog serves; checkout prioritized

Detection: External synthetic monitoring from multiple networks (not from the CDN itself); RUM error beacons.

Mitigation: Pre-approved failover runbook with a decision threshold (e.g., > 25% global error rate for > 3 min).

Prevention: Low TTL on edge hostnames; tested failover quarterly; origin capacity and shedding plan for cold-cache load; secondary CDN warmed with top objects if active multi-CDN.

Owner: Platform/SRE; incident commander makes the call per runbook.

4.5 Cached Errors and Deploy Ordering#

A deploy publishes HTML referencing app.9f3e.js before the asset is uploaded. The first requests get 404, which the CDN caches for its default negative TTL (sometimes minutes). The site is broken for everyone at those PoPs even after the asset appears.

Prevention: Upload assets before publishing HTML (and keep old assets for ≥ 1 week for clients with old HTML); negative-cache TTL ≤ 10s for asset paths; never cache 5xx beyond a few seconds.

Owner: Frontend platform (deploy ordering); edge platform (negative-caching defaults).

4.6 Hit-Ratio Erosion — The Slow Failure#

Over months, hit ratio drifts from 95% to 78%. Causes: new marketing query parameters, a Vary: User-Agent added by a framework upgrade, cookies from a new analytics vendor forwarded to origin and used in keys, an A/B test framework keyed on user ID. Origin cost and latency creep up; nobody notices until capacity planning.

Detection: hit_ratio per route with weekly trend alerts; top-N cache-key cardinality by route.

Prevention: Key allowlists; periodic cardinality reports; hit ratio as an SLO per route owner.

Owner: Route owners (their headers); platform (reporting).

4.7 Operational Reality Matrix#

5. Evaluation Rubric#

5.1 Level-Based Signals#

5.2 Strong Hire Signals#

5.3 Lean No-Hire Signals#

5.4 Common False Positives#

• Knowing anycast and BGP details ≠ edge design judgment. Useful if you're building a CDN; rarely the question.
• Vendor feature fluency ≠ design. Knowing every CloudFront behavior setting doesn't answer who owns freshness.
• High hit ratio ≠ good design. A 99% hit ratio achieved by caching personalized pages is a data breach.

6. Interview Flow & Pivots#

6.1 Typical 45-Minute Shape#

6.2 How Interviewers Pivot — And What They're Testing#

6.3 What to Deliberately Skip#

• BGP and anycast internals (unless building a CDN)
• HTTP caching spec corner cases (must-revalidate vs proxy-revalidate)
• Vendor feature lists
• Image optimization details beyond "format negotiation and resizing at edge"

6.4 Follow-Up Questions to Expect#

1. "How does request collapsing interact with a slow origin?" — Waiters queue behind one fetch; cap wait time and serve stale if available.
2. "What's the difference between max-age and s-maxage?" — s-maxage applies to shared caches; lets the CDN cache longer than browsers.
3. "How do you cache an API that requires an auth token?" — Only if the response doesn't depend on the caller; validate the token at the edge, strip it from the key and origin request.
4. "How do you know the purge worked?" — Purge acknowledgments plus sampled fetches from multiple PoPs asserting the new version header.
5. "How do you pick the TTL?" — From the freshness SLO product signs off on; purge as the fast path.
6. "Anycast or DNS-based routing?" — Anycast for simplicity and DDoS absorption; DNS for finer control; many CDNs use both.
7. "How do you prevent the origin from being hit directly?" — IP allowlist of CDN ranges plus a secret header or mTLS from edge to origin.

7. Active Drills#

Drill 1: The Opening#

Prompt: "Design the CDN strategy for our global e-commerce site."

Drill 2: Cache Key Design#

Prompt: "Product pages vary by language and currency. Marketing adds UTM parameters to every link. How do you design the cache key?"

Drill 3: Make It Concrete — Freshness#

Prompt: "A price change must be visible globally within 10 seconds. How?"

Drill 4: Origin Is Down#

Prompt: "Our origin is down for 20 minutes. What do users see?"

Drill 5: Purge at Scale#

Prompt: "A nightly bulk import updates prices for 2M products. How do you invalidate?"

Drill 6: Hot Object#

Prompt: "A single video segment gets 2M requests per second during a live event. What breaks?"

Drill 7: Build vs Buy#

Prompt: "Should we build our own CDN?"

Drill 8: Policy Change Without an Outage#

Prompt: "You need to change the cache key for all product pages to add a currency dimension. How do you roll it out?"

Drill 9: Cost#

Prompt: "Our CDN bill is $400K/month and growing 8% a month. What do you do?"

Drill 10: Multi-Region Origin#

Prompt: "We're adding a second origin region. How does the CDN use it?"

8. Deep Dive Scenarios#

Deep Dive 1: Launch-Day Origin Meltdown Behind a CDN#

Context: A limited-edition product launches at 10:00. The product page is behind the CDN with s-maxage=60. At 10:00:05, origin CPU hits 100% and error rates reach 40%. The CDN dashboard shows a 35% hit ratio for the product page, far below the expected 99%. You're pulled in.

Questions to Surface First:

• Why is the hit ratio 35% on a single hot URL — is the cache key fragmented?
• Is request collapsing enabled, and is there a shield?
• Is the page actually cacheable — is the origin emitting Set-Cookie or private under load (e.g., a session started for the queue)?
• Is the origin serving errors that are being cached, or bypassing cache?

Typical L5 Approach: Scales origin, raises the TTL. Scaling takes minutes; raising TTL doesn't help if requests are missing for another reason.

Staff Approach: Diagnoses fragmentation first. Finds the launch email links carried a unique ref parameter per recipient, which wasn't stripped — every click was a distinct cache key. Pushes an emergency edge rule stripping unknown query parameters for that route (staged to 10% of PoPs for 60s, then global), confirms hit ratio climbs above 95% within two minutes, and enables stale-if-error so remaining origin errors aren't shown.

Principal Approach: Treats "a marketing link format can take down the origin" as a platform defect. Makes query-parameter allowlists the default for every cacheable route, adds a pre-launch checklist that includes a synthetic load test through the CDN using the real campaign URLs, and establishes a joint marketing–engineering launch calendar.

Key Takeaway: "When a single hot URL has a low hit ratio, the cache key is fragmented until proven otherwise."

What clears the Staff bar:

• Looks at key cardinality before scaling
• Fixes at the edge in minutes with staging
• Makes the safe behavior the platform default

Deep Dive 2: The Silent Leak#

Context: A security researcher emails that loading /orders/recent.css on your site shows the page of whoever last requested it. Your team confirms: a new framework treats unknown extensions as the base route, so /orders/recent.css renders the user's order page — and the CDN caches anything ending in .css for 1 day. This is web cache deception.

Questions to Surface First:

• How long has this been possible, and which routes are affected?
• Do logs show exploitation (unusual extensions on personalized routes)?
• Why does the CDN cache by extension rather than by the origin's Cache-Control?
• Does the origin mark personalized responses as private?

Typical L5 Approach: Adds a rule to not cache /orders/*. Fixes one route; the pattern applies to every personalized route.

Staff Approach: Fixes the class: the edge must never override the origin's Cache-Control by extension; personalized responses always carry Cache-Control: private, no-store plus a private marker the edge refuses to cache; the framework must 404 unknown extensions on application routes. Purges all cached content on personalized paths; reviews logs for exploitation; engages security and privacy for disclosure assessment.

Principal Approach: Establishes that caching behavior is a security control. Edge caching rules go through security review; a quarterly external test for cache deception and poisoning is part of the security program; the "private by default" posture is written into the edge standard and verified in CI for every route.

Key Takeaway: "The edge should never be smarter than the origin about what's private."

What clears the Staff bar:

• Fixes the class, not the route
• Defense in depth across framework, origin and edge
• Engages security and privacy immediately

Deep Dive 3: Onboarding a Large Customer Surface — The Public API#

Context: The product API team wants to put its public read API (1.2M RPS peak, 40% of which are identical catalog queries) behind the CDN. Responses depend on the API key's plan tier (some fields hidden for free tier). The API team is worried about stale data and about the CDN seeing API keys.

Questions to Surface First:

• Does the response vary by API key, or only by plan tier?
• What freshness does the API contract promise?
• Where are rate limits enforced — would caching bypass them?
• Is the API key a secret that must not appear in cache keys or logs?

Typical L5 Approach: Adds the Authorization header to the cache key. Correct but fragments the cache by customer — hit ratio collapses to near zero for the long tail of keys.

Staff Approach: Validates the API key at the edge (edge function checks a signed token or looks up key → tier in a replicated edge KV), then keys the cache by tier (3 values), not by key. Rate limiting still runs per key at the edge before cache lookup, so cached responses count against quotas. Cache TTL matches the API's documented freshness (e.g., 30s) with Age header exposed to clients. The raw key is stripped from the origin request and never logged at edge.

Principal Approach: Frames it as a product decision: caching changes the API's freshness contract, which should be documented and versioned. Aligns API gateway and CDN responsibilities so there's one place for authentication and rate limiting (API Gateway), and one for caching — avoiding two teams implementing auth.

Key Takeaway: "Key the cache by the smallest dimension the response actually depends on — the tier, not the customer."

What clears the Staff bar:

• Keys by tier, not identity
• Keeps rate limiting ahead of cache
• Documents freshness as part of the API contract

Deep Dive 4: Post-Mortem — The Provider Outage#

Context: Your sole CDN provider had a 45-minute global outage. Your site was fully down. Leadership asks for a post-mortem and a plan so "this never happens again."

Questions to Surface First:

• Did we have a failover plan? Was it executed, and if not, why?
• What's the DNS TTL on the edge hostname, and who can change DNS in an incident?
• Could the origin have served traffic directly? At what fraction?
• What did 45 minutes cost?

Typical L5 Approach: Proposes adding a second CDN.

Staff Approach: Finds the real gaps: edge hostname had a 1-hour DNS TTL (failover would have taken an hour anyway), no runbook with a decision threshold, and origin only locked to CDN IPs — so direct traffic was impossible without a firewall change. Fixes: TTL 60s, runbook with decision criteria and pre-approved authority, origin able to accept direct traffic via a pre-provisioned LB behind a separate hostname, and shedding rules to cope with the load. Evaluates a secondary CDN as a separate decision with costs.

Principal Approach: Answers "never again" honestly: no plan makes a dependency infallible; the question is how much to spend on reducing expected downtime. Quantifies: expected provider outage time per year × revenue per minute versus the cost of active multi-CDN (config duplication, split volume discounts, ~1–2 additional engineers). Brings leadership a decision with numbers rather than a reflexive "add another vendor."

Key Takeaway: "A failover plan that's never been executed is a hypothesis."

What clears the Staff bar:

• Finds that the existing plan was unexecutable (TTL, lockdown)
• Separates quick fixes from the multi-CDN business decision
• Pre-approves authority

Deep Dive 5: Multi-Region Expansion with Data Residency#

Context: The company is entering the EU and must keep EU users' personal data in the EU. The CDN serves global traffic from all PoPs. Legal asks: "Does the CDN store personal data outside the EU?"

Questions to Surface First:

• What does the CDN cache — is any of it personal data? (Should be none, if personalized = no-store.)
• What about logs — edge logs include IP addresses, which are personal data under GDPR?
• Do EU users ever get routed to non-EU PoPs (anycast doesn't respect borders precisely)?
• Do edge functions process personal data (e.g., tokens with user IDs)?

Typical L5 Approach: Adds an EU origin and assumes the CDN is fine because it's "just a cache."

Staff Approach: Audits all three data flows: cache content (verify no personal data is cached — the private-by-default guardrail is now a compliance control), logs (configure regional log delivery and IP truncation), and edge compute (tokens validated at edge contain only pseudonymous IDs). For personalized routes, uses the CDN's regional services feature (where available) to restrict TLS termination and processing to EU PoPs for EU hostnames.

Principal Approach: Treats the edge as part of the company's data-processing footprint — it needs a data-processing agreement, a documented data map, and a review whenever new edge features (logging, compute, KV) are adopted. Establishes that the edge platform team is accountable to the privacy program, not only to engineering.

Key Takeaway: "'It's just a cache' is not a compliance position. Know what the edge stores, logs and processes."

What clears the Staff bar:

• Audits cache, logs and compute separately
• Reuses private-by-default as a compliance control
• Involves legal for sign-off

9. Level Expectations Summary#

After studying this case study, you should be able to:

• Classify content into immutable, semi-dynamic, personalized and never-cache, and give each a strategy
• Design an allowlist cache key with normalization, and explain its hit-ratio and security consequences
• Combine TTL, stale-while-revalidate, stale-if-error and tag purge to bound freshness twice
• Quantify origin protection from shields, collapsing and soft purge
• Identify cache poisoning, cache deception and private-content leakage and prevent each class
• Plan for CDN provider failure with DNS TTLs, runbooks and origin capacity
• Reason about CDN cost as egress × (1 − offload) plus contract strategy
• Decide what logic belongs at the edge and how to roll it out safely

The Bar for This Question#

Mid-level (L4): Knows CDNs cache static content near users; sets TTLs; mentions invalidation.

Senior (L5): Configures a CDN competently: versioned assets, TTLs, purge on deploy, TLS at edge. Misses cache-key design as a security boundary, purge stampedes, stale-if-error, vendor failure and cost.

Staff+ (L6): Classifies content, designs allowlist keys with leakage guardrails, bounds freshness with TTL plus purge, protects origin with shields and soft purge, plans for origin and CDN failure, treats edge config as code with staged rollout, and names owners for freshness and headers. The interviewer should learn something from the answer.

10. Staff Insiders: Controversial Opinions#

10.1 "The Cache Key Is a Security Boundary, Not a Performance Setting"#

The Staff position: Design the key with security review, default routes to non-cacheable, and test with two users in CI.

Why this matters in interviews: Candidates discuss keys only for hit ratio. Saying "leakage" first is a strong signal.

10.2 "Purge Is an Optimization; TTL Is the Guarantee"#

Purge pipelines fail silently — dropped events, rate limits, publishers who forget. Designs that rely on purge for correctness with multi-day TTLs are one lost message away from serving a 3-day-old price.

The Staff position: TTL bounds staleness; purge accelerates freshness. The exception is providers with reliable instant purge and purge verification — then "cache long, purge on change" is sound.

Why this matters in interviews: It demonstrates you think about failure of the freshness mechanism itself.

10.3 "Most Teams Should Cache Less at the Edge, Not More"#

Aggressive edge caching of semi-dynamic and personalized-adjacent content produces the most dangerous edge incidents (leaks, stale prices, poisoned pages), while the latency benefit for many of those routes is achieved by TLS termination and connection reuse alone.

The Staff position: Cache static aggressively; cache semi-dynamic deliberately with contracts; for dynamic routes, use the edge for TLS and connection reuse without caching.

Why this matters in interviews: It shows you distinguish the edge's two value propositions: caching and proximity.

10.4 "Multi-CDN Is Usually a Negotiating Tactic, Not a Reliability Strategy"#

The Staff position: A tested DNS failover path gets most of the reliability benefit for a fraction of the cost. Active multi-CDN is justified at high revenue per minute or when the pricing leverage pays for the effort.

Why this matters in interviews: Reflexively proposing multi-CDN is an L5 signal; pricing it is L6/L7.

10.5 "Edge Compute Is a Distributed System You Deploy to 300 Cities at Once"#

Running code at the edge moves you from one deploy target to hundreds, with limited observability, constrained state and a global blast radius. The latency win is real — tens to hundreds of milliseconds for distant users — but only for logic that is stateless and stable.

The Staff position: Token validation, routing, A/B bucketing, header normalization: yes. Business logic with state: no, until you have staged rollout, per-PoP observability and fast rollback.

Why this matters in interviews: It shows you evaluate edge compute as an operational commitment, not a feature.

11. The Principal Lens (L7)#

Why L7 Sees This Problem Differently#

At L6, the edge is a well-configured caching layer with safe keys and bounded freshness. At L7, the edge is the company's front door: the first code every customer request touches, a security control plane (TLS, WAF, bot defense, DDoS), a data processor subject to privacy law, a six- or seven-figure monthly line item, and a single vendor relationship whose outage is a total outage. The Principal decisions are about scope (what the edge owns), governance (who can change a rule that runs in 300 cities), money (egress contracts, cost allocation, build-vs-buy crossover), and vendor risk (lock-in via proprietary edge compute and config dialects).

The Org-Level Fault Line#

A central edge platform vs teams configuring the CDN directly.

🧭 Principal Move: "Teams own their headers; the platform owns the guardrails and the blast radius. No one — including the platform team — pushes a rule to all PoPs at once."

Cost Model#

Assumptions: blended CDN egress ~$0.01–0.05/GB depending on commitment and geography; origin/cloud egress ~$0.05–0.09/GB; fully loaded engineer ~$300K/year; figures are order-of-magnitude for planning, not quotes.

The strategic numbers: every point of byte offload above 95% saves roughly 1% of total bytes from origin egress, and image/format optimization often cuts total bytes 20–40%. Negotiated commits typically matter more than any single engineering optimization at large scale.

The 3-Year Evolution Path#

One-Way Doors vs Two-Way Doors#

The Standard I'd Write#

RFC: Edge Caching & Delivery Standard v1

Scope: All public hostnames served through the company edge.

Requirements:

• Routes MUST be non-cacheable in the shared cache unless declared cacheable in the route manifest.
• Responses that depend on user identity MUST send Cache-Control: private, no-store and the private marker header; the edge MUST refuse to cache marked responses.
• Cacheable routes MUST declare a cache-key allowlist; unknown query parameters, cookies and Authorization MUST be stripped before cache lookup and origin fetch.
• Static assets MUST use content-hashed URLs with max-age=31536000, immutable; they MUST NOT be purged.
• Cacheable routes SHOULD set stale-if-error ≥ 1 hour and stale-while-revalidate ≥ TTL.
• Purges SHOULD be soft by default; broad tags require platform approval; per-publisher purge rate limits apply.
• Edge config and code MUST roll out by PoP group with automated health gates; no global instantaneous deploys.
• Edge hostnames MUST have DNS TTL ≤ 60s and a tested failover target.

Exceptions: Filed with the edge platform; security review required for any exception to private-by-default.

Success metrics: Zero private-content cache incidents; byte offload ≥ 95% for static, ≥ 80% for semi-dynamic routes; purge p99 ≤ 5s; failover drill completed quarterly with time-to-shift ≤ 10 min; CDN $ per 1,000 sessions trending flat or down.

What I'd Tell the VP#

"Almost every customer request enters through our CDN, so it's both our biggest performance lever and a single point of failure. We're putting three things in place: rules that make it impossible for a team to accidentally cache one customer's private page for another, a way to update prices and content globally within seconds without overloading our servers, and a tested plan to move traffic elsewhere if the CDN provider has an outage like the ones that took down large parts of the web in 2019 and 2021. We'll also charge each team for the bandwidth their pages use, which historically cuts the bill 20–30% once people can see it. A second CDN is on the table — I'll bring you the cost versus the expected downtime it avoids so we can decide with numbers."

Principal Interview Signals#

Staff answers that L7 interviewers find insufficient:

• A perfectly configured CDN with no governance for the 30 teams who will change it next year.
• "Add a second CDN" without cost, feature-parity and purge fan-out analysis.
• No view of the contract — commit levels and pricing tiers are often the biggest cost lever.

Appendices

Appendix A: Mechanics in Depth#

A.1 Edge Request Handling#

handle(req):
  key = normalize(req)                         # allowlist params, normalized headers
  if route_not_cacheable(req): return proxy_to_origin(strip(req))
  obj = cache.get(key)
  if obj and obj.fresh():                      return obj                      # HIT
  if obj and obj.within_swr():                 async revalidate(key); return obj   # STALE, revalidating
  resp = collapse(key, lambda: fetch_from_shield(strip(req)))   # one fetch per key in flight
  if resp.is_error() and obj and obj.within_sie(): return obj   # serve stale on error
  if cacheable(resp) and not resp.has('Set-Cookie') and not resp.has('X-Private'):
      cache.put(key, resp, ttl=resp.s_maxage)
  return resp

A.2 Why Versioned URLs Beat Purge for Assets#

A.3 Admission and Eviction#

• LRU is the common baseline; video and large-object caches use segmented LRU or size-aware policies.
• Admission filters (e.g., only cache on second request within a window, tracked with a Bloom filter) keep one-hit wonders from evicting popular content — valuable because a large share of objects are requested once.
• Tiered storage: RAM for the hottest objects, SSD for the warm set, larger disk tiers at shields.

A.4 PoP Internals (If You're Building the CDN)#

Front servers terminate TLS and route by consistent hash of the cache key to the cache server that owns it, so each object is stored once per PoP. Hot objects are replicated to multiple cache servers to avoid single-server saturation. See Consistent Hashing.

Appendix B: Cache Keys and Variants#

B.1 Key Construction#

B.2 Vary Discipline#

B.3 Route Classes#

Appendix C: Invalidation Mechanisms#

C.1 Quick Comparison#

Appendix D: HTTP Contract and Client Behavior#

D.1 Headers Cheat Sheet#

D.2 Browser vs Edge TTL#

Set HTML's browser max-age=0 (or short) with a longer s-maxage — you can purge the edge but not browsers. Hashed assets can be long in both.

D.3 Origin Lockdown#

Accept origin traffic only from CDN egress IP ranges and require a secret header or mTLS certificate from the edge, rotated regularly. IP ranges alone are shared by every customer of the same CDN.

Appendix E: Observability#

E.1 Core Metrics — Non-Negotiable#

# Effectiveness
edge_hit_ratio{route}              byte_offload_pct{route}
shield_hit_ratio                   origin_rps / origin_egress_gbps
cache_key_cardinality{route}       # fragmentation detector

# User experience (RUM + synthetics)
ttfb_p75{country}                  lcp_p75{country}
edge_error_rate{pop}               external_synthetic_success{provider}

# Freshness
purge_latency_p99                  purge_requests_total{publisher}
served_stale_total{reason=swr|sie} content_version_mismatch_total   # sampled checks

# Safety
private_marker_in_cached_response_total   two_user_check_failures_total

E.2 Critical Alerts#

E.3 Control Plane vs Data Plane#

The CDN's config/purge APIs (control plane) can fail independently of serving (data plane). Keep an out-of-band way to change DNS and to disable caching rules, and never make publishing depend synchronously on purge success — queue and retry.

E.4 Debugging "Users See Old Content"#

1. Check Age and version headers from several PoPs.
2. Was a purge issued? Did it succeed? (purge logs, latency)
3. Is the browser caching it (max-age on HTML)?
4. Is the shield serving stale (SWR/SIE because origin is erroring)?
5. Is the cache key including something that splits old and new?

Appendix F: Scale Evolution#

F.1 What Works at Each Scale#

F.2 Multi-Region Path#

Single origin → regional origins with shield mapping → data-residency-aware hostnames → per-region edge policies.

F.3 What You Don't Build on Day One#

• Multi-CDN steering
• Edge compute beyond redirects and header normalization
• Your own PoPs
• Custom admission policies
• Real-time purge verification infrastructure (start with sampled checks)

Appendix G: Multi-Tenancy, Fairness and Cost#

G.1 Shared Cache Capacity#

In a shared edge, one team's large objects (video, downloads) can evict another team's hot pages. Separate cache namespaces or storage tiers per content class; admission filters for large objects.

G.2 Purge Fairness#

A single publisher issuing broad purges can degrade freshness for everyone by consuming purge capacity. Per-publisher rate limits; priority lanes for corrections and legal takedowns.

G.3 Cost Allocation#

Attribute egress by route prefix or hostname to owning teams; publish monthly. Visibility alone typically drives meaningful reductions (image optimization, dropping unused bundles).

G.4 Tradeoff Summary#