Technologies referenced in this case study: API Gateways — Technology Guide (Kong, Envoy, NGINX, AWS API Gateway) · Redis · ZooKeeper & etcd

Related case studies: Rate Limiting · Circuit Breakers · Service Discovery · Load Balancer · CDN & Edge Caching · API Design · Build vs Buy

Scope note: This case study is the design problem — what the gateway should own, how it fails, and who runs it. For product-by-product comparisons (Kong vs Envoy vs NGINX vs AWS API Gateway), see the technology guide; this page deliberately doesn't repeat them.

How to Use This Case Study#

Without a gateway:
t=0:      A JWT library vulnerability is disclosed (algorithm confusion)
t=+1d:    Security finds 40 services validating tokens with 6 different libraries
t=+2w:    31 services patched; 9 owned by teams busy with a launch
t=+5w:    Last service patched. Five weeks of exposure; a public API endpoint exploited in week 3

With a gateway:
t=0:      Same disclosure
t=+2h:    Gateway team patches the one validation path; staged rollout by region
t=+6h:    Fully deployed. Services receive a verified identity header over mTLS and never parse raw tokens
Result:   One change, one owner, six hours.

Executive Summary

If you only read one section, read this.

What This Interview Actually Tests#

An API gateway is not a "reverse proxy with plugins" question. Everyone can draw a box between the client and the services.

This is a centralization and blast-radius question that tests:

• Whether you can decide what to centralize (security, identity, traffic policy) and what to keep out (business logic, aggregation for one team)
• Whether you design a component in 100% of request paths so that it is more reliable than anything behind it
• Whether you can let 50 teams change routing daily without any of them being able to take down the other 49
• Whether you understand the gateway as an organizational boundary: the contract between a platform team and every product team

The key insight: The gateway's value comes from being shared, and its risk comes from being shared. Every feature you add to it gets consistency across all services — and puts every service behind that feature's bugs. Staff engineers keep the gateway thin, keep its config changes staged, and keep asking "does this need to be in the one component everyone depends on?"

The L5 vs L6 vs L7 Contrast — Start Here#

The Staff Positions#

The Three Intents#

🎯 Staff Move: "I'll design for first-party client ingress — web and mobile into a microservice fleet — because that's where the centralization tradeoffs are sharpest. If there's a public API product, I'd add API-management features on a separate hostname and plane, and I'd put aggregation in BFFs behind the gateway rather than in it."

The Five Fault Lines#

In the Wild: Real Production Systems#

Netflix — Zuul, Then Federated GraphQL#

Netflix's edge gateway Zuul routed all streaming API traffic into its microservices, with filters for authentication, routing, insights and resiliency testing (e.g., routing a slice of traffic to test failure behavior). Zuul 2 moved to an asynchronous, non-blocking architecture on Netty to handle large numbers of persistent connections efficiently. Separately, Netflix's engineering blog has described adopting federated GraphQL for parts of its product (notably Studio applications), where domain teams own their subgraphs and a gateway composes them — moving aggregation ownership to domain teams rather than a central API team.

Staff insight: The trajectory is instructive: a central edge for cross-cutting concerns stayed; client-specific aggregation moved to a model where domain teams own their slice. That's the thin-gateway-plus-owned-composition split, arrived at through scale.

Uber — Edge Gateway as an API Lifecycle Platform#

Uber's engineering blog describes its Edge Gateway as more than a proxy: a platform for the full API lifecycle — configuration-driven endpoint definitions, schema and protocol handling, authentication, rate limiting, and a self-service workflow so product teams can create and modify endpoints without the gateway team in the loop, with reviews and validation built into the pipeline.

Staff insight: At scale, the gateway's hard problem is not the proxy — it's the workflow by which thousands of engineers safely change it. Self-service with validation is the product.

AWS API Gateway — The Managed Contract#

Amazon API Gateway is a managed gateway whose published defaults teach good instincts: an account-level throttle of 10,000 requests per second with a 5,000-request burst per region by default (adjustable), per-stage and per-method throttles, usage plans with API keys and quotas for public APIs, and a 29-second default integration timeout. Stages and canary deployments are built into the model.

Staff insight: The managed model encodes what a gateway should own — throttling, keys and plans, staged deployment, timeouts — and makes the cost visible per request. When you design your own, those defaults are a sanity check: if your design has no throttle or no maximum timeout, you've left something out.

What Interviewers Probe#

System Architecture Overview#

Reading the diagram: The gateway data plane is stateless and has no synchronous dependency on the request path: tokens are validated locally against cached signing keys, rate limits are enforced locally with async sync to a quota store, and routes live in memory from the last good config snapshot. Each team owns a route manifest in its own repo; the pipeline validates prefix ownership and policy before the config server rolls it out cell by cell. Aggregation lives in BFFs owned by the web and mobile teams, not in the gateway. Domain services still verify the caller's mTLS identity and make their own authorization decisions — the gateway is the first check, not the only one.

Quick-Reference: The 30-Second Cheat Sheet#

Key Numbers Worth Memorizing#

Interview Walkthrough

Typical prompts: "Design an API gateway", "Design the entry point for our microservices", "We have 200 services and every team implements auth differently — fix it." The trap is to enumerate gateway features for 20 minutes. Spend 5 minutes on the request pipeline, then go to scope, hot-path dependencies, config safety and ownership.

Phase 1: Requirements & Framing (2–3 min)#

"First — who are the clients? First-party web and mobile apps, external partners, public developers, internal tools? They have different contracts. Second, how many services and teams sit behind it, and how often do routes change? Third, what's the auth model — who issues tokens? I'll assume web and mobile apps plus a partner API, 300 services owned by 60 teams, route changes several times a day, 150K RPS peak, and an existing OAuth identity provider issuing JWTs. Latency budget for the gateway itself: under 5ms p99."

Established:

• Client classes (first-party vs partner) → separate planes/policies
• Team count and change rate → self-service config is a requirement, not a nice-to-have
• Identity provider exists → the gateway validates, doesn't issue
• Overhead budget → forces a dependency-free hot path

Phase 2: Core Entities & API (1–2 min)#

A route manifest, owned by a product team:

# repo: orders-service/gateway/routes.yaml   (owner: team-orders)
host: api.example.com
routes:
  - prefix: /v1/orders            # prefix ownership registered to team-orders
    methods: [GET, POST]
    upstream: orders-svc
    timeout_ms: 800
    auth: { required: true, scopes: [orders:read, orders:write] }
    rate_limit: { per_user: 20/s, burst: 40 }
    retries: { idempotent_only: true }          # budget enforced by platform

🎯 Staff Move: "The most important API here isn't what clients call — it's the route manifest. That's the contract between the platform team and 60 product teams, and the pipeline that validates it is what keeps one team's typo from becoming everyone's outage."

Phase 3: High-Level Architecture (≤5 min)#

Draw the overview and name owners:

1. CDN/WAF in front — volumetric protection, coarse bot filtering (CDN & Edge Caching).
2. Gateway data plane — stateless proxies per region, in cells; TLS, auth, rate limit, routing, telemetry. Owner: platform.
3. Gateway control plane — manifests → validation pipeline → versioned snapshots → staged push. Owner: platform.
4. BFFs — per-client aggregation, owned by client teams.
5. Services — verify mTLS identity, authorize, own their business logic.

Phase 4: Transition to Depth#

"The request pipeline is standard; I'd rather spend time where gateways actually fail. Three places: hot-path dependencies — auth and rate limiting — that make the gateway only as available as its slowest dependency; configuration changes, which cause most gateway outages; and scope creep, which turns the gateway into a monolith. Then multi-region and the public API plane."

Phase 5: Deep Dives (25–30 min)#

Deep dive 1 — Dependency-free hot path (7 min).

Per request, no network calls except to the upstream:
  TLS:            session resumption for returning clients
  Authn:          JWT signature check with cached JWKS (refresh every 10 min, refresh-ahead)
                  opaque tokens (partners): cache introspection result for min(token TTL, 60s)
  Rate limit:     local token bucket per key; async sync to quota store every 500ms
  Routing:        in-memory radix tree from last good snapshot
  Identity out:   gateway strips client-supplied identity headers, adds signed x-identity,
                  upstream call over mTLS
Failure posture:
  IdP down        → keep validating with cached keys (tokens valid until expiry)
  Quota store down → local limits only (fail-open for abuse protection, conservative caps)
  Control plane down → serve last good config indefinitely

Deep dive 2 — Config safety (7 min). Validation (schema, prefix ownership, conflicts, policy: "no unauthenticated routes without an exception"), then staged rollout: canary cell (1–5% traffic) → 1 region → all regions, with automatic rollback on 5xx_by_route, 404_rate, or latency regression. A team's manifest can only affect its own prefixes.

Deep dive 3 — Scope (5 min). Criteria for whether a feature belongs in the gateway: same for every service? needed before routing? stateless? If not all three, it goes to a service or BFF.

Deep dive 4 — Isolation (5 min). Separate listeners/clusters (or separate gateways) for first-party and partner traffic so a partner spike can't exhaust capacity for the mobile app; per-client-class concurrency limits; cells so a poison request pattern takes out one cell.

Deep dive 5 — Multi-region (3 min). Gateways per region, anycast/GeoDNS to the nearest healthy region, routes prefer in-region upstreams; config staged by region.

Phase 6: Wrap-Up (2–3 min)#

"To summarize: a thin, stateless gateway per region and cell that owns TLS, identity, rate limits, routing and telemetry, with no synchronous dependencies on the hot path. Teams own their routes via manifests; a pipeline validates ownership and policy and rolls changes out by cell with automatic rollback. Aggregation lives in team-owned BFFs; services re-verify identity. Next I'd add per-route SLO dashboards for teams, a policy engine for exceptions, and the partner API plane with keys, plans and versioning. Biggest risk: a gateway-wide change — TLS cert, auth library, global plugin — that bypasses the staged pipeline. I'd make the pipeline the only path, including for the platform team."

Common Timing Mistakes#

1. The Staff Lens#

1.1 Why This Problem Exists in Staff Interviews#

The gateway is the most literal example of a platform decision: one team builds something that every other team depends on and changes. Senior engineers can build a proxy. Staff engineers decide what the proxy is for — and, just as importantly, what it will refuse to do — then design the workflow that lets dozens of teams use it without coordination meetings, and the rollout discipline that keeps the one component on every request path from being the thing that takes everything down.

1.2 The L5 vs L6 vs L7 Contrast — Visual#

1.3 The Staff Question That Cuts Through Everything#

"If we add this to the gateway, what happens to every other team the day it has a bug?"

Apply it to every proposed gateway feature. JWT validation: a bug affects everyone — but so does 40 different implementations; centralize, with extreme care. Aggregating the mobile home screen: a bug affects everyone to benefit one team; put it in the mobile BFF. Request body validation against 300 schemas: a bug or slow schema affects everyone; leave it in services.

2. Problem Framing & Intent#

2.1 The Three Intents — Explained#

Intent 1: First-party client ingress. Web and mobile apps call your backend. You control both sides, can ship client updates (slowly, for mobile), and care most about latency, reliability and team velocity. The gateway owns TLS, authentication, coarse authorization, rate limiting against abuse, routing (including canaries), and telemetry. Versioning is flexible because you own clients — but old mobile versions live for years, so routes stay backward compatible.

Intent 2: Public API product. Partners and developers integrate against a contract you can't change unilaterally. The gateway needs API keys and OAuth client credentials, plans and quotas (often tied to billing), usage metering, versioned routes (/v1, /v2), deprecation headers and windows (often 6–12+ months), and a developer portal. Quotas used for billing need accuracy that abuse-protection limits don't. Treat it as a product with a product manager, on its own hostname and capacity pool.

Intent 3: Aggregation and protocol translation. Clients want one call instead of seven; internal services speak gRPC; the web wants GraphQL. These are real needs, but they're client-specific and change at client-team velocity. Put them in a BFF per client (or a GraphQL layer with federated ownership) behind the gateway — not in the gateway.

🎯 Staff Move: "The gateway serves every team; a BFF serves one client. If a feature's requirements come from one client team, it belongs in something that team can deploy without asking the platform team."

2.2 When NOT to Use an API Gateway (or a New One)#

2.3 What the Interviewer Leaves Underspecified#

2.4 Precise Terminology#

3. The Five Fault Lines#

3.1 Fault Line 1: Thin vs Smart Gateway#

The Staff default: Thin, with a short, written list of what the gateway owns, and an extension mechanism for truly cross-cutting needs that goes through review, runs with a CPU/latency budget, and can be disabled per route.

The scope test: A feature belongs in the gateway only if it is (1) identical for all or most services, (2) needed before routing, and (3) stateless or cacheable locally. Authentication passes. Aggregation fails (1). Business validation fails (1) and (3).

🎯 Staff Move: "The gateway becoming the new monolith is the default outcome, not a risk. It happens one reasonable feature request at a time. I'd write the scope charter on day one and make every addition justify itself against it."

3.2 Fault Line 2: One Central Gateway vs Federated Gateways#

The Staff default: One gateway platform — same runtime, same pipeline, same policy engine — deployed as multiple isolated instances: at least first-party vs partner, and cells within each. Autonomy comes from self-service manifests, not from separate gateways.

When to deviate: Acquired companies or regulated business units may need separate gateways temporarily; plan convergence rather than accepting permanent divergence.

3.3 Fault Line 3: Auth at the Edge vs Auth in Every Service#

The Staff default: The gateway validates the external token locally (cached JWKS), strips any client-supplied identity headers, and forwards a signed internal identity (or an exchanged internal token) over mTLS. Services verify the mTLS peer is the gateway (or an authorized caller) and perform resource-level authorization — "can user 42 read order 9001?" — because only they have the data to answer it.

Client → Gateway:   Authorization: Bearer <external JWT, 15 min>
Gateway:            verify signature (cached JWKS), exp, aud, iss; check coarse scopes
                    strip: x-user-id, x-identity, x-tenant (anything client-supplied)
Gateway → Service:  mTLS (gateway workload identity)
                    x-identity: <signed internal assertion: sub, tenant, scopes, exp 60s>
Service:            verify mTLS peer ∈ allowed callers; verify assertion signature;
                    authorize resource access with domain data

Revocation tradeoff: Locally validated JWTs can't be revoked instantly. Keep lifetimes short (5–15 min), and for high-risk events (account compromise), push a small deny-list of token IDs or subjects to the gateway via the config channel.

3.4 Fault Line 4: Gateway Aggregation vs BFF vs Client Composition#

The Staff default: BFFs owned by client teams for first-party apps; federated GraphQL when many clients need flexible composition and the org can invest in schema governance. The gateway routes to them; it doesn't compose.

3.5 Fault Line 5: Self-Service Velocity vs Global Blast Radius#

Validation that matters:

• Prefix ownership: a manifest may only define routes under prefixes registered to that team; conflicts are rejected.
• Policy: routes require auth unless an approved exception exists; timeouts ≤ platform max (e.g., 30s); retries only for idempotent methods.
• Safety: regex routes are linted for catastrophic backtracking; wildcard routes need review.
• Rollout: canary cell → region → global, each with health gates and automatic rollback, ~30–60 min end to end; emergency path exists but still staged, just faster.

🎯 Staff Move: "Self-service doesn't mean unreviewed. It means the pipeline does the review — ownership, policy, safety — so humans only review exceptions. And the platform team uses the same pipeline; there is no side door."

4. Failure Modes & Operational Reality#

4.1 The Config Push That Took Down Every Route#

t=0:      Team-search merges a manifest with route prefix '/' (meant '/search/')
t=+1min:  Pipeline v1 has no prefix-ownership check; config server pushes globally
t=+1m30s: All gateway nodes load the new snapshot; '/' matches before longer prefixes
          due to an ordering bug → 70% of requests routed to search-svc
t=+2min:  404s and 400s everywhere; checkout, login, feed all failing
t=+6min:  On-call identifies config version change; rollback initiated
t=+9min:  Rollback propagates; errors subside

Detection: gw_5xx_by_route and gw_404_rate spike correlated with config_version change; per-route traffic share shifts abruptly.

Mitigation: One-click rollback to previous snapshot; config server keeps the last N signed snapshots.

Prevention: Prefix ownership registry; longest-prefix-match semantics tested in CI; staged rollout (canary cell 5% for 10 min with automatic rollback on route-level error or traffic-share anomalies); diff reports showing "routes whose upstream changed" for review.

Owner: Platform owns the pipeline and rollback; team-search owns the manifest. The post-mortem finding is the platform's: the pipeline allowed a change outside the team's ownership.

4.2 The Hot-Path Dependency — Auth Service Slowdown#

t=0:      Gateway validates partner opaque tokens via introspection on every request
t=+0s:    Identity service DB has lock contention; introspection p99 2ms → 1.5s
t=+10s:   Gateway worker connections pile up waiting on introspection;
          Little's law: 20K RPS × 1.5s = 30K in-flight vs ~8K capacity
t=+30s:   Gateway latency for ALL routes rises (shared event loop/connection pools)
t=+60s:   First-party mobile traffic (JWT, no introspection) also timing out
          because it shares gateway capacity with partner traffic
t=+8min:  Identity team fixes DB; recovery

Detection: gw_auth_latency_p99, gw_upstream_pending{cluster=idp}, overall gw_request_latency_overhead_ms.

Mitigation: Cache introspection results (min(token TTL, 60s)); bulkhead introspection calls (dedicated connection pool with a concurrency cap and a 50ms timeout); on timeout for a previously-seen valid token, allow within cache grace period.

Prevention: Prefer JWT for first-party (local validation); separate partner and first-party gateway deployments; never let a per-request dependency share capacity with traffic that doesn't need it.

Owner: Platform (gateway design); identity team (IdP performance).

4.3 Certificate Expiry#

t=0:      Wildcard cert for *.api.example.com expires at 00:00 UTC Saturday
t=+0s:    Every client TLS handshake fails; mobile apps show 'cannot connect'
t=+20min: Someone notices social media reports; on-call paged by synthetic monitors
t=+50min: New cert issued, deployed to all gateway nodes

Detection: cert_expiry_seconds per hostname per node; external synthetic TLS checks.

Prevention: Automated issuance and renewal (ACME or internal CA) at ≤ 2/3 of lifetime; alerts at 30, 14, 7 days; certs deployed through the staged config pipeline like everything else; inventory of every hostname's certificate.

Owner: Platform. This is a 100%-blast-radius failure with a 100%-preventable cause — which is why it keeps happening across the industry.

4.4 Key Rotation Lockout#

The IdP rotates its JWT signing key and immediately stops publishing the old one. Gateways that cached JWKS 8 minutes ago don't have the new key and reject every newly issued token; gateways that refresh see the new key but tokens signed with the old key (still valid for 15 minutes) now fail.

Prevention: Rotation protocol: publish new key → wait ≥ JWKS cache TTL → start signing with new key → keep old key published until the longest token lifetime passes → remove. Gateways refresh JWKS on unknown kid (rate-limited) rather than failing immediately.

Owner: Identity team (rotation protocol); platform (JWKS refresh behavior).

4.5 Noisy Client Class#

A partner's batch job sends 30K RPS of valid, within-quota requests to a partner endpoint that's expensive upstream. The shared gateway deployment's connection pools and CPU are consumed; first-party app latency rises 4×.

Prevention: Separate deployments (or at least separate listeners, worker pools and upstream connection pools) for first-party vs partner; per-client-class concurrency caps; quotas that account for cost, not just request count.

Owner: Platform (isolation); API product team (partner quotas).

4.6 Retry Amplification Through the Gateway#

The gateway retries 5xx on all routes twice; the BFF retries twice; services retry. A downstream brownout becomes a 27× load multiplier. See Circuit Breakers for the full analysis. Gateway rule: retries only on connection failure and idempotent methods, under a budget (≤ 10% of requests), and never when the upstream returned Retry-After or the remaining deadline is too short.

Owner: Platform.

4.7 Operational Reality Matrix#

5. Evaluation Rubric#

5.1 Level-Based Signals#

5.2 Strong Hire Signals#

5.3 Lean No-Hire Signals#

5.4 Common False Positives#

• Detailed plugin knowledge for one vendor ≠ gateway design. The questions are scope, dependencies and ownership.
• "Add GraphQL" ≠ solving aggregation. It moves the problem to schema governance and query cost control.
• Drawing a mesh and a gateway ≠ understanding their split. North-south vs east-west, and who owns each.

6. Interview Flow & Pivots#

6.1 Typical 45-Minute Shape#

6.2 How Interviewers Pivot — And What They're Testing#

6.3 What to Deliberately Skip#

• Vendor feature comparisons (use the technology guide)
• Rate-limit algorithm details (link to Rate Limiting)
• OAuth flow diagrams beyond "IdP issues tokens; gateway validates"
• Load balancer algorithms for upstreams

6.4 Follow-Up Questions to Expect#

1. "How do you revoke a token immediately?" — Short lifetimes plus a pushed deny-list of token IDs/subjects for emergencies.
2. "How do services know the request came through the gateway?" — mTLS peer identity plus a signed internal assertion with a short expiry.
3. "How do you do canary releases of a service through the gateway?" — Weighted routes in the manifest, header-based overrides for testers, automatic promotion on SLOs.
4. "Where do CORS and request size limits live?" — Gateway — they're uniform and needed before routing.
5. "What if the gateway itself needs to call a service to decide routing?" — Avoid; if unavoidable, cache aggressively and fail to a safe default route.
6. "How do you debug a slow request?" — Request ID at the gateway, trace span with per-filter timings, upstream timing header.
7. "How do you deprecate a route?" — Deprecation and Sunset headers, usage tracking per consumer, outreach, then 410 after the window.

7. Active Drills#

Drill 1: The Opening#

Prompt: "Design an API gateway for our company."

Drill 2: Auth Without a Hot-Path Dependency#

Prompt: "How does the gateway authenticate requests without depending on the identity service for every call?"

Drill 3: Make It Concrete — Self-Service Routes#

Prompt: "Sixty teams need to add and change routes daily. How do they do it without breaking each other?"

Drill 4: The Identity Service Is Down#

Prompt: "The identity provider has been down for 30 minutes. What works?"

Drill 5: Aggregation Request#

Prompt: "The mobile team wants the gateway to combine five service calls into one home-screen response."

Drill 6: One Partner Dominates#

Prompt: "One partner is 60% of partner-API traffic and its batch jobs spike to 10× at midnight."

Drill 7: Build vs Buy#

Prompt: "Should we build our own gateway?"

Drill 8: Policy Change Without an Outage#

Prompt: "Security wants every route to require authentication by default. Today 15% of routes are unauthenticated. Roll it out."

Drill 9: Cost#

Prompt: "Our managed gateway bill is $180K/month. Is that reasonable?"

Drill 10: Multi-Region#

Prompt: "We're going active-active in two regions. How does the gateway work?"

8. Deep Dive Scenarios#

Deep Dive 1: Peak-Traffic Latency Collapse#

Context: During the holiday peak (4× normal traffic), gateway p99 overhead rises from 3ms to 400ms and error rates climb to 8% across all routes. Backend services report normal latency. The gateway fleet autoscaled from 40 to 120 nodes, but it didn't help. You're pulled in.

Questions to Surface First:

• Is the time spent inside the gateway (filters) or waiting on something (upstream connections, a dependency)?
• Did anything change recently — a plugin, a route with a regex, a logging config?
• Are upstream connection pools exhausted, and did autoscaling multiply connection counts to backends?
• Is a shared dependency (rate-limit store, auth) in the hot path?

Typical L5 Approach: Scales further, raises instance sizes. The added nodes each open new connection pools to every upstream, making backend connection counts explode, and the true bottleneck — a synchronous call to a central rate-limit Redis per request, added two weeks earlier by a well-meaning plugin update — gets worse with more nodes.

Staff Approach: Uses per-filter timing to find that the rate-limit filter accounts for ~390ms: its Redis cluster is saturated at 4× load. Switches the rate-limit filter to local-first mode via config (staged, canary cell first), cutting overhead back to ~3ms within 15 minutes. Caps upstream connection pools per node to prevent the autoscaling connection explosion.

Principal Approach: Asks how a synchronous per-request dependency entered the gateway hot path without review. Establishes a rule that any filter making network calls requires platform architecture review, a latency budget, and a failure mode (fail-open with local fallback) — and adds peak-traffic load testing of the gateway with all production filters to the holiday readiness checklist.

Key Takeaway: "When the gateway is slow and backends are fine, look for a filter that makes a network call."

What clears the Staff bar:

• Per-filter timing before scaling
• Recognizes scaling can worsen a shared-dependency bottleneck
• Turns the fix into a review policy

Deep Dive 2: The Silent Auth Bypass#

Context: A penetration test finds that GET /internal/admin/users is reachable from the internet without authentication. Investigation shows a team added the route with auth: none eight months ago "temporarily for testing," and the downstream service trusted the gateway to have authenticated the call.

Questions to Surface First:

• How many other routes have auth: none, and which were approved?
• Did the service perform any authorization itself?
• Logs: was the route accessed by anyone other than the testers?
• Why does the pipeline allow auth: none without an exception?

Typical L5 Approach: Removes the route, adds auth, closes the finding.

Staff Approach: Fixes the class. Makes unauthenticated routes require an exception ID with security approval and an expiry; adds a CI test enumerating all public routes against the exception list; requires services to authorize using the verified identity (a request without a valid internal identity assertion is rejected by the service, regardless of gateway config); separates internal/admin routes onto a hostname not exposed to the internet at all.

Principal Approach: Treats "services trusted the gateway" as an organizational design flaw. Writes the zero-trust principle into the gateway standard: the gateway is a first check, never the only one. Makes the public-route inventory a quarterly security review artifact.

Key Takeaway: "The gateway is the first lock, not the only one."

What clears the Staff bar:

• Fixes the class with a pipeline policy
• Requires service-side authorization
• Adds expiry to exceptions

Deep Dive 3: Onboarding the Partner API Product#

Context: The business is launching a public partner API with three plans (Free, Pro, Enterprise), usage-based billing, and a promise of 12 months' notice before breaking changes. Today all traffic goes through one first-party gateway.

Questions to Surface First:

• Must partner traffic be isolated from first-party traffic? (Yes — different risk profile.)
• How accurate must usage counts be for billing?
• How will versions be expressed and tracked per consumer?
• Who is the product owner for the API contract?

Typical L5 Approach: Adds API-key auth and per-key rate limits to the existing gateway; uses gateway counters for billing.

Staff Approach: Separate hostname (api.partners.example.com) and separate gateway deployment on the same platform. API keys/OAuth client credentials mapped to plans; rate limits for protection enforced locally; billing from an exact metering pipeline built on access logs (idempotent aggregation per request ID) — gateway counters are approximate by design. Path versioning (/v1, /v2) with Deprecation/Sunset headers, per-consumer version usage dashboards, and a developer portal. Contract tests in CI block breaking changes to published versions.

Principal Approach: Establishes that the partner API is a product with an owner, SLAs, and an on-call — not a set of routes. Decides whether to use a managed API-management layer for keys, plans and portal (commodity) while keeping the gateway runtime shared, and sets the deprecation policy as a company commitment that legal and sales understand.

Key Takeaway: "A public API is a product contract; the gateway enforces it, but someone has to own it."

What clears the Staff bar:

• Isolates partner traffic on a shared platform
• Separates exact metering from approximate enforcement
• Designs versioning and deprecation as first-class

Deep Dive 4: Post-Mortem — The Global Outage From a Gateway Upgrade#

Context: The platform team upgraded the gateway runtime to a new minor version. It was tested in staging and rolled to all regions over 20 minutes. A change in default header-size limits caused ~12% of requests (those with large cookies from a marketing tool) to be rejected with 431 errors. It took 2 hours to identify because error dashboards were aggregated across all routes and the error rate looked like "elevated but not critical."

Questions to Surface First:

• Why did a 20-minute global rollout pass without detecting a 12% error rate?
• Why didn't staging catch it — what traffic does staging see?
• Were the health gates measuring the right things (status codes including 4xx)?
• Why did detection take 2 hours?

Typical L5 Approach: Raises the header limit, rolls forward, adds a staging test for large headers.

Staff Approach: Fixes the rollout. Runtime upgrades go through the same staged pipeline as config: canary cell with 1–5% of production traffic for at least an hour, health gates that include 4xx classes the gateway itself generates (431, 413, 400 from parsing), compared against the control cells. Adds shadow traffic replay (mirrored production requests to the new version, responses compared) before canary. Dashboards break out gateway-generated errors from upstream-generated errors.

Principal Approach: Treats runtime upgrades as the highest-risk change class for the platform, with their own error budget and change calendar (not during peak seasons). Asks how many teams would have noticed in minutes if they had per-route SLO alerts — and makes per-team route SLO dashboards part of the platform offering.

Key Takeaway: "Upgrading the gateway is a deploy to every service at once. Canary it with real traffic and measure the errors the gateway itself generates."

What clears the Staff bar:

• Distinguishes gateway-generated from upstream errors
• Pins defaults explicitly
• Extends the staged pipeline to runtime upgrades

Deep Dive 5: Multi-Region Expansion#

Context: The company is expanding to the EU with a new region. Leadership wants a single global API hostname, EU data residency for EU users, and no increase in latency for US users.

Questions to Surface First:

• How is a user's home region determined — by account, by location, by token claim?
• Which services are region-local (with residency requirements) and which are global?
• What happens when an EU user travels to the US?
• How does config roll out across three regions without correlated failure?

Typical L5 Approach: Deploys a gateway in the EU, uses GeoDNS to route EU IPs to it.

Staff Approach: Routes by account home region, not client IP: the token carries a home_region claim; any gateway can receive the request (nearest by anycast/latency DNS), and if the home region differs, forwards it to the home region's gateway over the backbone — so a traveling EU user's data stays in the EU. Region-local services are only routed within their region; global services (catalog) are served locally everywhere. Config staged per region; each region can absorb failover only for non-resident traffic.

Principal Approach: Aligns gateway routing with the company's data-residency architecture as a whole — identity, logging, analytics — and gets legal sign-off on the routing rules. Recognizes that "single global hostname" plus residency constraints means the gateway now enforces compliance, which raises its change-management bar.

Key Takeaway: "Route by where the user's data lives, not where the user is standing."

What clears the Staff bar:

• Uses token claims for home-region routing
• Separates resident and global services
• Makes residency violations observable

9. Level Expectations Summary#

After studying this case study, you should be able to:

• State and defend a thin-gateway scope with a concrete scope test
• Design a hot path with no synchronous dependencies: local JWT validation, local-first rate limits, fail-static config
• Propagate identity safely: strip client headers, signed internal identity, mTLS, service-side authorization
• Design self-service route manifests with prefix ownership, policy validation and staged rollout
• Separate first-party and partner traffic, and meter partner usage accurately
• Explain when aggregation belongs in a BFF, a GraphQL layer, a service, or the client
• Plan runtime upgrades, certificate renewal and key rotation as the high-risk changes they are
• Reason about multi-region routing, including residency

The Bar for This Question#

Mid-level (L4): Describes a reverse proxy that authenticates, rate limits and routes.

Senior (L5): Designs a functional gateway with auth, rate limiting, routing, logging and horizontal scaling. Tends to add aggregation and transformation. Misses hot-path dependencies, config blast radius, zero trust and ownership.

Staff+ (L6): Draws a scope boundary, removes hot-path dependencies, makes config self-service and staged, isolates client classes, propagates identity with defense in depth, puts aggregation in team-owned BFFs, and names owners for routes, policies and the platform. The interviewer should learn something from the answer.

10. Staff Insiders: Controversial Opinions#

10.1 "Your Gateway Is Becoming a Monolith, and You Did It One Reasonable Request at a Time"#

The Staff position: Write the scope charter first; every addition must pass the scope test. Say no to good ideas that belong elsewhere.

Why this matters in interviews: Proposing to keep things out of the gateway is a stronger signal than proposing features to add.

10.2 "Most Gateway Outages Are Config Changes, Not Traffic"#

Gateways are stateless and scale horizontally; traffic spikes rarely kill them. Config pushes, certificate expiry, runtime upgrades and key rotation — changes applied to every instance — are the dominant causes of gateway-wide outages.

The Staff position: Invest in the change pipeline before the scaling story.

Why this matters in interviews: Candidates spend their time on autoscaling; interviewers are waiting to hear about config.

10.3 "The Gateway Should Not Authorize"#

Coarse checks (scopes, tenant match) at the gateway are fine. Resource-level authorization — "can this user edit this document?" — needs domain data the gateway doesn't have and shouldn't fetch. Gateways that try (calling a policy service per request with resource lookups) add a hot-path dependency and still get it wrong.

The Staff position: Authenticate and coarse-filter at the gateway; authorize in services, possibly using a shared policy library or engine they call.

Why this matters in interviews: It shows precision about authentication vs authorization and about where data lives.

10.4 "You Probably Don't Need GraphQL at the Gateway — You Need Owned BFFs"#

GraphQL solves flexible composition for many clients, at the cost of schema governance, query-cost control and new failure modes (expensive queries, N+1 fan-out). Many organizations with 2–3 first-party clients get the benefit more cheaply with one BFF per client.

The Staff position: BFFs first; federated GraphQL when the number of clients and the variety of their needs justify the governance.

Why this matters in interviews: It shows you evaluate a popular technology on cost and ownership, not fashion.

10.5 "Per-Request Pricing Makes Managed Gateways a Trap at Scale — and a Gift Below It"#

At low volume, a managed gateway eliminates a platform team. At tens of billions of requests per month, per-request pricing can exceed the cost of the team and the compute to self-host. The crossover moves with your volume, not with vendor features.

The Staff position: Choose managed early, keep manifests portable, and recompute the crossover yearly.

Why this matters in interviews: Framing build vs buy as a curve rather than a verdict is an L6/L7 signal.

11. The Principal Lens (L7)#

Why L7 Sees This Problem Differently#

At L6, the gateway is a thin, reliable proxy with a safe change pipeline. At L7, it is the organizational boundary between the platform and every product team — the place where security policy, identity, API contracts, traffic governance and cost converge. The Principal decisions are about how many gateways the company should have, what the platform will and won't own, how 60 teams change it without coordination, which contracts (URL structure, identity format, versioning policy) are forever, and whether the company should rent or run the runtime. Get the boundary right and the gateway accelerates every team; get it wrong and it becomes the company's slowest-moving monolith and its largest correlated failure domain.

The Org-Level Fault Line#

Central API platform team vs domain-owned gateways.

🧭 Principal Move: "Centralize the mechanism and the policy; decentralize the routes and the composition. The platform team's success metric is how rarely product teams need to talk to it."

Cost Model#

Assumptions: self-hosted proxy instances ~8 vCPU at ~$30/vCPU-month; managed per-request pricing on the order of ~$1–3.50 per million requests; fully loaded engineer ~$300K/year. Order of magnitude only.

The pattern: managed is cheapest at startup scale, self-hosted almost always wins at large scale, and the growth tier is where the decision is genuinely close — dominated by whether you already have a platform team and whether managed API-management features are load-bearing.

The 3-Year Evolution Path#

One-Way Doors vs Two-Way Doors#

The Standard I'd Write#

RFC: API Gateway Platform Standard v1

Scope: All north-south HTTP/gRPC traffic entering production. East-west traffic is out of scope (see the Service Mesh standard).

Requirements:

• All public hostnames MUST be served by the gateway platform. New standalone gateways require architecture review.
• Routes MUST be declared in team-owned manifests under registered prefixes; changes MUST go through the validation and staged rollout pipeline. There is no manual path, including for the platform team.
• Routes MUST require authentication unless an approved exception with an expiry exists.
• The gateway MUST strip client-supplied identity headers and forward a signed internal identity over mTLS. Services MUST verify it and MUST perform resource-level authorization.
• Gateway filters MUST NOT make synchronous network calls on the request path without platform architecture review, a latency budget and a defined failure mode.
• The gateway MUST NOT contain client-specific aggregation or business logic; these belong in team-owned BFFs or services.
• Partner and first-party traffic MUST run on isolated deployments.
• Runtime upgrades, certificates and signing-key rotations MUST follow the staged change process with production canaries.

Exceptions: Filed with the platform team; security approval for auth exceptions; 6-month expiry.

Success metrics: Gateway availability ≥ 99.99%; p99 overhead ≤ 5ms; median route change lead time < 1 hour; zero cross-team config incidents; unauthenticated routes without exceptions = 0; number of non-platform gateways trending to zero.

What I'd Tell the VP#

"Every customer request goes through our API gateway, so it's both our strongest security control and a single point of failure. We're keeping it deliberately simple — it checks who you are, enforces limits and sends you to the right service — and moving everything else back to the teams that own it, so they can ship without waiting on us. Teams will change their own routes through an automated pipeline that tests every change on a small slice of traffic first; that's where most gateway outages come from today. We'll separate partner traffic from our apps so a partner's spike can't slow our customers down. At our current volume, running it ourselves is cheaper than the managed service by roughly a factor of five, and I'd like to revisit that decision in a year with real numbers."

Principal Interview Signals#

Staff answers that L7 interviewers find insufficient:

• A well-designed gateway with no plan for the four other gateways the company already runs.
• "We'll use managed" or "we'll self-host" without a volume-based cost comparison and a portability plan.
• Self-service routes without a governance model for prefix ownership, exceptions and audit.

Appendices

Appendix A: Mechanics in Depth#

A.1 Request Pipeline Pseudocode#

handle(req):
  enforce_limits(req.headers_size ≤ 16KB, req.body_size ≤ route.max_body)
  req.id = req.header('x-request-id') or uuid()
  route = trie.match(req.host, req.path, req.method)         # in-memory snapshot
  if not route: return 404
  strip(req, ['x-identity', 'x-user-id', 'x-tenant', ...])   # never trust client identity
  if route.auth.required:
      claims = verify_jwt(req.bearer, jwks_cache)             # local; ~0.1ms
      if not claims or not scopes_ok(claims, route.auth.scopes): return 401/403
  if not local_bucket(route.limit_key(claims, req)).take(): return 429 + Retry-After
  deadline = min(route.timeout, platform_max)
  resp = upstream(route.cluster, req + signed_identity(claims), deadline,
                  retry = idempotent(req) and budget_ok(route.cluster))
  scrub(resp, internal_headers)
  emit(access_log, metrics, span)                             # async
  return resp

A.2 Route Matching#

Rule: prefix routes by default; regex requires review and a linter check for catastrophic backtracking.

A.3 Connection Management#

• Client side: HTTP/2 multiplexing, TLS session resumption, idle timeouts (e.g., 60s), max connection age to rebalance.
• Upstream side: pooled keep-alive connections per cluster, capped per node (autoscaling multiplies pools — cap total upstream connections).
• Deploys: drain long-lived connections (HTTP/2 GOAWAY, WebSocket close with retry hints) over a window, not all at once.

Appendix B: Identity and Data Model#

B.1 Identity Propagation#

B.2 Rate-Limit and Quota Keys#

See Rate Limiting for algorithms and drift bounds.

B.3 Route Manifest Fields#

Appendix C: Where Each Concern Lives#

C.1 Quick Comparison#

Appendix D: API Contract and Client Behavior#

D.1 Standard Error Responses#

Distinguish gateway-generated errors from upstream errors in both metrics and a response header, so clients and dashboards can tell them apart.

D.2 Versioning and Deprecation#

GET /v1/orders/123
Deprecation: true
Sunset: Wed, 30 Sep 2027 00:00:00 GMT
Link: <https://developer.example.com/migrate-v2>; rel="deprecation"

Track requests per version per consumer; contact consumers directly before sunset.

D.3 Client Retry Guidance#

Clients retry only idempotent requests, on 503 with Retry-After or connection failure, with jittered backoff and a cap of 1–2 retries. Mobile SDKs should include an idempotency key on POSTs so the gateway/service can deduplicate (API Design).

Appendix E: Observability#

E.1 Core Metrics — Non-Negotiable#

gw_requests_total{route, code, origin=gateway|upstream}
gw_overhead_ms{quantile}                 # time spent in gateway, excluding upstream
gw_filter_duration_ms{filter}
gw_upstream_latency_ms{cluster}
gw_auth_failures_total{reason}           # expired, bad_sig, unknown_kid, scope
gw_ratelimit_rejections_total{route, key_class}
gw_config_version{node}                  # rollout visibility
gw_cert_expiry_seconds{hostname}
gw_upstream_cx_active{cluster}

E.2 Critical Alerts#

E.3 Control Plane vs Data Plane#

The config server and pipeline can be down without affecting traffic — nodes serve the last signed snapshot. Alert on the control plane separately, and never let a node start without a valid snapshot (bake a last-known-good into the image or local disk).

E.4 Debugging "The Gateway Is Slow"#

1. Overhead vs upstream: is the time inside the gateway?
2. Per-filter timings: which filter?
3. Does that filter make network calls?
4. Regex routes or large route tables recently added?
5. TLS: handshake rate spike (resumption broken, client reconnect storm)?

Appendix F: Scale Evolution#

F.1 What Works at Each Scale#

F.2 Multi-Region Path#

Single region → gateway per region with staged config → home-region routing via token claims → cells within regions.

F.3 What You Don't Build on Day One#

• Your own proxy
• GraphQL federation
• A policy engine (start with manifest validation rules)
• Per-tenant cells
• Response caching at the gateway (use the CDN)

Appendix G: Multi-Tenancy, Fairness and Cost#

G.1 Isolation Levels#

G.2 Cost Allocation#

Attribute gateway cost per route owner by request count and bytes; publish monthly. Hairpinned internal traffic and large file transfers through the gateway are usually the first things teams remove once they see the cost.

G.3 Tradeoff Summary#