Abuse & Bot Defence

Technologies that implement this pattern: Redis · API Gateway · Apache Kafka · Apache Flink · Elasticsearch · DynamoDB

Why This Matters#

Every endpoint that does something valuable for a human does the same thing for a script, at 10,000 times the rate and a fraction of a cent per attempt. The login form validates leaked passwords for whoever submits them. The signup form mints free-trial accounts. The checkout form tells a card tester which stolen numbers still work. The search page hands your whole catalogue to a scraper. None of these is a bug, and none of them shows up in a load test — they show up as a 40× spike in failed logins at 3am, a support queue full of "I didn't change my email," and a chargeback rate that gets your merchant account reviewed.

Most candidates treat this as a rate limiting question: "put a limit of 5 logins per minute per IP." Staff engineers treat it as an adversarial economics question. The attacker is not a traffic pattern; it is a person with a budget, a residential proxy pool of a million IPs at a few dollars per gigabyte, CAPTCHA-solving services priced per thousand solves, and a feedback loop that tells them within minutes which of your defences fired. The design question is not how many requests per IP but how do I make each successful abuse attempt cost more than it earns, without making legitimate users pay that cost, and how do I keep doing that after the attacker adapts?

The second reframe: every defence is a classifier, and every classifier has a false-positive budget. A rule that blocks 99% of credential stuffing and challenges 0.5% of real logins sounds great until you multiply: at 2M logins a day, that is 10,000 real people a day staring at a puzzle or locked out, some of whom are your highest-value customers on a carrier network that shares one IP across thousands of phones. Who signed off on that number is the interview question.

If you can walk an interviewer from "what does the attacker gain" to "which signals separate them from users" to "what action each risk level triggers" to "what happens when the scoring service is down" to "how the attacker will adapt and how we'll notice," you are answering at Staff level.

The 60-Second Version#

• Rate limits are the floor, not the defence. Per-IP limits stop naive scripts. A credential-stuffing run spread over 500K residential IPs at 2 attempts per IP per hour never trips them. Limit on several keys at once — IP, /24 or /64 prefix, ASN, account, device, and the username-password pair.
• Score, then choose an action from a ladder. Allow → silently add friction → step-up (MFA, email confirm) → challenge → block. Most traffic should see nothing; the ladder exists so a borderline score costs the user seconds, not their account.
• Fail-open vs fail-closed is decided per endpoint, in advance. If the risk service is down, login fails open to rate limits plus breached-password checks; money movement fails closed to step-up. Deciding during the incident means deciding wrong.
• Credential stuffing is detected in aggregate, not per request. A login success rate falling from ~95% to ~5%, or a spike in "unknown username" failures, is the signal. Breached-password screening of new passwords removes the fuel; a k-anonymity range query sends only a 5-character hash prefix and gets back a few hundred candidate suffixes to check locally.
• Fingerprints decay; plan for it. TLS and device fingerprints, behavioural timing and cookie-bound device IDs are strong for days to weeks before attackers rotate. Treat every signal as having a half-life and measure it.
• The attacker reads your responses. Instant, specific blocks teach them which signal fired. Delayed, batched or silent enforcement — shadow bans, deferred account suspension, uniform error messages — slows their iteration loop from minutes to days.

The Problem#

A consumer fintech app has 6M accounts and handles 2M logins a day. On a Saturday, failed logins jump from 80K to 3.1M in four hours. The traffic comes from 410K distinct IPs, most of them residential addresses in the same countries as real users, each making one to three attempts; the existing limit of 10 failures per IP per 15 minutes never fires. User agents are current mobile Chrome. 0.8% of attempts succeed — about 25K accounts, because people reuse passwords from other breaches. Within an hour the attackers start changing payout email addresses. The team's first fix, blocking a hosting-provider ASN, catches 4% of the traffic. The second, putting a CAPTCHA on every login, drops legitimate login success by 9% and barely dents the attack, because the attackers pay a solving service. Meanwhile the same app has a signup bonus being farmed by a script creating 20K accounts a day, and a card-testing ring using the add-card flow to validate stolen numbers in $0 authorizations. The job is to identify abuse with signals the attacker cannot cheaply fake, apply friction proportional to risk so real users rarely notice, protect the account even when the password is correct, decide in advance what happens when the scoring path fails, and build the loop that notices when the attacker adapts.

Case Studies That Use This Pattern#

• Rate Limiter — The first, cheapest layer: multi-key limits that stop naive automation before scoring runs
• Auth & Identity — Login, MFA step-up, session revocation and recovery flows are where account takeover is won or lost
• Edge Gateway — Where TLS fingerprints, IP reputation and challenges are cheapest to apply
• Payments — Card testing and fraudulent checkout, where risk scores decide allow, 3DS step-up, review or block
• Ticket Drops — Inventory-hoarding bots during a sale, where fairness to humans is the product
• Notifications — SMS pumping and OTP abuse turn your verification flow into someone else's revenue
• URL Shortener — Phishing and malware links abusing a trusted domain
• Dating Matches — Fake profiles and scam accounts, where signup abuse becomes user-safety harm

Which Problem Are We Solving?#

"Stop the bots" hides at least five different goals with different signals, actions and tolerances. Name them and commit.

🎯 Staff Move: "I'll focus on account takeover, because that's where users lose money and trust, and it's the hardest one: the attacker has the right password. That means detection in aggregate, step-up instead of blocks for the grey zone, and protecting the actions that matter — payout and email changes — even after a successful login. Signup abuse and card testing reuse the same signal platform with different actions."

The Core Tradeoff#

Staff Default Position#

Layer cheap multi-key limits under a risk score, map the score to a ladder of proportional actions, protect sensitive actions separately from login, decide fail-open or fail-closed per endpoint before you need to, and treat every defence as a classifier with an owner, a false-positive budget and a half-life.

The default stack: rate limits at the edge on IP, network prefix and ASN, and in the service on account, device and credential pair, with budgets in Redis and local counters for the hottest keys. A synchronous risk service on login, signup, add-payment and sensitive-change endpoints, with a p99 budget of ~20–30ms and a timeout that triggers the endpoint's pre-decided fallback. Signals come from a feature store fed by a streaming pipeline (velocities per IP, device, account and credential over 1m/1h/24h windows) and from the request (TLS fingerprint, device ID, IP reputation). Actions are a ladder — allow, step-up, challenge, block, shadow — with thresholds owned by trust and safety and changed through shadow → canary → enforce. Every decision is logged with its features so analysts can label, and labels from chargebacks, user reports and confirmed ATO feed the next model. Breached passwords are rejected at set and change time. Payout destination changes, email changes and new-device logins to high-value accounts get step-up regardless of score.

When to Deviate#

• Low-value, low-volume product. An internal tool or a B2B app with SSO and 2,000 users: rely on the identity provider's protections, per-account lockout with backoff, and MFA. A risk platform is overhead with nothing to learn from.
• The asset is scarce inventory, not accounts. For a ticket drop, the fairness mechanism (a virtual queue, per-account purchase caps, identity verification before the sale) matters more than real-time scoring; bots that can't jump a randomized queue gain little.
• You are early and the attack is current. Buy edge bot management and a hosted breached-password check this week; build the in-house signal platform when you have enough labelled abuse to train on and a team to own thresholds.
• Regulation demands explainability. Credit, insurance or employment decisions may require stating the reason for an adverse action. Prefer interpretable rules and scorecards over opaque models for those paths, and keep the reason codes.

One Question, Three Levels#

Where the Design Splits#

Fault Line 1: Block vs Challenge vs Shadow#

A block stops the request with an error. A challenge (CAPTCHA, proof-of-work, JavaScript check) asks the client to prove something. A step-up asks the user to prove something (MFA, email link). A shadow action lets the request appear to succeed while neutralizing it — a spam post visible only to its author, a fake account whose messages are never delivered, a scraper served stale or decoy data. Who pays: blocks make wrongly flagged users pay the full cost and tell the attacker exactly which request tripped; challenges make every challenged user pay seconds and some percentage of abandonment, while solving services make attackers pay only cents; shadow actions make the platform pay in complexity and in damage done during the delay, and make wrongly shadowed users pay invisibly. Staff default: step-up for account-level risk, challenges for anonymous high-volume abuse at the edge, shadow or delayed bulk enforcement for adaptive adversaries (spam, fake accounts), and hard blocks only for very high confidence (score > ~95 and corroborated). Deviate when: the action is irreversible and costly (payout to a new account, mass messaging) — block or hold first and explain later.

Fault Line 2: Fail-Open vs Fail-Closed#

The risk service, the feature store, the edge bot manager, the CAPTCHA provider — each can time out. Who pays: fail-open makes the users being attacked pay during an outage that coincides with an attack (and attackers do probe for degraded modes); fail-closed makes every legitimate user pay a full outage of the protected flow for a fault in a supporting system. Staff default: decide per endpoint and write it down. Login: fail open to the always-local layers — rate limits, breached-password check, per-account failure counters — with step-up for any new device. Signup: fail open but mark accounts "unscored" and hold their privileges (no messaging, no payouts) until rescored. Payout and credential changes: fail closed to step-up, never to hard denial. Alert when any endpoint is in fallback for more than 60 seconds. Deviate when: the endpoint is itself the attack target and has a cheap alternative (password reset can fall back to a delayed email-only flow).

Fault Line 3: Inline Scoring vs Asynchronous Detection#

Inline scoring runs before the action, inside a latency budget of tens of milliseconds, with the features you can fetch in that time. Asynchronous detection runs after, over minutes or hours of history, with graph features (accounts sharing devices, payment instruments, IPs) and expensive models. Who pays: inline makes the request path pay latency and availability coupling, and limits signals to what's precomputed; async makes victims pay for whatever happens before detection, and makes the system pay for unwinding (reversing payouts, deleting spam after it was seen). Staff default: both — inline for cheap, precomputed velocities and request signals on the critical endpoints, with a ~20–30ms budget; async graph and cluster detection that feeds back as features (a "device in a known farm" flag) and triggers bulk enforcement. Delay irreversible actions (payouts held for hours on new devices) so async detection has time to act. Deviate when: the action is cheap to reverse (a post can be removed) — then async-first keeps the write path fast.

Fault Line 4: Rules vs Models#

Rules ("more than 20 failed logins for distinct usernames from one device in 10 minutes → block device") are written by analysts, deploy in minutes and can be explained. Models generalize across combinations of signals and degrade more gracefully when the attacker changes one variable. Who pays: rules make analysts pay in maintenance — hundreds of rules accrete, overlap and nobody knows which still fire — and attackers learn their thresholds; models make the team pay for labels, training pipelines, drift monitoring and the inability to explain a single decision simply. Staff default: a model produces the score; rules handle hard policy (sanctioned countries, known-bad lists), fast response to a new attack (deploy in minutes, retire in weeks), and overrides. Every rule has an owner, a creation date and an expiry review. Deviate when: labels are scarce (new product, rare abuse) — rules and anomaly detection carry you until you can train.

Fault Line 5: Build vs Buy at the Edge#

Edge bot management sees traffic across many sites, fingerprints TLS and HTTP/2 behaviour, runs client-side detection scripts and returns a per-request score before your origin sees the request. Your own system knows things no vendor does: account age, purchase history, which devices a user has used for three years. Who pays: buying makes the infra budget pay per request and makes you depend on a generic score that may miss domain abuse (bonus farming looks like normal browsing); building makes your team pay for fingerprinting research that a vendor amortizes across millions of sites. Staff default: buy the edge layer (network and client fingerprints, generic bot score, challenges) and build the account layer (identity, history, device trust, domain-specific velocities). Feed the edge score into your own model as one feature rather than letting it decide alone. Deviate when: you are the scale where vendors learn from you — then the cross-customer signal is less valuable than control.

Common Interview Mistakes#

Quick Reference#

Staff Sentence Templates#

"The attacker gains [asset] at about [cost] per attempt. I want each success to cost more than [value], so I'm targeting [success rate / cost per attempt] rather than request volume."

"Scores under [X] are allowed silently, [X]–[Y] get step-up via [factor], above [Y] with corroboration get [block / hold / shadow]. Thresholds are owned by [team] and change through shadow, then a [N]% canary, then enforce."

"If the risk service times out after [N] ms, [endpoint] fails [open / closed] to [fallback], because [who pays]. We page if any endpoint is in fallback for more than [60] seconds."

"This signal has a half-life of about [days]. I'll track its precision weekly and expect to replace it; enforcement on it is [delayed / batched] so the attacker can't see which request tripped it."

Implementation Deep Dive#

1. Multi-Key Rate Limits — Redis, Keyed on What the Attacker Can't Cheaply Rotate#

One limit per IP is a single wall with a known height. Several limits on different dimensions force the attacker to rotate several things at once, and some of them (accounts, devices with history, phone numbers) cost real money.

# Evaluated in the login service before scoring; each is a token bucket in Redis
# (local in-process counters absorb the hottest keys; see Hot Keys)
limits = [
  # key                                   rate            burst   action on exhaust
  ("ip:{ip}",                             "20/min",       40,     "challenge"),
  ("net:{ip_prefix_24_or_64}",            "200/min",      400,    "challenge"),
  ("asn:{asn}:hosting",                   "500/min",      1000,   "challenge"),   # hosting ASNs only
  ("acct:{username_hash}:fail",           "5/15min",      10,     "step_up"),     # per target account
  ("device:{device_id}:distinct_users",   "3/hour",       5,      "block_device"),
  ("cred:{hash(username,password)}",      "1/day",        3,      "flag_stuffing"), # same pair from many IPs
  ("global:login_fail",                   None,           None,   "alert_only"),  # success-rate monitor
]

# Response is uniform regardless of which limit fired or whether the username exists:
#   "Incorrect email or password" + same latency (pad to ~300ms)
# so the attacker cannot enumerate accounts or learn thresholds.

Why the account-keyed limit steps up instead of locking: a hard lockout after N failures lets anyone lock any user out by guessing at their username. Progressive friction — delays that grow as failures approach the cap, then step-up — keeps the owner able to get in. Current US federal guidance for authenticators caps consecutive failures per authenticator at 100 and suggests bot challenges, increasing wait times and risk-based techniques to keep legitimate users from being locked out by the limit (NIST SP 800-63B-4).

🎯 Staff Insight: The cred:{hash(username,password)} key is the stuffing tell that per-IP limits miss: the same leaked pair tried from 30 different IPs in an hour is a list being worked, not a user who forgot their password.

2. The Risk Service — Features, Score, Ladder, Fallback#

POST /risk/evaluate     (p99 budget 25ms, hard timeout 40ms)
{ "event": "login", "account_id": "a_88123", "ip": "...", "device_id": "d_9f2...",
  "tls_fp": "t13d1516h2_...", "ua": "...", "outcome_so_far": "password_ok" }

features (precomputed by streaming jobs, fetched from the feature store in ~2–5ms):
  ip.reputation, ip.type (residential | mobile | hosting | proxy | tor)
  ip.fail_rate_1h, net24.distinct_accounts_1h, asn.fail_rate_1h
  device.age_days, device.accounts_seen_30d, device.first_seen_for_account
  account.age_days, account.value_tier, account.recent_password_reset
  cred_pair.distinct_ips_24h, tls_fp.share_of_failures_1h
  behaviour.typing_cadence_score, behaviour.time_on_page_ms   (from the client SDK)
  edge.bot_score                                               (vendor score as one feature)

score = model(features)            # gradient-boosted trees, 0–100
policy = ladder[event]             # owned by trust and safety, versioned config
  login:  < 30 allow | 30–70 step_up(email_link or TOTP) | 70–90 challenge then step_up | > 90 block
  overrides: new device AND account.value_tier = high  -> step_up regardless
             known-bad device cluster                  -> shadow (accept, then deny session)

on timeout or error:
  login  -> fallback "local": rate limits + breached-password + new-device step_up
  payout -> fallback "closed": step_up required
  emit risk.fallback{endpoint}; page if > 60s

log: decision_id, features, score, model_version, policy_version, action -> Kafka -> lake

Why the vendor score is a feature, not the decision: an edge bot score sees TLS and client behaviour across many sites and is excellent at "this is a script." It cannot know that this account has logged in from this device every day for three years. Letting either signal decide alone throws away the other.

3. Credential Stuffing and ATO — Remove the Fuel, Watch the Aggregate, Guard the Sensitive Actions#

# Remove the fuel: reject breached passwords at set/change time (k-anonymity range query)
prefix, suffix = sha1(password).hex().upper()[:5], [5:]
candidates = GET https://api.pwnedpasswords.com/range/{prefix}   # ~hundreds of suffixes back
if suffix in candidates: reject("This password has appeared in a data breach")
# on login: if a breached password is used successfully, force reset after step-up

# Watch the aggregate (streaming job, 1-minute windows)
login_success_rate = success / attempts                 # normal ~90–95%; stuffing drives it to < 20%
unknown_user_rate  = fails_unknown_username / fails      # spikes when attackers work a foreign list
alert if success_rate < 60% for 5 min or attempts > 5× trailing-week same-hour
on alert: raise step-up thresholds globally (policy flag), not per-IP whack-a-mole

# Guard sensitive actions independently of login
sensitive = {change_email, change_phone, add_payout_destination, disable_mfa, export_data}
for action in sensitive:
    require fresh auth (< 10 min) + step-up if device.first_seen < 7d
    notify old email/phone with a one-click "this wasn't me" that freezes the account
    hold payouts to newly added destinations for 24–72h on new devices

Why sensitive actions are the real perimeter: stuffing will produce some correct logins no matter what. An ATO only becomes a loss when the attacker changes the recovery email or adds a payout destination. Step-up plus a notification to the old contact plus a hold on new payout destinations converts most successful logins by attackers into dead ends — and the hold gives asynchronous detection hours to act.

4. Device and Behavioural Fingerprints — Strong, Then Stale#

signal                     stability for a real user    cost for attacker to rotate      typical half-life
TLS fingerprint (JA3/JA4)  high per client build         low with a TLS-spoofing library   days–weeks
HTTP/2 settings, order     high per client build         low–medium                        weeks
device ID (signed cookie,  high until cleared            very low (clear storage)          minutes for attackers;
  app install ID)                                                                            valuable as positive trust
platform attestation       high on real devices          high (needs real devices)         months
behavioural (typing,       medium                        medium–high to mimic well         weeks
  pointer, timing)
account/device history     high                          high (needs time)                 does not decay

policy:
  use fingerprints mainly as POSITIVE signals: "known device for this account for 90 days"
  use them as NEGATIVE signals only in aggregate: "this TLS fp is 70% of failures this hour"
  measure precision per signal weekly; retire or reweight when it drops below target

The asymmetry to say out loud: an attacker can make a device look new for free; they cannot make it look old and trusted without time and the victim's history. So fingerprints are most reliable as evidence for a user ("same device, same browser build, same city as the last 300 logins") and least reliable as evidence against one. Privacy review applies: fingerprinting is personal data processing in many jurisdictions, and the retention and purpose need sign-off.

5. Shadow Mode and Staged Enforcement — Changing a Threshold Without Locking Out 50,000 People#

new rule or model version:
  stage 1  shadow: compute decision, log it, take no action       (3–7 days)
           measure: would-have-acted rate, overlap with confirmed abuse labels,
                    share of affected users with long account history (false-positive proxy)
  stage 2  canary: enforce on 5% of traffic, randomized by account
           measure: login success, support contacts per 10K logins, step-up completion rate
  stage 3  enforce: 100%, with an automatic rollback if login success drops > 2 points
  every rule: owner, created_at, review_at (30 days), retire if precision < target

🎯 Staff Move: "I'd ship the new stuffing rule in shadow for three days and look at how many accounts older than a year it would have challenged. If that's more than a few hundred a day, the threshold is wrong — those are our best customers, not attackers."

Technique Comparison

The Adversarial Feedback Loop#

Abuse defence is the only part of system design where the load reads your code. Every visible response is a training signal for the attacker; every defence decays as they adapt. Design the loop on both sides.

The metrics that tell you the loop is working: time from a new attack pattern to a deployed response (target: under an hour for rules), share of abuse caught before monetization (before payout, before message delivery), false-positive rate measured by appeals overturned and challenges on long-tenured accounts, and loss per 100K accounts per month in dollars. "Requests blocked" is not on the list — it goes up when you are winning and when you are blocking customers.

Architecture Diagram#

How to narrate it: the edge removes cheap automation before it costs origin capacity and contributes a score. The application enforces limits on keys only it knows — account, device, credential pair — and calls the risk service only on endpoints where a decision matters. The risk service is fast because its features are precomputed by streaming jobs; when it times out, each endpoint falls back to its pre-decided mode. Everything is logged, and the async side turns history, graphs and labels into new features and bulk actions. Trust and safety owns policy and labels; the platform team owns the risk service's latency and availability; identity owns step-up and sensitive-action guards.

Failure Scenarios#

1. Low-and-Slow Stuffing — 25K Accounts Taken Over Past a Per-IP Limit#

Sat 01:00  Attack starts: 3.1M attempts over 4 hours from 410K residential IPs, 1–3 each.
Sat 01:00  Per-IP limit (10 fails / 15 min) never fires. Success rate drops 94% -> 31%.
Sat 01:40  No alert: dashboards track request rate per IP, not global success rate.
Sat 02:10  First payout-email changes on compromised accounts.
Sat 05:30  Support sees "I didn't change my email." Incident declared.
Sat 06:00  Global step-up enabled for all new-device logins. Attack success drops to ~0.
Sun        25K accounts compromised; 1,900 payout destinations changed; $310K in losses.

Detection: login.success_rate < 60% for 5 minutes (page); login.unknown_username_rate spike; cred_pair.distinct_ips_24h > 10 for many pairs. Blast radius: every account with a reused password; losses concentrated on accounts where payout destination changed without step-up. Mitigation: global step-up on new devices; force password reset with step-up for accounts that logged in during the window; freeze payouts to destinations added in the window. Prevention: aggregate stuffing monitor with an automatic policy flag; breached-password screening; step-up plus 48h hold on new payout destinations from new devices. Owner: trust and safety owns the stuffing monitor and policy flag; identity owns sensitive-action guards.

🎯 Staff Insight: The losses came from payout changes, not logins. A hold on new payout destinations would have turned 25K compromised logins into near-zero losses while detection caught up.

2. The Risk Service Brown-Out — Fail-Closed Takes Down Login#

t=0       Feature store node degrades; risk service p99 rises from 22ms to 900ms.
t=+30s    Login calls risk with a 1s timeout, default on error = deny (never decided).
t=+1min   Login success rate falls from 94% to 12%. Every user sees "try again later."
t=+4min   Retry storm: clients retry 3×, login traffic 3.5×, feature store worse.
t=+22min  Feature store node replaced; recovery as retries drain.

Detection: risk.p99_ms > 40; risk.fallback{endpoint} should have been firing; login.success_rate collapse. Blast radius: all logins for 22 minutes — a self-inflicted full outage of the product's front door. Mitigation: emergency config to bypass risk on login with local limits. Prevention: 40ms hard timeout; per-endpoint fallback written down and tested (login → local layer, payout → step-up); retry budgets on clients; game-day the risk service outage quarterly. Owner: risk platform owns latency and fallbacks; identity owns the login fallback behaviour.

3. The Carrier NAT False Positive — 180K Real Users Challenged#

Mon 08:00  New rule ships to 100%: > 50 distinct accounts per IP per hour -> challenge.
Mon 08:30  Major mobile carrier's CGNAT exit IPs each carry thousands of real users.
Mon 08:30  180K legitimate users get challenges; 11% abandon login.
Mon 10:15  Support volume 6×. Rule rolled back. 2 hours, ~20K users failed to log in.

Detection: login.challenge_rate by ASN; challenge.on_tenured_accounts (accounts > 1 year old) spike; support contacts per 10K logins. Blast radius: one carrier's users — skewed to mobile, often the highest-engagement segment. Mitigation: roll back; allowlist known mobile CGNAT ranges from IP-count rules. Prevention: shadow → 5% canary → enforce for every rule; IP type (mobile, residential, hosting) as a feature so shared-IP rules apply only where sharing is abnormal; automatic rollback on login success drop. Owner: trust and safety owns rule staging; platform owns the automatic rollback hook.

Operational Reality Matrix#

Beyond Staff: The Principal View#

Why L7 Sees This Problem Differently#

A Staff engineer defends one surface well: login gets a score, a ladder and a fallback. A Principal engineer looks across the company and finds login, signup, checkout, messaging, reviews and the referral program each running their own bot logic — five IP blocklists, three CAPTCHA vendors, two device-fingerprint SDKs, and no shared labels, so the fake accounts that signup missed are rediscovered by messaging a week later. The attacker sees one company; the defenders are organized as six teams. At L7 the work is a shared signal and decision platform with per-surface policies, a single labelling pipeline that turns chargebacks, reports and confirmed ATO into training data for everyone, and a business-level agreement on what false positives cost — because the tension between growth (fewer checks) and trust and safety (more checks) is otherwise settled by whoever escalates louder during each incident.

🧭 Principal Move: "We're fighting one adversary with six disconnected defences. I'd build one risk platform — shared signals, shared labels, one decision log — and let each surface own its policy ladder. And I'd get finance to put a dollar value on a false positive, so threshold changes are a business decision, not an argument."

The Org-Level Fault Line#

Central trust and safety platform vs per-product abuse teams vs outsourcing to vendors.

The Principal default is the third row, with vendors plugged in as signal providers at the edge rather than as the decision-maker.

Cost Model#

Assumptions: fully loaded engineer ~$25K/month, analyst ~$15K/month; edge bot management priced per request at roughly $0.50–$2 per million requests (illustrative; varies widely by plan); SMS OTP ~$0.01–0.08 per message depending on country; feature store and stream processing on shared infrastructure.

The Principal observation: the line that moves the business is losses avoided and conversion preserved, neither of which is in the table. A 1-point drop in login success at 50M logins a day is 500K failed sessions daily; a single stuffing wave at growth stage cost $310K in the scenario above. The platform pays for itself only if it is measured in those units — and the SMS line is a reminder that moving users to TOTP or passkeys is both a security and a cost decision.

The 3-Year Evolution Path#

The Year 2 trigger is the tell that per-surface defence has failed: the same device cluster that farmed signup bonuses shows up a week later testing cards at checkout, and nobody connected them.

One-Way Doors vs Two-Way Doors#

The Standard I'd Write#

RFC: Abuse Controls on User-Facing Endpoints (v1)

Scope: Every endpoint that creates accounts, authenticates, moves money, changes credentials or recovery contacts, or sends messages to other users.

MUST:

1. Each endpoint MUST enforce rate limits on at least the network, account and device dimensions, with uniform responses that do not reveal which limit fired or whether an account exists.
2. Each endpoint calling the risk service MUST document its fail-open or fail-closed behaviour and fallback, and MUST test it in a quarterly game day.
3. Changes of email, phone, MFA settings and payout destinations MUST require fresh authentication, step-up on new devices, and notification to the previous contact.
4. New rules and thresholds MUST run in shadow mode, then a canary of at most 5%, with automatic rollback on a defined drop in success rate.
5. Every decision MUST be logged with features, score, model and policy version, and retained for at least 13 months.

SHOULD: Prefer step-up over blocks in the grey zone; prefer TOTP or passkeys over SMS; reject breached passwords at set and change time; give every rule an owner and a 30-day review date.

Exceptions: Filed with trust and safety, time-boxed, with compensating controls.

Success metrics: ATO losses per 100K accounts; share of abuse caught before monetization; challenge rate on accounts older than one year; time from new attack pattern to deployed response.

What I'd Tell the VP#

Attackers are using passwords leaked from other companies to get into our customers' accounts, and today each of our products defends itself separately, so they simply move to whichever is weakest. I'm proposing one shared system that scores risk across all of them, asks for extra proof only when something looks wrong, and protects the moments that cost money — changing a payout account or an email — even when the password is correct. Last quarter's attack cost about $310,000 and a weekend of support; most of that came through a step we can close in weeks. The trade-off we need to own explicitly is how much friction we accept for real customers, so I'd like finance and product to agree a cost for a wrongly challenged user. Then threshold decisions become business decisions instead of incident-night arguments.

Principal Interview Signals#

Staff answers that L7 interviewers find insufficient:

• "We'll add a risk score to login." — Correct; silent on signup, checkout and messaging, where the same attacker goes next.
• "Trust and safety will tune the thresholds." — Doesn't say who arbitrates between growth and safety, or in what units.
• "We'll add more signals." — Doesn't plan for decay, privacy review or the labelling pipeline that makes signals useful.

How Real Companies Built It#

Facebook: Sigma, a Rule Engine Evaluated on Every Interaction#

Facebook's Sigma proactively identifies malicious actions such as spam, phishing and links to malware, and removes detected content automatically. It is a rule engine: every interaction, from posting a status update to clicking "like," triggers evaluation of a set of policies specific to that interaction type. After a two-year redesign replacing its in-house policy language with Haskell, Sigma ran at more than one million requests per second, and policies are continuously deployed — the source in the repository is the code running in Sigma — so engineers can push changes to production in minutes in response to new abuse (Fighting spam with Haskell).

Staff insight: The design goal that drove the rewrite is policy iteration speed, because the adversary iterates. "New rule live in minutes" is a requirement on the abuse platform, not a nicety — and it is why rule ownership and expiry matter, since a system that deploys rules that fast accumulates them just as fast.

Cloudflare: Bot Scores at the Edge, Customer-Chosen Actions#

Cloudflare's bot management assigns each request a bot score from 1 to 99 — 1 meaning almost certainly automated, scores below 30 grouped as likely automated — computed at the edge from several detection engines: heuristics that match known malicious fingerprints, machine learning trained on features across billions of daily requests, and lightweight JavaScript detections that identify headless browsers. The score is exposed to the customer's firewall rules, which decide whether to allow, challenge or block, and request fields such as the JA3/JA4 TLS fingerprints are available alongside it (Bot scores, Bot management variables).

Staff insight: This is the build-versus-buy split in one product: the vendor supplies a cross-site signal and fingerprints, and the customer owns the action ladder. That is the right division — feed the edge score into your own decision as one feature, and keep the policy yours.

Stripe Radar: Risk Scores Mapped to a Ladder of Actions#

Stripe Radar evaluates each payment in real time with models that use hundreds of risk factors and data across Stripe's network of businesses, producing a risk score from 0 to 99. By default a score of 65 or above indicates elevated risk and 75 or above high risk; high-risk payments are blocked by default, elevated-risk payments are allowed by default but can be routed to a manual review queue, and rules can also request 3D Secure authentication — a step-up. Businesses can adjust thresholds, and marking a refund as fraudulent feeds back into the models and adds the email and card fingerprint to block lists (Radar risk evaluation).

Staff insight: Allow, step-up (3DS), review, block — the action ladder, with thresholds the business can move and a feedback path from outcomes to labels. Notice the default: elevated risk is allowed, not blocked, because a false decline is a lost sale. That default is a statement of who pays.

Pwned Passwords and Cloudflare: Breached-Password Checks Without Revealing the Password#

When version 2 of the Pwned Passwords dataset launched with over half a billion leaked passwords, Cloudflare described the k-anonymity range API built for it: a client sends only the first five characters of the password's SHA-1 hash and receives every hash suffix in that range — 478 on average at launch, between 381 and 584 — then checks locally, so the service never learns the full hash of the password being checked, and the responses are simple enough to cache (Validating leaked passwords with k-anonymity). US federal guidance now requires verifiers to compare new passwords against a blocklist that includes passwords from previous breach corpuses (NIST SP 800-63B-4).

Staff insight: Credential stuffing runs on reused, already-leaked passwords. Screening them at set and change time removes the attacker's fuel instead of fighting each attempt — and the k-anonymity design answers the obvious privacy objection before the interviewer raises it.

Practice Drill#

Prompt: "We run a food-delivery marketplace with 15M customer accounts. New customers get a $20 first-order credit. Finance says first-order credits are up 4× this quarter with no matching growth in second orders, and support reports a wave of customers whose saved cards were used for orders they didn't place. The current defences are a per-IP limit on signup and login, and a CAPTCHA on signup. Design the abuse defences."

Staff Interview Application#

How to Introduce This Pattern#

"Before picking controls I want to know what the attacker gains and what each attempt costs them. Then I'd layer cheap multi-key limits, a risk score on the endpoints that matter, and a ladder of actions so most users see nothing and the grey zone gets step-up rather than a block. I'll decide per endpoint what happens when scoring is down, and I'll assume the attacker adapts — so enforcement is staged, partly delayed, and measured in losses, not blocked requests."

Lead with attacker economics, then signals, then the action ladder, then fail-open versus fail-closed, then the feedback loop and owners.

When NOT to Use This Pattern#

• Small, closed user bases: an internal or SSO-only B2B tool should rely on the identity provider, MFA and per-account backoff. A risk platform has nothing to learn from.
• The real problem is capacity, not abuse: if a legitimate partner's integration is overloading you, that's a rate limiting and backpressure question with quotas and 429s, not adversarial scoring.
• Noisy neighbours among paying tenants: fairness between customers belongs to multi-tenancy controls, not bot defence.
• Scarce inventory sales: a randomized virtual queue and purchase caps neutralize most bots more simply than real-time scoring.
• Regulated adverse decisions: where you must explain a denial, use interpretable rules and reason codes, not an opaque score.

Follow-Up Questions to Anticipate#

Scorecard#

Strong Hire Signals

Lean No-Hire Signals

Common False Positives: Knowing CAPTCHA vendors ≠ understanding attacker cost. Listing fingerprinting techniques ≠ knowing how fast they decay. "Machine learning will catch it" ≠ having labels, thresholds and an owner.

Capacity Planning Quick Reference#

Sizing the Defences#

attacker_cost_per_success  = cost_per_attempt / success_rate          # $0.001 / 0.008 ≈ $0.13 per ATO
target                     = cost_per_success > value_per_success      # raise cost or cut value
stuffing_attempts_per_ip   = total_attempts / distinct_ips             # 3.1M / 410K ≈ 7.5 over 4h
false_positive_users_day   = daily_events × challenge_rate × fp_share  # 2M × 2% × 25% ≈ 10K
risk_service_qps           = protected_events_per_sec × peak_factor    # 2M/day ≈ 23/s avg, 3–5× peak
feature_store_reads        = risk_qps × features_per_call              # 100/s × 30 ≈ 3K reads/s
decision_log_bytes_month   = events × ~2 KB × 30                       # 2M/day ≈ 120 GB/month
sms_cost_month             = otp_sends × price_per_sms                 # 300K × $0.03 ≈ $9K

Key Numbers Worth Memorizing#

Common Pitfalls Checklist#

• Limits keyed on network, account, device and credential pair — not IP alone
• Uniform error messages and latency; no account enumeration
• Progressive delays and step-up instead of hard lockout
• Aggregate login success-rate monitor wired to a global policy flag
• Breached passwords rejected at set and change time
• Sensitive actions require fresh auth, step-up on new devices, and notify the old contact
• Fail-open or fail-closed documented and tested per endpoint
• Every rule shadowed, canaried, owned and reviewed every 30 days
• Decision log retains features, score and versions for training and audit
• Success measured in losses and friction, not requests blocked