Technologies referenced in this case study: PostgreSQL · Kafka · Redis · DynamoDB · Cassandra

Related case studies: Blob Storage (S3) · Collaborative Editing · Real-Time Updates · Notification System · Handling Large Blobs

How to Use This Case Study#

Organized for interview use first, reference second. Read front-to-back once. Return to individual sections for targeted review.

Naive design (upload whole files, last-writer-wins by mtime):
t=0:       Laptop reconnects after 21 days offline. 4,200 local edits queued.
t=+5s:     Client re-uploads 38 GB of whole files, including a 9 GB video with a 1-byte tag change.
t=+40min:  Upload still running; home uplink saturated at 20 Mbps.
t=+41min:  Laptop's clock is 7 minutes fast. Its stale copy of budget.xlsx "wins" by mtime.
t=+42min:  Colleague's 3 weeks of edits to budget.xlsx silently overwritten on 6 devices.
t=+3 days: Colleague notices. Support ticket. Version history restore. Trust gone.

Staff design (content-addressed blocks, journal cursor, conflicted copies):
t=0:       Laptop reconnects. Client sends its cursor (journal position 88,412) for each namespace.
t=+1s:     Server returns 1,930 remote changes since 88,412. Client reconciles 3-way: base / local / remote.
t=+2s:     Client hashes changed files into 4 MB blocks; asks server which of 9,600 hashes are missing.
t=+3s:     Server: 212 blocks missing (the rest already exist — dedup + unchanged blocks).
t=+90s:    848 MB uploaded, 4,200 commits applied with base-revision checks.
t=+91s:    budget.xlsx commit rejected (base rev 17, server at rev 29) → "budget (Laptop's conflicted copy).xlsx".
t=+92s:    Zero data lost. One visible conflict. Every other device converges within 2s via push.

Executive Summary

If you only read one section, read this. Everything else in the case study elaborates on the contrast below.

What This Interview Actually Tests#

File sync is not a storage question. Blob storage is a solved, rentable commodity — S3 will hold your blocks at eleven nines.

This is a distributed state reconciliation question that tests:

• Whether you separate the block plane (immutable, content-addressed, dumb, huge) from the metadata plane (mutable, ordered, small, the hard part)
• Whether you can define a commit protocol that is idempotent, resumable, and conflict-detecting across flaky networks
• Whether you have a conflict policy a product manager has signed off on — and know that "last-writer-wins" silently destroys work
• Whether you can reason about a fleet of clients you do not control: old versions, wrong clocks, 3-week-offline laptops, 800K-file trees

The key insight: the metadata journal is the product. Blocks are just bytes. Every hard bug in file sync — lost edits, ghost files, infinite re-sync loops, "my folder disappeared" — lives in metadata and in the client's reconciliation of it.

The L5 → L6 → L7 Contrast — Start Here#

The Staff Positions#

The Three Intents#

Three intents hide under "Design Dropbox." Each leads to a different system.

🎯 Staff Move: "I'll design multi-device sync with shared folders. Backup is a strict subset of that design — one writer, no conflicts — so if we get sync right, backup is a configuration. The reverse isn't true: a backup system can't grow into shared folders without rebuilding the metadata layer."

The Five Fault Lines#

In the Wild: Real Production Systems#

Why this section belongs here: citing well-known production systems shows you've studied operational reality. Use these as one-sentence anchors in the interview.

Dropbox — Content-Addressed Blocks, Then Its Own Storage#

Dropbox's publicly described design splits files into 4 MB blocks addressed by SHA-256 hash, with metadata stored separately from block data. Blocks originally lived in Amazon S3; around 2015–2016 Dropbox migrated the bulk of its data into Magic Pocket, an in-house exabyte-scale block store. Later, Dropbox rewrote its desktop sync engine (publicly called Nucleus) in Rust, explicitly to make the client's sync state model testable and to eliminate classes of reconciliation bugs.

Staff insight: the two biggest Dropbox engineering investments were the storage economics (Magic Pocket) and the client sync engine (Nucleus) — not the upload API. That is exactly the split this case study argues for: the block plane is a cost problem, the sync engine is a correctness problem.

Google Drive and Microsoft OneDrive — Placeholders / Files On-Demand#

Both products (and Dropbox, via its smart-sync feature) ship placeholder files: the directory tree is fully synced, but file contents download only on open. This decouples "the namespace fits on the laptop" from "the bytes fit on the laptop."

Staff insight: placeholders change the client's hardest problem from bandwidth to metadata scale — a 2M-file tree must be listed, watched, and reconciled even when 0 bytes of content are local. Say this when the interviewer asks about enterprise customers with huge shared drives.

rsync — Rolling-Checksum Delta Transfer#

rsync (Tridgell & Mackerras) sends a weak rolling checksum plus a strong hash per block of the receiver's copy, letting the sender find matching blocks at any byte offset and send only literal differences.

Staff insight: rsync proves you can get delta efficiency on the wire without changing your storage format. Keep storage fixed-block and content-addressed; use rsync-style delta only as a transfer optimization for large files edited in place.

What Interviewers Probe#

System Architecture Overview#

Reading the diagram: three planes with three different scaling laws. The block plane scales with bytes (exabytes, cheap per GB, immutable, cacheable forever). The metadata plane scales with changes (billions of small ordered writes per day, one writer per namespace). The notification plane scales with connected devices (tens of millions of idle connections). A client never learns content from a notification — it only learns "go ask the journal." That separation is why a dropped notification costs seconds of latency, never correctness.

Quick-Reference: The 30-Second Cheat Sheet#

Key Numbers Worth Memorizing#

Interview Walkthrough

The most common mistake: candidates spend 20 minutes on upload APIs, S3 multipart, and a CDN for downloads — the parts that are genuinely easy — and never reach the journal, the reconciliation protocol, or conflicts. Compress the basics to ~10 minutes. Spend the rest where files actually get lost.

Phase 1: Requirements & Framing (2–3 minutes)#

State functional requirements in one breath:

"Users put files in a folder on any device; the same tree appears on every other device and for everyone the folder is shared with. Devices can be offline for a long time. Users can see and restore history."

Then spend the time on the non-functional requirements — that's where the design lives:

"Which product is this: personal multi-device sync, shared team folders, or backup? I'll design sync with shared folders, since backup is a subset. The requirements I'd commit to: one, we never silently lose an edit — conflicts become visible copies. Two, online devices converge within about 10 seconds p99. Three, we upload only bytes the server doesn't already have. Four, a device offline for 30 days reconnects and converges without a full rescan."

Scale it in 20 seconds:

"Say 500M registered, 50M daily active users, ~3 devices each, 20 file changes per active user per day. That's ~1B commits a day — ~12K/s average, ~40K/s peak. Storage in the high hundreds of petabytes to an exabyte. The byte volume is big but boring; the commit rate and the number of connected devices are what shape the design."

🎯 Staff Move: Naming "never silently lose an edit" as requirement #1 reframes the entire interview. It forces a base-revision commit protocol, rules out last-writer-wins, and gives you an SLO you can alert on. The interviewer now knows you think about correctness from the user's point of view.

Phase 2: Core Entities & API (1–2 minutes)#

Name the nouns in 30 seconds:

• Namespace — a root of a tree with its own journal: a user's home, or a shared folder. The unit of ordering, sharding, and permissions.
• Journal entry — (namespace_id, rev, path, file_id, blocklist, size, author, ts, deleted). Append-only; rev is monotonic per namespace.
• Block — immutable 4 MB byte range, addressed by SHA-256(content).
• Cursor — a client's position in a namespace journal: (namespace_id, rev).

Block plane (bulk, idempotent):

POST /blocks/missing   { hashes: [h1..hN] }            → { missing: [h3, h7] }
PUT  /blocks/{hash}    <bytes>                          → 201 (server verifies SHA-256)
GET  /blocks/{hash}                                     → bytes (cacheable forever)

Metadata plane (small, ordered):

POST /commit  { ns, path, blocklist, base_rev, client_op_id }
              → 200 { rev } | 409 { conflict, server_rev } | 412 { missing_blocks }
GET  /list_since?ns=..&cursor=88412&limit=2000  → { entries[], new_cursor, has_more }
GET  /notify?cursors=[..]  (long-poll, 60–90s)   → { changed: [ns...] }

🎯 Staff Move: "Two properties matter in this API. base_rev makes every commit a compare-and-set, so the server detects conflicts instead of resolving them by accident. client_op_id makes commits idempotent, so a client that loses the response can retry safely. Without both, flaky networks create either lost edits or duplicate files."

Phase 3: High-Level Architecture (≤5 minutes)#

Draw three planes, then walk the upload and the download.

Walk the flow in 90 seconds:

1. The watcher on Device A sees report.pdf change. The client chunks it into 4 MB blocks and hashes each.
2. The client asks the block service which hashes are missing; uploads only those. Block PUTs are idempotent — same hash, same bytes.
3. The client commits (path, blocklist, base_rev=41). The sync service checks permissions, verifies all blocks exist, and does a CAS: if current rev is 41, append rev 42. Otherwise, 409 conflict.
4. The journal append publishes "namespace 7 changed" to the notification service.
5. Device B's open long-poll returns with ns 7 changed — no payload.
6. Device B calls list_since(cursor=41) and gets rev 42's metadata.
7. Device B fetches only the blocks it doesn't already have locally, assembles the file, and atomically renames it into place.

Hit these explicitly:

1. Blocks and metadata are separate services with separate scaling laws.
2. The commit is the linearization point — a file doesn't exist until its journal entry does, and the journal entry can't exist until all its blocks do.
3. Notifications are hints — correctness comes from the cursor, not from push delivery.
4. The client holds three trees: last-synced base, local disk, and remote — reconciliation is a 3-way diff.

🎯 Staff Move: After drawing, say: "This is the design everyone draws. It works on the happy path. The interesting part is what the client does when it has been offline for a month and the three trees disagree — and what the server does when a client is wrong."

Phase 4: Transition to Depth (1 minute)#

"The block plane is basically Blob Storage — immutable, content-addressed, I'm happy to hand-wave it. The three places this design actually breaks are: conflict detection and resolution, the metadata journal under a huge shared folder, and the client fleet — especially deletes from a buggy client. Where would you like to go?"

If the interviewer has no preference, lead with conflicts — it's the most product-visible and immediately shows correctness thinking. Then go to the journal.

Phase 5: Deep Dives (25–30 minutes)#

For each: state the tradeoff → pick a position → quantify the cost → name who absorbs it.

Deep dive A: Conflicts and the commit protocol (7–8 min)

"Every commit is a compare-and-set on base revision. Two laptops both edit budget.xlsx from rev 17 while offline. Laptop 1 reconnects first: commit(base=17) succeeds, rev 18. Laptop 2 reconnects: commit(base=17) fails — server is at 18. Laptop 2's client doesn't retry blindly; it downloads rev 18 into place and renames its own version to budget (Laptop 2's conflicted copy 2026-09-29).xlsx, then commits that as a new file. Both edits survive. The user merges by hand."

Name who pays: "The user pays with a bit of friction. The alternative — last-writer-wins — makes the user pay with lost work, and they don't even find out. Product signs off on this; it's a product decision, not an engineering one."

Mention the edge cases:

• Rename vs edit: Device A renames a.txt → b.txt, Device B edits a.txt. Track file_id separately from path so the edit follows the rename instead of resurrecting a.txt.
• Delete vs edit: Edit wins. Resurrect the file with the edit rather than deleting unseen work.
• Directory move vs add: Moving a folder while another device adds files under it — the adds follow the folder by file_id of the parent, not by path string.

Deep dive B: The journal and the huge shared folder (7–8 min)

"The journal is sharded by namespace_id. One namespace = one ordered log on one shard. That gives us strong ordering where it matters — inside a folder tree — without distributed transactions. The failure is the monster namespace: a company-wide shared folder with 40M files and 3,000 active writers. At ~1–5K commits/s per journal shard, one namespace can saturate its shard."

Options and position:

• Split the namespace into child namespaces at mount points (each team subfolder becomes its own namespace, mounted into the parent). Ordering is only guaranteed within a child — acceptable, because users don't expect cross-folder ordering.
• Batch commits — a client uploading 10,000 photos sends 100 commits of 100 files, not 10,000.
• Admission control per namespace — throttle writes on a hot namespace rather than letting it degrade the whole shard's neighbors.

"I'd enforce a soft limit on namespace size — say 5M entries — and have the product guide admins to split. The alternative is a single-namespace design that works for 99.9% of users and fails for our biggest paying customer."

Deep dive C: The client fleet and mass deletes (6–7 min)

"The riskiest code in this system runs on 100M machines we don't control. A client bug that misreads the filesystem — say, an unmounted external drive looks like 'everything deleted' — would faithfully sync a mass delete to every device and every collaborator."

Staff mechanisms:

• Server-side delete brake: if a single client commits deletes of > 1,000 files or > 20% of a namespace in 10 minutes, pause propagation and ask the user to confirm.
• Soft deletes with 30-day retention — every delete is a journal entry, and a namespace can be rolled back to a rev.
• Staged client rollout: internal → 1% → 10% → 100% over ~2 weeks, with server-side feature gates by client version and a server kill switch that forces old behavior.
• Minimum supported client version with a server-enforced floor — so the protocol can evolve.

Deep dive D (if time): Bandwidth and the 2 GB file (3–4 min)

"Insert one byte at the start of a 2 GB file: with fixed 4 MB blocks, every boundary shifts, all 500 blocks get new hashes, and we'd re-upload 2 GB. Two fixes. Content-defined chunking in storage would re-upload ~1–2 chunks. Or keep fixed blocks in storage and use rsync-style rolling-checksum delta on the wire: the client computes that almost all bytes already exist at shifted offsets and sends a patch; the server materializes new blocks. I'd do the second — storage stays simple, and the case is rare for typical documents, common only for VM images, databases, and PST files."

Phase 6: Wrap-Up (2–3 minutes)#

Synthesize, don't restate:

"The shape is: an immutable, content-addressed block plane that's a cost problem; a per-namespace ordered journal that's a correctness problem; and a notification plane that's a connection-count problem. The product guarantee — never silently lose an edit — comes from base-revision commits and conflicted copies, and the operational guarantee comes from soft deletes and a delete brake, because the client fleet is where the scary bugs live."

Close with evolution:

"What I'd build later: placeholders so the namespace no longer has to fit on disk; LAN sync so two laptops on the same network exchange blocks directly; real merge for document types by handing them to a collaborative editor; and, at exabyte scale, a build-vs-buy review on the block store — at that size the storage bill is the company's biggest line item."

🎯 Staff Move: End on who owns what. "One team should own the sync protocol end-to-end — client and server. Most of the scary incidents I'd expect are protocol bugs that fall between a client team and a server team, each of which is green on its own dashboards."

Common Timing Mistakes#

1. The Staff Lens#

1.1 Why This Problem Exists in Staff Interviews#

File sync is a compact test of whether a candidate can identify the center of gravity of a system. Nearly everyone can design blob upload. Few candidates realize that the upload is the least risky part, and that the hard problems are: (1) establishing a total order of changes per tree, (2) detecting — not resolving — concurrent edits, (3) keeping a fleet of offline, clock-skewed, version-skewed clients convergent, and (4) making destructive operations recoverable.

It also tests ownership across a client/server boundary. Most system design interviews stop at the API. This one can't: half the correctness lives in a desktop binary that the company ships and cannot instantly roll back. A Staff candidate owns both halves.

1.2 The L5 vs L6 vs L7 Contrast — Visual#

The L5 path is not wrong — every box in it is a real component. The gap is emphasis: the L5 path spends its effort on bytes; the L6 path spends it on ordering and failure; the L7 path spends it on the org and the money.

1.3 The Staff Question That Cuts Through Everything#

"When two devices disagree about a file, who decides — and does the loser find out?"

Every major design decision follows from the answer:

• If the server decides by clock, you've chosen LWW and silent loss.
• If the server detects via base revision and the client materializes a copy, you've chosen visible conflicts, which requires a per-namespace total order (the journal), which requires a sharding unit (the namespace), which requires a limit on namespace size.
• If a merge engine decides, you've left file sync and entered Collaborative Editing — a different system for a narrower set of file types.

2. Problem Framing & Intent#

2.1 The Three Intents — Explained#

Personal multi-device sync. One human, 2–5 devices. Concurrency comes from offline devices, not from simultaneous humans. Conflicts are rare (a laptop and a phone both edited while offline). The hard problems are bandwidth (mobile uplinks, metered data), battery (don't hash a 10 GB folder on a phone), and convergence latency. Correctness bar: never lose an edit.

Shared team folders. Many humans, many devices, one tree. Concurrency is real: two people open the same spreadsheet. Add permissions (view/edit, inherited ACLs), audit ("who deleted the contracts folder?"), admin controls (remote wipe, legal hold, retention policies), and very large namespaces. Correctness bar: never lose an edit, and every change is attributable. This is where most enterprise revenue — and most escalations — come from.

One-way backup. One writer, no concurrent edits by construction. Optimize for cost per GB and restore correctness: content-defined chunking for better dedup across versions, cold storage tiers, periodic scrubbing, and restore drills. Correctness bar: bit-exact restore, proven by testing restores, not by trusting writes.

🎯 Staff Move: "If this were backup, I'd use content-defined chunking and global dedup within a customer — conflicts don't exist, and dedup ratio is the cost lever. For sync with shared folders, I'll keep fixed blocks and spend the complexity budget on the journal and conflicts instead."

2.2 When NOT to Use File Sync#

🎯 Staff Move: "I'd explicitly exclude live database files and VM images from sync, or sync them only when closed. Syncing a SQLite file mid-transaction is a corruption bug we'd own, and no amount of chunking cleverness fixes it."

2.3 What the Interviewer Leaves Underspecified#

2.4 Precise Terminology#

3. The Five Fault Lines#

Each fault line below gets: the options, who pays, the Staff default, and when to deviate.

3.1 Fault Line 1: Block Size & Chunking Strategy#

Staff default: fixed 4 MB blocks in storage, SHA-256 addressed, plus rsync-style delta as a transfer optimization for files > 64 MB that were modified in place.

"Block size is a cost lever that looks like a performance knob. Halving it doubles the block index and the number of has_blocks lookups; it rarely doubles dedup, because most dedup comes from whole identical files, not from shared sub-regions."

When to deviate: backup products (CDC wins: many versions of the same large files, dedup ratio is the business); media workflows with 50 GB files (bigger blocks, e.g., 16 MB, to cut request count).

🧭 Principal Move: "Block size is a one-way door. Once there's an exabyte stored under 4 MB SHA-256 addressing, changing it means re-chunking and re-hashing everything — a multi-quarter migration with double storage cost during the cutover. I'd make this decision with a cost model, not a benchmark."

3.2 Fault Line 2: Dedup Scope#

The side channel is not theoretical: a 2011 open-source tool (publicly known as "Dropship") demonstrated that Dropbox's cross-user dedup let anyone with a file's hashes retrieve the file, and Dropbox changed the protocol in response.

Staff default: dedup within a tenant (user or business account). Cross-tenant dedup only server-side, after the bytes are uploaded — which kills the bandwidth win but keeps the storage win without the side channel. Or require proof of possession: the server asks for hashes of random byte ranges before accepting a "I already have this hash" claim.

When to deviate: backup of public software/OS images where content is public by nature — global dedup of known-public blocks is fine.

🎯 Staff Move: "Dedup scope is a security decision wearing a cost-optimization costume. I'd get security to sign off on anything wider than per-tenant."

3.3 Fault Line 3: Metadata Consistency Model#

Staff default: per-namespace journal in a sharded relational store (MySQL/PostgreSQL), sharded by namespace_id, with a derived current-state table updated in the same transaction. The commit is:

BEGIN;
SELECT rev FROM ns_head WHERE ns_id = :ns FOR UPDATE;          -- serialize per namespace
-- if file's current rev != :base_rev → ROLLBACK, return 409
INSERT INTO journal (ns_id, rev, file_id, path, blocklist, ...) VALUES (:ns, head+1, ...);
UPSERT INTO current_files (ns_id, file_id, path, rev, ...) ...;
UPDATE ns_head SET rev = head+1 WHERE ns_id = :ns;
COMMIT;

"The row lock on ns_head serializes commits per namespace — that's the whole point. It caps a single namespace at roughly 1–5K commits/s on one shard, which is fine for 99.99% of namespaces and a known limit for the rest."

When to deviate: at extreme scale, the current-state index can move to a separately scaled store (e.g., a KV store per namespace shard), keeping the journal authoritative.

🧭 Principal Move: "The namespace is the unit of everything: ordering, sharding, cells, ACLs, restore, and blast radius. I'd write that down as an invariant — no feature is allowed to require a transaction across namespaces — because the first feature that violates it turns sharding into distributed transactions."

3.4 Fault Line 4: Conflict Resolution#

Staff default: base-rev CAS + conflicted copies for all opaque files. Offer optional advisory locks for business customers with binary formats. Route truly collaborative document types to a real-time editor rather than trying to merge them in the sync engine.

Conflict edge cases the client must handle:

🎯 Staff Move: "The rule I'd give the client team: when in doubt, keep both. Duplicate data is a support annoyance. Lost data is a trust incident."

3.5 Fault Line 5: Client Intelligence & Fleet Control#

Staff default: thick client with server-side guardrails. The server treats every client as potentially buggy:

• Validates every commit (blocks exist, paths legal, sizes sane).
• Enforces a minimum client version and can feature-gate by version.
• Rate-limits destructive operations per client (delete brake).
• Can put a single device into quarantine — read-only — when its behavior is anomalous (e.g., > 5 conflicts/minute, re-uploading the same file in a loop).

🧭 Principal Move: "Desktop clients are the one deploy target where 'roll back' takes weeks, not minutes. I'd budget for that explicitly: rollout rings, a server-side kill switch for every new client behavior, and a telemetry pipeline that can detect a bad client version in the 1% ring within an hour."

4. Failure Modes & Operational Reality#

4.1 Mass Delete from a Buggy Client — Full Timeline#

The scariest incident in file sync: the system works perfectly and propagates a wrong instruction everywhere.

t=0:        Client v212.4 rolls to 10% of desktops. A regression treats a temporarily
            unmounted network drive as "all files deleted."
t=+6min:    18,000 devices commit mass deletes. Journal faithfully records them.
t=+6min:    Notification fan-out tells collaborators' devices. Their clients delete local copies.
t=+9min:    client.delete_burst_count (deletes > 1,000 per device per 10 min) crosses 50× baseline.
t=+10min:   Page fires to sync-core on-call. Runbook: enable global delete brake.
t=+12min:   Delete brake on: new bulk deletes held pending user confirmation.
t=+15min:   Client v212.4 halted at 10%; server gate disables the new mount-detection path by version.
t=+2h:      Namespace rollback job restores affected namespaces to pre-incident rev, per-file,
            preserving any *legitimate* edits made after the deletes (a 3-way merge server-side).
t=+6h:      All affected users restored. No data lost because deletes are soft for 30 days.

Detection: client.delete_burst_count by client version; journal.delete_ratio (deletes / total commits, baseline ~5–8%); support ticket spike keyed on "files missing."

Blast radius: every namespace touched by an affected device — and every collaborator of those namespaces. A single device in a 3,000-person shared folder can propagate deletes to 3,000 people.

Mitigation: delete brake, version gate, namespace rollback.

Prevention: delete brake enabled by default (threshold: > 1,000 files or > 20% of a namespace in 10 min); staged rollout with a delete-ratio canary check between rings; client-side sanity check "is the sync root mounted?" before any bulk delete.

Owner: sync-core on-call pages first; client team owns the root cause; support owns user comms.

4.2 Re-Sync Storm After a Cursor or Journal Bug#

t=0:        A migration of journal shard 41 renumbers revs (bug: rev not preserved).
t=+1min:    3.2M clients with cursors on shard-41 namespaces call list_since(cursor).
            Server says cursor invalid → clients fall back to FULL re-list.
t=+2min:    Full re-list of 3.2M namespaces × average 30K entries = ~10¹¹ rows read.
t=+3min:    Metadata DB read replicas at 100% CPU. list_since p99 goes from 80ms to 30s.
t=+5min:    Clients time out and retry — without jitter — doubling load.
t=+8min:    Unrelated shards starve because the API tier's connection pools are full.

Detection: sync.cursor_reset_total (baseline near 0), api.list_since.full_relist_ratio, metadata replica CPU.

Mitigation: server returns 503 Retry-After: <jittered 5–30 min> for full re-lists over a budget; admission control of full re-lists per shard (e.g., max 500 concurrent).

Prevention: revs are never renumbered — treat them as part of the public protocol. Migration dry-runs compare cursor validity on a shadow copy. Clients use exponential backoff with full jitter.

Owner: sync-core (journal), with the client team owning retry behavior.

4.3 Hot Namespace#

A company-wide shared folder with 40M entries and 3,000 writers — plus an automated script that writes 200 files/s — saturates its journal shard.

Detection: journal.commit_latency_p99{shard}, journal.lock_wait_ms{ns} — top-N namespaces by commit rate.

Blast radius: every other namespace co-located on that shard (~thousands) sees commit latency climb from 20ms to seconds.

Mitigation: per-namespace commit rate limit (e.g., 500 commits/s with batching encouraged); move the hot namespace to a dedicated shard (namespace migration is a supported operation — copy journal, flip routing, drain).

Prevention: soft namespace size limits with admin warnings; sub-namespace mount points; API batching for automated writers.

Owner: sync-core owns shard placement; the customer's admin owns the folder structure — a success/support engineer carries that conversation.

4.4 Silent Block Corruption#

A disk returns a bit-flipped block; a client downloads it, verifies SHA-256, detects mismatch — good. But an older client version doesn't verify hashes on download, writes corrupted bytes to disk, and then re-uploads a "modified" file.

Detection: block.download_hash_mismatch_total; background scrubber block.scrub_mismatch_total.

Mitigation: the block store fetches from another replica / reconstructs from erasure coding; quarantine the bad copy.

Prevention: every client verifies hashes on download (version floor enforces it); background scrubbing of all blocks every ~2–4 weeks; end-to-end checksums from client to disk.

Owner: storage team (scrub, repair), client team (verification).

4.5 Notification Plane Partial Outage#

One notification cluster (~200 hosts) loses its connection to the change bus. Clients stay connected but receive no hints.

Detection: this is the silent one — no errors anywhere. Use sync.convergence_lag_p99 measured by synthetic canaries: robot accounts on each cluster that commit and time arrival on a second device. Alert if p99 > 60s for 5 min.

Mitigation: clients also run a slow safety-net poll (every 5–10 min) on the cursor; notification hosts health-check their bus subscription, not just their listen socket.

Owner: realtime/notifications team.

4.6 Operational Reality Matrix#

🎯 Staff Move: "Dedup multiplies blast radius. One corrupted or wrongly-GC'd block can be referenced by a million files across a tenant. That's why the block store needs scrubbing, a GC grace period, and a quarantine rather than immediate deletes."

5. Evaluation Rubric#

5.1 Level-Based Signals#

5.2 Strong Hire Signals#

5.3 Lean No-Hire Signals#

5.4 Common False Positives#

• Deep knowledge of Rabin fingerprinting ≠ file sync design. CDC math is impressive but rarely the deciding factor for sync.
• Drawing a CDN for downloads ≠ scale thinking. Blocks are cacheable, sure — the hard scale is metadata and connections.
• "We'll use CRDTs" ≠ solving conflicts. CRDTs don't exist for arbitrary binary files; saying this for a PSD file is a red flag.
• Mentioning Magic Pocket ≠ build-vs-buy judgment. The question is when building your own storage pays off, not that Dropbox did it.

6. Interview Flow & Pivots#

6.1 Typical 45-Minute Shape#

6.2 How Interviewers Pivot — And What They're Testing#

6.3 What to Deliberately Skip#

• Presigned URL mechanics and S3 multipart details — acknowledge, don't design.
• Thumbnail/preview generation — a downstream consumer of the change bus; one sentence.
• Search across file contents — a separate indexing pipeline (Search Indexing).
• Billing and quota — mention the per-user quota check at commit time, move on.

6.4 Follow-Up Questions to Expect#

1. "How does a client know which blocks it must download?" — diff the new blocklist against its local block cache (keyed by hash).
2. "What happens if the client crashes mid-upload?" — blocks are idempotent; the commit hasn't happened; retry resumes by asking missing again.
3. "How do you garbage-collect blocks?" — refcounts from live and retained revisions; mark-and-sweep with a grace period ≥ max in-flight upload time (e.g., 7 days).
4. "How do you handle moves across namespaces?" — copy blocklist refs into target (no byte copy), commit in target, then delete in source — two commits, idempotent by client_op_id.
5. "What about very large directories on the client?" — the local DB, not the filesystem, is the source of truth for sync state; watchers overflow, so periodic reconciliation scans are required.
6. "How do you migrate a namespace between shards?" — copy journal up to rev N, dual-write/lock tail, flip routing, clients keep cursors because revs are preserved.
7. "How do you test this?" — deterministic simulation of client + server with randomized network, clock, and filesystem faults — the approach Dropbox publicly described for its Nucleus sync engine.

7. Active Drills#

Drill 1: The Opening#

Prompt: "Design Dropbox."

Drill 2: The Commit Protocol#

Prompt: "Walk me through exactly what happens when I save a 20 MB file."

Drill 3: Make the Journal Concrete#

Prompt: "You said 'journal.' What's the schema, and how does a device that was offline for 3 weeks catch up?"

Drill 4: The Conflict#

Prompt: "Two laptops, both offline, both edit plan.docx. They reconnect 10 minutes apart. What happens?"

Drill 5: The Dependency Goes Down#

Prompt: "The metadata database for 1/16 of namespaces is down for 20 minutes. What do users see?"

Drill 6: The Hot Namespace#

Prompt: "Our largest customer has one shared folder with 40M files and 3,000 editors. It's slow for them and for others on the same shard."

Drill 7: Dedup and Security#

Prompt: "We could save 30% storage with global dedup. Why not?"

Drill 8: Build vs Buy the Block Store#

Prompt: "We're on S3. Finance says storage is our largest cost. Should we build our own?"

Drill 9: Policy Change Without Outage — New Client Behavior#

Prompt: "We want clients to start using content-defined chunking for large files. How do you roll it out?"

Drill 10: Multi-Region#

Prompt: "EU customers require data residency. How does the design change?"

8. Deep Dive Scenarios#

Deep Dive 1: Peak Traffic — Monday Morning Reconnect Storm#

Context: Every Monday at 08:00–09:30 local time, 20M laptops wake up. This Monday, after a weekend in which a large customer bulk-imported 2B files, list_since p99 hits 25s and the notification tier drops connections. The on-call escalates to you.

Questions to Surface First:

• Is the load from catch-up volume (big cursors gaps) or reconnect volume (connection churn)?
• Are clients retrying with jitter, or synchronized?
• Which shards hold the bulk-imported namespaces?
• Is the degradation global or cell-local?

Typical L5 Approach: Scale out the API tier and add read replicas. Maybe increase the notification cluster size. Treats it as a capacity problem.

Staff Approach: Separates the two loads. Admission-controls expensive catch-ups per shard, returns jittered Retry-After to spread reconnection over 10–15 minutes, and serves compacted catch-up (latest state per file) instead of full journal replay for large gaps. Moves the bulk-imported namespaces' shard off shared pools.

Principal Approach: Asks why a single customer's weekend import could affect Monday for everyone — a cell-isolation failure. Funds cell boundaries that include API pools, not just DB shards, and adds a "bulk import" product path that writes through a throttled pipeline with its own capacity, so large customer operations can't consume the interactive tier's budget.

Key Takeaway: "Catch-up cost is proportional to the gap, and gaps align on Monday morning. Admission-control the expensive path and compact what you replay."

What clears the Staff bar:

• Distinguishes connection load from catch-up load
• Uses compaction and admission control, not only more hardware
• Traces the cross-customer impact to a missing isolation boundary

Deep Dive 2: Silent Failure — Files That Never Arrive#

Context: Support reports a trickle of "my file never showed up on my other computer." All dashboards are green. Error rates are normal. It has been happening for ~2 weeks.

Questions to Surface First:

• Is the file in the journal? (server-side truth)
• Is the receiving device's cursor past that rev? (did it think it applied it?)
• Which client versions and platforms are affected?
• Is there a common pattern in paths (unicode, case, length)?

Typical L5 Approach: Asks users to reinstall/re-link. Looks at server error logs. Finds nothing, because nothing errored.

Staff Approach: Treats it as a convergence-correctness bug. Builds a consistency checker: for a sample of devices, compare the device's reported tree hash for a namespace (reported via telemetry) against the server's current state at the same rev. Finds a mismatch pattern: files with Unicode names in NFD form on macOS are "applied" but written to a normalized path, so the local DB thinks they're present while the scanner considers them different files and silently ignores them.

Principal Approach: Makes convergence verification a permanent, org-level SLO: every client periodically reports a Merkle-style tree hash per namespace; the server compares. "Divergent namespaces per million" becomes a tracked metric with an owner and a quarterly target. The org stops relying on users to detect correctness bugs.

Key Takeaway: "In sync, the worst bugs don't error. You need a metric that compares what the client has with what the server says it should have."

What clears the Staff bar:

• Builds a verification signal rather than chasing logs
• Finds platform-specific path identity issues (Unicode normalization, case sensitivity)
• Pushes toward a permanent convergence SLO

Deep Dive 3: Large Customer Onboarding#

Context: A 60,000-employee company is migrating from on-prem file servers: 1.2 PB, 900M files, deeply nested permission structures, cutover planned over one weekend.

Questions to Surface First:

• What's the largest single folder tree, in files and in writers?
• What permission model do they have (per-folder ACLs, inherited, deny rules)?
• How many desktops will link on Monday, and with what default sync settings?
• What are their compliance needs: legal hold, retention, residency?

Typical L5 Approach: Provision more storage and more DB shards. Run the upload over the weekend.

Staff Approach: Re-plans the migration as a data-model exercise. Maps their shares to namespaces with a cap of ~5M entries each; imports via a server-side bulk path (no desktop clients involved); stages the desktop rollout over two weeks by department; defaults every desktop to placeholders so Monday doesn't pull 1.2 PB × 60,000 devices; pre-provisions dedicated shards for the three largest namespaces.

Principal Approach: Turns it into a repeatable enterprise onboarding program: a namespace-shaping tool, a capacity reservation process, a migration runbook owned by professional services, and a commercial guardrail — contracts above 500 TB include a staged migration schedule. The first bespoke migration becomes the template for the next hundred.

Key Takeaway: "A large customer is a data-model problem before it's a capacity problem. Shape the namespaces, then size the shards."

What clears the Staff bar:

• Uses placeholders to decouple onboarding from bandwidth
• Imports server-side rather than through a million client uploads
• Plans verification and rollback, not just ingestion

Deep Dive 4: Post-Mortem — Garbage Collection Deleted Live Blocks#

Context: 0.002% of files return errors on download: their blocks are gone. The block GC job, rewritten last month for performance, is the suspect.

Questions to Surface First:

• Were the blocks referenced by a live or retained revision at GC time?
• Was there an in-flight upload that referenced the block before commit?
• Do we have the blocks in a GC quarantine?

Typical L5 Approach: Roll back the GC job; restore from backups if they exist.

Staff Approach: Identifies the race: a client uploaded a block (refcount 0, not yet committed), the new GC ran with a 1-hour grace window instead of 7 days, and deleted it before a slow client committed 3 hours later. The commit validation checked block existence at commit time — but a different file already referencing the same hash (via dedup) had been committed, which bypassed the check because the index lookup hit a stale cache. Fix: restore from quarantine; GC grace ≥ max upload session lifetime (7 days); GC reads index from primary; deletes go to a 14-day quarantine before physical deletion.

Principal Approach: Classifies GC as a destructive-change process with the same controls as schema migrations: design review, shadow mode (GC computes deletions but logs only, for 2 weeks, compared against an independent mark phase), and staged enablement per cell. Any system that permanently deletes customer data needs two independent implementations to agree.

Key Takeaway: "Content addressing makes blocks immutable — not immortal. GC is the one place the block plane can lose data, so it gets the most paranoia."

What clears the Staff bar:

• Finds the upload-before-commit reachability gap
• Separates logical deletion from physical deletion with quarantine
• Brings cross-team sign-off to a destructive operation

Deep Dive 5: Multi-Region Expansion#

Context: The product is single-region (US). Leadership wants an EU region within 12 months for residency and latency.

Questions to Surface First:

• Residency for all users or for business tenants that request it?
• Can a namespace span regions (US owner, EU collaborators)?
• What is global today (identity, billing, sharing links) and must stay global?

Typical L5 Approach: Replicate the whole stack to EU; use multi-master DB replication.

Staff Approach: Home-region per namespace. EU namespaces keep journal and blocks in the EU; cross-region collaborators access via their home region's API, which proxies to the namespace's region. Global control plane: identity, namespace → region routing directory (small, strongly consistent, heavily cached). No multi-master journal: it breaks the per-namespace total order. Migration tool moves namespaces between regions with preserved revs.

Principal Approach: Defines region as a cell with a legal boundary and publishes a residency standard: which data classes can leave the region (none of content; limited metadata like billing), how cross-region shares are logged, and how namespace moves are approved. Sequences the build: EU region with new tenants only in months 1–6, migration of existing tenants in 6–12, with a revenue-weighted migration order.

Key Takeaway: "Multi-region for file sync means home-region namespaces, not multi-master journals. Ordering lives in exactly one place."

What clears the Staff bar:

• Keeps the per-namespace ordering invariant
• Minimizes the global control plane
• Plans migration with preserved cursors

9. Level Expectations Summary#

After studying this case study, you should be able to:

• Split file sync into a block plane, a metadata journal and a notification plane, and say which scaling law governs each
• Define a commit protocol that is idempotent (client_op_id), conflict-detecting (base_rev) and resumable (blocks before metadata)
• Explain why last-writer-wins is a data-loss policy, and defend conflicted copies to a product manager
• Quantify commits/s, block index size, notification connections and polling cost from a DAU assumption
• Explain the three-tree client model and the rename/delete/case-folding edge cases
• Name the mass-delete, cursor-storm, hot-namespace, GC and silent-divergence failure modes, each with a metric and an owner
• Argue dedup scope as a security decision and block size as a one-way door
• Describe how you'd roll out a client behavior change to 100M devices you can't roll back

The Bar for This Question#

Mid-level (L4): Designs upload/download with blocks and a metadata DB. Mentions dedup by hash. Handles the happy path correctly. Conflicts, deletes and client rollout come up only when prompted.

Senior (L5): Adds delta sync, dedup, a notification mechanism and replication. The design works. Conflicts are handled by timestamp. The client is treated as a black box. Numbers appear for storage but not for commits or connections.

Staff+ (L6): Frames intents, names "never silently lose an edit" as the governing requirement, and builds the per-namespace journal, base-rev CAS and conflicted copies from it. Treats the client fleet as the riskiest deploy target and designs server-side brakes. Quantifies the metadata plane. Assigns owners to every failure mode. The interviewer should learn something from the answer, whether that's the dedup side channel, NFD/NFC path divergence, or why a GC grace window must exceed the upload session lifetime.

10. Staff Insiders: Controversial Opinions#

10.1 "Delta Sync Is Overrated for Most Users"#

The Staff position: ship fixed blocks + dedup first. Add wire-level delta only for files > 64 MB with measured in-place edit patterns. The engineering budget belongs in reconciliation correctness.

Why this matters in interviews: candidates who spend 10 minutes on rolling checksums are optimizing the 1% case while the 99% case (conflicts, deletes) goes undesigned.

10.2 "Conflicted Copies Are a Feature, Not a Failure"#

The Staff position: product should advertise "we never lose your edits." Conflicted copies are the implementation of that promise.

Why this matters in interviews: defending an "ugly" UX on correctness grounds, and naming who signs off, is a clean Staff-level signal.

10.3 "The Client Is the Most Dangerous Component You Own"#

The Staff position: budget for deterministic simulation testing, rollout rings, server-side kill switches, and a delete brake — before adding features.

Why this matters in interviews: most candidates never mention the client. Mentioning its rollout discipline marks you as someone who has operated one.

10.4 "Global Dedup Is a Security Bug With a Cost Justification"#

The Staff position: per-tenant dedup by default. Cross-tenant only server-side, or with proof of possession.

Why this matters in interviews: it shows you reason about a cost optimization's security surface, not only its savings.

10.5 "Building Your Own Storage Is Almost Always Wrong — Until It's Obviously Right"#

The Staff position: write the break-even model down and revisit it yearly. Don't argue it from ideology.

Why this matters in interviews: build-vs-buy questions reward thresholds, not opinions.

11. The Principal Lens (L7)#

Why L7 Sees This Problem Differently#

A Staff engineer designs a correct sync system. A Principal engineer sees that file sync is really three businesses sharing one protocol: a storage-cost business (the block plane, whose unit economics decide gross margin), a correctness business (the journal and client, whose failures decide churn and trust), and an enterprise business (permissions, residency, legal hold, whose features decide revenue per seat). Each has a different owner, a different time horizon and a different failure posture. The L7 job is to put the invariants that must never break (per-namespace ordering, no silent loss, soft deletes) into standards, and to let everything else evolve independently.

The Org-Level Fault Line#

One sync protocol owned end-to-end vs. separate client and server teams. Conway's law pushes toward a desktop team, a mobile team, a sync-API team and a storage team, each with its own roadmap. Protocol bugs such as path normalization, cursor semantics or rename handling fall into the seams, and every team's dashboard stays green while users lose files. The alternative, one "sync core" team with authority over the protocol spec, the server journal and the client reconciliation engine, concentrates risk and makes that team a bottleneck.

The Principal position: one team owns the protocol spec and the reconciliation engine (shared code across desktop and mobile, as a library), plus the journal. Platform-specific shell teams own UI, OS integration and the file watchers. Storage stays separate behind a narrow, immutable-block API.

🧭 Principal Move: "I'd draw the team boundary where the invariants are. Everything that can lose a file (reconciliation, the journal, GC reachability) sits under one owner. Everything that can only make sync slower or uglier can be distributed."

Cost Model#

Assumptions: 4 MB blocks; 60% of stored bytes cold after 30 days; object storage at ~$21/TB-month hot, ~$10/TB-month infrequent-access list price (owned hardware ~3–5× cheaper at scale); fully loaded engineer ~$300K/year; metadata on sharded SQL.

Reading it: at growth scale, headcount costs more than infrastructure. At hyperscale, the rent-vs-own delta (~$2–3M/month) pays for a storage team several times over. That is the trigger for building your own storage.

The 3-Year Evolution Path#

One-Way Doors vs Two-Way Doors#

The Standard I'd Write#

RFC: Sync Correctness Standard v1

Scope: All clients and services that read or write the namespace journal or block store.

MUST

1. Every mutating commit carries base_rev and client_op_id. The server rejects commits missing either.
2. No operation may silently overwrite a revision the client has not observed. Concurrent edits produce conflicted copies.
3. Deletes are soft for ≥ 30 days (consumer) / ≥ 180 days (business). Physical block deletion requires a ≥ 7-day GC grace period and a ≥ 14-day quarantine.
4. No feature may require a transaction across namespaces.
5. Every new client behavior ships behind a server-controlled flag, through ≥ 3 rollout rings, with a delete-ratio and conflict-rate canary gate between rings.
6. Clients verify SHA-256 on every block download.

SHOULD

• Clients report per-namespace tree hashes for convergence verification at least daily.
• Bulk writers (> 100 commits/s) use batched commits.

Exceptions: Filed with sync-core. Approved by the sync-core lead plus one security reviewer. Exceptions expire after 2 quarters.

Success metrics: zero confirmed silent data-loss incidents per quarter; conflict rate < 0.1% of commits; sync.divergent_namespaces_per_million < 10; p99 convergence < 10s.

What I'd Tell the VP#

"Our product's promise is that files are never lost and always show up everywhere. Storage is the biggest cost but the smallest risk. The real risk is the sync software running on our customers' computers, because a bad release there can delete files for millions of people and takes weeks to undo. I'm asking for three things. First, a single team accountable for sync correctness end-to-end. Second, safety brakes that stop mass deletions automatically. Third, a yearly build-vs-rent review of storage, since at our growth rate owning it could save ~$25–35M a year by year 3. In return we commit to a measurable standard: zero silent data-loss incidents, conflicts under 0.1%, and changes visible on every device within 10 seconds."

Principal Interview Signals#

Staff answers that L7 interviewers find insufficient:

• "We'll add a delete brake." Correct, but it doesn't say who can override it, how the threshold is governed, or how it's tested org-wide.
• "We'd consider building our own storage later." That gives no trigger, no cost model and no migration plan.
• "The client team and server team will coordinate on the protocol." Coordination isn't ownership. It's how protocol bugs fall between teams.

Appendices

Appendix A: Chunking Mechanics in Depth#

A.1 Fixed-Size Blocks#

def chunk_fixed(file, size=4*MB):
    blocks = []
    while (data := file.read(size)):
        h = sha256(data)
        blocks.append(h)
        local_block_cache.put(h, data_location)   # for later dedup/download
    return blocks                                   # the blocklist

Why it's right for sync: O(1) boundary logic, perfectly parallel hashing, and an easy capacity model (bytes / 4 MB = blocks). Why it's wrong for backup: an insert shifts all later boundaries.

A.2 Content-Defined Chunking#

def chunk_cdc(stream, min=1*MB, avg=4*MB, max=8*MB):
    mask = avg - 1                     # boundary when rolling hash & mask == 0
    start, h = 0, 0
    for i, byte in enumerate(stream):
        h = gear_roll(h, byte)
        length = i - start + 1
        if (length >= min and (h & mask) == 0) or length >= max:
            emit(start, i); start = i + 1; h = 0

Boundaries depend on content, so an insert disturbs only the chunk it lands in and possibly its neighbor. It costs ~2–3× the CPU of fixed chunking, and the size distribution needs min/max clamps.

A.3 rsync-Style Wire Delta#

# Receiver (server) has old version's block signatures:
sigs = [(adler32(b), sha256(b)) for b in old_blocks]
# Sender (client) slides a window over the new file:
for each offset: if adler32(window) in weak_set and sha256(window) matches:
        emit COPY(block_index); advance by block
    else: emit LITERAL(byte); advance by 1   # rolling update is O(1)

Use it only as a transfer optimization. The server reconstructs full 4 MB blocks and stores them content-addressed as usual.

Appendix B: Data Model#

B.1 Tables (per journal shard)#

B.2 Block Index (separate KV store)#

At 1 EB that's ~250B entries at ~64 B each, or ~16 TB, sharded by hash prefix across a KV store (see DynamoDB or Cassandra-style stores). Hash prefixes are uniformly distributed, so there are no hot keys except for extremely popular content.

B.3 Path Identity Rules#

• Paths are keyed by file_id. The path is an attribute, never an identity.
• Normalize Unicode to NFC on the server. Keep the client's original bytes for display.
• Track case-insensitive collisions per namespace and surface them as conflicts on case-insensitive platforms.
• Reject paths that are invalid on any supported OS (e.g., CON, trailing dots on Windows), or map them reversibly.

Appendix C: Client Reconciliation — The Three-Tree Model#

Rules that prevent the classic bugs:

• Apply remote changes atomically: write to a temp file in the same directory, fsync, then rename, so apps never see torn files.
• Advance the cursor only after the local DB commits the applied state. A crash between them causes a re-apply (idempotent), never a skip.
• Don't trust the watcher alone. inotify/FSEvents queues overflow on large trees, so run a periodic full scan (e.g., every 6–24h) that compares disk to the local DB.
• Ignore echoes. Changes the client itself wrote into place must not be re-detected as local edits (compare against the expected hash).

Appendix D: API Contract & Client Behavior#

Retries: exponential backoff with full jitter, base 1s, cap 5 min. Long-poll: 60–90s server hold with ±20% jitter so reconnections don't synchronize.

Appendix E: Observability#

E.1 Core Metrics#

# Correctness (the ones that matter)
sync.commit_conflict_rate                 # conflicts / commits, by client_version, os
sync.divergent_namespaces_per_million     # tree-hash mismatch from client reports
sync.convergence_lag_p99                  # synthetic canary: commit on A → visible on B
client.delete_burst_count                 # devices exceeding delete threshold, by version
# Throughput & health
journal.commit_latency_p99{shard}, journal.lock_wait_ms{ns} (top-N)
api.list_since.full_relist_ratio, sync.cursor_reset_total
block.missing_ratio                       # fraction of hashes needing upload (dedup effectiveness)
block.scrub_mismatch_total, block.get_404_total{referenced=true}
notify.connections_open, notify.reconnect_rate

E.2 Critical Alerts#

E.3 Debugging "My File Is Missing"#

1. Is it in the journal? If not, the upload/commit never happened, so check client logs and quota.
2. Is it deleted in the journal? Check author and device. If it came from a burst, look at the delete brake logs.
3. Is the device's cursor past the rev? If not, it's a notification or catch-up problem.
4. Cursor past but file absent locally? That's a reconciliation bug (path normalization, ignore rules, watcher overflow).

Appendix F: Scale Evolution#

F.1 What Works at Each Scale#

F.2 What You Don't Build on Day One#

• LAN sync, global dedup, CDC chunking, wire delta, owned storage, multi-region.
• Do build on day one: ns_id in every key, base_rev + client_op_id, soft deletes, the delete brake, rollout rings. These are expensive to retrofit.

Appendix G: Multi-Tenancy, Fairness & Cost#

• Per-tenant quotas checked at commit time (bytes counted after per-tenant dedup, so users aren't charged for duplicates).
• Per-namespace commit rate limits protect shard neighbors (see Rate Limiting).
• Dedicated shards for the largest tenants as a paid enterprise feature.
• Cost attribution: bytes stored per tenant, commits per tenant and connected devices per tenant together explain ~90% of cost. Report them monthly to finance so pricing tracks cost drivers.