System Design #06: Scalable A/B Testing Platform

SYSTEM DESIGN #06 · INTERVIEW GUIDE

Scalable A/B Testing Platform

Most companies run A/B tests wrong: they peek at results early, ship winners that regress in production, and run experiments on overlapping user segments that invalidate each other’s results. A truly scalable experimentation platform eliminates all of these failure modes — it assigns users to variants in under 1ms using MurmurHash3, detects Sample Ratio Mismatch with a Chi-square test before any metric is trusted, and applies Sequential Probability Ratio Testing (SPRT) so you can stop experiments early without inflating false positive rates. This system is designed to run 10,000 simultaneous experiments across web, mobile, and server-side surfaces — the architecture used by Netflix, Airbnb, and LinkedIn. You will learn how to build the assignment service, the event pipeline, the stats engine, and the guardrail metrics that prevent a ‘winning’ feature from silently breaking revenue.

MurmurHash3SPRTKafkaClickHouseChi-Square

💡

The Gist — What Problem Are We Solving?

Running thousands of experiments simultaneously without them interfering

Want to test whether a red button converts better than a blue one — while running 9,999 other tests simultaneously? This system assigns each user to the right variant in under 5ms, ensures the same user always sees the same variant, and tells you with mathematical proof when a result is reliable enough to act on.

💬

Think of it as a giant science lab that runs thousands of controlled experiments at once and tells you which ideas actually work.

Functional Requirements

These are the capabilities the system must deliver — what users and operators can actually do with it.

🔬
Experiment Management

🔬Create, configure, start, pause, stop experiments; define variants and metrics


Assignment Service

⚡Deterministic variant assignment in <5ms; same user always gets same variant

📊
Metrics Collection

📊Track assignment and conversion events per variant per experiment

📈
Stats Engine

📈Running t-test and SPRT for early stopping; p-value, CIs, lift calculation

🚨
Auto-stop

🚨Pause automatically on statistical significance or guardrail metric violation

Non-Functional Requirements

These define how well the system must perform — the quality attributes that separate a toy from a production system.

⚡ Assignment Latency

⚡<5ms p99 — must not slow down ad serving

📈 Throughput

📈100K assignment decisions/sec

🔒 Consistency

🔒Same user always sees same variant across sessions

🔀 Mutual Exclusion

🔀Users in at most 1 experiment per namespace

⏱️ Config Propagation

⏱️<30s from experiment change to all servers

📊

Key Metrics — The Numbers That Define This System

The headline numbers to know cold — and be ready to explain how each one is achieved.

<5ms
assignment p99
10K
concurrent experiments
100K/sec
decisions
p<0.05
threshold
<30s
config propagation
🏗️

System Architecture Diagram

Full data flow from source to serving. Each layer scales independently.

Ingestion
Experiment Config
UI

Config Store
Redis pub/sub

Assignment Service
MurmurHash3

Event Stream
Kafka

Processing
Stats Engine
Flink SPRT

Results Store
Postgres

Dashboard + Auto-stop

🗺️

End-to-End User Journey

Trace a single request end-to-end — the story interviewers want you to tell fluently.

1
Marketer creates experiment

— Defines: variants, traffic split 50/50, primary metric CTR, guardrail metric page-load time

2
User hits ad serving

— hash(user_id + experiment_id) % 10000; user gets variant based on bucket range

3
Consistent assignment

— Same user returns: same hash → same bucket → same variant always

4
Events tracked

— Impression + click tagged with exp_id and variant_id; published to Kafka

5
Stats engine runs

— Flink computes running z-test every 5 min; SPRT checks early stopping every 15 min

🔭

High-Level Design — Component Breakdown

Core components — each with a single, well-defined responsibility. The key architectural insight: each layer scales independently, and failure in one component is isolated from the rest.

1
Experiment UI
2
Config Store
3
Assignment
4
Kafka
5
Stats Engine
6
Results DB
1 — Experiment UI

Handles responsibilities for the Experiment UI layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.

2 — Config Store

Handles responsibilities for the Config Store layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.

3 — Assignment

Handles responsibilities for the Assignment layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.

4 — Kafka

Distributed event bus with RF=3 for durability. Partitioned by user_id_hash for per-user ordering. LZ4 compression reduces storage cost by 60%. Exactly-once semantics via idempotent producers and transactional consumers.

5 — Stats Engine

Handles responsibilities for the Stats Engine layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.

6 — Results DB

Handles responsibilities for the Results DB layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.

🔬

Low-Level Design — Deep Dives

Deep dives worth explaining in detail in any senior engineering interview. For each: know the data structure, the algorithm, the why, and the trade-off you made.

1 — Assignment Service
MurmurHash3 · <1ms

Stateless assignment service. Given (user_id, experiment_id), computes variant assignment as: bucket = MurmurHash3(user_id + experiment_id) % 10000. Variant thresholds stored in Redis (e.g., control=0-4999, treatment=5000-9999 for 50/50 split). Redis read <1ms including network. Assignment logged to Kafka experiment_assignments topic. Sticky assignment: bucket is deterministic — same user always gets same variant for the same experiment.

import mmh3
def assign(user_id, experiment):
bucket = mmh3.hash(f'{user_id}{experiment.id}’) % 10000
for variant, threshold in experiment.thresholds:
if bucket < threshold: return variant

2 — Metric Event Pipeline
Kafka · Flink · ClickHouse

User actions (page views, clicks, conversions, revenue) consumed from Kafka. Flink job joins events with assignment log by user_id and experiment_id. Joined events written to ClickHouse experiment_metrics table partitioned by experiment_id. SPRT computation runs as a Flink aggregate: counts and sums per variant updated in real time. Dashboard polls SPRT state API every 60 seconds.

events
.connect(assignments)
.keyBy(e->e.user_id, a->a.user_id)
.process(new ExperimentJoiner())
.sinkTo(clickhouseSink(‘experiment_metrics’))

3 — SPRT Stats Engine
Sequential Probability Ratio Test

SPRT computes likelihood ratio Λ = L(H1)/L(H0) after each new observation. Stop condition: Λ ≥ B=(1-β)/α → reject H0 (significant effect). Λ ≤ A=β/(1-α) → accept H0 (no effect). Otherwise continue. Parameters: α=0.05 (Type I error), β=0.2 (Type II error). B=19, A=0.0526. For conversion rate experiments: Λ updated as Bernoulli likelihood ratio. For continuous metrics (revenue): normal SPRT with known variance from pilot data.

def sprt_update(lambda_prev, x, p0, p1):
lr = (p1**x * (1-p1)**(1-x)) / (p0**x * (1-p0)**(1-x))
lambda_new = lambda_prev * lr
if lambda_new >= B: return ‘REJECT_H0’
if lambda_new <= A: return 'ACCEPT_H0' return 'CONTINUE'

4 — SRM Detector
Chi-Square · Auto-Pause

Sample Ratio Mismatch detection runs every hour. Chi-square test: expected_control = total × split_control; observed_control = actual_assignments_control. χ² = Σ (observed-expected)²/expected. If χ² > critical value (p<0.001 threshold), experiment is auto-paused and an alert sent. Common SRM causes: caching serving stale variant, bot filtering applied to one variant only, SDK version differences. SRM is checked before any metric results are shared to prevent acting on contaminated data.

from scipy import stats
def detect_srm(observed, expected_ratio, alpha=0.001):
expected = [sum(observed) * r for r in expected_ratio]
chi2, p = stats.chisquare(observed, expected)
return p < alpha # True = SRM detected

⚖️

Trade-offs & Decision Log

Every senior interview comes down to these decisions. Know the exact trade-off, the reasoning, and the specific numbers that justify each choice.

⚖️ SPRT (Sequential) vs Fixed-Horizon Testing


SPRT — Sequential Testing ✅ Chosen
  • Can stop experiments early (2-3× faster average runtime)
  • Controls false positive rate at any stopping point
  • Handles optional stopping without inflating Type I error
  • More complex to implement and explain to stakeholders

Fixed-Horizon (classical)
  • Simpler — set sample size upfront, run to completion
  • Well-understood by data scientists
  • Peeking inflates false positive rate to 26%+ (vs 5% nominal)
  • Slow — must wait for full sample even when result is obvious

💡

Decision: SPRT with α=0.05, β=0.2 for most experiments; fixed-horizon for pre-registered regulatory/finance experiments

⚖️ Client-Side vs Server-Side Assignment


Server-Side Assignment ✅ Chosen
  • No flicker — user sees correct variant from first render
  • Works for API and backend experiments
  • Assignment persists across devices with same user_id
  • Adds 10-20ms to server response time

Client-Side (JS SDK)
  • Zero server-side latency impact
  • Flicker on first load — user briefly sees control before variant
  • Easy to implement with third-party tools
  • Assignment cannot be trusted for server-computed metrics

💡

Decision: Server-side for revenue/checkout/API experiments; client-side JS SDK only for pure UI cosmetic tests

🎯

Interview Questions — Answered
The exact questions interviewers ask — with production-grade answers

Q1
How do you prevent experiments from interfering with each other (interaction effects)?

Three-layer isolation: (1) Orthogonal assignment — users hashed with experiment_id salt (MurmurHash3(user_id + experiment_id)) ensures independent uniform distribution per experiment. (2) Mutex groups — experiments that share the same UI surface are placed in a mutual exclusion group; a user can only be in one experiment per mutex group. (3) CUPED variance reduction — uses pre-experiment metric values as covariates to reduce noise, allowing detection of smaller effects with the same sample size. Interaction effects between experiments in separate mutex groups are measured post-hoc using regression analysis on the metric event log.

Q2
What is Sample Ratio Mismatch (SRM) and why does it matter?

SRM occurs when the observed split between control and treatment differs from the intended split (e.g., 51% / 49% instead of 50% / 50%). This indicates a systematic bug in assignment, logging, or filtering that makes the groups non-comparable — any metric difference is confounded by the selection bias. Detection: Chi-square test with p-value threshold of 0.001 (very conservative to avoid false SRM flags). Common causes: bots filtered from one variant only, page caching serving stale variant assignments, or logging failures in one variant. Any experiment with SRM detected is immediately paused and flagged for investigation before results are shared.

Q3
How long does an experiment need to run before results are trustworthy?

Minimum runtime is determined by three constraints: (1) Statistical power: sample size must reach the pre-calculated minimum based on MDE (minimum detectable effect), baseline conversion rate, α, and β. (2) Novelty effect: minimum 1 full week to avoid users behaving differently just because the UI changed. (3) Business cycles: minimum 1 full week to capture Mon-Sun variation (e.g., weekend shoppers behave differently). SPRT can signal a winner before the minimum runtime if the effect size is large — but the experiment continues to the minimum runtime anyway to confirm novelty effect has dissipated.

System Design Series · Every Tuesday & Thursday

Level up your system design interviews

Each post covers Gist, Functional & Non-Functional Requirements, Key Metrics, System Diagram, User Journey, HLD, LLD, and Trade-offs & FAQs.

Subscribe to never miss a post →


Categories: System Design

Tags: , , , , , ,

Leave a Reply

Discover more from Cloud Wizard Inc.

Subscribe now to keep reading and get access to the full archive.

Continue reading