SYSTEM DESIGN #01 · INTERVIEW GUIDE
Multi-Touch Attribution System
Every marketer asks the same impossible question: which ad actually drove the sale? When a customer sees a Facebook ad on Monday, clicks a Google ad on Wednesday, and converts after an email on Thursday, traditional last-touch attribution hands 100% of the credit to the email — and your entire Facebook budget gets cut the next day. This system solves that by capturing every touchpoint across web, mobile, and server-to-server, stitching identities across devices without third-party cookies, and running six attribution models in parallel so you can see the complete conversion story. By the end of this guide you will know how to ingest 500K events/sec, resolve cross-device identity with Union-Find graphs, and serve channel-level ROAS reports in under 3 seconds — the exact architecture used at companies like Google, Meta, and the Trade Desk.
KafkaApache FlinkClickHouseUnion-FindCloudflare Workers
💡
The Gist — What Problem Are We Solving?
Figuring out which ad actually made you buy something
When you see a Facebook ad on Monday, click a Google ad on Wednesday, and buy on Thursday after an email — who gets the credit? This system watches every ad touchpoint before a purchase, then fairly splits credit between all of them using different mathematical models so marketers know where to spend next month’s budget.
💬Think of it as the referee that decides how much credit Facebook, Google, and email each deserve for a sale.
These are the capabilities the system must deliver — what users and operators can actually do with it.
✅
Event Collection
✅Track impressions, clicks, conversions across web, mobile, server-to-server
🔗
Identity Stitching
🔗Link sessions across devices via cookies, hashed email, IDFA/GAID
🧮
Attribution Engine
🧮Run 6 models: First-touch, Last-touch, Linear, Time-Decay, Position-Based, Data-Driven
📊
Reporting API
📊Serve breakdowns by channel, campaign, model with <3s latency
🔐
Privacy Controls
🔐GDPR/CCPA consent gate; erasure within 30 days; no raw PII in store
⚡
Non-Functional Requirements
These define how well the system must perform — the quality attributes that separate a toy from a production system.
⚡ Throughput
⚡500K events/sec peak; 5B+ events/day
⏱️ Ingestion Latency
⏱️<200ms p99 from event to Kafka
🔍 Query Latency
🔍<3s p95 for standard reports over 90 days
🛡️ Durability
🛡️Zero event loss; RF=3 Kafka replication
📅 Retention
📅2 years raw; 5 years aggregated
📊
Key Metrics — The Numbers That Define This System
The headline numbers to know cold — and be ready to explain how each one is achieved.
🏗️
System Architecture Diagram
Full data flow from source to serving. Each layer scales independently.
Ingestion
→
Edge Collector
Cloudflare Workers
→
→
↓
Processing
→
→
ClickHouse OLAP + S3 Iceberg
→
🗺️
End-to-End User Journey
Trace a single request end-to-end — the story interviewers want you to tell fluently.
1
User sees ad
— Impression event fired from SDK; edge collector responds in <50ms; Bloom filter dedup check
2
User clicks ad
— Click event with gclid/fbclid captured; server-side cookie set with 365-day expiry
3
User converts
— Conversion event sent with canonical user_id; identity stitcher resolves cross-device
4
Attribution runs
— Flink processes touchpoint window; all 6 models compute credit weights in real time
🔭
High-Level Design — Component Breakdown
Core components — each with a single, well-defined responsibility. The key architectural insight: each layer scales independently, and failure in one component is isolated from the rest.
1 — Client SDKs
Collects raw events from browser, iOS, and Android surfaces. Batches events every 500ms or 50 events and sends via sendBeacon() for unload-safe delivery. Generates client-side ULIDs as event IDs for deduplication.
2 — Edge Collector
Cloudflare Worker running at 250+ PoPs worldwide. Validates schema, stamps server_received_at, applies IP-based pre-bid IVT filter, and proxies to Kafka REST proxy in <5ms CPU time.
3 — Kafka
Distributed event bus with RF=3 for durability. Partitioned by user_id_hash for per-user ordering. LZ4 compression reduces storage cost by 60%. Exactly-once semantics via idempotent producers and transactional consumers.
4 — Flink
Stateful stream processor with 30-second checkpointing to S3. KeyedProcessFunction provides per-key state (Union-Find, EWMA baseline, experiment assignment). Exactly-once processing via two-phase commit sink.
5 — Attribution
Handles responsibilities for the Attribution layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
6 — ClickHouse
Column-oriented OLAP database. MergeTree engine with per-day partitioning and (entity_id, date) sort key. Pre-aggregated Materialised Views reduce query time for common dashboard patterns from seconds to milliseconds.
🔬
Low-Level Design — Deep Dives
Deep dives worth explaining in detail in any senior engineering interview. For each: know the data structure, the algorithm, the why, and the trade-off you made.
1 — Event Ingestion SDK
Web · iOS · Android · S2S
Browser SDK uses navigator.sendBeacon() for unload-safe fire-and-forget delivery. Payload is a JSON array batched every 500ms or 50 events, whichever comes first. S2S endpoint accepts the same schema, authenticated via HMAC-SHA256 request signature. Edge collector on Cloudflare Workers validates schema, stamps server_received_at, and proxies to Kafka.
POST /v1/events
{“event_id”:”01HX…”,”type”:”click”,”ts”:1720000000,
“campaign_id”:”cmp_123″,”user_agent_hash”:”a3f…”}
2 — Identity Stitcher
Union-Find + Neptune
Path-compressed Union-Find stored in Redis as two hashes: parent[] and rank[]. find(x) follows parent pointers with path compression. union(x,y) merges by rank (smaller rank attaches to larger). Deterministic signals (hashed email) always win over probabilistic — deterministic node is made root. Neptune stores the full graph for audit and complex traversal queries.
def find(r, x):
if r.hget(‘parent’,x) != x:
r.hset(‘parent’,x, find(r, r.hget(‘parent’,x)))
return r.hget(‘parent’,x)
3 — Attribution Engine
6 Models · Flink Stateful
Flink KeyedProcessFunction keyed on canonical_id. State: list of touchpoints within a 30-day lookback window. On each conversion event, all 6 attribution models are evaluated against the window and credit splits emitted to ClickHouse. Shapley model uses pre-computed power-set weights stored in a broadcast state from nightly Spark training job.
class AttributionFn(KeyedProcessFunction):
def process(self, event, ctx):
window = self.state.touchpoints.value()
for model in MODELS:
emit(model.credit(window, event))
4 — OLAP Query Layer
ClickHouse · MergeTree
Attribution results stored in a ClickHouse ReplacingMergeTree table partitioned by toStartOfDay(conversion_ts) and ordered by (campaign_id, channel, date). Pre-aggregated materialised views for common query patterns (channel-level daily ROAS). Query API uses Redis 5-minute TTL cache keyed by (org_id, query_hash). Reports with >1M rows streamed via HTTP chunked transfer.
CREATE TABLE attribution (
conversion_id UUID,
channel LowCardinality(String),
model LowCardinality(String),
credit Float32
) ENGINE = ReplacingMergeTree
PARTITION BY toStartOfDay(ts)
ORDER BY (campaign_id, channel, date)
⚖️
Trade-offs & Decision Log
Every senior interview comes down to these decisions. Know the exact trade-off, the reasoning, and the specific numbers that justify each choice.
⚖️ Real-Time vs Batch Attribution
✓
Real-Time (Flink) ✅ Chosen
- Results available in seconds — enables same-session optimisation
- Catches multi-touch journeys that complete within minutes
- Higher infrastructure cost (Flink cluster, stateful processing)
- More complex exactly-once semantics to implement
→
Batch (Spark nightly)
- Simpler to build and operate — standard Spark jobs
- Full lookback window available for data-driven models
- Results only available next morning
- Misses real-time budget-pacing signals
💡Decision: Use Flink for <24h attribution; nightly Spark retrains the data-driven Shapley model on full 30-day history
⚖️ Deterministic vs Probabilistic Identity
✓
Deterministic (hashed email) ✅ Primary
- 100% accurate when login signal present
- Privacy-safe — HMAC-SHA256, never raw PII stored
- Works across all browsers and devices
- Requires user to be logged in / provide email at checkout
→
Probabilistic (device fingerprint)
- Works for anonymous sessions — no login required
- ~75% cross-device match accuracy
- Degraded by privacy regulations — GDPR restricts without consent
- Fingerprint entropy shrinks as browsers block signals
💡Decision: Deterministic first; probabilistic fallback only for anonymous sessions with explicit GDPR consent
🎯Interview Questions — Answered
The exact questions interviewers ask — with production-grade answers
Q1
Why 6 attribution models instead of just data-driven?
Different models answer different business questions. Last-touch tells you which channels close deals (optimise bottom-funnel). First-touch tells you which channels create awareness (optimise top-funnel). Data-driven (Shapley) gives the statistically fairest split but requires 30+ days of conversion history to train reliably. Small advertisers with <100 conversions/month cannot use data-driven — they fall back to rule-based models. Running all six simultaneously lets you compare and build trust in the data-driven model before committing budget decisions to it.
Q2
How do you handle attribution for cross-device journeys without third-party cookies?
Three-tier signal waterfall: (1) Deterministic — hashed email collected at checkout via Enhanced Conversions/CAPI, matched against the identity graph. (2) First-party cookie — server-set 365-day cookie via Set-Cookie header (not blocked by ITP/ETP). (3) Probabilistic — cosine similarity ≥0.85 on device fingerprint vector (IP + UA + timezone + screen resolution) for anonymous sessions with explicit GDPR consent. Signal tier 1 matches ~60% of conversions, tier 2 adds ~25%, tier 3 covers the remaining ~15% on consented traffic.
Q3
What happens to attribution when a conversion event arrives 25 hours late?
Events within the 24-hour Flink watermark window are processed inline. Events arriving after the watermark (25h+) are routed to a Kafka side-input topic. A nightly Spark job reads this late-event topic and runs a retroactive attribution correction pass: it re-opens the affected conversion windows, recalculates credit splits, and writes a correction record to ClickHouse. The correction overwrites the original attribution row using a ReplacingMergeTree engine (final=1 at query time merges duplicates). Total correction latency: <24h after the late event arrives.
System Design Series · Every Tuesday & Thursday
Level up your system design interviews
Each post covers Gist, Functional & Non-Functional Requirements, Key Metrics, System Diagram, User Journey, HLD, LLD, and Trade-offs & FAQs.
Subscribe to never miss a post →
Previous Articles
Categories: System Design
Tags: adtech, clickhouse, flink, identity graph, interview prep, kafka, multi-touch attribution, stream processing, system design
Leave a Reply