SYSTEM DESIGN #04 · INTERVIEW GUIDE
Measurement Readiness Dashboard
A single misconfigured tracking tag can silently corrupt weeks of marketing data — and by the time someone notices, the campaign budget has already been misallocated. The Measurement Readiness Dashboard is a continuous auditing system that crawls every page across every market, validates every tag against its expected schema, computes a 0–100 readiness score, and fires alerts to Slack and PagerDuty the moment something drifts. It handles the full complexity of real-world deployments: consent-gated tags, geo-restricted pixels, A/B test variants, and Single Page Applications where traditional crawlers go blind. This guide shows you how to build a production-grade compliance monitoring platform that scales to 50,000 pages/day with Playwright, SQS, and a custom scoring engine.
PlaywrightAWS SQSTimescaleDBPagerDutyChromium
💡
The Gist — What Problem Are We Solving?
A robot QA tester that visits your site from 50 countries every 15 minutes
A tracking tag breaks silently on a Friday. Nobody notices until Monday. Two days of data gone and two days of wrong decisions made. This system sends a headless browser robot to every page on your site from 50 countries every 15 minutes, checks every tag fires correctly, tests consent flows, and raises an alert within 5 minutes if anything breaks.
💬Think of it as a health check that tells you if your tracking is broken — before the missing data costs you.
These are the capabilities the system must deliver — what users and operators can actually do with it.
🤖
Tag Validation
🤖Check GA4, Meta Pixel, TikTok tag presence, payload schema, firing order
✅
Consent Testing
✅Test accept and decline flows; verify no pixels fire on rejection
🔗
Integration Health
🔗API reachability, data freshness, schema match for each vendor
📊
Health Score
📊0–100 score per market: Tagging 40% + Integration 30% + Compliance 30%
🔔
Alerting
🔔Alert within 5 min on >10pt score drop or critical tag failure
⚡
Non-Functional Requirements
These define how well the system must perform — the quality attributes that separate a toy from a production system.
⏱️ Refresh Cadence
⏱️Score updated every 15 minutes per market
🌍 Coverage
🌍50+ markets; 500 URLs/market = 25K total URLs
⚡ Alert SLA
⚡<5 min from failure to Slack/PagerDuty notification
🖥️ Dashboard Load
🖥️<2s p95 for 50-market overview
🛡️ Uptime
🛡️99.9% crawler uptime; markets independent — one failure does not block others
📊
Key Metrics — The Numbers That Define This System
The headline numbers to know cold — and be ready to explain how each one is achieved.
🏗️
System Architecture Diagram
Full data flow from source to serving. Each layer scales independently.
Ingestion
→
→
Playwright Crawler Fleet
per-market
→
↓
Processing
→
→
Dashboard UI + Alert Engine
Slack/PagerDuty
🗺️
End-to-End User Journey
Trace a single request end-to-end — the story interviewers want you to tell fluently.
1
Scheduler triggers crawl
— Cron fires every 15 min; SQS messages created for each market × URL batch
2
Playwright crawl runs
— Headless Chrome visits URL, intercepts network requests, records all tag payloads
3
Consent flows tested
— Click Accept: record which tags fire. Reload, click Decline: verify no ad tags fire
4
Score computed
— Tag score + Integration score + Compliance score weighted to 0–100
🔭
High-Level Design — Component Breakdown
Core components — each with a single, well-defined responsibility. The key architectural insight: each layer scales independently, and failure in one component is isolated from the rest.
1 — Cron
Handles responsibilities for the Cron layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
2 — SQS
Handles responsibilities for the SQS layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
3 — Playwright
Handles responsibilities for the Playwright layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
4 — Tag Validator
Handles responsibilities for the Tag Validator layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
5 — Score Engine
Handles responsibilities for the Score Engine layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
6 — Postgres
Relational source of truth for transactional data. ACID guarantees for order records, account state, and financial ledgers. Read replicas serve analytics queries; connection pooling via PgBouncer.
🔬
Low-Level Design — Deep Dives
Deep dives worth explaining in detail in any senior engineering interview. For each: know the data structure, the algorithm, the why, and the trade-off you made.
1 — Scheduler & Job Dispatcher
SQS · Priority Queue
Scheduler runs every 15 minutes, reads URL manifest from S3 (JSON list of {url, priority, last_score, last_crawled_at}). High-priority URLs (checkout, conversion pages) dispatched to SQS FIFO queue. Standard URLs go to SQS Standard queue. SQS visibility timeout prevents parallel processing of the same URL. Dead letter queue after 3 failed attempts triggers manual review alert.
for url in urls:
sqs.send_message(
QueueUrl=PRIORITY_QUEUE if url.priority==’high’ else STANDARD_QUEUE,
MessageBody=json.dumps(url),
MessageGroupId=url.domain # FIFO group
)
2 — Playwright Crawler
Headless Chromium · Tag Extraction
Playwright launches Chromium in headless mode. Network interception captures all outgoing requests during page load. Tag extraction pipeline: (1) Intercept all requests to known tag endpoints (GTM, GA4, Meta Pixel, etc.). (2) Parse request payload against expected schema for each tag type. (3) Execute synthetic interactions (click CTA, scroll 90%, fill form) to trigger interaction-dependent tags. (4) Capture final network log and compare against expected manifest.
async with playwright.chromium.launch() as browser:
page = await browser.new_page()
page.on(‘request’, capture_tag)
await page.goto(url, wait_until=’networkidle’)
await page.click(‘[data-testid=”cta”]’)
return collected_tags
3 — Scoring Engine
0–100 · Weighted by Impact
Scoring formula: score = 100 – sum(penalty for missing/invalid tag). Penalty weights: conversion_tag=40pts, audience_tag=15pts, analytics_tag=5pts, performance_tag=1pt. Each tag is assessed as PASS / WARN (present but invalid schema) / FAIL (absent). WARN deducts 50% of the full penalty. Score computed per-page, then aggregated to domain-level (weighted by page priority). Score history stored in TimescaleDB for trend analysis.
def score_page(tags_found, expected_manifest):
score = 100
for expected in expected_manifest:
found = next((t for t in tags_found if t.type == expected.type), None)
if not found:
score -= WEIGHTS[expected.type]
elif not validate_schema(found, expected.schema):
score -= WEIGHTS[expected.type] * 0.5
return max(0, score)
4 — Alert Router
Slack · PagerDuty · Email
Alert conditions evaluated after every crawl: score < 70 → PagerDuty P2 alert. Score 70-89 → Slack #measurement-alerts. Score drop >20pts vs previous crawl → PagerDuty P1 regardless of absolute score. Redis key alert:{domain} with 30-min TTL prevents duplicate alerts. Alert message includes: current score, previous score, list of failing tags with example URLs, suggested fix.
if score < THRESHOLDS['red'] and not redis.exists(f'alert:{domain}'):
pagerduty.create_incident(severity='P2', details=report)
redis.setex(f'alert:{domain}', 1800, '1')
⚖️
Trade-offs & Decision Log
Every senior interview comes down to these decisions. Know the exact trade-off, the reasoning, and the specific numbers that justify each choice.
⚖️ Headless Browser vs Lightweight HTTP Crawler
✓
Playwright (headless) ✅ Chosen
- Executes JavaScript — catches GTM, dynamic tag loaders
- Sees the final DOM state that real users see
- Validates consent banners and CMP interactions
- 5-10× slower and more resource-intensive than HTTP
→
HTTP crawler (curl/requests)
- 10× faster — no JS execution overhead
- Misses 80%+ of modern tag implementations (all GTM-based)
- Cannot validate consent gate behaviour
- Cannot detect SPA route-change tags
💡Decision: Playwright for all tag validation; lightweight HTTP pre-fetch for robots.txt and sitemap parsing only
⚖️ Push Alerts vs Pull Dashboard Checks
✓
Push (Kafka → PagerDuty/Slack) ✅ Chosen
- Alert fires within 2 minutes of score drop
- On-call team notified before users are affected
- Requires alert deduplication (Redis 30min window)
- Alert fatigue risk if thresholds not tuned
→
Pull (users check dashboard)
- No alert infrastructure needed
- Issues only caught when someone happens to look
- Average detection lag: hours to days
- Unsuitable for data-quality SLAs
💡Decision: Push alerts for score drops below threshold; daily digest email for gradual drift; dashboard for manual investigation
🎯Interview Questions — Answered
The exact questions interviewers ask — with production-grade answers
Q1
How do you validate tags that only fire after user interaction (click, scroll)?
Playwright’s page.evaluate() injects synthetic interaction events: programmatic click on CTA buttons, scroll to 50%/90% page depth, and form fill on key input fields. These trigger interaction-dependent tags (scroll tracking pixels, button click events, form abandonment). For each interaction, the Playwright script captures the network waterfall before and after, comparing against the expected tag manifest. Tags that require real user data (e.g. purchase confirmation tags) are validated in a sandbox environment with synthetic order IDs that are excluded from reporting.
Q2
How does the scoring engine weight different tag failures?
Tags are scored by business impact: (1) Conversion tags (purchase, lead) = 40 points each — these directly impact attribution accuracy. (2) Audience tags (remarketing pixels) = 15 points each. (3) Analytics tags (pageview) = 5 points each. (4) Performance tags (heatmap, session replay) = 1 point each. Maximum score = 100. A missing purchase tag drops the score from 100 to 60 immediately. Score thresholds: 90-100 = Green, 70-89 = Amber (alert), <70 = Red (page). This weighting ensures high-severity issues are immediately visible rather than averaged into a misleading aggregate.
Q3
How often does the crawler run and how is it rate-limited?
Each URL is crawled every 15 minutes for critical pages (checkout, purchase confirmation, homepage) and every 4 hours for content pages. Rate limiting is enforced at two levels: (1) Per-domain: maximum 2 concurrent Playwright instances per domain to avoid triggering bot detection. (2) Global: SQS visibility timeout prevents parallel processing of the same URL. Crawl budget is allocated by page priority score (critical > high > medium > low). Total crawl capacity: 200 concurrent Playwright instances across 20 workers = ~50K pages/day.
System Design Series · Every Tuesday & Thursday
Level up your system design interviews
Each post covers Gist, Functional & Non-Functional Requirements, Key Metrics, System Diagram, User Journey, HLD, LLD, and Trade-offs & FAQs.
Subscribe to never miss a post →
Previous Articles
Categories: System Design
Tags: adtech, alerting, dashboard, interview prep, measurement readiness, playwright, system design, tag validation
Leave a Reply