SYSTEM DESIGN #10 · INTERVIEW GUIDE
Marketing ROI Optimization Engine
Most marketing teams allocate budget based on last month’s performance — which means they are always optimising for the past. This Marketing ROI Optimization Engine changes that: it pulls hourly spend and conversion data from every ad platform, trains a LightGBM revenue prediction model, runs SLSQP constrained optimisation to find the maximum-ROI budget allocation, and serves SHAP explanations so marketers understand exactly why the system recommended shifting $50K from Facebook to YouTube. The system is designed for safety as well as speed: every recommendation goes through a human approval gate before budgets are touched, and a shadow-mode back-test validates every model change before it goes live. After this guide you will know how to build a production ML pipeline that moves budgets automatically and auditably.
LightGBMSLSQPSHAPTimescaleDBPython
💡
The Gist — What Problem Are We Solving?
Telling you exactly where to move your budget to make more money
You have $10M across Meta, Google, TikTok, and TV. How do you split it to maximise revenue? Gut feel does not scale. This system analyses the historical return of every dollar per channel, models diminishing returns, and recommends exactly how to reallocate — in plain English, with the maths shown, waiting for human approval before touching any budget.
💬Think of it as a financial advisor for your ad budget — it shows its working, explains its reasoning, and waits for your sign-off before touching a single pound.
These are the capabilities the system must deliver — what users and operators can actually do with it.
📈
Spend Ingestion
📈Hourly pull from Meta, Google, TikTok APIs; attribution revenue per channel
🧮
Optimizer
🧮Constrained optimisation: maximise revenue subject to total budget constraint
💡
Explainability
💡SHAP values plus plain-English explanation for every recommendation
✅
Human Approval
✅Budget manager approves or rejects; full audit trail; no auto-execution above threshold
📊
Performance Tracking
📊Track predicted vs actual ROAS per recommendation; retrain on drift
⚡
Non-Functional Requirements
These define how well the system must perform — the quality attributes that separate a toy from a production system.
⏱️ Recommendation Latency
⏱️New recommendation within 30 minutes of latest spend data
⚡ Optimizer Runtime
⚡<60s for 1,000 campaigns (scipy SLSQP)
💰 Budget Scale
💰Handles $1B+ annual budgets without numerical precision issues
🛡️ Safety
🛡️Human approval for all changes; auto-execute only <5% daily budget with guardrails
🔄 Retraining
🔄Auto-retrain if MAPE >15% on prediction accuracy
📊
Key Metrics — The Numbers That Define This System
The headline numbers to know cold — and be ready to explain how each one is achieved.
30min
recommendation cycle
🏗️
System Architecture Diagram
Full data flow from source to serving. Each layer scales independently.
↓
🗺️
End-to-End User Journey
Trace a single request end-to-end — the story interviewers want you to tell fluently.
1
Spend data ingested
— Hourly pull from all ad platform APIs; attribution revenue per channel written to Feature Store
2
Features engineered
— Per-campaign: trailing_7d_roas, trailing_30d_roas, spend_velocity, audience_saturation, day_of_week_index
3
Revenue model runs
— LightGBM predicts marginal revenue for each $1K spend increment — builds diminishing returns curve per channel
4
Optimizer solves
— scipy SLSQP: maximise Σ(predicted_revenue_i) subject to Σ(budget_i) = total_budget and per-channel bounds
5
Explanation generated
— SHAP values → top 3 reasons: ‘Move £15K from TikTok to Google. Predicted uplift +£42K. Google 7d ROAS 4.8× vs TikTok 2.1×’
6
Human approves
— Budget manager reviews in dashboard; approves or modifies; sent via Integration Hub to ad platforms
🔭
High-Level Design — Component Breakdown
Core components — each with a single, well-defined responsibility. The key architectural insight: each layer scales independently, and failure in one component is isolated from the rest.
1 — Ad APIs
Handles responsibilities for the Ad APIs layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
2 — Feature Store
Handles responsibilities for the Feature Store layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
3 — LightGBM
Handles responsibilities for the LightGBM layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
4 — SLSQP
Handles responsibilities for the SLSQP layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
5 — SHAP
Handles responsibilities for the SHAP layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
6 — Approval UI
Handles responsibilities for the Approval UI layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
🔬
Low-Level Design — Deep Dives
Deep dives worth explaining in detail in any senior engineering interview. For each: know the data structure, the algorithm, the why, and the trade-off you made.
1 — Data Collector
Hourly API Pull
Connector jobs pull spend + conversion data from ad platform APIs hourly. Each connector is a separate Kubernetes CronJob: Meta Ads API, Google Ads API, TikTok Ads API, LinkedIn Campaign Manager. Data normalised to canonical schema on ingest. Connector uses OAuth2 with token rotation; secrets in AWS Secrets Manager. Connector errors trigger retry with jitter; after 3 failures, alert fired and manual refresh UI shown to user.
class MetaConnector:
def fetch_hourly(self, account_id, date):
params = {‘fields’: ‘spend,impressions,clicks,conversions’,
‘time_range’: {‘since’: date, ‘until’: date}}
return self.api.get_insights(account_id, params)
2 — Feature Store
Time-series · 90-day Window
Feature store: Postgres TimescaleDB for time-series spend/revenue features + Redis for real-time lag features (last 7 days). Features per channel: spend_7d, spend_30d, revenue_7d, roas_7d, roas_30d, log_spend, spend_saturation_ratio (spend/max_historical_spend). Feature pipeline runs hourly after connector ingestion. Features versioned — model retrained on consistent feature set; old features retained for 180 days for back-testing.
SELECT
channel,
SUM(spend) OVER (PARTITION BY channel ORDER BY date ROWS 6 PRECEDING) AS spend_7d,
SUM(revenue) OVER (PARTITION BY channel ORDER BY date ROWS 29 PRECEDING) AS revenue_30d
FROM channel_daily
3 — LightGBM Revenue Model
ROAS Prediction · Retrained Weekly
LightGBM regression model predicts incremental revenue per $ of spend per channel. Input features: spend_7d, spend_30d, log_spend, spend_saturation_ratio, day_of_week, is_holiday, channel_type. Trained on 90 days of daily channel-level data. Objective: minimize MAE on revenue. Validation: time-series split (most recent 14 days as holdout). Deployed as PMML model file loaded into prediction service — no model server required.
import lightgbm as lgb
params = {‘objective’: ‘regression_l1’,
‘num_leaves’: 31, ‘learning_rate’: 0.05}
model = lgb.train(params, train_data,
num_boost_round=200,
valid_sets=[val_data])
4 — SLSQP Budget Optimiser
Constrained Optimisation
SLSQP (Sequential Least Squares Programming) maximises total predicted revenue subject to: (1) sum(allocations) = total_budget. (2) min_budget[channel] ≤ allocation[channel] ≤ max_budget[channel]. (3) No single channel >40% of total budget (diversification). Objective function: -sum(model.predict(allocation[c]) for c in channels). SLSQP finds local optimum in <1s for 20 channels. SHAP values computed post-optimisation to explain recommendations.
from scipy.optimize import minimize
def optimise(budget, model, bounds):
def neg_revenue(allocs):
return -sum(model.predict(allocs))
return minimize(neg_revenue,
x0=[budget/n_channels]*n_channels,
method=’SLSQP’,
bounds=bounds,
constraints={‘type’:’eq’,’fun’:lambda x:sum(x)-budget})
⚖️
Trade-offs & Decision Log
Every senior interview comes down to these decisions. Know the exact trade-off, the reasoning, and the specific numbers that justify each choice.
⚖️ ML Budget Optimisation vs Rule-Based Allocation
✓
LightGBM + SLSQP ✅ Chosen
- Captures non-linear diminishing returns — ROAS curves are not linear
- SLSQP finds global optimum under budget constraints in <1s
- SHAP values explain every recommendation to stakeholders
- Requires 90 days of historical data for reliable predictions
→
Rule-based (last-period pacing)
- Simple — scale up winners, scale down losers
- No historical data dependency
- Misses cross-channel interaction effects
- Slow to respond to market changes — 1-2 week lag
💡Decision: LightGBM+SLSQP with human approval gate; rule-based fallback when <30 days history available for a channel
⚖️ Automated Execution vs Human Approval Gate
✓
Human Approval Gate ✅ Chosen
- Prevents catastrophic budget shifts from model errors
- Builds marketer trust in the system over time
- Allows domain knowledge override (brand safety, PR events)
- Slows execution — approvals can take hours
→
Fully automated execution
- Fastest possible optimisation response
- No human bottleneck on budget changes
- Single model bug can drain budgets in minutes
- Difficult to audit and explain to finance
💡Decision: Human approval for changes >15% of channel budget; auto-execute micro-adjustments <5% within daily guard rails
🎯Interview Questions — Answered
The exact questions interviewers ask — with production-grade answers
Q1
How does the model handle diminishing returns on ad spend?
LightGBM captures diminishing returns naturally through its tree-based structure — trees can represent non-linear spend-to-revenue curves without explicit feature engineering. To improve accuracy, spend is featurised as log(spend) and spend/historical_spend_ratio (saturation indicator). The training objective is revenue per dollar (ROAS), not total revenue — this prevents the model from always recommending maximum spend. SLSQP then maximises total predicted revenue subject to: total budget ≤ constraint, per-channel min/max bounds, and a ‘diversification’ constraint preventing >40% allocation to any single channel.
Q2
How is the model validated before budget recommendations go live?
Three validation gates: (1) Back-test on holdout period — model predicts last 30 days of revenue using only data from 31-90 days ago; MAPE <15% required to proceed. (2) Shadow mode — for 2 weeks, model runs in parallel making recommendations that are logged but not actioned; human reviewers compare recommendations to actual outcomes. (3) Gradual rollout — first recommendations apply to ≤5% of total budget; expanded to 25%, 50%, 100% over 4 weeks if ROAS improvement is statistically significant (p<0.05). Model is retrained weekly on the most recent 90 days; major architecture changes require full re-validation.
Q3
What happens if the ML model recommends a budget shift that violates brand strategy?
The human approval gate is the explicit override mechanism. Every recommendation above the 5% auto-execute threshold is shown to the budget manager with: (1) Recommended allocation vs current allocation. (2) SHAP explanation: ‘Google Search recommended increase because ROAS=4.2 vs channel average of 2.8, driven by branded keyword performance uplift last 7 days.’ (3) Historical accuracy of model on this channel (e.g., ‘82% of Google Search recommendations improved ROAS in the last 90 days’). (4) One-click override with reason code (brand safety, PR event, test in progress). Overrides feed back into the model as constraints for the next cycle.
System Design Series · Every Tuesday & Thursday
Level up your system design interviews
Each post covers Gist, Functional & Non-Functional Requirements, Key Metrics, System Diagram, User Journey, HLD, LLD, and Trade-offs & FAQs.
Subscribe to never miss a post →
Previous Articles
Categories: System Design
Tags: interview prep, lightgbm, machine learning, marketing roi, optimization, shapley, system design
Leave a Reply