§ 01 Problem
Predict marketing-campaign conversion on 48,000 synthetic customer records where only 2.04% convert. The first model came back with ROC-AUC 1.0000 — a score that is physically impossible on marketing data. Most pipelines would have shipped that number; the real work of the thesis was refusing to believe it. The model wasn't learning behaviour, it was reading the answer key through features derived from the outcome.
§ 02 Approach
- A six-scenario ablation hunted the leak systematically: removing the obviously suspicious derived features changed nothing, until dropping CPA_Proxy alone collapsed AUC from 1.0000 to 0.7071 — one feature was carrying the entire illusion.
- The leaking proxies were rebuilt as leakage-free engineered features (ROI_v2, CPA_v2) computed strictly from pre-conversion inputs, so nothing downstream of the label could flow back into training.
- Model choice followed the use case, not the leaderboard: Logistic Regression (ROC-AUC 0.713) won over XGBoost (0.705) and Random Forest (0.698) because the prescriptive layer needed coefficients a human can read — and the honest gap between them was two points, not twenty.
- SMOTE — the textbook answer to 2% positives — was tested and rejected on evidence: at that imbalance it made every metric slightly worse, so it stayed out of the final pipeline.
§ 03 Outcome
The honest model feeds a prescriptive recommender that simulates 49 channel-platform combinations per customer and picks the best, producing a ~26.5% simulated relative lift in conversion probability on the 9,600-customer test set. Supported by TÜBİTAK 2209-B; the leakage story became the defense's centerpiece rather than its embarrassment.
+26.5%
simulated relative lift, 9,600-customer test set
0.713
honest ROC-AUC after leakage removal
