Telco Customer Churn

§ 01 Problem

Predicting which telecom customers are about to leave is a solved tutorial. Keeping the model honest after deployment isn't: without run tracking and drift monitoring, you can't tell which model version produced which number, or whether the live data still looks like the training data.

§ 02 Approach

• Logistic Regression with the saga solver and elastic-net regularization, class imbalance handled with SMOTE — the winner of a 12-candidate screening (4 models × 3 imbalance strategies) tuned with 120 Optuna trials, every experiment logged run-by-run in MLflow.
• The model isn't just a notebook — it ships as a FastAPI service (api/) with a prediction form, Swagger docs at /docs, and single + batch (max 500) prediction endpoints.
• The decision threshold isn't hardcoded: training optimizes it for F1 on out-of-fold probabilities (0.565 for the current run), stores it in model_metadata.json, and the API returns threshold_used with every response — every prediction traceable to an exact model version and cutoff.
• The tuned model had to earn its place: acceptance checks in metadata require a minimum F1 improvement over baseline before a run is accepted; CI runs ruff + black + 20 data-independent tests + API smoke tests on every push.
• A JSON drift report (reports/drift/) compares live feature distributions against the training snapshot, turning 'the model feels stale' into an inspectable artifact.

§ 03 Outcome

A reproducible pipeline where no result is an orphan: every model version is tracked, every input shift is visible, and retraining is a decision backed by a report rather than a hunch.

§ 04 Evidence

∎