Business Challenge
A fast-growing e-commerce company, noticed a 20% increase in customer churn over six months. Their existing analytics system provided post-churn insights but failed to predict at-risk customers early. They needed a real-time predictive model to:
- Identify high-risk customers before churn
- Enable targeted retention campaigns (discounts, personalized offers)
- Reduce customer acquisition costs by improving retention
Solution: Automated ML Pipeline for Churn Prediction
We designed a scalable data pipeline that ingests transactional, behavioral, and engagement data to generate churn probability scores updated daily.
Architecture Overview:
High-Level
Key Components
1. Data Ingestion
- PostgreSQL: Historical orders, returns, and customer metadata (updated hourly).
- CRM API: Real-time customer service interactions (complaints, refunds).
- S3 Buckets: User clickstreams (page views, cart abandonment) processed daily.
Tools:
- Python (Boto3, Psycopg2, Requests) for extraction
- Airflow to manage dependencies (e.g., “Wait for S3 data before feature engineering”)
2. Transformation & Feature Engineering
- Pandas: Cleaned null values, standardized formats (e.g., USD currencies).
- PySpark: Computed aggregated features:
- 30-day_purchase_frequency
- avg_cart_abandonment_rate
- customer_service_complaints_last_week
3. Machine Learning Model
- Algorithm: XGBoost (via scikit-learn API) for handling imbalanced data.
- Optuna: Automated hyperparameter tuning (optimized for precision@top-10% to focus on highest-risk customers).
- Validation: Time-based split (train on 6 months, test on next 30 days).
Key Features:
- Recency/frequency metrics (RFM)
- Engagement decay rate (e.g., “Days since last login”)
- Sentiment score from customer support tickets
4. Deployment & Output
- AWS Lambda: Served predictions via API (cost-effective for sporadic retraining).
- Snowflake: Stored predictions with customer IDs for joinable analytics.
- Downstream: Marketing teams used Tableau to filter customers by churn risk and LTV.
Results
| Metric | Before | After |
|---|---|---|
| Churn Rate | 22% | 16% |
| Retention Campaign ROI | 1.5x | 3.8x |
| Model Accuracy (AUC-ROC) | — | 0.89 |
Business Impact:
- Saved $2.3M/year by reducing churn in high-LTV segments.
- Enabled dynamic email campaigns
(e.g., “We miss you!” discounts for 50% predicted churn risk).
Lessons Learned
- Cold-start problem:
Added synthetic data for new users. - Lambda limitations:
Switched to batch predictions for >10K users to avoid timeouts. - Feature drift:
Implemented Evidently.ai monitors to track data shifts.