Sparkify, a fictional music streaming startup, aims to predict user churn using Apache Spark. In this context, "churn" refers to customers leaving the service over a given period. Identifying potential churners is crucial, as acquiring new customers can cost five to six times more than retaining existing ones.
- Free plan users: Stream music with ads
- Paid plan users: Ad-free music streaming
- User actions: Thumb up/down songs, create playlists, add friends
- Churn types:
- Downgrade: Paid to free plan
- Cancel: Stop using the service entirely
Develop a machine learning model to accurately identify potential churners of both types (downgraded and cancelled users) based on their interaction data.
- Data: 12GB log file with 18 fields per user interaction
- Tools: Apache Spark, PySpark MLlib
- Process: Data loading, exploration, feature creation, model building, churn prediction
- Models Tested: Logistic Regression, Support Vector Machine and Gradient Boosted Trees
- Best Model: Tuned Logistic Regression
- Addresses critical business challenge of customer retention
- Demonstrates handling of large datasets with Spark
- Applies machine learning techniques to real-world problem
The winning model effectively identifies users at risk of churning, enabling targeted retention strategies to potentially save millions in revenue.