Authors: Isaac Tabor and Kimberly L
This project explores the use of machine learning to predict NCAA March Madness basketball outcomes using a comprehensive dataset of historical team performance and game results. Leveraging data from the 2003 to 2024 seasons, we built a supervised binary classification model using XGBoost, with the goal of estimating the probability that a given team will win a specific tournament matchup. We also used an XGBRegressor, which is typically used for regression tasks; however, we used the log loss metric, which aligned with the classification nature of our study, and our outputs predicted probabilities of one class (Team A winning), and we used a threshold of .5 to classify each game into the appropriate binary class (Win vs. Lose).
Each observation in our final dataset represents a single NCAA tournament game, enriched with detailed team and opponent statistics aggregated at the regular season level. To capture matchup dynamics more effectively, we engineered differential features, such as the difference in win percentage, shooting efficiency, and rebounds, between competing teams.
To evaluate our model, we split the data temporally, training on seasons prior to 2025 and validating on more recent years. We used accuracy as our primary evaluation metric, supported by log loss, ROC AUC and confusion matrices. We trained the model using a low learning rate and a fixed maximum of 1,000 trees to balance learning capacity and overfitting risk. We also used a GroupKFold cross-validation strategy to ensure robust seasonal generalization. Our final model achieved an average CV accuracy of .6568 and on average was outperformed by our “chalk” benchmark that had a baseline accuracy of .69744. The “chalk” benchmark predicts winners based solely on seed ranking.
Our findings show that even with just box score statistics, our machine learning model was unable to outperform a purely seed ranking tournament prediction strategy. Future improvements could include exploring including a rankings differential to the features we used, removing unhelpful features we used, adding possession-based metrics, advanced efficiency ratings, or combining models through ensembling. Overall, this work explores the value of structured feature engineering and thoughtful validation in sports analytics.