VinDatathon 2026 — Time Series Forecasting Entry

Project Structure

submission/
├── README.md
├── requirements.txt
├── prepare_tableau_data.py       # Generates tableau_data/ CSVs from raw data
├── notebooks/
│   ├── 01_mcq_analysis.ipynb     # Part 1 — Business intelligence / MCQ answers
│   ├── 03_tableau_data_prep.ipynb # Part 2 — Tableau dashboard data preparation
│   ├── 04_forecasting.ipynb     # Part 3 — Time series forecasting (visualization)
│   └── run_pipeline.py          # Part 3 — Full forecasting pipeline
├── output/
│   ├── features_train.csv       # Engineered training features (81 columns)
│   ├── features_test.csv        # Engineered test features
│   ├── submission.csv           # Final submission (Revenue & COGS, 548 rows)
│   ├── run_summary.json         # Machine-readable final run configuration/metrics
│   └── model_cache/             # Cached model predictions and diagnostics (.npz)
├── tableau_data/                 # Pre-computed CSVs for Tableau dashboards (18 files)
├── tableau_dashboard/           # Tableau packaged workbooks (.twbx) (5 files)
├── src/
│   └── config.py                # Shared configuration constants
└── data/                       # Raw competition data (15 CSV files)

Quick Start

Step 1 — Create the environment

# Create a new conda environment (recommended)
conda create -n vin_forecast python=3.11 -y

# Or create from scratch with pip
python3 -m venv vin_forecast_env
source vin_forecast_env/bin/activate   # Linux/macOS
# vin_forecast_env\Scripts\activate     # Windows

Step 2 — Activate the environment

conda activate vin_forecast
# or: source vin_forecast_env/bin/activate

Step 3 — Install all required packages

pip install -r requirements.txt

Step 4 — Verify the data is in place

Make sure the data/ folder contains all 15 raw CSV files provided by the competition.

ls data/
# customers.csv  geography.csv  inventory.csv  order_items.csv  orders.csv
# payments.csv  products.csv  promotions.csv  returns.csv  reviews.csv
# sales.csv  sample_submission.csv  shipments.csv  web_traffic.csv

Step 5 — Generate Tableau data (optional — only if you want to re-run from scratch)

The tableau_data/ folder is pre-computed. If you need to regenerate it:

python prepare_tableau_data.py

This reads raw CSVs from data/ and outputs 18 aggregate CSVs into tableau_data/.

Step 6 — Run the forecasting pipeline

# Must be run from the root directory (same level as data/)
python notebooks/run_pipeline.py

Outputs are saved under output/:

• output/submission.csv — 548 rows of daily Revenue and COGS predictions for 2023-01-01 to 2024-07-01.
• output/run_summary.json — selected objective, ensemble settings, CV metrics, margin checks, and submission totals.

---

Notebooks Overview

File	Part	Description
01_mcq_analysis.ipynb	Part 1	Business intelligence analysis & MCQ answers with data visualizations
03_tableau_data_prep.ipynb	Part 2	Prepares 18 aggregate CSVs consumed by the Tableau dashboards
04_forecasting.ipynb	Part 3	Exploratory visualization of time series patterns before modeling
run_pipeline.py	Part 3	Full end-to-end forecasting pipeline — features → models → ensemble → submission

Forecasting Pipeline Details

The pipeline (notebooks/run_pipeline.py) produces the submission in 13 steps:

1. Load Data — Training (2012–2022) and test (2023–07-01 to 2024-07-01) CSVs
2. Feature Engineering — 81 calendar, holiday, Tet, promo, and Fourier features
3. Sample Weighting — Era-based weighting (high_era scheme: 2014–2018 weight=1.0)
4. CV Folds — Three temporal folds (A=2022, B=2021, C=rolling 12-month)
5. M1 — Ridge Regression — Linear baseline on log-scale targets
6. M2 — LightGBM Helpers / Initial Base Models — Gradient-boosted base model family over three sample-weight schemes
7. M3 — Prophet — Facebook Prophet with multiplicative seasonality and promo regressors
8. Q-Specialists — 8 LightGBM models (4 quarters × Revenue/COGS), each quarter-boosted
9. Objective and Ensemble Selection
   • Compare LightGBM regression, regression_l1, and Huber candidates on Fold A
   • Select by a balanced MAE/RMSE/R² composite while tracking public MAE as a guardrail
   • Test NNLS against the best single LightGBM sample-weight scheme
   • Retune alpha_Q, CR, and CC after the selected objective is locked
10. Final Full-Data Retrain — Retrain final LightGBM base models and Q-specialists with the selected objective before submission
11. Post-Processing — Margin sanity check (COGS/Revenue ratio by quarter)
12. Generate Submission — output/submission.csv and output/run_summary.json
13. SHAP Explainability — Component-level LightGBM/Ridge diagnostics saved under output/

Key hyperparameters (discovered / fixed)

Parameter	Value	Source
LightGBM objective	huber, alpha=0.7	Fold A objective search
alpha_Q (per-quarter blend)	[0.7, 0.6, 0.7, 0.3]	Fold A coordinate descent after objective selection
LGB_BLEND_MODE	high_era	Fold A balanced composite vs. NNLS
NNLS Revenue weights	Stored in output/run_summary.json	Fold A diagnostic
NNLS COGS weights	Stored in output/run_summary.json	Fold A diagnostic
Calibration CR (Revenue)	1.025	Fold A balanced composite
Calibration CC (COGS)	1.025	Fold A balanced composite
Ridge alpha	3.0	Fixed
LGB num_leaves	63, learning_rate=0.03	Fixed

The public MAE score from the earlier leaderboard is tracked only as a sanity guardrail. The corrected final pipeline is optimized for the final leaderboard's three metrics: MAE, RMSE, and R².

Tableau Dashboards

Five packaged workbooks in tableau_dashboard/:

File	Topic
D1_Revenue_Profitability.twbx	Revenue trends, YoY growth, channel efficiency
D2_Customer_Segmentation_LatestVersion.twbx	RFM segmentation, CLV, cohort retention
D3_Product_Analysis.twbx	ABC classification, dead stock, inventory leakage
D4_Marketing.twbx	Promo ROI matrix, borrowed demand, channel attribution
D5_Operations_Supply_Chain.twbx	Service quality, fulfillment, refund, and operational diagnostics