🧠 Neural Networks in C++

Developed by Matthew Bernardi, Pietro Celati, and Gabriele Marsili > For the Machine Learning course project @ University of Pisa.

---

✨ Core Features & Implementations

We built this framework to be highly optimized and extensible, implementing several advanced deep learning algorithms and techniques from the ground up:

• Advanced Optimizers: Beyond standard Backpropagation via SGD with Momentum, we implemented Nesterov Accelerated Gradient, RProp, and QuickProp.
• Robust Regularization: Integrated L2 Weight Decay and Dropout (applicable to both input and hidden layers) to prevent model overfitting.
• Smart Training Mechanisms: Implemented Early Stopping based on validation loss monitoring, alongside a custom plateau detection system (patience_threshold) for training loss.
• High-Performance Concurrency: Accelerated forward and backward passes using a custom Thread Pool, which parallelizes node computations within each layer.
• Data Processing Pipeline: Automated Z-score normalization (fitted strictly on the training set to prevent data leakage, then applied across all splits) and 1-of-k (One-Hot) encoding.
• Validation & Tuning: Fully integrated K-Fold Cross-Validation seamlessly coupled with a Cartesian Grid Search for hyperparameter exploration.

---

🚀 Getting Started

Build

Compile the project using the provided Makefile:

make

Execution Modes

The framework includes dedicated pipelines for our target benchmarks. You can run them using the following flags:

./neural_net --monk      # Run the MONK dataset benchmark (tasks 1, 2, and 3)
./neural_net --cup       # Execute the full ML-CUP25 pipeline
./neural_net --grid      # Perform hyperparameter Grid Search on the CUP dataset

---

🏗️ Architecture & Modules

The codebase is structured to ensure a clean separation of concerns:

Module	Path	Description
Core Architecture	src/NN_architecture/	Base neural network logic and orchestration.
Nodes & Weights	NN_node.cpp	Low-level math, weight updates, and optimizers (SGD+Momentum, Nesterov, RProp, QuickProp).
Layers	NN_layer.cpp	Online and Batch Backpropagation, parallelized via Thread Pool.
Dataset Handling	src/dataset_handling/dataset_operator.cpp	Data parsers, MONK 1-of-k encoding, CUP loading, and Z-score standardization.
Metrics & Math	src/math_utils/calculator.cpp	Calculation of MSE, MEE, and classification Accuracy.
Configuration	src/config/config_orchestrator.*	JSON parsing for dynamic network instantiation and hyperparameter loading.
Grid Search	src/utils/grid_search.*	K-Fold CV and Cartesian product generation for hyperparameter tuning.
Visualization	scripts/plot.py	Python scripts for learning curves, grid heatmaps, MONK summaries, and CUP MEE plotting.

---

⚙️ Configuration

Network topologies and training hyperparameters are completely decoupled from the C++ source code and managed via JSON files:

• src/config/cup_config.json — Setup for the ML-CUP25 dataset (Regression task, 4 output nodes).
• src/config/monk_config.json — Setup for the MONK's problems (Binary classification).

---

📊 Output Artifacts

All experiment results and logs are automatically tracked and saved in the results/<run_name>/ directory:

• 📄 epoch_metrics.csv — Detailed metrics tracked per epoch.
• 📉 mee_summary.json — Mean Euclidean Error (MEE) on TR/VAL/intTS splits, restored to the original scale.
• 📦 MarsiliBernardiCelati_ML-CUP25-TS.csv — Blind test predictions formatted for official submission.
• 🖼️ plots/ — Graphical visualizations (learning curves, heatmaps) generated by the scripts/plot.py utility.