Housing Price Prediction using Linear, Ridge, and Lasso Regression

This repository contains a comprehensive machine learning workflow to predict housing prices using the classic Boston Housing dataset. The project demonstrates data preprocessing, feature scaling, model training, and hyperparameter tuning using regularization techniques.

Project Overview

The goal of this project is to build and evaluate predictive models for housing prices (MEDV) based on various neighborhood and environmental features (such as crime rates, number of rooms, pupil-teacher ratios, etc.).

To combat overfitting and handle multicollinearity, the performance of standard Linear Regression is compared against regularized variants (Ridge and Lasso).

---

Tech Stack & Libraries

• Language: Python
• Data Analysis & Manipulation: Pandas, NumPy
• Data Visualization: Matplotlib, Seaborn
• Machine Learning: Scikit-Learn

---

Dataset Features

The dataset includes the following architectural and socioeconomic attributes:

• CRIM: Per capita crime rate by town
• ZN: Proportion of residential land zoned for lots over 25,000 sq. ft.
• INDUS: Proportion of non-retail business acres per town
• CHAS: Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
• NOX: Nitric oxides concentration (parts per 10 million)
• RM: Average number of rooms per dwelling
• AGE: Proportion of owner-occupied units built prior to 1940
• DIS: Weighted distances to five Boston employment centers
• RAD: Index of accessibility to radial highways
• TAX: Full-value property-tax rate per $10,000
• PTRATIO: Pupil-teacher ratio by town
• B: $1000(Bk - 0.63)^2$ where $Bk$ is the proportion of Black residents by town
• LSTAT: Percentage of lower status of the population
• MEDV / price: Median value of owner-occupied homes in $1000s (Target Variable)

---

Workflow & Implementation

1. Data Cleaning & Exploration

• Checked for missing values and analyzed data types.
• Separated features ($X$) and target ($y$) variables.

2. Feature Scaling

• Applied StandardScaler to normalize features, ensuring that the penalty terms in Ridge and Lasso regression treat all features equitably.

3. Model Training & Hyperparameter Tuning

• Linear Regression: Baseline model evaluated with 5-fold cross-validation.
• Ridge Regression ($L_2$ Regularization): Tuned using GridSearchCV to optimize the penalty strength ($\alpha$).
• Lasso Regression ($L_1$ Regularization): Tuned using GridSearchCV to perform automatic feature selection by shrinking less important coefficients to zero.

---

Evaluation Metrics

Models are evaluated and compared using standard regression evaluation metrics:

• R-Squared ($R^2$) Score: To measure the proportion of variance explained by the model.
• Mean Absolute Error (MAE): To evaluate average absolute prediction errors.
• Root Mean Squared Error (RMSE): To penalize larger errors and evaluate variance.