A collection of biweekly laboratory assignments focusing on statistical analysis, machine learning classification, and regression techniques I completed for my data science module coursework (COMP13212). These applied mathematical concepts in the class computationally, developing experience from hypothesis tests to model training for classification and target prediction.
Below is a description of all 4 graded labs completed, their properties and procedure.
- Clone the repository
- Install dependencies:
pip install pandas numpy matplotlib scipy sklearn wordcloud - Launch Jupyter:
jupyter notebook
Key Concepts: Random Sampling, Sampling with Replacement, Standard Error of the Mean (SEM), Confidence Intervals.
- Description: This lab uses the Gapminder dataset to analyze global life expectancy. It explores how different sampling methods affect the accuracy of population estimates and visualizes uncertainty through confidence intervals.
Key Concepts: Data Cleaning, Feature Engineering, Data Smoothing, Hypothesis Testing (t-tests/Z-tests).
- Description: Investigates 70 years of Heathrow weather data to find local evidence of climate change. Includes processing raw data (e.g., millimeter of rain, air frost days) and performing statistical tests to determine if weather attributes have significantly changed over time.
Key Concepts: Natural Language Processing (NLP), Regex Cleaning, Tokenization, Probability Theory (Priors/Posteriors).
- Description: Builds a text classifier to categorize resumes into job roles (e.g., Data Science, Accountant, Advocate). The project involves text preprocessing, creating token frequency distributions, and visualizing class-specific language using Word Clouds.
Key Concepts: High-Dimensional Data, Missing Value Imputation, Feature Selection, Linear Regression.
- Description: Performs regression on a complex dataset to predict age based on biological features (CpG sites). It addresses challenges like the "curse of dimensionality" and employs mean imputation to handle missing values before model training.
This project is is not licensed for modification or addition.