Neural Vision Classification

Image classification project built around two noisy, imbalanced vision datasets: dermoscopic lesion diagnosis and facial expression recognition. The work starts from simple MLP baselines, moves through a custom CNN with ablations, then finishes with ResNet18 fine-tuning, test-time augmentation, seed ensembling, and a curriculum learning bonus experiment.

The final Kaggle-oriented pipeline passed the required threshold, with a remote score above 0.77 on the facial expression task. The project is written as a reproducible notebook plus a polished technical report.

What This Project Tries To Solve

The goal is to classify images into 7 classes under realistic constraints:

• moles: dermoscopic images where color, lesion texture, shape and class imbalance matter a lot.
• faces: aligned facial images where expressions depend on local structure around eyes, eyebrows and mouth.

Both datasets are imbalanced, so the main metric is F1-macro, not plain accuracy. Accuracy is still reported, but it can hide poor performance on rare classes.

Highlights

• MLP baseline with and without dropout.
• Custom SmallCNN with 4 convolutional blocks.
• CNN ablations for BatchNorm, augmentation regimes and class imbalance.
• ResNet18 fine-tuning with frozen early layers, differential learning rates and warmup.
• Focal loss for the skin-lesion task.
• Test-time augmentation for final inference.
• Seed ensemble for Kaggle remote predictions.
• Bonus curriculum learning with explicit image difficulty and pacing.

Visual Data Checks

The first thing the project does is inspect the data. The two domains need different augmentation strategies: lesions can be flipped/rotated naturally, while faces need gentler transforms.

Dataset balance	Raw samples

Augmentations

For CNN training, four augmentation modes are compared:

• none: resize, tensor conversion, normalization.
• geom: crop/resize, flips and small rotations.
• color: mild color jitter and rare blur.
• all: geometric + color transforms + small random erasing.

For the lesion dataset, rotations and vertical flips are valid because lesion orientation is not semantic. For faces, vertical flips and large rotations are avoided because they create unrealistic faces and damage the expression signal.

Model Families

MLP

The MLP receives resized 64x64 RGB images flattened into a vector:

3 x 64 x 64 -> Flatten -> Linear(12288, 512) -> BatchNorm -> ReLU
             -> Dropout(optional) -> Linear(512, 256) -> BatchNorm -> ReLU
             -> Dropout(optional) -> Linear(256, 7)

SmallCNN

The CNN uses 4 convolutional blocks and a compact classifier head:

3 -> 48 -> 96 -> 192 -> 288 -> AdaptiveAvgPool -> Dropout -> Linear(288, 7)

It stays inside the assignment constraints while still giving the model spatial inductive bias.

ResNet18 Fine-Tuning

The strongest local models use ResNet18 pretrained on ImageNet. Early layers stay frozen, later layers are adapted, and the final head is replaced for 7-class classification.

Curriculum Learning Bonus

The curriculum experiment trains SmallCNN from scratch on faces. It does not initialize from the baseline and does not use a pretrained model.

The curriculum has:

• a static difficulty score per image based on low contrast and extreme brightness;
• class-balanced pacing, so rare classes are not removed from early stages;
• an augmentation curriculum: none -> geom -> all;
• evaluation on the full local test set at every epoch.

The curriculum run is a strong and interpretable bonus experiment. The full all baseline still wins, likely because it sees all hard examples and all augmentations from the start, while the curriculum spends many epochs on easier subsets. That is a useful result: curriculum learning depends heavily on the quality of the difficulty proxy.

Results Snapshot

Family	Dataset	Best local result
MLP	faces	F1-macro 0.5635 with dropout
MLP	moles	F1-macro 0.4454 without dropout
SmallCNN + BatchNorm	moles	F1-macro 0.5159
SmallCNN + sqrt sampler	moles	F1-macro 0.6086
SmallCNN + geom aug	faces	F1-macro 0.5633
Curriculum SmallCNN	faces	F1-macro 0.5138
ResNet18 + warmup + TTA	moles	F1-macro 0.7283
ResNet18 + warmup + TTA	faces	F1-macro 0.7106 locally, with final Kaggle pipeline above 0.77

Detailed plots and interpretation are in the report.

Repository Structure

.
├── main.ipynb                         # Main notebook: setup, training, evaluation, submissions
├── src/                               # Standalone submission / ensemble scripts
├── docs/                              # Report and technical notes
├── assets/report/                     # Selected plots used by README and report
├── results/                           # Compact CSV summaries
├── requirements.txt
└── .gitignore                         # Excludes raw datasets, checkpoints and generated outputs

Raw datasets and trained checkpoints are intentionally not committed. The notebook expects the datasets locally in their original folders:

vai-de-pielea-mea/
you-re-on-candid-camera/

Running Locally

Create an environment and install dependencies:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Then open:

jupyter notebook main.ipynb

For full CNN and fine-tuning runs, use a GPU runtime. CPU/MPS is fine for inspection and some smaller runs, but the project was designed around accelerated training.

Final Report

The polished report is available here:

docs/report.pdf

It includes dataset analysis, preprocessing, augmentation previews, model architectures, ablation tables, training curves, confusion matrices, curriculum learning interpretation and the final Kaggle prediction strategy.