👁️ Eye-Tracking-Based Reading Time Prediction (⌐ ͡■ ͜ʖ ͡■)

A complex machine learning pipeline designed to predict human Total Reading Time (TRT) for specific words within a text. Developed during the 24-hour sprint at the NitroNLP Hackathon (April 25-26, 2026) by a team of four, this project bridges the gap between natural language processing and cognitive science.

🧠 Project Overview

To accurately predict how long a person looks at a word, the model must understand both the physical appearance of the text and its cognitive weight. This project builds a multi-layered feature extraction pipeline that culminates in an optimized LightGBM regressor.

The environment and memory management (especially regarding VRAM during HuggingFace inference) were natively developed and optimized for Ubuntu Linux.

⚙️ Pipeline Architecture

The feature engineering is split into four progressive cognitive layers:

1. Surface Features

Captures the visual and physical footprint of the words.

• Word length, title case detection, uppercase formatting, and digit/punctuation presence.
• Spatial positioning via page and word indexing extracted from word_id.

2. Linguistic Features

Maps the grammatical role and commonality of the text.

• Zipf Frequency: Calculates how common a word is in the Romanian language using the wordfreq library.
• POS Tagging: Implements the ro_core_news_sm model via spaCy to map Part-of-Speech tags (transformed into categorical codes for the ML algorithm).

3. Cognitive "Surprisal" Extraction

Measures how "surprised" the human brain is by a word in a specific context.

• Utilizes the readerbench/RoGPT2-base Transformer model.
• Computes the Cross-Entropy Loss at each token to mathematically represent predictability.
• Anti-Crash Optimization: Implements custom batching (chunks of 200 words) and manual VRAM clearing (torch.cuda.empty_cache()) to prevent Out-Of-Memory errors during deep learning inference.

4. Advanced Contextual Spillover

Accounts for human reading rhythm and attention momentum[cite: 1].

• Target Encoding (Text Difficulty): Calculates the mean reading time per text category.
• Attention Spillover: Passes the length and "surprisal" of the previous word to the current word, simulating the cognitive lag of processing difficult terms.

🚀 Model Details

The final predictions are generated using a highly tuned LightGBM Regressor.

• Target Transformation: Applied np.log1p to the target variable to stabilize variance caused by reading time outliers, reverting to real milliseconds with np.expm1 for the final submission.
• Data Aggregation: Aggregates training data by word_id (answer: mean) to filter out human noise (e.g., blinking, skipping) and learn the "universal" mean reading time.
• Hyperparameters: Tuned for maximum generalizability with n_estimators=2000, learning_rate=0.005, num_leaves=50, and strict regularization (reg_alpha=0.1, reg_lambda=0.1).

🛠️ Installation & Usage

1. Clone the repository:

   git clone https://github.com/ValentinB01/Eye-Tracking-Based-Reading-Time-Prediction.git

2. Install the required dependencies:

   pip install pandas numpy lightgbm wordfreq spacy transformers torch
   python -m spacy download ro_core_news_sm

3. Ensure your train_data.csv and test_data.csv are in the project root.

4. Run the Jupyter Notebook EyeTrackingPrediciton.ipynb sequentially to generate the final submission.csv.