Twitter Sentiment Analysis

A comprehensive machine learning project for classifying Twitter sentiment (positive/negative) using multiple feature extraction techniques and classifier models.

Project Overview

This project implements a complete machine learning pipeline for sentiment analysis on tweet data. It compares different approaches to feature extraction and classification to identify the best performing model combinations.

Dataset

• Training Data: train_E6oV3lV.csv - 31,962 tweets with sentiment labels (balanced/imbalanced)
• Test Data: test_tweets_anuFYb8.csv - Unlabeled tweets for prediction
• Data Format: CSV files with columns id, tweet, and label (0 = negative, 1 = positive)

Project Structure

twitter_sentiment/
├── data_train.ipynb              # Main notebook with full pipeline
├── train_E6oV3lV.csv             # Training dataset
├── test_tweets_anuFYb8.csv       # Test dataset
├── sub_lreg_bow.csv              # Logistic Regression (BoW) predictions
├── sub_rf_bow.csv                # Random Forest (BoW) predictions
├── sub_svc_bow.csv               # SVM (BoW) predictions
├── sub_xgb_bow.csv               # XGBoost (BoW) predictions
├── sub_xgb_tuned_w2v.csv         # XGBoost tuned (Word2Vec) predictions
└── README.md                      # This file

Methodology

1. Data Preprocessing

The pipeline includes comprehensive text preprocessing:

• Mention Removal: Removes @mentions using regex pattern @[\w]*
• Special Character Cleaning: Removes punctuation, numbers, and special characters
• Short Word Removal: Filters out words with length ≤ 3 characters
• Text Normalization:
  • Tokenization
  • Stemming (using Porter Stemmer)
  • Lowercasification

Sample Processing Flow:

Original: "@user Hey! Check #food out!!!"
After Processing: "check food"

2. Feature Extraction Techniques

The project explores four different feature extraction methods:

Bag of Words (BoW)

• Method: CountVectorizer from scikit-learn
• Parameters:
  • Max features: 1,000
  • Max document frequency: 0.90
  • Min document frequency: 2
  • English stop words removed
• Output Shape: Sparse matrix (31,962 tweets × 1,000 features)

TF-IDF (Term Frequency-Inverse Document Frequency)

• Method: TfidfVectorizer from scikit-learn
• Parameters: Same as BoW
• Advantage: Weighs word importance by document frequency
• Output Shape: Sparse matrix (31,962 tweets × 1,000 features)

Word2Vec (W2V)

• Method: Gensim Word2Vec with Skip-gram model
• Parameters:
  • Vector size: 200 dimensions
  • Context window: 5 words
  • Min count: 2 (minimum word frequency)
  • Negative sampling: 10
  • Training epochs: 20
• Approach: Creates document-level vectors by averaging word embeddings
• Output Shape: Dense matrix (31,962 tweets × 200 features)
• Example: Word similarity analysis shows meaningful relationships:
  • Similar to "dinner": spaghetti, cookout, noodle, spinach
  • Similar to "trump": donald, unfit, unstable, unelected

Doc2Vec (Document to Vector)

• Method: Gensim Doc2Vec with Distributed Memory (PV-DM)
• Parameters:
  • Vector size: 200 dimensions
  • Context window: 5 words
  • Min count: 5 (minimum word frequency)
  • Negative sampling: 7
  • Training epochs: 15
• Advantage: Captures document-level semantic meaning
• Output Shape: Dense matrix (31,962 tweets × 200 features)

3. Classification Models

The project trains four machine learning classifiers:

Logistic Regression

• Simple baseline model
• Fast training and prediction
• Works with all feature types
• Hyperparameter: Decision threshold = 0.3

Support Vector Machine (SVM)

• Kernel: Linear
• Hyperparameter C: Various values tested (1, default)
• Probability: Enabled for probability estimates
• Threshold: 0.3 for binary classification

Random Forest

• Ensemble method using multiple decision trees
• Captures non-linear relationships
• Robust to imbalanced datasets
• Submission: sub_rf_bow.csv

XGBoost

• Gradient boosting method
• High performance on tabular data
• Two versions:
  • Standard XGBoost with BoW features: sub_xgb_bow.csv
  • Tuned XGBoost with Word2Vec features: sub_xgb_tuned_w2v.csv
• Better predictive performance than other models

4. Model Evaluation

• Metric: F1-Score (on validation set, 30% hold-out)
• Threshold: Classification threshold set to 0.3 (adjusted for imbalanced data)
• Validation Strategy: Train-test split with random_state=42 for reproducibility

Results Summary

Model	Feature Type	Status
Logistic Regression	Bag of Words	✓ Submitted
Logistic Regression	TF-IDF	Evaluated
Logistic Regression	Word2Vec	Evaluated
Logistic Regression	Doc2Vec	Evaluated
SVM	Bag of Words	✓ Submitted
SVM	TF-IDF	Evaluated
SVM	Word2Vec	Evaluated
Random Forest	Bag of Words	✓ Submitted
XGBoost	Bag of Words	✓ Submitted
XGBoost (Tuned)	Word2Vec	✓ Submitted

Key Findings:

• BoW features provide consistent performance across models
• Word2Vec embeddings capture semantic relationships better
• XGBoost with tuned hyperparameters achieves best performance
• Simple threshold of 0.3 helps handle class imbalance

Output Files

Submission Files (CSV Format)

All submission files contain two columns: id (tweet id) and label (predicted sentiment):

• sub_lreg_bow.csv - Logistic Regression predictions (BoW)
• sub_svc_bow.csv - SVM predictions (BoW)
• sub_rf_bow.csv - Random Forest predictions (BoW)
• sub_xgb_bow.csv - XGBoost predictions (BoW)
• sub_xgb_tuned_w2v.csv - XGBoost predictions (Word2Vec) - Recommended

Key Libraries

• Data Processing: pandas, numpy
• Text Processing: nltk, re, string
• Feature Extraction: scikit-learn (CountVectorizer, TfidfVectorizer), gensim
• Machine Learning: scikit-learn (LogisticRegression, SVM, RandomForest), XGBoost
• Visualization: matplotlib, seaborn
• Embeddings: gensim (Word2Vec, Doc2Vec)

How to Run

Prerequisites

pip install numpy pandas nltk scikit-learn gensim xgboost matplotlib seaborn

Execution

1. Open data_train.ipynb in Jupyter Notebook or JupyterLab
2. Download required NLTK data (if needed) - models handle this automatically
3. Run all cells sequentially to:
   • Load and explore data
   • Preprocess tweets
   • Extract features using all four methods
   • Train all classifier models
   • Generate predictions and save submission files

Data Analysis Insights

Dataset Characteristics

• Imbalanced Dataset: Class distribution shows more samples of one sentiment
• Tweet Length Distribution: Tweets range from very short to longer format
• Data Quality: Contains URLs, mentions, hashtags, and special characters

Most Important Hashtags

The analysis identifies frequently occurring hashtags indicating discussion topics:

• Negative sentiment tweets show distinct hashtag patterns
• Positive sentiment tweets reveal different conversation themes
• Hashtag frequency analysis provides domain context

Feature Engineering Notes

Threshold Selection

The decision threshold was set at 0.3 (instead of default 0.5) to optimize performance on the imbalanced dataset:

• Improves recall for the minority class
• Better F1-Score overall
• Adjustable based on business requirements (precision vs. recall trade-off)

Dimensionality

• BoW/TF-IDF: 1,000 features (sparsity ~99%)
• Word2Vec/Doc2Vec: 200 dimensions (dense representation)
• Trade-off: BoW is interpretable; embeddings are more compact

Performance Considerations

• Training Time: Word2Vec and Doc2Vec models take significant time for corpus of 32K+ tweets
• Memory Usage: Sparse matrices (BoW/TF-IDF) are memory-efficient
• Inference Speed: Dense embeddings (W2V/D2V) are faster for prediction
• Model Size: Embedding models are smaller than BoW for the same features

Future Improvements

1. Advanced Preprocessing:

   • Handle contractions (e.g., "don't" → "do not")
   • Domain-specific stop words
   • Spelling correction

2. Feature Enhancement:

   • BERT/RoBERTa embeddings
   • Sentiment lexicon features
   • Subword tokenization (BPE)

3. Model Improvements:

   • Neural networks (CNN, LSTM, Transformers)
   • Ensemble methods
   • Hyperparameter tuning with GridSearchCV/RandomizedSearchCV

4. Class Balancing:

   • SMOTE (Synthetic Minority Over-sampling)
   • class_weight parameter in classifiers
   • Stratified cross-validation

5. Evaluation:

   • Cross-validation instead of single train-test split
   • ROC-AUC, Precision-Recall curves
   • Confusion matrix analysis by feature type

Author

Created as a comprehensive machine learning project for Twitter sentiment classification using multiple feature extraction and modeling techniques.

---

Last Updated: 2026
Status: Complete - All models trained and submissions generated