Skip to content

itzjunayed/CSE499-Project-IdentiFind

Repository files navigation

IdentiFind: A Knowledge Base-Free Framework for Person Profile Disambiguation and Generation

Senior Design Project (CSE499A & CSE499B)
Department of Electrical & Computer Engineering
North South University, Bangladesh
Spring 2025 (Data Collection) + Summer 2025 (System Development)


Table of Contents


Overview

IdentiFind is a comprehensive two-phase research project that addresses the challenging problem of person profile disambiguation and biographical generation for individuals with limited online presence. The project evolved through two semesters of intensive work:

The Journey

CSE499A (Spring 2025)          →          CSE499B (Summer 2025)
─────────────────────                     ─────────────────────
Data Collection                           System Development
Web Scraping (32K+ entries)               Face Recognition Pipeline
Dataset Validation (9.4K profiles)        NLP & LLM Integration
GPT-4o-mini Summaries                     Multi-Source Validation
                                          Comprehensive Evaluation

The Problem

Traditional multimodal entity linking systems face critical limitations:

  1. Heavy reliance on knowledge bases that often lack information about ordinary individuals
  2. Vulnerability to noisy, misleading, or contradictory information across web sources
  3. Inability to handle zero-shot scenarios for unknown entities
  4. Poor performance on individuals with weak digital footprints
  5. Existing search engines (Google Lens) require clear, individual portraits and fail with group photos

Our Complete Solution

IdentiFind provides an end-to-end solution requiring only:

  • ✅ A single input image (individual portrait or group photo)
  • ✅ Unlabeled web links (automatically retrieved via FaceCheck API)

Key Innovation: Zero-shot biographical profiling without pre-existing knowledge bases, achieving high-confidence profiles through multi-source validation and Natural Language Inference.


Team

Name Student ID
Tamim Ishrak Sanjid 2122405642
Md. Shakib Shahariar Junayed 2121095042
MD. Sakib Sami 2121422042
Md. Fahim Morshed 2122279642

Faculty Advisor: Dr. Nabeel Mohammed
Associate Professor, Department of Electrical & Computer Engineering

Research Assistants: Md. Mohibur Rahman Nabil, Afsara Waziha


Results & Evaluation

Processing Statistics

Face2Profile Benchmark Evaluation:

Metric Mistral-7B DeepSeek-7B
Total Profiles 10,000 10,000
Successfully Processed 3,474 (34.7%) 3,474 (34.7%)
Failed (Anti-scraping) 6,526 (65.3%) 6,526 (65.3%)
Avg. Processing Time 3.2 min/profile 3.5 min/profile
Total Processing Time 185 hours 202 hours

Entity Recognition Performance

Individual Component Matching:

Match Type Mistral-7B DeepSeek-7B Difference
Full Name 2,329 (67.0%) 2,274 (65.5%) +55 (+2.4%)
First Name 3,130 (90.1%) 3,066 (88.3%) +64 (+2.1%)
Last Name 3,020 (86.9%) 3,062 (88.1%) -42 (-1.4%)
First + Last 2,440 (70.2%) 2,392 (68.8%) +48 (+2.0%)

Professional Information Extraction:

Match Type Mistral-7B DeepSeek-7B Difference % Improvement
Name + Company 1,558 (44.8%) 1,483 (42.7%) +75 +5.1%
Name + Job 957 (27.5%) 1,383 (39.8%) -426 +44.5%
Name + Company + Job 433 (12.5%) 606 (17.4%) -173 +40.0%

Key Finding: DeepSeek-7B significantly outperforms Mistral-7B in professional context extraction


Comprehensive Metrics Comparison

Metric Category Metric Mistral-7B DeepSeek-7B Winner
Lexical Quality ROUGE-1 0.500 0.472 Mistral
ROUGE-2 0.243 0.197 Mistral
ROUGE-L 0.315 0.272 Mistral
Jaccard Similarity 0.312 0.272 Mistral
METEOR 0.359 0.337 Mistral
Semantic BERTScore 0.897 0.901 DeepSeek
Length Ratio 0.732 0.731 Tie
Precision (BLEU) BLEU-1 0.347 0.323 Mistral
BLEU-2 0.160 0.123 Mistral
BLEU-4 0.068 0.043 Mistral
C-BLEU-1 0.313 0.295 Mistral
C-BLEU-2 0.145 0.113 Mistral
C-BLEU-4 0.062 0.039 Mistral
Entity Coverage ECS_overall 0.467 0.530 DeepSeek
ECS_noname 0.290 0.425 DeepSeek
ECS_nojob 0.645 0.634 Mistral
Diversity Bigram Diversity 0.981 0.994 DeepSeek
Trigram Diversity 0.992 0.995 DeepSeek
Semantic Diversity 1.075 1.044 Mistral
Vocab Overlap 0.263 0.217 Mistral

Model Specialization Analysis

Mistral-7B Strengths:

  • Superior linguistic coherence (ROUGE-1: 0.500 vs 0.472)
  • Better basic name extraction (67.0% vs 65.5%)
  • Higher lexical precision across all BLEU metrics (58% better BLEU-4)
  • Enhanced semantic diversity (1.075 vs 1.044)
  • Better vocabulary richness (26.3% vs 21.7% overlap)

DeepSeek-7B Strengths:

  • Exceptional professional info extraction (44.5% better name+job matching)
  • Superior job title identification (46.6% improvement in ECS_noname)
  • Higher overall entity coverage (ECS: 0.530 vs 0.467 = +13.5%)
  • Better complex entity combinations (40.0% improvement in name+company+job)
  • Stronger semantic understanding (BERTScore: 0.901 vs 0.897)

Visualization: Key Findings

Entity Extraction Performance:

Name + Job Title Extraction
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Mistral-7B:  ████████████░░░░░░░░░░░░░░░░░░  27.5% (957)
DeepSeek-7B: █████████████████░░░░░░░░░░░░░  39.8% (1,383) ⭐

Improvement: +44.5%

Lexical Quality (ROUGE-1):

ROUGE-1 Score (Higher = Better)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Mistral-7B:  ██████████████████████████░░░░  0.500 ⭐
DeepSeek-7B: ████████████████████████░░░░░░  0.472

Difference: +5.9%

Overall Entity Coverage (ECS):

Entity Coverage Score
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Mistral-7B:  ███████████████████░░░░░░░░░░░  0.467
DeepSeek-7B: █████████████████████░░░░░░░░░  0.530 ⭐

Improvement: +13.5%

Complementary Architecture Insight

Decision Matrix: When to Use Each Model

Use Case Recommended Model Reason
Professional recruitment DeepSeek-7B Superior job title + company extraction
Biography writing Mistral-7B Better linguistic quality and coherence
LinkedIn profile generation DeepSeek-7B Excellent professional context understanding
Academic profiles Mistral-7B More fluent summaries
Legal/medical verification DeepSeek-7B Higher factual accuracy (ECS)
Creative writing Mistral-7B Better vocabulary diversity
Data extraction tasks DeepSeek-7B Superior entity extraction
Public communication Mistral-7B More readable output

Ethical Considerations

Privacy & Consent

IdentiFind processes publicly available data only. However, users must be aware of ethical responsibilities:

✅ Acceptable Use:

  • Legitimate identity verification (with consent)
  • Missing person identification (law enforcement)
  • Academic research (IRB approved)
  • Professional background checks (with authorization)

❌ Prohibited Use:

  • Unauthorized surveillance or stalking
  • Discriminatory profiling or hiring practices
  • Invasion of privacy without consent
  • Malicious impersonation or identity theft

Data Collection Ethics (CSE499A)

CSE499A Dataset (9,472 profiles):

  • ✅ Collected from publicly accessible professional platforms
  • ✅ No scraping of private social media (Facebook, Instagram)
  • ✅ Respect for robots.txt and platform ToS
  • ✅ No storage of sensitive PII (SSN, credit cards, etc.)

Bias & Fairness

We acknowledge and actively work to mitigate biases in:

  1. Facial Recognition Systems

    • Demographic disparities (see Limitations section)
    • Age bias (better for 25-55 age range)
    • Ongoing bias auditing framework development
  2. Web Data Availability

    • Digital divide (well-documented individuals favored)
    • Geographic bias (US profiles more accessible)
    • Language bias (English-centric processing)
  3. Language Processing

    • English-only support (planned multilingual expansion)
    • Cultural context understanding
    • Name recognition across cultures

Transparency & Accountability

We commit to:

  • 🔍 Open-source codebase for community review
  • 📊 Public bias audit reports (quarterly)
  • 📝 Detailed methodology documentation
  • 🔐 Security vulnerability disclosure program

Users have the right to:

  • 🚫 Opt-out of profiling (if detected)
  • 📜 Request explanation of generated profile
  • ✏️ Correct inaccurate information
  • 🗑️ Request profile deletion

UN Sustainable Development Goals

This project aligns with:

SDG #9: Industry, Innovation and Infrastructure

  • Innovation in AI-powered identity verification
  • Accessible technology for ordinary individuals
  • Scalable infrastructure for information processing

SDG #16: Peace, Justice and Strong Institutions

  • Legitimate identity verification for legal systems
  • Missing person identification capabilities
  • Combating misinformation through source validation

About

IdentiFind: A Knowledge Base-Free Framework for Person Profile Disambiguation and Generation

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages