Redrob Candidate Ranker

An AI-powered candidate ranking system for the INDIA RUNS Data & AI Challenge (Track 1). Ranks 100,000 candidates across 5 dimensions using local ML inference — zero network calls at runtime.

Overview

Redrob Ranker parses job descriptions and scores candidates using a multi-dimensional approach:

• Skills Match (30%): TF-IDF cosine similarity with proficiency weighting
• Career Trajectory (25%): Role relevance and progression analysis
• Platform Signals (20%): Redrob behavioral signals (GitHub, response rates, etc.)
• Education Tier (15%): Institution tier and field relevance
• Experience Fit (10%): Years of experience and work mode alignment

Architecture

┌─────────────────┐
│  JD Parser      │ → Extract keywords, target YOE, work mode
└────────┬────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────┐
│              Score Aggregator                            │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐  │
│  │ Skills   │ │ Career   │ │ Signals  │ │ Education│  │
│  │ Scorer   │ │ Scorer   │ │ Scorer   │ │ Scorer   │  │
│  └──────────┘ └──────────┘ └──────────┘ └──────────┘  │
│                    ┌──────────┐                          │
│                    │Experience│                          │
│                    │ Scorer   │                          │
│                    └──────────┘                          │
└─────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────┐
│  Top-100 Heap   │ → Ranked candidates with reasoning
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ submission.csv  │ → Validated output
└─────────────────┘

Quick Start

Reproducible Command

python rank.py --candidates ./candidates.jsonl --jd ./job_description.docx --out ./divyanshu-redrob-ranker.csv

Note: If --out is not specified, the output filename is automatically generated from submission_metadata.yaml's team_name field.

Installation

pip install -r requirements.txt

Basic Usage

python rank.py --candidates ./candidates.jsonl --jd ./job_description.docx

The output filename will be auto-generated from submission_metadata.yaml (e.g., divyanshu-redrob-ranker.csv).

Custom Weights

python rank.py --candidates ./candidates.jsonl --jd ./job_description.docx \
  --w-skills 0.30 --w-career 0.25 --w-signals 0.20 --w-edu 0.15 --w-exp 0.10

Verbose Mode

python rank.py --candidates ./candidates.jsonl --jd ./job_description.docx --verbose

Disable Honeypot Filter (Debugging)

python rank.py --candidates ./candidates.jsonl --jd ./job_description.docx --no-honeypot-filter

Score Formula

final_score = (skills × 0.30) + (career × 0.25) + (signals × 0.20) + 
              (education × 0.15) + (experience × 0.10)

Skills Score

coverage × 0.6 + avg_proficiency_weighted × 0.4 + assessment_bonus

• Coverage: Fraction of JD keywords matched
• Proficiency weights: beginner=0.2, intermediate=0.5, advanced=0.8, expert=1.0
• Trust multiplier: log1p(endorsements) / log1p(100) × min(1, duration_months/24)

Career Score

weighted_relevance × 0.6 + progression × 0.2 + consistency × 0.2 + industry_bonus

• Relevance: TF-IDF cosine similarity of role title/description to JD
• Recency weights: most recent × 0.5, second × 0.3, third × 0.2
• Consistency: Penalize gaps > 6 months between roles

Signals Score

github_activity × 0.25 + response_rate × 0.20 + interview_rate × 0.20 +
offer_rate × 0.15 + completeness × 0.10 + open_to_work × 0.05 + saved × 0.05

Education Score

max_tier + field_relevance_bonus

• Tier weights: tier_1=1.0, tier_2=0.8, tier_3=0.6, tier_4=0.4, unknown=0.35

Experience Score

years_fit + notice_penalty + relocation_bonus + work_mode_bonus

• Years fit: 1 - min(1, |candidate_yoe - target_yoe| / 8)
• Notice penalty: -0.1 if notice_period_days > 90

Performance

• Runtime: ~4 minutes for 100,000 candidates on 8-core CPU, 16GB RAM
• Memory: Uses streaming (ijson) to avoid loading full 487MB dataset
• Optimization:
  • Pre-computed JD TF-IDF vector (computed fresh each run, not cached)
  • Batch processing (1000 candidates/batch)
  • heapq.nlargest for top-100 selection (never sort all 100K)
  • Honeypot filter runs during scoring (cheap checks, no significant perf impact)

File Structure

redrob-ranker/
├── rank.py                    # Main CLI script
├── scorer/
│   ├── __init__.py
│   ├── jd_parser.py           # Parse job_description.docx
│   ├── skills_scorer.py       # Dimension 1: Skills match
│   ├── career_scorer.py       # Dimension 2: Career trajectory
│   ├── signals_scorer.py      # Dimension 3: Platform signals
│   ├── education_scorer.py    # Dimension 4: Education tier
│   ├── experience_scorer.py   # Dimension 5: Experience fit
│   ├── honeypot_filter.py     # Honeypot detection (6 plausibility checks)
│   └── aggregator.py          # Combine all 5 scores
├── sandbox/
│   ├── app.py                 # Streamlit sandbox app
│   └── README.md              # Sandbox deployment instructions
├── ui/
│   └── app/                   # Next.js demo UI
├── requirements.txt           # Core dependencies (pinned versions)
├── requirements-sandbox.txt   # Sandbox dependencies
├── submission_metadata.yaml   # Challenge metadata
├── sample_candidates.json     # Sample data for testing
├── sample_job_description.txt # Sample JD for testing
└── README.md

Dataset Schema

Each candidate in candidates.jsonl has:

{
  "candidate_id": "CAND_0000001",
  "profile": {
    "headline": "string",
    "years_of_experience": 6.5,
    "current_industry": "string"
  },
  "career_history": [...],
  "education": [...],
  "skills": [...],
  "redrob_signals": {
    "github_activity_score": 72.0,
    "recruiter_response_rate": 0.76,
    "skill_assessment_scores": {"Python": 88},
    ...
  }
}

Submission Format

Output submission.csv with:

• Header: candidate_id,rank,score,reasoning
• Exactly 100 rows (ranks 1-100)
• Scores non-increasing by rank
• Tie-break: candidate_id ascending

Validation

python validate_submission.py submission.csv

Constraints

• ✅ Zero network calls during ranking
• ✅ All models pre-loaded locally
• ✅ Runtime < 5 minutes on 16GB RAM CPU
• ✅ Output passes validation script
• ✅ Exactly 100 ranked candidates
• ✅ Honeypot filter enabled by default (excludes impossible profiles)
• ✅ Tie-breaking by candidate_id ascending (deterministic)

Known Limitations

Honeypot Filter

• The honeypot filter uses heuristic checks that may have false positives/negatives:
  • False positive risk: Candidates with unusual but legitimate career paths (e.g., rapid promotions, concurrent part-time roles) may be flagged
  • False negative risk: Sophisticated honeypots designed to pass plausibility checks may not be caught
• The 2+ check threshold and 48-month severe gap rule balance precision vs recall
• Audit log (honeypot_audit.jsonl) allows manual review of flagged candidates

Scoring Limitations

• TF-IDF similarity doesn't capture semantic meaning beyond keyword overlap
• Career trajectory scoring assumes linear progression; non-linear paths may be undervalued
• Education tier scoring is based on institution reputation lists, which may be incomplete
• Signals scoring depends on Redrob platform data quality and completeness

Performance

• Runtime is optimized for 100K candidates but may exceed 5 minutes on slower hardware (<4 cores)
• Memory usage scales with batch size; default 1000 is safe for 16GB RAM

Sandbox

A Streamlit sandbox is available for testing with small samples:

pip install -r requirements-sandbox.txt
streamlit run sandbox/app.py

See sandbox/README.md for deployment instructions to Streamlit Cloud.

License

MIT License

Challenge

INDIA RUNS Data & AI Challenge — Track 1 Organized by Hack2Skill / Redrob