RareSight

RareSight is a reference implementation for rare disease ranking from phenotype terms. It combines a heterogeneous phenotype, gene, and disease knowledge graph with graph embeddings, few-shot learning, retrieval, and evaluation baselines.

This repository is the most research oriented project in this workspace. It is written as an experimental pipeline, not as a finished web product.

What it includes

• HPO ontology parsing from OBO files
• HPOA disease to phenotype loading
• Disease level train, seen-test, and unseen-test splitting
• Leakage checks for unseen disease evaluation
• Heterogeneous knowledge graph construction
• Plain PyTorch R-GCN style link prediction pretraining
• Prototypical few-shot patient encoder
• TF-IDF disease retrieval baseline
• Ontology similarity baseline
• RareSight model ablations
• Bootstrap confidence intervals and paired significance testing
• Smoke tests for the full pipeline

Tech stack

• Python
• PyTorch
• NumPy, SciPy, pandas
• scikit-learn
• NetworkX
• PyYAML
• pytest

Repository structure

configs/       Default experiment configuration
data/sample/   Small sample ontology files for smoke tests
scripts/       Download, preprocess, evaluation, and baseline scripts
src/           Pipeline source code
tests/         Smoke and integration tests
results/       Evaluation outputs

Quickstart

Create and activate a virtual environment:

python -m venv .venv
.venv\Scripts\activate

Install dependencies:

pip install -r requirements.txt

Run the smoke tests:

pytest tests -v -s

Full experiment flow

The full run needs real benchmark data. Start with the scripts in order:

bash scripts/01_download_data.sh --full
bash scripts/02_preprocess.sh
bash scripts/03_run_full_evaluation.sh

The evaluation writes results to results/full_run.json and detailed rankings to results/full_run_with_rankings.json.

Deployment

Do not deploy this like a static website. For recruiters, the best free presentation is:

• GitHub for the code
• A short technical write-up in the README
• Optional Colab notebook or Kaggle notebook for a small demo run
• Optional Hugging Face Space only if you build an inference UI around a trained artifact

Recruiter note

This is the strongest project in this folder from a computer science perspective. It shows architecture, ML evaluation, baselines, tests, and research discipline. Put it near the top of your resume, but make sure the README has real results once you run the full pipeline.

Current status

The codebase includes smoke tests and sample data. The checked-in sample flow is meant to verify wiring, not to claim research performance. Do not report smoke-test numbers as final model results.

Limitations

• Full-scale results are not included yet
• Some benchmark downloads require manual access or accounts
• The fine-tuned LLM baseline needs stronger compute than a basic laptop
• This is a research pipeline, not an end-user medical tool