Conversational Document Retrieval (BM25 + MonoT5)

University of Stavanger — Information Retrieval and Text Mining (DAT640)

Overview

Built a multi-stage retrieval pipeline on the MS MARCO corpus (8.8M passages) combining BM25 for initial candidate retrieval, MonoT5 for neural re-ranking, and hybrid T5 query rewriting for conversational context. Improved NDCG@3 from 0.098 to 0.274 over the BM25 baseline.

Pipeline

Query → T5 Query Rewriting → BM25 (First-stage Retrieval) → MonoT5 Re-ranking → Ranked Results

Results

Method	NDCG@3
BM25 Baseline	0.098
BM25 + MonoT5 Re-ranking	0.274
Improvement	+180%

Tech Stack

• Python
• BM25 (rank_bm25)
• MonoT5 (Hugging Face Transformers — castorini/pygaggle)
• T5 Query Rewriting (castorini/t5-base-canard)
• SQLite (inverted index)
• NLTK
• Pandas

Dataset

MS MARCO Passage Ranking dataset (8.8M passages) — available at microsoft.github.io/msmarco.

Dataset not included due to size. Download collection.tsv and place in the project root.

Files

File	Description
Baseline Code.py	BM25 baseline retrieval implementation
Advanced Method Code.py	Full pipeline — BM25 + MonoT5 re-ranking
Query ReWriting.py	T5-based query rewriting for conversational context
queries_train.csv	Training queries
TREC_RUNFILE.txt	TREC-format evaluation output
Final_DAT640___Group_1.pdf	Full project report

How to Run

Note: Advanced Method Code.py and Query ReWriting.py were developed in Google Colab and contain Colab-specific commands (e.g., pip install, drive.mount). They are best run in a Colab environment.

Option 1 — Google Colab (Recommended)

Upload the .py files to Google Colab, mount your Drive with the dataset, and run cell by cell.

Option 2 — Local (Baseline only)

pip install -r requirements.txt
python "Baseline Code.py"

Note on pygaggle: The MonoT5 re-ranker depends on castorini/pygaggle, which is not in requirements.txt because it requires a manual Git clone and is only practical in a Colab/GPU environment. See Advanced Method Code.py for the full install steps.

For the full MonoT5 pipeline, use Colab with GPU for reasonable performance.