Text Summarizer — Fine-tuned T5 on SAMSum

Fine-tuning t5-small(Model size 60.5M params) for dialogue summarization using the SAMSum dataset. Trained on Google Collab

Overview

This project fine-tunes a T5-small model to summarize conversations/dialogues into concise summaries. The model is trained on 4,000 dialogue-summary pairs from the SAMSum corpus.

Dataset

Split	Samples used
Train	4,000 (sampled)
Validation	500 (sampled)

Source: knkarthick/samsum on Hugging Face.

Model

• Base model: t5-small (~60M parameters)
• Task: Sequence-to-sequence summarization
• Tokenizer: T5Tokenizer with input max length 256, target max length 64

Training Setup

Parameter	Value
Epochs	6
Batch size	8
Precision	bfloat16
Warmup steps	500
Weight decay	0.01

Requirements

pip install transformers datasets torch pandas

Usage

1. Clone the repo and open text_summarizer.ipynb in Jupyter or VS Code.
2. Run all cells in order — the notebook handles data loading, preprocessing, tokenization, and training.
3. Trained checkpoints are saved to ./results/.

# Quick inference after training
from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("./results/checkpoint-best")
model = T5ForConditionalGeneration.from_pretrained("./results/checkpoint-best")

dialogue = "Hannah: Hey, can we meet tomorrow? Alex: Sure, what time? Hannah: Around 3pm?"
inputs = tokenizer("summarize: " + dialogue, return_tensors="pt", max_length=256, truncation=True)
summary_ids = model.generate(inputs["input_ids"], max_length=64)
print(tokenizer.decode(summary_ids[0], skip_special_tokens=True))

Project Structure

text_summarizer/
├── text_summarizer.ipynb          # Main training notebook
├── train.csv            # Preprocessed cleaned training data
├── validation.csv       # Preprocessed  cleaned validation data
└── results/             # Saved model checkpoints

Notes

• Trained on Google COLLAB