🏆 3rd Place Overall · 🥇 1st Place Nuclear Segmentation (PUMA Track 1)
This repository contains the implementation of HoverLink, a deep learning approach for the Panoptic Segmentation of Nuclei and Tissue in Advanced Melanoma (PUMA) challenge.
HoverLink is a dual-branch architecture designed for the joint segmentation of tissue and nuclei.
- Backbone: Two parallel U-Net branches using ConvNeXt-Base backbones pretrained on ImageNet.
- Structure:
- Branch A: Dedicated to tissue segmentation.
- Branch B: Dedicated to nuclear segmentation (based on the HoVerNeXt architecture).
- Feature Linkage: Inspired by PanopTILs, the model links these branches via bottleneck feature concatenation. A learned feature transformation matrix further refines the nuclear output by integrating weighted per-pixel features from the tissue branch into the final activation of the nuclear branch.
preprocessing/: Utilities for training data preprocessing.hp_search.ipynb: Notebook for hyperparameter optimization.training.py: Main training script.inference.py: Script for running inference.
-
Input Pipeline: The model is trained on
$1024 \times 1024$ patches extracted from the center of larger$2048 \times 2048$ regions. -
Augmentation: Spatial augmentations, including mirroring, translation, scaling, rotation, shearing, and elastic deformations, are applied to the full
$2048 \times 2048$ region prior to cropping. Color jitter in the HED color space is applied to mitigate staining variability. - Sampling: Weighted sampling is used during training, with weights inversely proportional to the area of each tissue class.
- Normalization: Inputs are normalized per RGB channel using ImageNet mean and standard deviation.
This project was developed as part of the Panoptic Segmentation of nUclei and tissue in advanced MelanomA (PUMA) challenge [PUMA Challenge 2024].
This project includes code from the following sources:
- HoverNeXt – Licensed under the GNU General Public License (GPL). Any portions of this project incorporating HoverNeXt code are subject to the terms of the GPL.
- muTILS – Licensed under the MIT License. Code from muTILS retains its original MIT license and copyright notice.
For details on the respective licenses, please refer to: