Official implementaiton of RefAM: Attention Magnets for Zero-Shot Referral Segmentaiton
-
Updated
Feb 6, 2026 - Jupyter Notebook
#
attention-sinks
Here are 2 public repositories matching this topic...
Faithful from-scratch PyTorch reproduction of OpenAI's GPT-OSS architecture (sliding/full attention alternation, learned attention sinks, YaRN 128K, top-2-of-8 MoE), scaled to Chinchilla-optimal 502M total / 247M active training on a single A100 80GB
-
Updated
Jun 29, 2026 - Python
Improve this page
Add a description, image, and links to the attention-sinks topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the attention-sinks topic, visit your repo's landing page and select "manage topics."