This library implements all techniques to manipulate attention mechanisms, including attention-head sparsification and weight manipulations, that were used in Some Attention is All You Need for Retrieval
For questions, please reach out to the corresponding author in the mentioned paper.