https://github.com/jonsmirl/ssa
Subquadratic Sparse Attention (SSA)
Content-routed sparse attention for long contexts — exact softmax over a selected subset of keys (a bounded, dropped-mass error versus full attention), with the theory that says when it is sound and the experiments that measure it.
https://github.com/jonsmirl/ssa
Subquadratic Sparse Attention (SSA)
Content-routed sparse attention for long contexts — exact softmax over a selected subset of keys (a bounded, dropped-mass error versus full attention), with the theory that says when it is sound and the experiments that measure it.