No AI summary available for this article.
Why It Matters
Triangular attention is a core computation in AlphaFold3-style biomolecular models, with cubic cost in token count.
Provenance
Discovered via ArXiv and published by ArXiv.
Key Claims
Original description
Triangular attention is a core computation in AlphaFold3-style biomolecular models, with cubic cost in token count. Its shared pair bias adds a gradient reduction across attention slices to the usual reductions over queries and keys. The open-source backends we examine handle these reductions through repeated probability recomputation, floating-point atomics, or full score-gradient storage. Separately, computing the softmax backward correction from BF16-rounded forward outputs loses numerical precision. We present TurboPairFormer, a triangular attention implementation for NVIDIA Hopper GPUs th...
Discovered via ArXiv
Research papers and preprints from arXiv.
Publisher: arxiv.org
ID: http://arxiv.org/abs/2610.05854v1 · Indexed 42 minutes ago