MiniMax-01: Scaling Foundation Models with Lightning Attention
Paper
•
2501.08313
•
Published
•
301
Attention
-
-
Lizard: An Efficient Linearization Framework for Large Language Models
Paper • 2507.09025 • Published • 18 -
On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective
Paper • 2507.23632 • Published • 6 -
Causal Attention with Lookahead Keys
Paper • 2509.07301 • Published • 21 -
Hybrid Architectures for Language Models: Systematic Analysis and Design Insights
Paper • 2510.04800 • Published • 36 -
Less is More: Recursive Reasoning with Tiny Networks
Paper • 2510.04871 • Published • 497 -
Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention
Paper • 2510.04212 • Published • 23 -
Reactive Transformer (RxT) -- Stateful Real-Time Processing for Event-Driven Reactive Language Models
Paper • 2510.03561 • Published • 24 -
Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
Paper • 2510.19338 • Published • 114 -
Kimi Linear: An Expressive, Efficient Attention Architecture
Paper • 2510.26692 • Published • 117 -
DoPE: Denoising Rotary Position Embedding
Paper • 2511.09146 • Published • 92 -
SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space
Paper • 2511.20102 • Published • 26 -
InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Models
Paper • 2512.08829 • Published • 12