PANews reported on February 18 that the DeepSeek team recently released a technical paper titled "Native Sparse Attention: Hardware-aligned and Natively Trainable Sparse Attention Mechanism", which introduced their proposed NSA (Natively Sparse Attention) mechanism. NSA combines algorithmic innovation and hardware optimization to achieve efficient long text modeling. Its core innovations include:

1. A dynamic hierarchical sparse strategy that combines coarse-grained token compression with fine-grained token selection to preserve global context information and local accuracy;

2. Significantly accelerate computations through balanced algorithm design and modern hardware optimization;

3. Support end-to-end training, reducing pre-training computational costs while maintaining model performance.

Experimental results show that NSA performs well in areas such as long text tasks and instruction reasoning, especially in 64k-length sequence processing, achieving significant acceleration of decoding, forward propagation and backpropagation.