Tag: Transformer Alternative
All the articles with the tag "Transformer Alternative".
-
From Attention to Atoms: Spectral Dictionary Learning for Fast, Interpretable Language Models
Published: at 04:33 PM
75.71 🤔本文提出光谱字典生成模型（SDGM），通过学习全局傅里叶字典和 token 混合系数替换自注意力机制，实现 O(KL) 复杂度的高效语言建模，并在基准数据集上取得竞争性 perplexity 和显著的资源节省。