Tag: Long Context

All the articles with the tag "Long Context".

• The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs

  Published: 4 May, 2025 |  at  04:29 PM

  

  85.10 👍

  论文通过大规模实验分析了Transformer LLMs中稀疏注意力的效率-准确性权衡，揭示了长序列下更大稀疏模型的优势，并建立了可推广的缩放定律。

• LIFT: Improving Long Context Understanding of Large Language Models through Long Input Fine-Tuning

  Published: 4 May, 2025 |  at  04:27 PM

  

  82.62 👍

  本文提出LIFT框架，通过长输入微调和Gated Memory适配器提升短上下文LLMs的长上下文理解能力，实验显示显著性能改进。