Paper Digest
RSS FeedPaper Digest is a totally free and open-source project that automatically generates a personalized paper digest for you based on your preferences. This project is powered by lots of great projects, and many thanks to them.
Recent Posts
-
Attention Retrieves, MLP Memorizes: Disentangling Trainable Components in the Transformer
Published: at 11:59 AM
93.14 🤔本文通过冻结Transformer组件并提出MixiT模型，揭示了自注意力机制在检索和语言建模中的输入依赖性必要性，以及MLP层在记忆中的主导作用，强调了架构异质性对任务解决的重要性。
-
TL;DR: Too Long, Do Re-weighting for Effcient LLM Reasoning Compression
Published: at 11:22 AM 92.10 🤔本文提出TLDR方法，通过动态再加权系统1和系统2推理数据，显著压缩大型语言模型的推理输出token数量（约40%），同时在多难度数学任务上基本保持准确性。
-
Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning
Published: at 11:24 AM 91.44 🤔本文提出双向对齐（BiAlign）方法，通过对齐学生模型与教师模型的令牌级输出分布和输入偏好，显著提升了学生模型的上下文学习能力，并在多种任务上取得了优于基线的结果。