Posts
All the articles I've posted.
-
AdaReasoner: Adaptive Reasoning Enables More Flexible Thinking
Published: at 11:22 AM
87.06 🤔AdaReasoner通过强化学习框架自适应调整大型语言模型的推理配置（生成温度、推理步骤数和指令格式），在多样化任务上显著优于固定配置的基线方法，展现了快速收敛和分布外鲁棒性。
-
MoL for LLMs: Dual-Loss Optimization to Enhance Domain Expertise While Preserving General Capabilities
Published: at 11:10 AM 87.03 🤔本文提出MoL框架，通过对领域语料使用CE损失和对通用语料使用KL散度损失的双重优化策略，显著提升大型语言模型的领域专长，同时有效保留通用能力，并在医学领域任务中取得优异表现。
-
ATLAS: Learning to Optimally Memorize the Context at Test Time
Published: at 11:22 AM 86.98 🤔本文提出Atlas，一种高容量长期内存模块，通过滑动窗口Omega规则和Muon优化器优化上下文记忆，在语言建模和长上下文理解任务中显著优于Transformer和现代RNN。
-
Not All Correct Answers Are Equal: Why Your Distillation Source Matters
Published: at 11:11 AM 86.97 🤔本文通过从三个顶尖大语言模型中提炼189万推理数据，系统研究了提炼源对学生模型性能的影响，发现AM-Thinking-v1提炼数据在多个推理基准上显著提升学生模型表现，并展现出适应性生成长度特性。
-
Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization
Published: at 11:08 AM 86.94 🤔This paper introduces a fine-tuning strategy for LLMs that leverages the unequal importance of attention matrices and customized learning rates to enhance efficiency, demonstrating through theoretical analysis and experiments on GLUE benchmarks that fine-tuning only Wq and Wv with higher learning rates for Wv can match or exceed full fine-tuning performance with fewer parameters.