Tag: Representation Learning
All the articles with the tag "Representation Learning".
-
Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon
Published: at 10:59 AM
85.36 🤔This paper introduces a taxonomy of language model memorization into recitation, reconstruction, and recollection, demonstrating through experiments with Pythia models that different factors influence each category, with a taxonomy-based predictive model outperforming baselines in predicting memorization likelihood.
-
Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions
Published: at 09:31 AM 85.11 🤔本文通过提出AI记忆系统的分类（参数、上下文结构化和非结构化）和六种基本操作（整合、更新、索引、遗忘、检索、压缩），系统化地综述了长期记忆、长上下文、参数修改和多源记忆等研究主题，并展望了未来方向。
-
Unveiling the Mechanisms of Explicit CoT Training: How CoT Enhances Reasoning Generalization
Published: at 11:21 PM 85.04 🤔本文通过控制实验、内部机制分析和理论推导，揭示了显式思维链（CoT）训练通过形成二阶段泛化电路显著提升大型语言模型的分布内（ID）和分布外（OOD）推理泛化能力，并验证了其在噪声数据下的鲁棒性。
-
ICLR: In-Context Learning of Representations
Published: at 08:41 AM 84.18 🤔本文通过上下文图追踪任务揭示了大型语言模型能随上下文规模增加而突现地重组概念表示以适应新语义，并提出能量最小化假设解释这一过程。
-
Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
Published: at 01:18 AM 81.62 🤔本文提出DPE，一种无需训练的长文本外推方法，通过检测RoPE不同维度组的有效相对距离并识别关键维度，有选择地调整这些关键维度的位置索引，显著扩展了LLM的上下文窗口并提升了长文本任务性能。