Tag: Long Context

All the articles with the tag "Long Context".

• Recall with Reasoning: Chain-of-Thought Distillation for Mamba's Long-Context Memory and Extrapolation

  Published: 8 May, 2025 |  at  06:13 PM

  

  86.84 🤔

  This paper proposes Recall with Reasoning (RwR), a method that enhances Mamba's long-context memory and extrapolation by distilling chain-of-thought summarization from a teacher model, achieving significant performance improvements on LONGMEMEVAL and HELMET benchmarks while preserving short-context capabilities.

• Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions

  Published: 7 May, 2025 |  at  09:31 AM

  

  85.11 🤔

  本文通过提出AI记忆系统的分类（参数、上下文结构化和非结构化）和六种基本操作（整合、更新、索引、遗忘、检索、压缩），系统化地综述了长期记忆、长上下文、参数修改和多源记忆等研究主题，并展望了未来方向。

• ICLR: In-Context Learning of Representations

  Published: 7 May, 2025 |  at  08:41 AM

  

  84.18 🤔

  本文通过上下文图追踪任务揭示了大型语言模型能随上下文规模增加而突现地重组概念表示以适应新语义，并提出能量最小化假设解释这一过程。

• Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models

  Published: 8 May, 2025 |  at  06:19 PM

  

  83.98 🤔

  This paper introduces a recursive summarization method to enhance long-term dialogue memory in LLMs, achieving marginal quantitative improvements and notable qualitative gains in consistency and coherence across multiple models and datasets.

• Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation

  Published: 6 May, 2025 |  at  01:18 AM

  

  81.62 🤔

  本文提出DPE，一种无需训练的长文本外推方法，通过检测RoPE不同维度组的有效相对距离并识别关键维度，有选择地调整这些关键维度的位置索引，显著扩展了LLM的上下文窗口并提升了长文本任务性能。