Tag: Sequence Modeling

All the articles with the tag "Sequence Modeling".

• Competition Dynamics Shape Algorithmic Phases of In-Context Learning

  Published: 9 May, 2025 |  at  11:08 AM

  

  85.25 🤔

  This paper introduces a synthetic sequence modeling task using finite Markov mixtures to unify the study of in-context learning (ICL), identifying four competing algorithms that explain model behavior and phase transitions, thus offering insights into ICL's transient nature and phenomenology.

• Test-time regression: a unifying framework for designing sequence models with associative memory

  Published: 7 May, 2025 |  at  12:16 AM

  

  71.71 🤔

  本文提出一个基于测试时回归的统一框架，通过将关联回忆形式化为回归问题，推导出多种序列模型（如线性注意力、状态空间模型、softmax注意力），并通过合成实验验证其回归能力，同时提出高阶注意力泛化。