Tag: Long-Term Dependencies

All the articles with the tag "Long-Term Dependencies".

• MoM: Linear Sequence Modeling with Mixture-of-Memories

  Published: 8 May, 2025 |  at  06:19 PM

  

  89.33 🤔

  The Mixture-of-Memories (MoM) architecture introduces multiple independent memory states with a routing mechanism to enhance memory capacity and reduce interference in linear sequence modeling, achieving significant performance gains over other linear models on recall-intensive tasks and nearing Transformer performance at larger scales while maintaining efficiency.