Tag: Catastrophic Forgetting
All the articles with the tag "Catastrophic Forgetting".
-
Recurrent Knowledge Identification and Fusion for Language Model Continual Learning
Published: at 11:26 AM
88.00 🤔本文提出Recurrent-KIF框架，通过内外循环机制动态估计参数重要性并迭代融合新旧知识，在持续学习中有效缓解灾难性遗忘并促进知识转移，实验验证其在多个大语言模型上的性能优势。
-
LoKI: Low-damage Knowledge Implanting of Large Language Models
Published: at 11:23 AM 87.82 🤔本文提出LoKI，一种参数高效微调框架，通过分析Transformer FFN层的知识存储机制和层平衡参数选择策略，在下游任务适应和预训练知识保留之间实现了竞争性平衡。
-
Replay to Remember: Retaining Domain Knowledge in Streaming Language Models
Published: at 04:30 PM 65.14 🤔本文通过结合LoRA和轻量级重放机制的方法，在流式学习条件下帮助大型语言模型减轻灾难性遗忘，同时实现了实时域适应。