Tag: Efficiency
All the articles with the tag "Efficiency".
-
HINT: Hypernetwork Approach to Training Weight Interval Regions in Continual Learning
Published: at 11:21 AM
65.63 🤔HINT proposes a continual learning framework using interval arithmetic in embedding space with a hypernetwork to generate target network weights, achieving improved scalability and non-forgetting guarantees over InterContiNet while outperforming several benchmarks, though struggling with complex datasets.
-
GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance
Published: at 11:18 AM 65.19 🤔GuidedQuant通过整合最终损失梯度信息并保留输出通道内权重依赖性，结合LNQ算法显著提升了大型语言模型在权重和激活量化下的性能，实现了更高效的后训练量化。
-
Exploring the Role of Diversity in Example Selection for In-Context Learning
Published: at 09:33 AM 64.87 🤔本文提出基于多样性的上下文学习（DICL）方法，通过最大边际相关性（MMR）算法重新排序示例以平衡相关性和多样性，在多个数据集和大型语言模型上实现了约70%的下游任务性能提升或维持。
-
SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillation
Published: at 11:15 PM 64.11 🤔本文提出了 SmallPlan 框架，通过结合 LLM 指导的蒸馏、模拟环境反馈的 SFT 和 RL，训练轻量级的小型语言模型 (SLM) 进行高效的机器人高层路径规划，使其在资源受限的边缘设备上实现接近大型模型 (LLM) 的性能。
-
Adaptive Layer-skipping in Pre-trained LLMs
Published: at 04:28 PM 62.55 🤔本文提出FlexiDepth方法，通过插件式路由器和适配器实现预训练LLM的自适应层跳过，提高计算效率同时保持生成性能，并通过实验揭示了token类型对计算需求的影响。