Tag: Fine-tuning
All the articles with the tag "Fine-tuning".
-
LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection
Published: at 06:16 PM
86.80 🤔LENSLLM introduces a Hessian-based PAC-Bayes framework and NTK-based scaling model for LLM selection, achieving up to 91.1% accuracy and 88.5% computational cost reduction by modeling fine-tuning dynamics across diverse tasks.
-
On the Generalization vs Fidelity Paradox in Knowledge Distillation
Published: at 11:23 AM 86.63 🤔本文通过大规模实证分析揭示知识蒸馏（KD）显著提升小型语言模型的零样本推理性能（高达10%），但对大型模型收益有限，且性能提升与推理保真度存在脱节，强调任务专长和适度参数调整的重要性。
-
Not All Adapters Matter: Selective Adapter Freezing for Memory-Efficient Fine-Tuning of Language Models
Published: at 11:20 PM 86.60 🤔本文提出SAFE方法，通过选择性冻结对任务贡献较小的适配器，实现资源高效的语言模型微调，在显著降低内存使用和计算成本的同时，保持甚至提升模型性能。
-
RARE: Retrieval-Augmented Reasoning Modeling
Published: at 11:28 AM 86.57 🤔RARE提出了一种新范式，通过将领域知识存储外部化并优化推理能力，使轻量级模型在多领域基准测试中实现最先进的性能，超越检索增强的GPT-4和DeepSeek-R1。
-
Skip-Thinking: Chunk-wise Chain-of-Thought Distillation Enable Smaller Language Models to Reason Better and Faster
Published: at 11:53 AM 86.49 🤔本文提出分块训练（CWT）和跳跃思维训练（STT），通过将推理过程分块并跳过非核心块，显著提升小型语言模型在链式思维蒸馏中的推理准确性和速度。