Tag: Fine-tuning
All the articles with the tag "Fine-tuning".
-
ULFine: Unbiased Lightweight Fine-tuning for Foundation-Model-Assisted Long-Tailed Semi-Supervised Learning
Published: at 11:06 AM
98.62 👍This paper introduces ULFine, an unbiased lightweight fine-tuning strategy for foundation-model-assisted long-tailed semi-supervised learning, which mitigates 'minority bottleneck' and 'majority overconfidence' issues using Prototype Adaptive Fitting and Dual Logit Fusion, achieving significant performance improvements and over 10x training cost reduction on benchmark datasets.
-
Think, Prune, Train, Improve: Scaling Reasoning without Scaling Models
Published: at 04:31 PM 82.99 👍本文提出 Think, Prune, Train 框架，通过迭代监督微调和基于正确性的数据修剪，实现模型在不增加规模的情况下提升推理能力，避免模型坍缩。
-
LIFT: Improving Long Context Understanding of Large Language Models through Long Input Fine-Tuning
Published: at 04:27 PM 82.62 👍本文提出LIFT框架，通过长输入微调和Gated Memory适配器提升短上下文LLMs的长上下文理解能力，实验显示显著性能改进。
-
Weight Ensembling Improves Reasoning in Language Models
Published: at 01:27 AM 88.15 😐本文发现监督微调导致推理模型多样性坍塌损害 Pass@K，并提出通过插值早期与后期 SFT 检查点（WiSE-FT）的方法，有效提升模型多样性，同时提高 Pass@1 和 Pass@K，进而改善测试时缩放和强化学习效果。
-
Towards Reasoning Ability of Small Language Models
Published: at 04:26 PM 86.53 😐本文通过系统基准测试72个SLMs，证明小型语言模型可以通过结构化训练和压缩技术实现与大型模型相当的推理能力，从而挑战了规模依赖的传统观点。