Tag: Supervised Learning
All the articles with the tag "Supervised Learning".
-
Does Knowledge Distillation Matter for Large Language Model based Bundle Generation?
Published: at 04:30 PM
72.97 🤔本文首次系统探索知识蒸馏技术在基于大语言模型的捆绑生成任务中的应用，通过提出一个全面的 KD 框架和实验验证，证明了在减少计算需求的同时能保持甚至提升性能。
-
Contextures: Representations from Contexts
Published: at 11:05 AM 69.00 🤔This paper introduces the contexture theory, unifying representation learning across paradigms by targeting top singular functions of a context-induced expectation operator, demonstrating high alignment in neural representations and proposing a task-agnostic metric for context evaluation with strong empirical correlation to performance on various datasets.
-
SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillation
Published: at 11:15 PM 64.11 🤔本文提出了 SmallPlan 框架，通过结合 LLM 指导的蒸馏、模拟环境反馈的 SFT 和 RL，训练轻量级的小型语言模型 (SLM) 进行高效的机器人高层路径规划，使其在资源受限的边缘设备上实现接近大型模型 (LLM) 的性能。
-
Constraint Back-translation Improves Complex Instruction Following of Large Language Models
Published: at 09:32 AM 61.24 👍本文提出约束反向翻译方法，通过从现有指令-响应对中提取隐含约束构建高质量复杂指令数据集CRAB，并结合反向训练显著提升大型语言模型在复杂指令跟随任务上的性能。
-
Weight Ensembling Improves Reasoning in Language Models
Published: at 01:27 AM 88.15 😐本文发现监督微调导致推理模型多样性坍塌损害 Pass@K，并提出通过插值早期与后期 SFT 检查点（WiSE-FT）的方法，有效提升模型多样性，同时提高 Pass@1 和 Pass@K，进而改善测试时缩放和强化学习效果。