Tag: Data Augmentation
All the articles with the tag "Data Augmentation".
-
On the generalization of language models from in-context learning and finetuning: a controlled study
Published: at 04:33 PM
82.00 🤔本文通过控制实验比较了语言模型在上下文学习和微调下的泛化能力，发现上下文学习更灵活，并提出通过数据增强方法显著改善微调的泛化性能。
-
Beyond Next Token Prediction: Patch-Level Training for Large Language Models
Published: at 11:18 AM 79.83 🤔本文提出patch级训练方法，通过将多个token聚合成高信息密度patch并分阶段训练大型语言模型，在训练成本减半的情况下保持甚至略提升模型性能。
-
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning
Published: at 11:16 AM 79.36 🤔This paper introduces SEFE, a method combining Answer Style Diversification (ASD) to mitigate superficial forgetting and RegLoRA to address essential forgetting in Multimodal Continual Instruction Tuning, achieving state-of-the-art performance on the CoIN benchmark.
-
R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Training
Published: at 11:18 PM 73.79 🤔R&B框架通过基于语义相似性的数据重新分组和梯度驱动的动态权重调整，以极低的计算开销（0.01%）在自然语言和多模态任务中匹配或超越现有数据混合策略，提升了基础模型训练效率。
-
Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
Published: at 12:16 AM 70.15 🤔本文提出了一种奖励增强数据集方法，通过对偏好对进行重新标记使大型语言模型条件化于奖励值学习响应质量全谱，显著提升了直接偏好优化（DPO）的性能并缓解了其遗忘高质被拒响应和无差别学习低质选中响应的局限性。