Tag: Large Language Model
All the articles with the tag "Large Language Model".
-
On the generalization of language models from in-context learning and finetuning: a controlled study
Published: at 04:33 PM
82.00 🤔本文通过控制实验比较了语言模型在上下文学习和微调下的泛化能力，发现上下文学习更灵活，并提出通过数据增强方法显著改善微调的泛化性能。
-
Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
Published: at 01:18 AM 81.62 🤔本文提出DPE，一种无需训练的长文本外推方法，通过检测RoPE不同维度组的有效相对距离并识别关键维度，有选择地调整这些关键维度的位置索引，显著扩展了LLM的上下文窗口并提升了长文本任务性能。
-
Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data
Published: at 11:18 AM 81.58 🤔本文提出判别式微调（DFT）框架，通过判别式概率模型优化大型语言模型的输出概率，无需人类偏好数据或奖励模型，在数学推理和通用语言任务上显著优于SFT并与SFT→PO方法相当。
-
Don't be lazy: CompleteP enables compute-efficient deep transformers
Published: at 11:16 AM 81.10 🤔This paper introduces CompleteP, a parameterization for transformers with α = 1, which ensures depth-wise hyperparameter transfer and complete feature learning, achieving 12-34% compute efficiency improvements and enabling a wider range of compute-optimal width-to-depth ratios.
-
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
Published: at 11:18 AM 80.55 🤔This paper demonstrates through meta-analysis and experiments that Chain-of-Thought (CoT) prompting significantly enhances large language model performance on math and symbolic reasoning tasks, but offers limited benefits for non-symbolic tasks and underperforms compared to tool-augmented approaches.