Tag: Large Language Model
All the articles with the tag "Large Language Model".
-
Latent Principle Discovery for Language Model Self-Improvement
Published: at 11:25 AM
85.30 🤔本文提出STaPLe算法，通过Monte Carlo EM方法自动化发现和学习语言模型自我改进的潜在原则，在多个指令跟随基准上显著提升小型模型性能，同时通过聚类生成人类可解释的宪法。
-
Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giant
Published: at 11:12 AM 85.28 🤔This paper comprehensively evaluates the impact of four quantization methods (GPTQ, AWQ, SmoothQuant, FP8) on instruction-tuned LLMs and SLMs from 1B to 405B parameters across 13 datasets, revealing that quantized models often outperform smaller baselines but struggle with instruction-following and hallucination detection, with FP8 showing robustness and task difficulty not always correlating with accuracy loss.
-
The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants
Published: at 11:43 AM 85.27 🤔本文提出*Avengers*框架，通过无训练的嵌入、聚类、评分和投票操作，整合多个小型开源语言模型的集体智能，在15个多样化数据集上平均性能超越GPT-4.1，展现了开源模型挑战专有巨头的潜力。
-
MateICL: Mitigating Attention Dispersion in Large-Scale In-Context Learning
Published: at 11:13 PM 85.27 🤔本文提出 MateICL 框架，通过分割上下文窗口并引入注意力校准层解决大型语言模型在大规模上下文学习中的注意力分散问题，实验证明其在多种 NLP 任务中有效提升性能并保持稳定性。
-
Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning
Published: at 11:28 AM 85.25 🤔本文提出强化蒸馏（REDI）框架，通过两阶段训练利用正向和负向推理轨迹，显著提升小型语言模型的数学推理性能，Qwen-REDI-1.5B在公开数据上达到1.5B模型的最新水平。