Tag: Efficiency
All the articles with the tag "Efficiency".
-
Graph Attention is Not Always Beneficial: A Theoretical Analysis of Graph Attention Mechanisms via Contextual Stochastic Block Models
Published: at 11:30 AM
89.11 🤔This paper provides a theoretical analysis using Contextual Stochastic Block Models to demonstrate that graph attention mechanisms are beneficial for node classification only when structure noise exceeds feature noise, proposes a multi-layer GAT to achieve perfect classification at lower SNR thresholds, and validates these findings through synthetic and real-world experiments.
-
Model Merging in Pre-training of Large Language Models
Published: at 11:14 AM 89.09 🤔本文提出预训练模型平均（PMA）策略，通过融合预训练阶段的检查点显著提升大型语言模型性能、预测退火效果并增强训练稳定性，为高效模型开发提供了新方法和实用指南。
-
SLearnLLM: A Self-Learning Framework for Efficient Domain-Specific Adaptation of Large Language Models
Published: at 11:24 AM 89.09 🤔SLearnLLM提出了一种自学习框架，通过让大语言模型自我评估并筛选错误回答的QA对进行微调，在农业和医疗领域实现了与全数据集微调相当的性能提升，同时显著降低了训练时间成本。
-
Self-Data Distillation for Recovering Quality in Pruned Large Language Models
Published: at 11:19 PM 89.06 🤔本文提出自数据蒸馏微调方法，通过利用未剪枝模型生成蒸馏数据集恢复剪枝后大型语言模型的质量，在HuggingFace OpenLLM Leaderboard v1上显著优于标准监督微调，并通过模型合并和推测解码进一步提升性能和效率。
-
Log-Augmented Generation: Scaling Test-Time Reasoning with Reusable Computation
Published: at 11:18 AM 89.05 🤔本文提出日志增强生成（LAG）框架，通过使用KV缓存直接复用过去的推理计算，显著提升大型语言模型在知识和推理密集型任务上的准确性和效率，优于标准代理系统及现有反思和KV缓存方法。