Tag: Fine-tuning
All the articles with the tag "Fine-tuning".
-
Pre-Act: Multi-Step Planning and Reasoning Improves Acting in LLM Agents
Published: at 11:24 AM
87.43 🤔本文提出Pre-Act方法，通过多步骤规划和详细推理提升LLM代理性能，并通过微调小型模型（如Llama 3.1 70B）在Almita数据集上实现比GPT-4高69.5%的行动准确率和28%的目标完成率。
-
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning
Published: at 01:15 PM 87.30 🤔本文提出LoRA-SB方法，通过基于全参数微调第一步梯度近似的初始化策略优化低秩微调，在参数量减少27-90倍的情况下，显著超越LoRA-XS并接近全参数微调性能。
-
Foundation Models For Seismic Data Processing: An Extensive Review
Published: at 11:07 AM 87.29 🤔This paper conducts an extensive review of natural image foundation models for seismic data processing, demonstrating that hierarchical models like Swin and ConvNeXt, especially with self-supervised pre-training, outperform non-hierarchical ones in demultiple, interpolation, and denoising tasks, while highlighting the benefits and limitations of natural image pre-training for seismic applications.
-
FlashThink: An Early Exit Method For Efficient Reasoning
Published: at 11:16 AM 87.28 🤔FlashThink方法通过验证模型动态判断推理过程是否提前结束，在保持大型语言模型准确率的同时显著减少推理内容长度（平均效率提升约77%），并通过FT²微调进一步优化性能。
-
Mini-batch Coresets for Memory-efficient Language Model Training on Data Mixtures
Published: at 01:14 PM 87.28 🤔本文提出 CoLM 方法，通过构建小批量核心集匹配大批量梯度，在内存需求减少 2 倍的情况下，使 LLM 微调性能优于 4 倍批大小的常规训练，同时提升收敛速度。