Tag: Efficiency

All the articles with the tag "Efficiency".

• Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts

  Published: 2 Jun, 2025 |  at  11:24 AM

  

  88.22 🤔

  本文提出LayerMoE算法，通过基于层间语言相似性的专家分配和路由分类器，实现了多语言LLM的高效扩展，以更少的参数显著提升新语言性能并减少旧语言遗忘。

• Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks

  Published: 8 May, 2025 |  at  10:21 AM

  

  88.19 🤔

  This paper proposes a task-oriented semantic communication framework for LMM-based vehicle AI, using LLaVA with Semantic Matching for efficient image slicing and Fusion Attention-based power allocation to prioritize critical data transmission, achieving significant accuracy improvements (up to 33.1% at low SNR) in traffic VQA tasks.

• SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization

  Published: 21 May, 2025 |  at  11:24 AM

  

  88.16 🤔

  SoLoPO通过将长上下文偏好优化分解为短上下文优化和短到长奖励对齐，显著提升了大型语言模型在长上下文任务中的性能和训练效率，同时保持短上下文能力。

• You Do Not Fully Utilize Transformer's Representation Capacity

  Published: 31 May, 2025 |  at  11:35 AM

  

  88.14 🤔

  本文提出Layer-Integrated Memory (LIMe)，通过学习跨层路由机制整合之前所有层的Key-Value表示，显著缓解Transformer的表示崩塌问题，并在语言建模、推理任务和深层网络中实现更快收敛和更高准确率。

• ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation

  Published: 25 May, 2025 |  at  11:25 AM

  

  87.99 🤔

  ShareLoRA通过在模型层间共享低秩矩阵A或B，显著减少可训练参数量（相较LoRA减少44%-96%），并在多种模型和任务中保持甚至超越LoRA的性能，展现出高效性、适应性和跨域鲁棒性。