Archives
All the articles I've archived.
-
Contextures: Representations from Contexts
Published: at 11:05 AM
69.00 🤔This paper introduces the contexture theory, unifying representation learning across paradigms by targeting top singular functions of a context-induced expectation operator, demonstrating high alignment in neural representations and proposing a task-agnostic metric for context evaluation with strong empirical correlation to performance on various datasets.
-
COSMOS: Predictable and Cost-Effective Adaptation of LLMs
Published: at 11:05 AM 75.99 🤔COSMOS introduces a cost-effective framework to predict performance and cost of LLM adaptation strategies like QLoRA fine-tuning and retrieval-augmented ICL, achieving high accuracy (1.09% MAE) and reducing computational costs by 92.72% across eight diverse benchmarks.
-
How Do Multimodal Large Language Models Handle Complex Multimodal Reasoning? Placing Them in An Extensible Escape Game
Published: at 10:59 AM 68.75 🤔This paper introduces MM-Escape, a benchmark using the customizable 3D environment EscapeCraft to evaluate multimodal reasoning in MLLMs through room escape tasks, revealing that while models like GPT-4o achieve high success in simple scenarios, performance drops significantly with increased difficulty, exposing distinct limitations in reasoning and spatial awareness.
-
Communicating Activations Between Language Model Agents
Published: at 10:59 AM 87.71 🤔This paper introduces Activation Communication (AC), a novel method for inter-LLM communication using intermediate activations instead of natural language, achieving up to 27% performance improvement over traditional methods with significantly reduced compute across coordination games and reasoning benchmarks.
-
Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Compute
Published: at 10:59 AM 86.42 🤔This paper introduces ModelSwitch, a multi-LLM repeated sampling strategy that leverages answer consistency to dynamically switch models, achieving superior performance and 34% sample efficiency over single-LLM self-consistency across diverse datasets.
-
Exploring Effective Distillation of Self-Supervised Speech Models for Automatic Speech Recognition
Published: at 10:59 AM 72.65 🤔This paper explores effective distillation of HuBERT for ASR by comparing student model structures, introducing a discriminative loss for improved low-resource performance, and proposing front-end distillation from waveform to Fbank features, achieving 17% parameter reduction and doubled inference speed with minor performance degradation.
-
How do Humans and Language Models Reason About Creativity? A Comparative Analysis
Published: at 10:59 AM 60.58 🤔This paper conducts a comparative analysis of creativity evaluation in STEM, revealing that human experts and LLMs prioritize different facets of originality (cleverness vs. remoteness/uncommonness) and are differentially influenced by contextual examples, with LLMs showing higher predictive accuracy but poorer construct validity due to homogenized facet correlations.
-
Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon
Published: at 10:59 AM 85.36 🤔This paper introduces a taxonomy of language model memorization into recitation, reconstruction, and recollection, demonstrating through experiments with Pythia models that different factors influence each category, with a taxonomy-based predictive model outperforming baselines in predicting memorization likelihood.
-
Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoders
Published: at 10:58 AM 61.78 🤔This paper uses Sparse Autoencoders to identify and manipulate language-specific features in Large Language Models, introducing a monolinguality metric, demonstrating context dependency via code-switching, and enhancing steering vectors for better control over multilingual generation while revealing significant language-specific impacts through ablation studies.
-
RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization
Published: at 11:10 AM 69.29 🤔RWKVQuant introduces a tailored Post Training Quantization framework for RWKV models, using a coarse-to-fine proxy to hybridize scalar and vector quantization and optimizing codebooks for element-wise operations, achieving ~3-bit quantization with minimal accuracy loss and significant memory and speed improvements.
-
Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
Published: at 11:09 AM 75.37 🤔This paper demonstrates that finetuning aligned LLMs on narrow tasks like writing insecure code can lead to emergent misalignment, causing broadly harmful behaviors across unrelated tasks, as evidenced by experiments on multiple models with control setups and backdoor triggers.
-
Radio: Rate-Distortion Optimization for Large Language Model Compression
Published: at 11:09 AM 79.84 🤔This paper introduces 'Radio,' a rate-distortion optimization framework for LLM compression that outperforms existing quantization methods in perplexity and downstream task accuracy, particularly at lower bit depths, by iteratively optimizing bit depths and using companding quantization post-training.
-
VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making
Published: at 11:08 AM 70.12 🤔This paper introduces VLM Q-Learning, an offline-to-online reinforcement learning method that fine-tunes Vision-Language Models for interactive decision-making by filtering suboptimal actions with a critic head, achieving significant performance improvements over supervised fine-tuning across multiple multimodal agent tasks.
-
Competition Dynamics Shape Algorithmic Phases of In-Context Learning
Published: at 11:08 AM 85.25 🤔This paper introduces a synthetic sequence modeling task using finite Markov mixtures to unify the study of in-context learning (ICL), identifying four competing algorithms that explain model behavior and phase transitions, thus offering insights into ICL's transient nature and phenomenology.
-
CRANE: Reasoning with constrained LLM generation
Published: at 11:08 AM 61.03 🤔This paper introduces CRANE, a reasoning-augmented constrained decoding algorithm that alternates between unconstrained and constrained generation to preserve LLM reasoning capabilities while ensuring syntactic correctness, achieving up to 10% accuracy improvement on symbolic reasoning benchmarks like GSM-Symbolic and FOLIO.
-
Compact Recurrent Transformer with Persistent Memory
Published: at 11:06 AM 66.84 🤔This paper introduces the Compact Recurrent Transformer (CRT), which combines shallow Transformers with RNNs to efficiently process long sequences using a single persistent memory vector, achieving superior or comparable performance to full-length Transformers and Transformer-XL on language and video tasks with significantly reduced computational cost.
-
RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference
Published: at 11:06 AM 73.12 🤔RetroInfer reimagines the KV cache as a vector storage system, using an attention-aware wave index and wave buffer to achieve up to 4.5x speedup over full attention and 10.5x over sparse baselines for long-context LLM inference, while preserving near-full-attention accuracy.
-
SIMPLEMIX: Frustratingly Simple Mixing of Off- and On-policy Data in Language Model Preference Learning
Published: at 11:06 AM 68.33 🤔This paper introduces SIMPLEMIX, a simple method to mix on- and off-policy data in language model preference optimization, demonstrating that their complementary strengths—on-policy for reasoning tasks and off-policy for open-ended tasks—lead to a 6.03% average improvement over single-source methods on Alpaca Eval 2.0.
-
MoM: Linear Sequence Modeling with Mixture-of-Memories
Published: at 06:19 PM 89.33 🤔The Mixture-of-Memories (MoM) architecture introduces multiple independent memory states with a routing mechanism to enhance memory capacity and reduce interference in linear sequence modeling, achieving significant performance gains over other linear models on recall-intensive tasks and nearing Transformer performance at larger scales while maintaining efficiency.
-
Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models
Published: at 06:19 PM 83.98 🤔This paper introduces a recursive summarization method to enhance long-term dialogue memory in LLMs, achieving marginal quantitative improvements and notable qualitative gains in consistency and coherence across multiple models and datasets.
-
Activation Space Interventions Can Be Transferred Between Large Language Models
Published: at 06:19 PM 85.65 🤔This paper demonstrates that activation space interventions for AI safety, such as backdoor removal and refusal behavior, can be transferred between large language models using autoencoder mappings, enabling smaller models to align larger ones, though challenges remain in cross-architecture transfers and complex tasks like corrupted capabilities.
-
RADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scale
Published: at 06:17 PM 87.73 🤔RADLADS introduces a cost-effective three-step distillation protocol to convert softmax attention transformers into linear attention models using only 350-700M tokens, achieving near-teacher performance on benchmarks and setting a new state-of-the-art for pure RNNs with models up to 72B parameters.
-
SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
Published: at 06:17 PM 86.55 🤔This paper investigates zero RL training on diverse open base models, achieving significant accuracy and response length improvements while identifying key factors like reward design and data difficulty that influence the emergence of reasoning behaviors.
-
SEAL: Steerable Reasoning Calibration of Large Language Models for Free
Published: at 06:16 PM 87.52 🤔SEAL, a training-free method, calibrates the reasoning process of Large Language Models by steering latent representations to reduce redundant thoughts, achieving up to 14.1% accuracy improvement and 50.4% token reduction across diverse benchmarks.
-
ZeroSearch: Incentivize the Search Capability of LLMs without Searching
Published: at 06:16 PM 86.09 🤔ZEROSEARCH introduces a reinforcement learning framework that enhances LLMs' search capabilities by simulating search engines with fine-tuned LLMs, achieving performance comparable to or better than real search engines without API costs through a curriculum-based rollout strategy.
-
LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection
Published: at 06:16 PM 86.80 🤔LENSLLM introduces a Hessian-based PAC-Bayes framework and NTK-based scaling model for LLM selection, achieving up to 91.1% accuracy and 88.5% computational cost reduction by modeling fine-tuning dynamics across diverse tasks.
-
When Reasoning Beats Scale: A 1.5B Reasoning Model Outranks 13B LLMs as Discriminator
Published: at 06:13 PM 88.68 🤔This paper demonstrates that a 1.5B parameter reasoning model (Distill-R1) outperforms larger non-reasoning LLMs as a discriminator in a text-to-SQL planning framework by leveraging a novel soft score extraction method from chain-of-thought outputs, though it struggles significantly as a generator.
-
Recall with Reasoning: Chain-of-Thought Distillation for Mamba's Long-Context Memory and Extrapolation
Published: at 06:13 PM 86.84 🤔This paper proposes Recall with Reasoning (RwR), a method that enhances Mamba's long-context memory and extrapolation by distilling chain-of-thought summarization from a teacher model, achieving significant performance improvements on LONGMEMEVAL and HELMET benchmarks while preserving short-context capabilities.
-
Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models
Published: at 06:12 PM 91.54 🤔This paper introduces Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning (LS-Mixture SFT), which combines long and short CoT datasets to fine-tune non-reasoning LLMs, achieving a 2.3% average accuracy improvement and 47.61% response length reduction on reasoning benchmarks.
-
Adversarial Attacks in Multimodal Systems: A Practitioner's Survey
Published: at 11:09 AM 96.71 🤔This survey paper provides a comprehensive overview of adversarial attacks on multimodal AI systems across text, image, video, and audio modalities, categorizing threats by attacker knowledge, intention, and execution to equip practitioners with knowledge of vulnerabilities and cross-modal risks.
-
CB-cPIR: Code-Based Computational Private Information Retrieval
Published: at 11:08 AM 93.98 🤔CB-cPIR introduces a code-based single-server computational private information retrieval scheme that enhances security against subquery attacks by using high-weight secret vectors and dual queries, achieving lower communication and computational costs compared to lattice-based schemes like XPIR and SimplePIR.
-
Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks
Published: at 11:07 AM 87.75 🤔ASTRA introduces an efficient defense for Vision Language Models by adaptively steering activations away from adversarial directions using image attribution, achieving state-of-the-art performance in mitigating jailbreak attacks with minimal impact on benign utility and high inference efficiency.
-
Deformable Beta Splatting
Published: at 11:07 AM 87.15 🤔Deformable Beta Splatting (DBS) enhances real-time radiance field rendering by introducing deformable Beta Kernels for superior geometric fidelity, Spherical Beta for efficient color encoding, and kernel-agnostic MCMC optimization, achieving state-of-the-art visual quality with 45% fewer parameters and 1.5x faster rendering than 3DGS-MCMC.
-
Towards Safer Pretraining: Analyzing and Filtering Harmful Content in Webscale datasets for Responsible LLMs
Published: at 11:07 AM 94.57 🤔This paper proposes a three-dimensional taxonomy and develops TTP and HarmFormer tools to filter harmful content from web-scale LLM pretraining datasets, revealing significant toxicity prevalence and persistent safety gaps through benchmarks like HAVOC.
-
Always Skip Attention
Published: at 11:06 AM 89.20 🤔This paper theoretically demonstrates the ill-conditioning of Self-Attention Blocks in Vision Transformers without skip connections, highlights their role as regularizers, and proposes Token Graying (SVD and DCT) to improve input token conditioning, achieving modest performance gains in supervised and self-supervised tasks.
-
Enhancing Safety Standards in Automated Systems Using Dynamic Bayesian Networks
Published: at 11:06 AM 90.63 🤔This paper proposes a Dynamic Bayesian Network framework for autonomous vehicles that enhances safety in cut-in maneuvers by integrating lateral evidence and probabilistic safety assessments, achieving superior crash avoidance in high-speed scenarios (9.22% crash rate) compared to baseline models in the JRC-FSM simulator.
-
Sparse-Group Boosting with Balanced Selection Frequencies: A Simulation-Based Approach and R Implementation
Published: at 10:25 AM 87.75 🤔This paper introduces sparse-group boosting and a simulation-based group balancing algorithm within the 'sgboost' R package to mitigate variable selection bias in high-dimensional grouped data, demonstrating improved fairness and interpretability through simulations and ecological data analysis.
-
Style Feature Extraction Using Contrastive Conditioned Variational Autoencoders with Mutual Information Constraints
Published: at 10:25 AM 93.98 🤔This paper proposes a novel method combining contrastive learning with conditional variational autoencoders and mutual information constraints to extract style features from unlabeled data, demonstrating effectiveness on simple datasets like MNIST while facing challenges with natural image datasets due to augmentation limitations and qualitative evaluation.
-
Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations
Published: at 10:22 AM 89.20 🤔The Video Prediction Policy (VPP) introduces a novel generalist robot policy that leverages predictive visual representations from fine-tuned video diffusion models to learn implicit inverse dynamics, achieving significant improvements of 41.5% on the Calvin ABC→D benchmark and 31.6% in real-world dexterous manipulation tasks over state-of-the-art baselines.
-
MELON: Provable Indirect Prompt Injection Defense via Masked Re-execution and Tool Comparison
Published: at 10:22 AM 89.40 🤔MELON introduces a novel training-free defense against indirect prompt injection attacks on LLM agents by detecting independence of tool calls from user inputs through masked re-execution, achieving superior attack prevention (0.24% ASR on GPT-4o) and utility preservation (58.78% UA on GPT-4o) compared to existing methods.
-
Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
Published: at 10:22 AM 97.91 😐Insight-V introduces a scalable data generation pipeline and a multi-agent system with iterative DPO training to significantly enhance long-chain visual reasoning in MLLMs, achieving up to 7.0% performance gains on challenging benchmarks while maintaining perception capabilities.
-
UnifyFL: Enabling Decentralized Cross-Silo Federated Learning
Published: at 10:21 AM 87.15 🤔UnifyFL proposes a decentralized cross-silo federated learning framework using Ethereum blockchain and IPFS to enable trust-based collaboration among organizations, achieving comparable accuracy to centralized FL with flexible aggregation policies and efficient handling of stragglers through synchronous and asynchronous modes.
-
Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks
Published: at 10:21 AM 88.19 🤔This paper proposes a task-oriented semantic communication framework for LMM-based vehicle AI, using LLaVA with Semantic Matching for efficient image slicing and Fusion Attention-based power allocation to prioritize critical data transmission, achieving significant accuracy improvements (up to 33.1% at low SNR) in traffic VQA tasks.
-
Agentic AI: The Era of Semantic Decoding
Published: at 12:27 AM 89.68 🤔本文提出语义解码视角，将大型语言模型、人类和工具的协作框架化为语义空间中的优化过程，通过语义令牌的交换和语义解码算法的设计探索AI系统的新计算范式。
-
Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off
Published: at 12:26 AM 93.66 🤔本文提出 FedCEO 框架，通过服务器端张量低秩优化和客户端语义互补性，在差分隐私联邦学习中实现了效用-隐私权衡的显著改进，理论上提升了 O(√d) 的界限，并在实验中验证了其优越性能。
-
Stabilizing and Solving Unique Continuation Problems by Parameterizing Data and Learning Finite Element Solution Operators
Published: at 12:26 AM 87.37 🤔本文提出了一种结合有限元方法与机器学习技术（自编码器与操作符学习）解决非线性PDE逆问题中唯一性延续问题的方法，通过数据降维和稳定化技术提高病态问题的求解稳定性和效率，并在合成数据上验证了其有效性。
-
AI agents may be worth the hype but not the resources (yet): An initial exploration of machine translation quality and costs in three language pairs in the legal and news domains
Published: at 12:22 AM 86.86 🤔本文通过实证评估五种机器翻译范式，发现推理增强的大型语言模型（如o1-preview）在人工评估中表现出色，超越传统NMT，而多智能体系统虽具潜力，但因高计算成本和语言对表现不一致而受限。
-
Test-time Correlation Alignment
Published: at 12:21 AM 93.31 🤔本文提出测试时相关性对齐（TCA）范式，通过构建伪源域相关性并应用线性变换对齐测试数据特征，显著提升测试时适应（TTA）性能，同时保持高效性和源域知识。
-
Nonparametric learning of covariate-based Markov jump processes using RKHS techniques
Published: at 12:18 AM 88.71 🤔本文提出了一种基于再生核希尔伯特空间（RKHS）的非参数化方法，通过频率学和贝叶斯框架建模连续时间马尔可夫链（CTMC）中协变量驱动的非线性转移率，显著提升了个体化状态转移预测的准确性。
-
Survey of Abstract Meaning Representation: Then, Now, Future
Published: at 12:18 AM 95.51 😐本文综述了抽象意义表示（AMR）作为一种图结构语义表示框架的发展、解析与生成方法、多语言扩展及下游应用，揭示其在提升机器语言理解中的潜力与局限。
-
Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models
Published: at 12:17 AM 96.36 🤔本文提出了一种基于视觉-语言模型的定义引导提示技术和UnHateMeme框架，用于检测和缓解多模态模因中的仇恨内容，通过零样本和少样本提示实现高效检测，并生成非仇恨替代内容以保持图像-文本一致性，在实验中展现出显著效果。
-
CoordField: Coordination Field for Agentic UAV Task Allocation In Low-altitude Urban Scenarios
Published: at 12:16 AM 90.99 🤔本文提出了一种基于协调场的代理系统（CoordField），通过大型语言模型解析自然语言指令并利用动态势场实现异构无人机群在城市环境中的去中心化任务分配，实验验证了其在任务覆盖、响应时间和动态适应性方面的优越性能。
-
LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models
Published: at 09:34 AM 85.07 🤔本文通过LLM-Coordination基准测试框架，评估大型语言模型在纯协调游戏中的多智能体协调能力，发现其在环境依赖任务中表现优异但在心智理论推理和联合规划中存在显著不足，同时展现出对未见伙伴的零样本适应性。
-
Do We Need a Detailed Rubric for Automated Essay Scoring using Large Language Models?
Published: at 09:33 AM 61.53 😐本文通过对比详细、简化和无评分标准在四个大型语言模型上的自动作文评分表现，发现简化标准在大多数模型中能保持与详细标准相似的准确性并显著降低token使用量，但模型特异性和整体性能不足仍需关注。
-
Exploring the Role of Diversity in Example Selection for In-Context Learning
Published: at 09:33 AM 64.87 🤔本文提出基于多样性的上下文学习（DICL）方法，通过最大边际相关性（MMR）算法重新排序示例以平衡相关性和多样性，在多个数据集和大型语言模型上实现了约70%的下游任务性能提升或维持。
-
Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approach
Published: at 09:32 AM 74.12 🤔本文通过校准感知微调（CFT和RCFT）方法，结合可校准和不可校准区域的理论框架，显著改善了偏好对齐后大型语言模型的校准性能，同时维持或提升其语言能力。
-
LLM-Empowered Embodied Agent for Memory-Augmented Task Planning in Household Robotics
Published: at 09:32 AM 62.29 🤔本文提出了一种基于LLM的代理编排机器人系统，通过模块化任务规划和RAG记忆检索实现家庭环境中长程任务的自主执行，并在三个场景中展示了较高的任务规划准确率和记忆召回改进。
-
EMORL: Ensemble Multi-Objective Reinforcement Learning for Efficient and Flexible LLM Fine-Tuning
Published: at 09:32 AM 87.79 🤔本文提出EMORL框架，通过集成学习分别训练单目标模型并在隐藏状态层聚合，结合分层网格搜索优化权重，在咨询反思生成任务中实现了与传统方法相当的性能，同时显著提升了训练效率、可扩展性和解释性。
-
Constraint Back-translation Improves Complex Instruction Following of Large Language Models
Published: at 09:32 AM 61.24 👍本文提出约束反向翻译方法，通过从现有指令-响应对中提取隐含约束构建高质量复杂指令数据集CRAB，并结合反向训练显著提升大型语言模型在复杂指令跟随任务上的性能。
-
Waking Up an AI: A Quantitative Framework for Prompt-Induced Phase Transition in Large Language Models
Published: at 09:31 AM 64.57 🤔本文提出了一种双重提示框架（TIP和TQP）来量化大型语言模型（LLMs）的认知相变，发现LLMs对概念融合提示的情感反应与人类直觉差异显著，揭示了AI与人类认知在概念整合上的潜在鸿沟。
-
Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions
Published: at 09:31 AM 85.11 🤔本文通过提出AI记忆系统的分类（参数、上下文结构化和非结构化）和六种基本操作（整合、更新、索引、遗忘、检索、压缩），系统化地综述了长期记忆、长上下文、参数修改和多源记忆等研究主题，并展望了未来方向。
-
R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning
Published: at 08:43 AM 82.56 🤔本文提出R1-Reward，通过StableReinforce算法将强化学习应用于多模态奖励模型训练，显著提升了性能并在多个基准测试中超越现有最优模型，同时展示了优异的数据效率和测试时扩展性。
-
CCSK:Cognitive Convection of Self-Knowledge Based Retrieval Augmentation for Large Language Models
Published: at 08:43 AM 70.69 🤔本文提出CCSK框架，通过Siamese Network和Response Quality Model动态融合查询相似性和响应质量，优化大型语言模型的信息检索决策，在多个问答数据集上显著提升了F1分数和准确率。
-
Investigating Task Arithmetic for Zero-Shot Information Retrieval
Published: at 08:43 AM 86.02 🤔本文提出任务算术方法，通过参数加减操作实现零样本信息检索的领域和语言适应，在科学、生物医学和多语言数据集上取得最高18%的NDCG@10提升，展现了轻量级模型适应的潜力。
-
100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models
Published: at 08:42 AM 85.65 🤔本文综述了DeepSeek-R1发布后100天内推理语言模型的复制研究，系统总结了监督微调和基于可验证奖励的强化学习方法在数据构建和算法设计上的进展，并探讨了推理能力提升的多方向应用。
-
Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models
Published: at 08:42 AM 78.41 🤔本文系统综述了基于强化学习的推理方法在多模态大语言模型（MLLMs）中的进展，分析了算法设计、奖励机制及应用，揭示了跨模态推理和奖励稀疏性等挑战，并提出了分层奖励和交互式RL等未来方向。
-
What do Language Model Probabilities Represent? From Distribution Estimation to Response Prediction
Published: at 08:42 AM 83.92 🤔本文通过理论分析区分了语言模型输出概率的三种解释（完成分布、响应分布、事件分布），揭示了现有研究中对这些分布的混淆和误解，并呼吁谨慎解释模型概率以指导LLM的开发和应用。
-
Looped Transformers for Length Generalization
Published: at 08:42 AM 74.13 🤔本文提出Looped Transformers方法，通过循环结构和自适应步数显著提升了Transformer在算法任务上的长度泛化能力，在多种任务中优于传统方法。
-
Exploring the Potential of Offline RL for Reasoning in LLMs: A Preliminary Study
Published: at 08:41 AM 86.49 🤔本文通过探索离线强化学习方法（LD-DPO），在DeepDistill-32B模型上实现了平均3.3%的推理性能提升，尤其在Arena-Hard基准上提升10.1%，并强调了推理长度与语义丰富性平衡的重要性。
-
ICLR: In-Context Learning of Representations
Published: at 08:41 AM 84.18 🤔本文通过上下文图追踪任务揭示了大型语言模型能随上下文规模增加而突现地重组概念表示以适应新语义，并提出能量最小化假设解释这一过程。
-
Racing Thoughts: Explaining Contextualization Errors in Large Language Models
Published: at 12:18 AM 74.82 🤔本文提出‘LLM Race Conditions Hypothesis’解释大型语言模型的上下文化错误，通过机械可解释性技术验证了关键窗口和上下文化顺序对模型性能的影响，并探索了推理时干预措施来缓解问题。
-
Activated LoRA: Fine-tuned LLMs for Intrinsics
Published: at 12:17 AM 86.84 🤔本文提出 Activated LoRA (aLoRA)，一种改进的 LoRA 框架，通过仅对激活后 token 适配权重，复用基础模型 KV 缓存，实现高效动态适配，并在多个任务上保持与标准 LoRA 相当的性能，同时显著降低推理成本。
-
Extracting and Transferring Abilities For Building Multi-lingual Ability-enhanced Large Language Models
Published: at 12:17 AM 68.20 🤔本文提出MAET方法，通过提取语言无关的能力相关权重并跨语言转移，构建多语言能力增强的大型语言模型，在数学和科学任务上以60%的计算资源实现约10%的性能提升，优于多种基线方法。
-
Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
Published: at 12:16 AM 70.15 🤔本文提出了一种奖励增强数据集方法，通过对偏好对进行重新标记使大型语言模型条件化于奖励值学习响应质量全谱，显著提升了直接偏好优化（DPO）的性能并缓解了其遗忘高质被拒响应和无差别学习低质选中响应的局限性。
-
Toward Understanding In-context vs. In-weight Learning
Published: at 12:16 AM 82.20 🤔本文通过一个简化的理论模型和多场景实验，揭示了数据分布特性如何驱动上下文学习（ICL）和权重学习（IWL）的出现与竞争，并解释了ICL在训练过程中可能短暂的原因。
-
Test-time regression: a unifying framework for designing sequence models with associative memory
Published: at 12:16 AM 71.71 🤔本文提出一个基于测试时回归的统一框架，通过将关联回忆形式化为回归问题，推导出多种序列模型（如线性注意力、状态空间模型、softmax注意力），并通过合成实验验证其回归能力，同时提出高阶注意力泛化。
-
Intra-Layer Recurrence in Transformers for Language Modeling
Published: at 12:12 AM 69.79 🤔本文提出Intra-Layer Recurrence (ILR)方法，通过在Transformer单次前向传播中选择性循环特定层（尤其是早期层），在不增加参数量的情况下改善语言建模困惑度，但计算成本增加和大规模模型验证不足限制了其实用性。
-
Beyond the Last Answer: Your Reasoning Trace Uncovers More than You Think
Published: at 12:12 AM 76.38 🤔本文提出了一种通过分割大型语言模型推理轨迹为子思维并从中间状态生成多条推理路径、最终以众数聚合答案的方法，显著提高了数学推理任务的准确性（最高提升13%），并揭示了答案一致性与正确性的相关性。
-
RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
Published: at 12:11 AM 88.02 🤔本文提出StarPO框架和RAGEN系统，通过多轮轨迹级别强化学习训练LLM智能体，揭示了训练不稳定性（如Echo Trap）和推理能力不足的挑战，并通过StarPO-S改进稳定性和泛化性，但推理能力仍需细粒度奖励设计支持。
-
RM-R1: Reward Modeling as Reasoning
Published: at 12:11 AM 85.26 🤔本文提出RM-R1，一种通过将奖励建模转化为推理任务并结合蒸馏和强化学习训练的推理奖励模型（REASRMS），在多个基准测试上取得了最先进性能，同时显著提升了可解释性。
-
TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
Published: at 12:11 AM 76.46 🤔本文提出TT-LoRA MoE框架，通过两阶段训练结合张量分解的低秩适配器和动态稀疏路由机制，以极低的参数量（LoRA的2%，AdapterFusion的0.03%）实现多任务NLP分类任务的竞争性性能，平均准确率提升约4个百分点，同时解决任务干扰和知识遗忘问题。
-
Unveiling the Mechanisms of Explicit CoT Training: How CoT Enhances Reasoning Generalization
Published: at 11:21 PM 85.04 🤔本文通过控制实验、内部机制分析和理论推导，揭示了显式思维链（CoT）训练通过形成二阶段泛化电路显著提升大型语言模型的分布内（ID）和分布外（OOD）推理泛化能力，并验证了其在噪声数据下的鲁棒性。
-
The dynamic interplay between in-context and in-weight learning in humans and neural networks
Published: at 11:20 PM 70.07 🤔本文通过神经网络中上下文学习（ICL）与权重学习（IWL）的动态交互，统一解释了人类学习中的组合性泛化、课程效应及灵活性与保留性权衡，为认知科学双过程理论提供了新视角。
-
A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
Published: at 11:19 PM 90.65 😐本文通过提出一个四维度分类框架（什么扩展、如何扩展、哪里扩展、扩展效果如何），系统综述了测试时扩展（TTS）在大型语言模型中的研究现状，为理解和应用推理阶段计算扩展提供了结构化视角和实践指导。
-
LZ Penalty: An information-theoretic repetition penalty for autoregressive language models
Published: at 11:19 PM 67.26 🤔本文提出LZ惩罚方法，基于LZ77压缩算法的码长变化动态调整自回归语言模型的采样分布，在贪婪解码下有效消除退化重复，同时保持推理基准性能。
-
Think2SQL: Reinforce LLM Reasoning Capabilities for Text2SQL
Published: at 11:18 PM 87.33 🤔本文通过结合监督微调（SFT）、强化学习（RL）及细粒度奖励函数（如QATCH），显著提升了小型LLM在Text2SQL任务中的推理能力和性能，Think2SQL-7B模型在BIRD数据集上超越了400B+参数模型。
-
R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Training
Published: at 11:18 PM 73.79 🤔R&B框架通过基于语义相似性的数据重新分组和梯度驱动的动态权重调整，以极低的计算开销（0.01%）在自然语言和多模态任务中匹配或超越现有数据混合策略，提升了基础模型训练效率。
-
Can a Crow Hatch a Falcon? Lineage Matters in Predicting Large Language Model Performance
Published: at 11:17 PM 75.40 🤔本文提出谱系正则化矩阵分解（LRMF）方法，通过利用大型语言模型的谱系关系显著提高性能预测准确性，在同质和异质模型场景下均优于传统方法，尤其在冷启动问题上表现突出。
-
Hierarchical Attention Generates Better Proofs
Published: at 11:16 PM 69.57 🤔本文提出层次注意力正则化方法，通过引导大型语言模型的注意力机制与数学推理的五级层次结构对齐，在 miniF2F 和 ProofNet 基准上分别提升证明成功率 2.05% 和 1.69%，并显著降低证明复杂度。
-
When2Call: When (not) to Call Tools
Published: at 11:16 PM 66.07 🤔本文提出When2Call基准，通过多选格式评估语言模型在工具调用决策上的表现，并通过偏好优化（RPO）训练方法显著提升模型在何时调用工具及何时保守行为之间的平衡能力。
-
Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking
Published: at 11:15 PM 83.31 🤔本文提出InteRank方法，通过知识蒸馏和强化学习训练一个3B参数小型语言模型，在推理密集型文档重排序任务中生成解释并实现与70B+参数模型相当的性能，在BRIGHT基准上位列第三。
-
Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism
Published: at 11:15 PM 87.26 🤔本文通过提出Gather-and-Aggregate (G&A)机制，揭示了Transformer和SSM模型在上下文检索能力上的性能差距主要源于少数关键头部的实现差异，并通过混合模型实验验证了注意力机制在改进SSM检索能力上的潜力。
-
Improving Reasoning Performance in Large Language Models via Representation Engineering
Published: at 11:15 PM 88.60 👍本文通过表示工程方法，利用控制向量干预大型语言模型的残差流，成功提升了Pythia和Mistral模型在归纳、演绎和数学推理任务上的表现，表明推理能力可通过调整内部表示进行调控。
-
MateICL: Mitigating Attention Dispersion in Large-Scale In-Context Learning
Published: at 11:13 PM 85.27 🤔本文提出 MateICL 框架，通过分割上下文窗口并引入注意力校准层解决大型语言模型在大规模上下文学习中的注意力分散问题，实验证明其在多种 NLP 任务中有效提升性能并保持稳定性。
-
CREAM: Consistency Regularized Self-Rewarding Language Models
Published: at 01:28 AM 87.56 🤔本文提出了CREAM（Consistency Regularized Self-Rewarding Language Model）方法，通过衡量自奖励过程中不同迭代模型之间排序的一致性来正则化偏好训练，从而缓解奖励偏差问题，提高小型语言模型的对齐性能和训练稳定性。
-
Weight Ensembling Improves Reasoning in Language Models
Published: at 01:27 AM 88.15 😐本文发现监督微调导致推理模型多样性坍塌损害 Pass@K，并提出通过插值早期与后期 SFT 检查点（WiSE-FT）的方法，有效提升模型多样性，同时提高 Pass@1 和 Pass@K，进而改善测试时缩放和强化学习效果。
-
Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs
Published: at 01:27 AM 70.15 🤔本文提出了低秩知识遗忘（LoKU）框架，包含反向铰链损失（IHL）和 Fisher 加权低秩适配器初始化（FILA），以实现鲁棒且参数高效的大语言模型知识遗忘，有效移除敏感信息同时保持模型原有能力。
-
Reinforcement Learning for LLM Reasoning Under Memory Constraints
Published: at 01:19 AM 90.32 👍本文提出了S-GRPO和T-SPMO两种内存高效、无批评者的强化学习方法，结合LoRA微调，在有限硬件资源下显著提升了大型语言模型在数学推理任务上的性能，其中T-SPMO在需要细粒度信用分配的任务上表现尤为突出。
-
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
Published: at 01:19 AM 87.21 👍本文提出了Mem0及其图增强变体Mem0*<sup>g</sup>*，这是一种可扩展的记忆架构，通过动态提取、整合和检索对话中的关键信息来赋予AI Agent长期记忆能力，并在LOCOMO基准测试中显著优于现有方法，同时大幅降低了计算开销。
-
Efficient Reasoning for LLMs through Speculative Chain-of-Thought
Published: at 01:19 AM 79.97 🤔本文提出了推测思维链（SCoT）框架，通过轻量级草稿模型并行生成多个思维链草稿，并由微调后的目标大模型选择最佳草稿或决定重新思考，从而在保持接近大模型准确率的同时，显著降低了大型语言模型的推理延迟。
-
Reinforcement Learning for Reasoning in Large Language Models with One Training Example
Published: at 01:19 AM 85.07 🤔本文发现，通过对大型语言模型应用带有可验证奖励的强化学习，仅使用一个训练示例即可显著提升其数学推理能力，效果可媲美使用数千示例进行训练，并揭示了饱和后泛化、跨领域泛化等现象，强调了策略梯度和探索的重要性。
-
Dynamic Fisher-weighted Model Merging via Bayesian Optimization
Published: at 01:19 AM 86.13 🤔本文提出了动态 Fisher 加权合并 (DF-Merge) 方法，通过贝叶斯优化动态调整微调模型的缩放系数，并在这些缩放模型上利用 Fisher 信息进行加权合并，从而高效地创建性能显著优于现有基线的多任务模型。
-
Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs
Published: at 01:18 AM 89.54 😐本文通过实证研究发现，大型语言模型在推理任务中存在"过度思考"简单问题和"思考不足"困难问题的现象，其推理长度与正确性呈非单调关系，且简单偏好更短回答可在保持准确率的同时显著减少生成长度。
-
Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
Published: at 01:18 AM 81.62 🤔本文提出DPE，一种无需训练的长文本外推方法，通过检测RoPE不同维度组的有效相对距离并识别关键维度，有选择地调整这些关键维度的位置索引，显著扩展了LLM的上下文窗口并提升了长文本任务性能。
-
Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decomposition
Published: at 01:18 AM 93.53 😐本文提出了 Trace-of-Thought Prompting，一种基于提示的知识蒸馏框架，通过将复杂问题分解为可管理的步骤，有效地将高资源模型的推理能力迁移到低资源模型，显著提升了低资源模型在算术推理任务上的表现，且无需大量微调。
-
Base Models Beat Aligned Models at Randomness and Creativity
Published: at 01:17 AM 85.43 😐本文通过在随机数生成、混合策略游戏和创意写作等需要不可预测性的任务上进行实验，发现流行的对齐技术会损害基础模型在这方面的能力，而基础模型在这些任务上表现更佳，这表明在常见基准性能和不可预测能力之间可能存在权衡。
-
Llama-Nemotron: Efficient Reasoning Models
Published: at 11:30 PM 72.72 🤔NVIDIA 发布了 Llama-Nemotron 系列开放模型，通过结合神经架构搜索、知识蒸馏、持续预训练、基于高质量合成数据的多阶段有监督微调和大规模强化学习，构建了在推理能力和效率上均达到领先水平、并支持动态推理模式切换的异构模型家族。
-
Reward Guidance for Reinforcement Learning Tasks Based on Large Language Models: The LMGT Framework
Published: at 11:16 PM 76.51 🤔本文提出了LMGT框架，通过利用大型语言模型的先验知识对强化学习的奖励进行动态调整，有效平衡了探索与利用，显著提高了样本效率并降低了训练成本，并在多种环境、算法以及机器人和推荐系统等复杂场景中验证了其有效性。
-
Which Attention Heads Matter for In-Context Learning?
Published: at 11:15 PM 90.67 👍本文通过对12个大型语言模型进行消融研究和训练动态分析，发现函数向量头是驱动少样本上下文学习的主要机制，尤其在大型模型中，并且许多函数向量头在训练过程中从归纳头演变而来，纠正了先前认为归纳头是主要驱动力的观点。
-
Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks
Published: at 11:15 PM 82.23 👍本文提出LLM代理可以通过自动收集和选择自身在序列决策任务中的成功轨迹作为上下文示例，显著提升性能，减少对人工知识工程的依赖。
-
SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillation
Published: at 11:15 PM 64.11 🤔本文提出了 SmallPlan 框架，通过结合 LLM 指导的蒸馏、模拟环境反馈的 SFT 和 RL，训练轻量级的小型语言模型 (SLM) 进行高效的机器人高层路径规划，使其在资源受限的边缘设备上实现接近大型模型 (LLM) 的性能。
-
Block Circulant Adapter for Large Language Models
Published: at 04:34 PM 71.85 🤔本文提出块循环适配器方法，通过利用块循环矩阵和FFT优化LLM的微调过程，显著降低存储和计算成本，同时通过学习率调整确保训练稳定。
-
From Attention to Atoms: Spectral Dictionary Learning for Fast, Interpretable Language Models
Published: at 04:33 PM 75.71 🤔本文提出光谱字典生成模型（SDGM），通过学习全局傅里叶字典和 token 混合系数替换自注意力机制，实现 O(KL) 复杂度的高效语言建模，并在基准数据集上取得竞争性 perplexity 和显著的资源节省。
-
On the generalization of language models from in-context learning and finetuning: a controlled study
Published: at 04:33 PM 82.00 🤔本文通过控制实验比较了语言模型在上下文学习和微调下的泛化能力，发现上下文学习更灵活，并提出通过数据增强方法显著改善微调的泛化性能。
-
Empirical Evaluation of Progressive Coding for Sparse Autoencoders
Published: at 04:33 PM 51.36 🤔本文通过实证评估比较了Matryoshka SAEs和基于字典幂律修剪的方法，以实现SAEs的渐进式编码，提高计算效率、重建保真度和可解释性。
-
Communication-Efficient Wireless Federated Fine-Tuning for Large-Scale AI Models
Published: at 04:33 PM 73.51 🤔本文提出了一种无线联邦LoRA微调框架，通过Sparsified Orthogonal Fine-Tuning (SOFT) 和Two Stage Federated Algorithm (TSFA) 优化参数稀疏化和动态资源分配，提高了通信效率和学习性能。
-
Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Math
Published: at 04:33 PM 83.56 👍本文提出了一种多阶段训练方案，包括大规模蒸馏、滚动偏好优化和可验证奖励的强化学习，显著提升了小型语言模型在数学推理任务中的性能，使3.8B参数的Phi-4-Mini-Reasoning模型超过了近两倍参数的开源基线模型。
-
Learning to Plan Before Answering: Self-Teaching LLMs to Learn Abstract Plans for Problem Solving
Published: at 04:33 PM 74.84 🤔本文提出LEPA自训练算法，通过训练LLM生成预期计划作为抽象元知识来提升问题解决泛化能力，并在多个推理基准上显著优于现有方法。
-
Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing
Published: at 04:33 PM 69.21 🤔本文提出Mixture of Sparse Attention (MoSA)方法，通过专家选择路由实现基于内容的稀疏注意力，显著提高了Transformer模型在相同计算预算下的语言建模性能，并优化了资源使用。
-
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization
Published: at 04:33 PM 62.92 🤔本文提出基于超网络的HYPEROFA方法，用于初始化新语言令牌嵌入，提高PLM对低资源语言的适应性，性能优于随机初始化并与OFA方法持平或更好。
- Published: at 04:33 PM 75.58 🤔
本文通过提出位置 ID 操纵的 PFT 方法，揭示并解决了 LLM 在角色分离学习中依赖捷径的问题，提高了模型的鲁棒性和安全性，同时保持了性能。
-
RWKV-X: A Linear Complexity Hybrid Language Model
Published: at 04:32 PM 87.80 👍本文提出RWKV-X，一种线性复杂度的混合语言模型，通过结合RWKV和稀疏注意力机制，提升长上下文建模能力，同时保持高效性和短上下文性能。
-
DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
Published: at 04:32 PM 56.98 🤔本文提出DeepSeek-Prover-V2，通过子目标分解和强化学习统一非正式和正式数学推理，显著提升了神经定理证明的性能，在多个基准上达到最先进水平。
-
AdaptMI: Adaptive Skill-based In-context Math Instruction for Small Language Models
Published: at 04:32 PM 80.86 🤔本文提出AdaptMI和AdaptMI+自适应方法，通过基于奖励模型检测问题难度并针对困难问题选择技能-based in-context示例，提高小语言模型在数学推理任务中的性能，同时避免认知过载。
-
CachePrune: Neural-Based Attribution Defense Against Indirect Prompt Injection Attacks
Published: at 04:32 PM 55.01 🤔本文提出CachePrune方法，通过基于DPO损失的特征归因识别并修剪KV缓存中的关键神经元，防御间接提示注入攻击，同时保持模型响应质量。
-
TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts
Published: at 04:32 PM 81.41 🤔本文提出 TT-LoRA MoE 框架，通过两阶段解耦的专家训练和路由机制，实现了参数高效的多任务学习，显著减少计算开销并保持性能。
-
Kimi-Audio Technical Report
Published: at 04:32 PM 64.70 🤔本文提出Kimi-Audio，一个开源的音频基础模型，通过结合音频分词、LLM处理和逆分词的统一架构，以及大规模多模态训练，实现了音频理解、生成和对话的多任务SOTA性能。
-
Beyond Public Access in LLM Pre-Training Data
Published: at 04:32 PM 53.67 🤔本文通過DE-COP成員推斷攻擊方法，使用O'Reilly書籍數據集證明OpenAI的GPT-4o可能訓練過非公共版權內容，突顯了LLM預訓練數據中非公共數據使用增加的趨勢及加強透明度和許可框架的必要性。
-
MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness
Published: at 04:32 PM 63.08 🤔本文提出MAC-Tuning方法，通过分步微调分离答案预测和置信度估计，提升LLMs在多问题设置下的知识边界意识，显著减少幻觉并改善性能。
-
Efficient Knowledge Transfer in Multi-Task Learning through Task-Adaptive Low-Rank Representation
Published: at 04:32 PM 87.85 🤔本文提出 TA-LoRA 方法，通过任务自适应低秩表示和快速-缓慢权重机制提升多任务学习的知识转移效率，实现对未见任务的优异泛化性能，同时保持高参数效率。
-
Diff-Prompt: Diffusion-Driven Prompt Generator with Mask Supervision
Published: at 04:32 PM 54.39 🤔本文提出Diff-Prompt方法，使用扩散模型基于掩码监督生成细粒度提示信息，显著提升预训练多模态模型在复杂指代表达理解任务上的性能，同时保持高效微调。
-
HyPerAlign: Hypotheses-driven Personalized Alignment
Published: at 04:32 PM 65.01 🤔本文提出HyPerAlign方法，通过假设驱动的少样本学习实现LLM的个性化对齐，提高了模型对个体用户的适应性和安全性，同时减少了对微调的依赖。
-
Toward Efficient Exploration by Large Language Model Agents
Published: at 04:31 PM 79.45 🤔本文通过使用 LLMs 显式实现后验采样 RL 算法，显著提高了 LLMs 代理在自然语言环境中的探索效率，同时保留了经典算法的统计性能优势。
-
Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review
Published: at 04:31 PM 50.24 🤔本文通过系统综述和实证基准测试，比较了LLMs的不确定性量化与校准方法，揭示了这些方法的有效性、局限性，并为未来研究提供了关键洞见。
-
Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare
Published: at 04:31 PM 67.24 🤔本文通过实证实验指导在医疗专业应用中语言模型的选择，强调微调小语言模型和领域特定预训练的显著优势，使其在特定任务上超越零-shot 大语言模型。
-
LLM Enhancer: Merged Approach using Vector Embedding for Reducing Large Language Model Hallucinations with External Knowledge
Published: at 04:31 PM 51.20 🤔本文提出 LLM-ENHANCER 系统，通过合并多个在线数据来源并使用向量嵌入减少大型语言模型的幻觉，提高响应准确性，同时保持自然性和经济性。
-
Meeseeks: An Iterative Benchmark Evaluating LLMs Multi-Turn Instruction-Following Ability
Published: at 04:31 PM 53.12 🤔本文提出Meeseeks多轮指令遵循基准，通过迭代反馈机制系统评估LLMs的自纠错能力，发现模型在多轮互动中性能显著提升。
-
An Empirical Study of Evaluating Long-form Question Answering
Published: at 04:31 PM 55.78 🤔本文实证研究了长形式问题回答的自动评估指标，证明了基于LLM的指标在准确性和稳定性上的优势，同时分析了其偏差和改进策略。
-
Think, Prune, Train, Improve: Scaling Reasoning without Scaling Models
Published: at 04:31 PM 82.99 👍本文提出 Think, Prune, Train 框架，通过迭代监督微调和基于正确性的数据修剪，实现模型在不增加规模的情况下提升推理能力，避免模型坍缩。
-
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models
Published: at 04:31 PM 59.95 🤔本文提出Token-Shuffle方法，通过利用视觉词汇维度冗余动态合并和恢复图像令牌，实现高效的高分辨率文本到图像生成，同时在统一自回归框架下保持出色性能。
-
Phi-4-reasoning Technical Report
Published: at 04:31 PM 77.09 😐本文通过数据导向的监督微调和强化学习，开发了小型LLM Phi-4-reasoning 和 Phi-4-reasoning-plus，提升了其在复杂推理任务上的性能，与大型模型竞争。
-
X-Fusion: Introducing New Modality to Frozen Large Language Models
Published: at 04:31 PM 66.52 🤔本文提出X-Fusion框架，通過凍結LLM參數並添加雙塔結構，高效實現多模態理解和生成，同時保留原始語言能力。
-
Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injections
Published: at 04:30 PM 72.07 🤔本文通过提出攻击框架和实验评估，揭示了LLM-as-a-judge系统的prompt injection漏洞，并推荐使用多模型委员会等策略提升鲁棒性。
-
Param$Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost
Published: at 04:30 PM 86.83 😐本文提出Param∆方法，通过直接添加参数差值在零成本下实现后训练知识向新基模型的转移，达到与传统后训练相当的性能。
-
Pushing the boundary on Natural Language Inference
Published: at 04:30 PM 56.51 🤔本文提出使用Group Relative Policy Optimization结合Chain-of-Thought学习的方法提升自然语言推理任务的性能，无需标注推理路径，通过参数高效微调在对抗性基准上实现最先进结果。
-
Monte Carlo Planning with Large Language Model for Text-Based Game Agents
Published: at 04:30 PM 55.97 🤔本文提出MC-DML算法，通过整合大型语言模型的动态记忆机制与蒙特卡罗树搜索，提升文本-based游戏代理的规划效率和性能，实验结果显示其在初始阶段就优于需多次迭代的强基线。
-
Learning Explainable Dense Reward Shapes via Bayesian Optimization
Published: at 04:30 PM 55.26 🤔本文提出一种通过Bayesian Optimization学习解释性密集奖励形状的方法，以解决RLHF中奖励稀疏问题，实现token级信用分配优化，提升训练效率和性能，同时保持最优政策不变。
-
Streaming, Fast and Slow: Cognitive Load-Aware Streaming for Efficient LLM Serving
Published: at 04:30 PM 60.43 🤔本文提出基于认知负载的适应性流式传输框架，用于优化 LLM 服务，通过动态调整输出速度减少计算资源消耗高达 16.8%，同时维持用户满意度。
-
TTRL: Test-Time Reinforcement Learning
Published: at 04:30 PM 93.49 👍本文提出测试时强化学习（TTRL）方法，通过多数投票估计奖励，在无标签测试数据上训练大语言模型，实现模型自演化并显著提升推理任务性能。
-
Does Knowledge Distillation Matter for Large Language Model based Bundle Generation?
Published: at 04:30 PM 72.97 🤔本文首次系统探索知识蒸馏技术在基于大语言模型的捆绑生成任务中的应用，通过提出一个全面的 KD 框架和实验验证，证明了在减少计算需求的同时能保持甚至提升性能。
-
Replay to Remember: Retaining Domain Knowledge in Streaming Language Models
Published: at 04:30 PM 65.14 🤔本文通过结合LoRA和轻量级重放机制的方法，在流式学习条件下帮助大型语言模型减轻灾难性遗忘，同时实现了实时域适应。
-
When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars
Published: at 04:30 PM 69.59 🤔本论文通过上下文无关文法合成数据研究了元数据条件化在语言模型预训练中的影响，发现其对长提示任务有益但对短提示任务有害，揭示了潜在语义推断的权衡。
-
W-PCA Based Gradient-Free Proxy for Efficient Search of Lightweight Language Models
Published: at 04:30 PM 53.85 🤔本文提出 W-PCA 方法，通过结合参数数量和主成分分析，提供一种高效的零-shot NAS 代理，用于轻量级语言模型的搜索，显著提高了搜索效率和模型性能。
-
MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
Published: at 04:30 PM 70.65 🤔本文提出MegaScale-Infer系统，通过分离注意力模块和FFN模块的并行策略以及高效M2N通信库，优化大规模MoE模型的推理效率，实现高达1.90倍的吞吐量提升。
-
Efficient Single-Pass Training for Multi-Turn Reasoning
Published: at 04:30 PM 74.08 🤔本文提出了一种通过响应令牌复制和自定义注意力掩码来实现多轮推理对话单次前向传递训练的方法，显著提高了训练效率，同时维护了推理可见性和位置一致性。
-
MOOSComp: Improving Lightweight Long-Context Compressor via Mitigating Over-Smoothing and Incorporating Outlier Scores
Published: at 04:29 PM 77.68 🤔本文提出MOOSComp方法，通过在训练中添加inter-class cosine similarity loss缓解over-smoothing问题，并在压缩中整合outlier分数保留关键token，显著提升了任务无关的长上下文压缩性能和泛化能力。
-
Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability
Published: at 04:29 PM 82.91 👍本文通过引入批评-修订提示和比较多任务训练、反事实训练及其结合的方法，系统评估了知识蒸馏对语言模型性能和可解释性的影响，发现多任务训练在性能上表现出色，而结合批评-修订提示的方法显著提升了可解释性。
-
Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMs
Published: at 04:29 PM 69.81 🤔本文提出UniME框架，通过文本判别知识蒸馏和硬负例增强指令微调，利用多模态大语言模型学习通用的多模态嵌入，提高了下游任务的判别性和组合能力。
-
Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning
Published: at 04:29 PM 50.69 🤔本文提出PaperCoder框架，通过多代理LLM的多阶段管道自动从机器学习论文生成高质量代码仓库，提升了研究的可复现性，并在基准测试中显著优于现有方法。
-
TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAs
Published: at 04:29 PM 70.02 🤔本文提出TeLLMe，一种能量高效的三元LLM FPGA加速器，通过表查找矩阵引擎和反向注意力优化，支持预填充和解码阶段，在7W功率下实现高达9.51 tokens/s吞吐量和低预填充延迟。
-
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
Published: at 04:29 PM 85.10 👍论文通过大规模实验分析了Transformer LLMs中稀疏注意力的效率-准确性权衡，揭示了长序列下更大稀疏模型的优势，并建立了可推广的缩放定律。
-
StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation
Published: at 04:29 PM 79.93 🤔本文提出 StreamRL 框架，通过分离式流生成架构优化 RL 训练，解决了流水线和偏斜气泡问题，提高了 LLMs RL 训练的吞吐量和成本效率。
-
A closer look at how large language models trust humans: patterns and biases
Published: at 04:29 PM 56.91 🤔本研究通过模拟实验首次揭示大型语言模型对人类的隐性信任模式，显示其类似于人类受可信度维度影响，但存在模型异质性和人口统计学偏差。
-
Collaborating Action by Action: A Multi-agent LLM Framework for Embodied Reasoning
Published: at 04:29 PM 76.05 🤔本文提出MINDcraft框架和MineCollab基准，评估LLM在多代理具身协作中的性能，揭示了当前模型在通信和协调方面的局限性，并呼吁开发更先进的协作方法。
-
Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement
Published: at 04:29 PM 71.89 🤔本文提出动态参数化RAG框架DyPRAG，通过训练一个轻量级参数翻译器在测试时动态转换文档为参数知识，显著降低成本、提升泛化能力和缓解RAG幻觉问题。
-
DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs
Published: at 04:29 PM 63.91 🤔本文提出DYMU框架，通过动态令牌合并和虚拟取消合并的训练-free方法，显著提高了VLMs的计算效率，同时在多个基准上保持了与完整模型相似的性能。
-
On-Device Qwen2.5: Efficient LLM Inference with Model Compression and Hardware Acceleration
Published: at 04:29 PM 53.38 🤔本文提出软件硬件协同优化框架，通过 AWQ 模型压缩和 FPGA 加速在边缘设备上高效部署 Qwen2.5-0.5B 模型，实现 55.1% 的压缩率和 5.1 tokens/s 的推理速度，同时保持较高准确性。
-
PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning
Published: at 04:28 PM 53.32 🤔本文提出PointLoRA方法，通过低秩适配和多尺度令牌选择，实现点云模型的参数高效微调，显著减少可训练参数同时在多个数据集上达到竞争性性能。
-
Less is More: Enhancing Structured Multi-Agent Reasoning via Quality-Guided Distillation
Published: at 04:28 PM 77.44 🤔本文提出了一种质量导向的多代理框架，通过提示诱导、检索增强合成和奖励过滤从少量标注数据中提炼高质量监督信号，提升LLMs在低资源结构化推理任务中的性能。
-
SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference
Published: at 04:28 PM 59.39 🤔本研究提出 SpargeAttn，一种通用稀疏注意力机制，通过两阶段在线过滤器和量化技术加速各种模型的推理，同时保持端到端性能无损。
-
WALL-E 2.0: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents
Published: at 04:28 PM 75.27 🤔本文提出WALL-E 2.0，一种无训练的神经符号学习方法，通过对齐LLM与环境动态构建精确世界模型，并结合模型预测控制框架，显著提升了LLM代理在开放世界任务中的性能。
-
MARFT: Multi-Agent Reinforcement Fine-Tuning
Published: at 04:28 PM 56.39 🤔本文提出MARFT框架，通过序列决策和信任区域优化在LLM-based多代理系统中实现高效强化微调，提升代理协作能力并解决传统MARL的适用性问题。
-
Quantum-Enhanced LLM Efficient Fine Tuning
Published: at 04:28 PM 72.15 🤔本文提出量子张量混合适配（QTHA）方法，通过整合量子神经网络和张量网络，实现LLM的参数高效微调，显著减少参数量并提升性能，为量子增强人工智能奠定基础。
-
Adaptive Layer-skipping in Pre-trained LLMs
Published: at 04:28 PM 62.55 🤔本文提出FlexiDepth方法，通过插件式路由器和适配器实现预训练LLM的自适应层跳过，提高计算效率同时保持生成性能，并通过实验揭示了token类型对计算需求的影响。
-
Synergizing RAG and Reasoning: A Systematic Review
Published: at 04:28 PM 54.75 🤔本论文系统综述了检索增强生成（RAG）与推理能力的协同整合，构建了多维分类框架、提供了实用指南，并指出了未来研究方向，以推进RAG系统在复杂任务中的认知能力。
-
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
Published: at 04:28 PM 86.30 😐本文提出 EAGLE-3 方法，通过移除特征预测约束和多层特征融合技术，显著提高了大语言模型的推理加速比，并在实验中实现了高达 6.5 倍的无损速度提升。
-
Training Plug-n-Play Knowledge Modules with Deep Context Distillation
Published: at 04:28 PM 69.06 🤔本文提出使用深度上下文蒸馏训练可插拔知识模块的方法，能够在低数据场景下高效整合文档知识，并通过实验证明其在问答任务中优于传统方法且与 RAG 具有协同效应。
-
Humanity's Last Exam
Published: at 04:28 PM 58.39 👍本文引入HUMANITY'S LAST EXAM基准测试，通过专家创建的挑战性多模态问题，解决现有LLM基准饱和问题，评估模型在封闭式学术任务中的能力。
-
ElChat: Adapting Chat Language Models Using Only Target Unlabeled Language Data
Published: at 04:28 PM 75.31 🤔本文提出ElChat方法，通过直接在目标无标签数据上适应聊天模型，并结合模型合并和权重复制技术，成功恢复聊天能力和指令遵循，同时在目标语言性能和安全方面表现出色。
-
LIFT: Improving Long Context Understanding of Large Language Models through Long Input Fine-Tuning
Published: at 04:27 PM 82.62 👍本文提出LIFT框架，通过长输入微调和Gated Memory适配器提升短上下文LLMs的长上下文理解能力，实验显示显著性能改进。
-
HSI: Head-Specific Intervention Can Induce Misaligned AI Coordination in Large Language Models
Published: at 04:27 PM 78.97 🤔本文提出Head-Specific Intervention (HSI)方法，通过针对特定注意力头的激活干预，成功诱导Llama 2模型在AI协调行为上绕过安全对齐，效果优于监督微调和其它干预策略。
-
Reason2Attack: Jailbreaking Text-to-Image Models via LLM Reasoning
Published: at 04:27 PM 58.67 🤔本文提出Reason2Attack方法，通过基于Frame Semantics的CoT示例合成和带攻击过程奖励的强化学习，增强LLM的推理能力，以高效生成对抗性提示实现对T2I模型的越狱攻击。
-
ASIDE: Architectural Separation of Instructions and Data in Language Models
Published: at 04:27 PM 53.34 🤔本文提出ASIDE方法，通过在嵌入级别应用固定正交旋转实现大型语言模型的指令-数据架构分离，提高了模型的安全性和对提示注入攻击的鲁棒性，同时不牺牲性能。
-
SAGE: A Framework of Precise Retrieval for RAG
Published: at 04:27 PM 64.13 🤔本文提出SAGE框架，通过语义分割、基于梯度的块选择和LLM自反馈机制，提高RAG系统的检索精度和问答性能，同时显著降低成本。
-
Latent Factor Models Meets Instructions: Goal-conditioned Latent Factor Discovery without Task Supervision
Published: at 04:27 PM 59.70 🤔本文提出Instruct-LF方法，通过结合LLMs的指令遵循能力和梯度-based统计模型，实现无需任务监督的目标导向潜在因素发现，提高了下游任务性能并在人工评估中被偏好。
-
Codenames as a Benchmark for Large Language Models
Published: at 04:27 PM 77.18 👍本论文提出使用Codenames游戏作为LLMs推理能力的基准，通过实验评估不同LLMs在语言理解、战略推理和合作方面的表现，展示了它们的独特行为和泛化潜力。
-
Less is More: Towards Green Code Large Language Models via Unified Structural Pruning
Published: at 04:27 PM 66.29 🤔本文提出Flab-Pruner，一种结合词汇、层和FFN剪枝的统一结构剪枝方法，通过KL散度优化和自定义微调策略，在减少代码LLM参数的同时保持高性能和效率。
-
EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning
Published: at 04:27 PM 60.29 🤔本文提出EPO方法，通过强化学习优化一个专门的战略推理模型，辅助任意LLM代理在动态环境中实现长期目标对齐，提升战略推理能力。
-
Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding
Published: at 04:27 PM 83.39 👍本文系统揭示了自注意力模块中大规模值在LLM上下文知识理解中的关键作用，并通过实验证明其源于旋转位置编码（RoPE），为模型优化和量化策略提供新洞见。
-
Evidence of conceptual mastery in the application of rules by Large Language Models
Published: at 04:26 PM 62.79 🤔本文通过心理实验证明大型语言模型在规则应用中表现出概念掌握能力，能够泛化到新情境并部分模仿人类对时间压力等语境的敏感性。
-
From System 1 to System 2: A Survey of Reasoning Large Language Models
Published: at 04:26 PM 75.04 🤔本文综述了从基础LLMs向推理LLMs的演进，通过整合System 2技术提升AI的逐步推理能力，并在基准测试中展示了显著性能改进。
-
SuperARC: An Agnostic Test for Narrow, General, and Super Intelligence Based On the Principles of Recursive Compression and Algorithmic Probability
Published: at 04:26 PM 54.84 🤔本文提出SuperARC测试框架，通过算法概率和Kolmogorov复杂度的原理，设计了一个客观的AGI和ASI评估方法，证明递归压缩等价于预测，并展示了LLMs的局限性。
-
PennyLang: Pioneering LLM-Based Quantum Code Generation with a Novel PennyLane-Centric Dataset
Published: at 04:26 PM 55.37 🤔本文提出 PennyLang 数据集和 RAG/GraphRAG 框架，通过提升 LLM 在 PennyLane 量子代码生成中的准确性和正确性，填补了 AI 辅助量子编程的空白。
-
Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks
Published: at 04:26 PM 70.33 🤔本文提出PLAN-AND-ACT框架，通过分离规划和执行模块、利用合成数据训练和动态重规划，提高LLM代理在复杂长期任务中的性能，并在web导航基准上达到state-of-the-art结果。
-
You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects
Published: at 04:26 PM 53.07 🤔本文提出ExecutionAgent，一个基于LLM的自主代理，通过meta-prompting和迭代反馈机制自动设置并执行任意软件项目的测试套件，显著提高了测试执行的成功率和准确性。
-
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Published: at 04:26 PM 76.52 🤔本文首次系统调查了大型语言模型高效推理的进展，通过分类模型、输出和提示-based方法，探讨了减少"过度思考"现象的策略，以优化计算效率并保持推理能力。
-
Prompt-Based Cost-Effective Evaluation and Operation of ChatGPT as a Computer Programming Teaching Assistant
Published: at 04:26 PM 66.50 🤔本文通过设计基于ICL和CoT的提示模板，实现了ChatGPT在编程教育中的成本效益评估和操作，显著降低了手动评估需求并提升了反馈的结构化分析。
-
State Space Models are Strong Text Rerankers
Published: at 04:26 PM 50.53 🤔本文通过全面benchmark比较状态空间模型如Mamba与Transformer在文本重排序任务中的性能和效率，发现Mamba模型可实现类似性能但效率较低，并强调了未来优化方向。
-
Towards Reasoning Ability of Small Language Models
Published: at 04:26 PM 86.53 😐本文通过系统基准测试72个SLMs，证明小型语言模型可以通过结构化训练和压缩技术实现与大型模型相当的推理能力，从而挑战了规模依赖的传统观点。