Tag: Representation Learning
All the articles with the tag "Representation Learning".
-
Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
Published: at 11:14 AM
76.90 🤔This paper introduces Latent Preference Coding (LPC), a framework that uses discrete latent codes to model multifaceted human preferences, consistently improving the performance of offline alignment algorithms like DPO, SimPO, and IPO across multiple LLMs and benchmarks.
-
Domain Regeneration: How well do LLMs match syntactic properties of text domains?
Published: at 11:19 AM 75.86 🤔本文通过‘LLM-regeneration’范式，使用Llama模型生成Wikipedia和新闻文本，发现生成文本在句法复杂性指标上表现出均值偏移、方差降低和长尾减少的系统性差异，揭示了模型在域匹配能力上的局限性。
-
Temporal Scaling Law for Large Language Models
Published: at 11:16 AM 75.44 🤔本文提出时间缩放定律（Temporal Scaling Law），通过动态双曲线法则建模LLM预训练中每个token位置的损失变化，精准预测整体测试损失演变，支持直接在目标模型上选择超参数并揭示学习动态。
-
Can a Crow Hatch a Falcon? Lineage Matters in Predicting Large Language Model Performance
Published: at 11:17 PM 75.40 🤔本文提出谱系正则化矩阵分解（LRMF）方法，通过利用大型语言模型的谱系关系显著提高性能预测准确性，在同质和异质模型场景下均优于传统方法，尤其在冷启动问题上表现突出。
-
I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?
Published: at 11:22 AM 74.26 🤔本文通过潜在变量模型和可识别性分析，证明大型语言模型通过下一词预测学习的表示近似为潜在概念后验概率对数的线性变换，支持线性表示假设，并提出结构化稀疏自编码器改进概念提取效果。