语言模型的训练目标浓缩为一个损失函数——交叉熵。但围绕它的变体（label smoothing、辅助损失、对比损失）各自解决不同的问题。本篇拆解交叉熵的直觉与数学、label smoothing 为什么能防过自信、辅助损失在 MoE 与多任务中的角色，以及对比损失在 Embedding 训练里的位置。

Evaluating an AI system requires disaggregated assessment, as performance varies across domains such as benchmark task t…

Diffusion Transformers deliver state-of-the-art video generation, but their long spatiotemporal sequences make attention…

正则化（防止过拟合的技术）在经典机器学习里是必修课，但在大模型训练里却「存在感稀薄」——预训练几乎不用 Dropout，微调却常开。这个反差本身是个有趣的信号。本篇拆解 Dropout 与权重衰减的机制、它们在大模型时代角色变化的原因、以及微调场景的正则化实践。

九年过去，Transformer 的骨架纹丝未动，外围部件却几乎换了一遍：归一化从 Post-LN 走到 QK-Norm，位置编码从正弦函数走到 RoPE 与 NoPE 混布，注意力从 MHA 走到 GQA 与 MLA。本文以 12 家旗舰模型的官方配置为据梳理这条演进主线，并回答一个问题——模型之间的差距，还剩多少在架构里？

Vision-language-action (VLA) models face a timing gap between low-rate inference and high-rate robot execution. We chara…

Video diffusion models repeatedly process long spatiotemporal token sequences during denoising, making attention a major…

Nunchux AI has released VC-Attention, a training-free low-bit attention kernel built for video Diffusion Transformers (D…

Ian Buck, vice president of hyperscale and high-performance computing at NVIDIA, Tuesday spoke on AI factory efficiency …

Iterative reference-conditioned image editing can introduce grid-like and granular textures, commonly described as digit…

Autoregressive (AR) video diffusion models have shown great potential in real-time video generation. Recent methods dist…

Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-b…