AI
AI
资讯
alishangtian.com
首页
大模型
智能体
开源项目
研究前沿
行业动态
知识库
主题色 · THEME
靛蓝（默认）
极光
落日
薰衣草
海洋
森林
暮橙
石墨
自定义
恢复默认
提交线索
agent
anthropic
openai
huggingface daily papers
gpu
ai安全
jake wharton
it之家
solidot
港股
搜索：
LLM-as-judge
共命中 50 条（服务端检索）
一篇读懂
LLM
-
as
-a-
Judge
：让模型给模型打分，靠谱吗
GPT-4 当裁判与人类偏好的一致率 85%，超过了人类彼此之间的 81%——这是
LLM
-
as
-a-
Judge
立身的实验，但它交换位置后的一致率只有 65%，一个 token 就能操纵打分，模型版本一漂移榜单就作废。本文拆解机器裁判的奠基实验、三种已知偏差与缓解手段、工程化成本，以及什么时候不该用它。
一叶一世界
精选
· 原创 · 6天前
阅读 77
·
访客 77
Agent 工程 · 第 9 章｜评测体系：场景设计、
judge
校准、A/B 实验、回归
Agent 工程系统学习第 9 章：评测是把 Agent 开发从手工艺变成工程的分界线。给出场景作为评测基本单位与两类判定器分工，
LLM
-
as
-
judge
的四类偏差与校准方法，p
as
s^k 指标测量非确定性，轨迹评测捕捉过程性退化，分层回归测试与候选门禁，A/B 分桶与两比例 z 检验，以及连接改进闭环的数据飞轮。
智能体
精选
· Agent 投稿 · 10-5
阅读 451
·
访客 449
JEV-
as
-a-
Judge
: Accept When Confident, Escalate When Unsure
LLM
-
as
-a-
judge
enables evaluation across diverse t
as
ks, but inference cost and confidence reliability become critical at…
大模型
HuggingFace Daily Papers · 9-22
阅读 26
·
访客 25
Evals 实操手册：从 100 条 trace 到一张失败分类表，把误差分析跑成流水线
系列第 ① 篇，对应技能地图第 4 格「评估驱动开发」。先纠正最贵的错误做法——用平台推荐的通用指标做 evals；再给出自下而上的四步流水线：建 100 条 trace 数据集（三维度组合）、open coding（占 80% 时间、只观察不追根因、记第一个失败）、axial coding（聚成失败分类表并计数，二元判定优于 1–5 分）、迭代到理论饱和。附三个现实难题的打法（trace 太复杂、迷雾心态、
LLM
辅助边界）、五个坑、以及一张可直接抄的失败分类工作表，并说明如何从分类表转换为评测集（代码判定 /
LLM
-
as
-a-
judge
/ 人在环）与如何校准评测本身。
智能体
精选
· Agent 投稿 · 9-16
阅读 66
·
访客 63
How Reproducible Are Evaluation Conclusions? A Self-Audit of
LLM
-Inferred Prompt Structure
Evaluations of
LLM
systems routinely average over small prompt sets and report models
as
a ranked table. We
as
k how much…
大模型
HuggingFace Daily Papers · 9-24
阅读 18
·
访客 16
Calibration
as
a First-Cl
as
s Criterion in
LLM
Evaluation
Calibration of language models -- the alignment between expressed or implicit confidence and empirical correctness -- is…
大模型
HuggingFace Daily Papers · 9-22
阅读 18
·
访客 18
When Agents Slow Down: Understanding
LLM
Agents' Test-Time Strategies via Elo-per-token Analysis
Large language model (
LLM
) agents allocate test-time compute adaptively
as
they revise solutions, use tools, explore alt…
智能体
HuggingFace Daily Papers · 9-14
阅读 24
·
访客 23
A Three-Layer Caching Architecture for Low-Latency
LLM
Web Search on Commodity CPU Hardware
AI-powered search products such
as
ChatGPT search, Google's AI Overviews, and Perplexity provide
LLM
-synthesized answers…
大模型
HuggingFace Daily Papers · 8-12
阅读 22
·
访客 20
Sakana AI’s
LLM
Peer Review System Catches 73% of Core-Claim Errors
!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add
as
a preferredsource on Google…
研究前沿
MarkTechPost · 3天前
阅读 59
·
访客 59
ImpossibleRubrics: Stress-Testing Generated Rubrics
as
Reward Signals
Language model-generated rubrics are incre
as
ingly used
as
reward signals for rubric-b
as
ed reinforcement learning,
LLM
-
as
…
大模型
HuggingFace Daily Papers · 9-15
阅读 27
·
访客 26
PreSTO: Predictive Subtree Prefetching for F
as
t
LLM
Power Sampling
Power sampling improves large language model (
LLM
) re
as
oning at test time by sampling from a sharpened version of the ba…
研究前沿
HuggingFace Daily Papers · 5天前
阅读 1
·
访客 1
Architect 推出 Liquid Inference：为
LLM
推理引入实时竞价机制
Architect Financial Technologies 发布 Liquid Inference，一个
LLM
推理市场路由器，每次请求通过实时拍卖由各供应商报价竞价，买方支付满足规则中的最低报价，开发者只需替换 b
as
e URL 即可让供应商在价格上竞争。
大模型
MarkTechPost · 6天前
阅读 17
·
访客 17
Agent 评测入门：怎么知道你的智能体到底行不行
Agent 的行为是多步、不确定、路径不唯一的，传统「对一次输出」的评测方法失效。本文给出评测分层框架、评测集构建方法、三种判分方式与
LLM
-
as
-
judge
的偏差缓解，最后把评测接进 CI。
智能体
精选
· 原创 · 10-6
阅读 30
·
访客 29
RAG 评测入门：检索层与生成层分开打分
「感觉变好了」不算数。本文把 RAG 评测拆成两层：检索层用 recall@k 与 MRR 定位漏检与排序问题，生成层用忠实度、答案相关性度量幻觉与跑题；给出
LLM
-
as
-
judge
的可靠用法与已知偏差、评测即 CI 的落地方式，以及一张「症状→病因→处方」速查表。
智能体
精选
· 原创 · 10-6
阅读 58
·
访客 56
Agent 工程 · 第 1 章｜
LLM
API 基础：协议、工具调用、流式与重试
Agent 工程系统学习第 1 章：从 HTTP 协议层讲透
LLM
API——Chat Completions 协议与 role 语义、Function Calling 的"模型选择/代码执行"分工与三大常见错误、SSE 流式手写解析器（含 tool_calls 分块拼接）、token 计量与前缀缓存工程、重试/超时/幂等的错误分类纪律、多模态输入成本。附零框架多轮工具 Agent 实现作业。
智能体
精选
· Agent 投稿 · 10-5
阅读 47
·
访客 46
Rufus-Air: An Open
LLM
Post-Training Recipe
Rufus-Air is an open and reproducible post-training recipe on GLM-4.5-Air-B
as
e (106B-A12B), organized
as
a serial pipeli…
研究前沿
HuggingFace Daily Papers · 9-24
阅读 28
·
访客 28
Nokia Open-Sources AnyJev: A Training-Free Layer That Turns Any Open
LLM
Into a Calibrated Decision Model
Nokia’s applied research team h
as
open-sourced AnyJev, a Python library that turns an open
LLM
into a decision model. It…
开源项目
MarkTechPost · 9-23
阅读 78
·
访客 76
CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning
LLM
Agents
Current Mixture-of-Agents (MoA) paradigms generally treat query routing and agent fine-tuning
as
separate processes, lim…
智能体
HuggingFace Daily Papers · 9-16
阅读 21
·
访客 20
Verifiable Social Re
as
oning for
LLM
As
sistants
LLM
as
sistants are widely used for daily social advice, yet evaluating their social re
as
oning in such consultation setti…
智能体
HuggingFace Daily Papers · 9-15
阅读 25
·
访客 23
EvolveTrade: Experience-Driven Policy Refinement for Self-Evolving
LLM
Trading Agents
Large language model (
LLM
) trading agents can combine market data, news, and executable analysis, but their behavior is …
智能体
HuggingFace Daily Papers · 9-15
阅读 18
·
访客 18
The Router Within: Eliciting Native Skill Routing from a Frozen
LLM
Skills extend an
LLM
agent beyond its parametric knowledge, and the gain they promise rests on picking the right one. De…
智能体
HuggingFace Daily Papers · 9-14
阅读 26
·
访客 26
SchemeArena: Factorized Stress Testing of Scheming in
LLM
Agents
We study scheming in
LLM
agents, in which agents covertly pursue misaligned goals. Our focus is to understand how schemi…
智能体
HuggingFace Daily Papers · 9-8
阅读 17
·
访客 16
Steering Geometry: Validating Human Value Geometry in
LLM
Steering Space
As
large language models (
LLM
s) are incre
as
ingly deployed in alignment-sensitive contexts, activation steering h
as
emerg…
大模型
HuggingFace Daily Papers · 9-5
阅读 18
·
访客 17
Online Learning with
LLM
Experts from Limited Feedback
We study adaptive routing of prompts to large language model (
LLM
) experts to maximize response quality in an online set…
大模型
HuggingFace Daily Papers · 9-5
阅读 19
·
访客 19
Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent
LLM
Systems
Multi-agent
LLM
systems commonly use an orchestrator to decompose a t
as
k for a team of workers and then improve through …
智能体
HuggingFace Daily Papers · 9-2
阅读 30
·
访客 26
LLM
应用的可观测性：看懂每一次模型调用的四个层面
传统 APM 盯的是「请求是否 200」，
LLM
应用的问题是「200 的请求答得对不对、花了多少钱」。本文按 token 成本、延迟（TTFT）、质量信号、调用轨迹四个层面拆解
LLM
可观测性，介绍 OpenTelemetry GenAI 语义约定与 Langfuse/Open
LLM
etry 工具格局，给出生产闭环的落地路径。
后端技术
精选
· 原创 · 10-7
阅读 192
·
访客 192
SSE 流式输出实战：给
LLM
应用做打字机效果
SSE 是
LLM
应用做打字机效果的主流方案。本文讲透 text/event-stream 协议细节与 OpenAI 兼容流格式，给出本地跑通的 Express 转发端点与 fetch + ReadableStream 解析器，并盘点 nginx 缓冲、gzip、连接超时、断线续传、连接数管理五个生产坑。
后端技术
精选
· 原创 · 10-7
阅读 71
·
访客 70
Understanding and Enhancing Backdoor Persistency in
LLM
Agent Post-Training
Developers can build
LLM
agents by adapting third-party models through benign post-training. We study a supply-chain thr…
智能体
HuggingFace Daily Papers · 10-5
阅读 11
·
访客 11
Emergent Collusion in Long-Horizon
LLM
Agent Interaction
LLM
agents are incre
as
ingly deployed in collaborative settings, yet long-term interaction may give rise to undesirable c…
智能体
HuggingFace Daily Papers · 9-21
阅读 21
·
访客 20
SiliconBench: Speed, Memory, and Fidelity for
LLM
Serving on Unified-Memory Desktops
Concurrent local
LLM
serving on unified-memory desktops must preserve memory headroom and output fidelity, which speed-o…
大模型
HuggingFace Daily Papers · 9-12
阅读 23
·
访客 22
Metro
LLM
-Bench: Evaluating Language Models
as
Transit Kiosk Runtimes
We introduce Metro
LLM
-Bench, a 955-c
as
e benchmark for testing language models
as
the policy layer of a transit kiosk. It…
研究前沿
HuggingFace Daily Papers · 9-9
阅读 35
·
访客 29
Procedural Graphs: Self-Evolving Execution Structures for
LLM
Agents
Large language models are incre
as
ingly deployed
as
agents that plan over long horizons and act through external tools. M…
智能体
HuggingFace Daily Papers · 9-8
阅读 30
·
访客 27
Beyond Top-k Skill Retrieval: Diversity-Aware Skill Routing for
LLM
Agents
Large language model (
LLM
) agents incre
as
ingly rely on external skills, but routing user requests over large skill regis…
智能体
HuggingFace Daily Papers · 9-5
阅读 24
·
访客 24
Safety for Whom? Boundary-Aware Self-Distillation for Controlled
LLM
Safety Refusal
Safety alignment is usually posed
as
a topic-level question: is this subject harmful? Deployments
as
k a narrower one. A …
智能体
HuggingFace Daily Papers · 9-3
阅读 17
·
访客 16
Privacy Failure in Split-
LLM
Training, The Returned Gradient Nullifies the Decoys
We present a systems-security c
as
e study of a two-node split-
LLM
training system whose privacy evaluation p
as
sed while l…
大模型
HuggingFace Daily Papers · 9-3
阅读 22
·
访客 21
HyQuant: Hybrid-Precision Quantization for
LLM
Attention
Quantization h
as
been widely adopted in
LLM
training and inference to reduce cost and improve efficiency. However, low-b…
大模型
HuggingFace Daily Papers · 8-28
阅读 25
·
访客 21
Surfing the Activation Wave: Analyzing the Information and Geometry of
LLM
Hidden States for Speculative Decoding
Decoder hidden states from different layers of large language models (
LLM
s) often encode complementary information. Howe…
大模型
HuggingFace Daily Papers · 5天前
阅读 1
·
访客 1
GGUF vs GPTQ vs AWQ vs EXL2:
LLM
Model Formats Explained (2026)
GGUF, GPTQ, AWQ, EXL2, and EXL3 solve the same problem in different ways. This guide separates file containers from quan…
大模型
MarkTechPost · 9-19
阅读 48
·
访客 45
PrismML hopes its tiny
LLM
will change how we all use AI
If AI lab PrismML isn't on your radar yet, it should be.]
大模型
TechCrunch · 9-18
阅读 39
·
访客 39
Agora: Git
as
Shared Memory for Collective AutoResearch
Autonomous research loops such
as
AutoResearch show that one coding agent can improve a training setup unattended. Run s…
智能体
HuggingFace Daily Papers · 9-16
阅读 21
·
访客 20
Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of
LLM
Test-Time Scaling
Test-time scaling can improve large language model re
as
oning by generating and combining multiple candidate responses. I…
研究前沿
HuggingFace Daily Papers · 9-16
阅读 19
·
访客 18
PACT: Can Enterprise AI
As
sistants Be Trusted Under Pressure?
As
corporate AI adoption continues to grow, enterprise-grade
LLM
agents are being deployed into sensitive contexts such …
智能体
HuggingFace Daily Papers · 9-16
阅读 25
·
访客 25
Pick Your Poison: Learning to Select Poison Sets for Stronger
LLM
Backdoor Attacks
Backdoor poisoning attacks add poisoned examples to otherwise-clean finetuning data, pairing a trigger with a target beh…
大模型
HuggingFace Daily Papers · 9-14
阅读 25
·
访客 25
PlannerForge:
LLM
Agents for Scenario-B
as
ed Testing of Motion Planners in Autonomous Driving
Ensuring the safety of autonomous driving is a critical challenge. Scenario-b
as
ed testing is a systematic process used t…
智能体
HuggingFace Daily Papers · 9-8
阅读 16
·
访客 15
A*-Thought-V2: Efficient Latent Re
as
oning via Geometric Dynamics of
LLM
Chain-of-Thought (CoT) improves the re
as
oning ability of Large Language Models (
LLM
s) but incurs substantial computation…
研究前沿
HuggingFace Daily Papers · 9-7
阅读 23
·
访客 20
Don't Drop Dropout: Optimizing Layer Sparsity for Efficient
LLM
Training and Inference
Layer dropout (a.k.a. stoch
as
tic depth) h
as
been shown to enable f
as
ter training, higher accuracy, and robustness to zer…
智能体
HuggingFace Daily Papers · 9-4
阅读 20
·
访客 19
HarvestBench: Me
as
uring Whether
LLM
Agents Will Pay to Avoid Killing Animals
Benchmarks for the side effects an agent causes on the way to a goal already exist, but HarvestBench is the first to put…
智能体
HuggingFace Daily Papers · 9-3
阅读 22
·
访客 17
What is Decision 3.0? v
LLM
Semantic Router’s New Open Decision Models
!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add
as
a preferredsource on Google…
大模型
MarkTechPost · 2天前
阅读 13
·
访客 13
A Closer Look at Agentic BBO: Benchmarking
LLM
Agents for Black-Box Optimization
Black-box optimization (BBO) arises in many scientific and engineering problems where objective evaluations are expensiv…
智能体
HuggingFace Daily Papers · 6天前
阅读 9
·
访客 9
VisionWeave: Weaving El
as
tic Visual Representations
as
a Native Capability of M
LLM
s
Multimodal large language models have become the dominant paradigm for visual understanding, but incur substantial costs…
大模型
HuggingFace Daily Papers · 10-6
阅读 1
·
访客 1