The Evaluation-Feedback Gap Is Quietly Widening

2026-09-03

This is no longer on the current shelf — shelves rotate as new material clears the bar. The analysis below is unchanged. See what is featured now.

Why it earned a slot

We're getting spectacularly good at making evaluation cheaper while becoming increasingly blind to what users actually do. EarlyEval predicts agent outcomes early to cut evaluation costs dramatically. Incremental Pooled LLM Evaluation does the same for retrieval model selection. Meanwhile, the user feedback paper demonstrates that LLMs fundamentally cannot detect unique signals from organic user interactions — the very signals that would tell us if our cheap synthetic benchmarks are misleading us. This creates a dangerous bifurcation: the evaluation stack is splitting into cheap automated benchmarks that we can iterate against, and expensive, hard-to-interpret user signals that we're structurally unable to leverage. The result is an optimization loop that gets tighter and faster against synthetic metrics while drifting further from real-world performance. If you're building production AI, you should be deeply suspicious of any evaluation pipeline that doesn't include some path to organic user signal, no matter how elegant the benchmark.

Topics: evaluation, agents, production-ai