Pruning Quietly Breaks Smart-Home Agents
A September 2026 study of 19,500 smart-home tool-calling instances finds that dense models shatter under pruning while mixture-of-experts systems hold up — and the field isn't measuring degradation correctly.
September 2026, arXiv 2609.17515: Congjing Zhang and co-authors run 19,500 smart-home tool-calling cases through four LLMs, spanning dense Transformer, dense hybrid, and mixture-of-experts architectures. They prune these models by depth, width, hybrid, and expert methods, then fine-tune and measure what actually breaks in deployment.
Dense models don't degrade gracefully. They have a narrow safe pruning region, then fall off a cliff. Mixture-of-experts models survive much deeper pruning before they collapse. If you're picking an architecture for production agents, that's not a subtle trade-off; it's a structural difference.
The methodological point is sharper. Zhang et al. show that aggregate task accuracy hides the real failures. Pruning knocks out grounded specificity — the ability to name the right device or argument — long before it ruins the overall task structure. Dense models, when pruned too hard, start refusing to answer instead of guessing wrong. This kind of silent failure never shows up in a headline benchmark.
A separate Emergence World study follows eight worlds of ten agents and applies controlled stress events during the run. Its phishing analysis records hostile content entering persistent memory and an agent attempting to revisit an attack link approximately 46 hours after the campaign ended. Early recognition did not ensure later restraint. These are lasting effects in the tested environment, not an established early-cascade timetable for arbitrary production deployments. I would test persistence and recovery separately from immediate threat recognition.
I think this paper should force teams to rethink pruning for deployment. Most people treat pruning as a clean compression knob, but Zhang et al. make it clear: with dense models, you're walking a tightrope, and the safe zone is smaller than you think.
I don't know if these findings generalize beyond smart-home tool calling. The four models tested are a narrow slice, and this paper doesn't answer what happens in open-ended generation or long-document reasoning. Those may break in completely different ways. What I'd want to see next is whether the same sharp drop-off shows up when pruning retrieval-augmented systems, where the context window is the thing being trimmed. Until then, this is a warning, not a verdict.