🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing

Latent Space · 2026-08-26

🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of ComputingLatent Space ▶ Play the video

This item is no longer on the current shelf. Its page remains available, and editorial corrections may be made. See what is featured now.

Why it earned a slot

Anima Anandkumar’s critique of foundation models for physics—especially weather—isn’t just a niche critique; it’s a fundamental challenge to the current AI hype cycle. Her work at Accelerated Understanding proves that physics simulations require structured, interpretable representations, not just scaling LLMs. The skepticism she faced isn’t just academic; it’s a warning that foundation models for domains beyond language are still in their infancy. This isn’t about ‘AI can’t do physics’—it’s about why we’re still treating physics as an afterthought in AI research. The episode’s tension—between the promise of general-purpose models and the reality of domain-specific constraints—mirrors the same debates we’ve seen in reinforcement learning (RL) vs. supervised learning.

Anandkumar’s emphasis on physics-informed neural networks (PINNs) and hybrid architectures feels like a necessary corrective to the ‘bigger model = better’ narrative. If this discussion doesn’t land in the next wave of AI papers, we’re doomed to repeat the same mistakes we made with language models—only now with higher stakes (climate, engineering, etc.). One glaring omission: no direct comparison to recent work on physics-guided diffusion models (e.g., those from DeepMind or Meta). That’s a gap—because those projects are trying to bridge the gap she’s highlighting. Also, the ‘open-source weather model’ claim feels premature without a clear roadmap for deployment. Skepticism is fair, but so is the need for concrete benchmarks.

The short version

Foundation models for physics? Anima Anandkumar just dropped a reality check—here’s why it matters for everyone building AI systems.

Why it matters

Foundation models for language have dominated AI, but physics, chemistry, and engineering domains require different architectures—ones that respect causality, not just correlations. Anandkumar’s work exposes a critical flaw in the ‘one-size-fits-all’ AI paradigm, forcing us to ask: When is a foundation model not a foundation model? This is urgent because climate modeling, drug discovery, and robotics are all waiting for AI that understands how the world works, not just how to predict it. The stakes aren’t just academic. Climate models alone cost billions to run, and if we’re chasing LLMs for physics, we’re repeating the same mistakes we made with NLP—where ‘foundation models’ turned into black boxes that fail at nuance.

Anandkumar’s critique is a call to arms for domain-specific AI, not just another scaling contest. Right now, the AI ecosystem is fixated on ‘AGI’ and ‘general intelligence,’ but Anandkumar’s focus on structured reasoning aligns with what agentic systems actually demand—interpretability and modularity are non-negotiable. The question isn’t whether AI can do physics; it’s whether we’re willing to build the right tools for it. The broader trend here is the shift from ‘data hunger’ to knowledge-aware models. Anandkumar’s work is a blueprint for how that might look—hybrid systems that combine deep learning with symbolic reasoning. That’s where the real breakthroughs will happen, not in another GPT-5.

My take

In agentic systems, interpretability isn’t optional—it’s a requirement for real-world deployment. Anandkumar’s critique resonates because ‘foundation models’ fail the moment you need to explain why a system made a decision, not just what it predicted. For example, in autonomous systems, you can’t just say ‘the model predicted a collision’—you need to know why it predicted it, and how to mitigate it. The real irony? The same teams pushing LLMs for physics are the ones who’ve already admitted their models don’t understand language. So why would we expect them to ‘understand’ physics? Anandkumar’s work is a wake-up call that we need new architectures—ones that combine the strengths of deep learning with the rigor of physics.

That’s where the next generation of AI will come from, not another round of scaling. The other angle? This is a test case for AI alignment. If we can’t get physics right, how can we expect AI to align with human values in high-stakes domains? Anandkumar’s skepticism isn’t just about models—it’s about capability. And that’s a conversation we’re not having enough.

How it connects

• Physics-informed neural networks (PINNs) are the missing link between deep learning and scientific modeling—but they’re still treated as a niche. If we don’t invest in them now, we’ll be playing catch-up when climate models or drug discovery demand AI that understands causality.
• The ‘foundation model’ label is a red herring—it implies generality, but physics, chemistry, and engineering require specialized architectures. This is why we’re seeing a resurgence of hybrid models (e.g., neural-symbolic systems, graph-based approaches).
• Anandkumar’s critique mirrors the debates in reinforcement learning—where ‘general’ RL agents failed because they didn’t account for domain-specific constraints. The same will happen with physics.

Bottom line

If you’re building AI for domains beyond language, ignore Anandkumar’s critique at your peril—start designing architectures that respect physics, not just data.