🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing

Latent Space · 2026-08-26

▶ Play the video

This is no longer on the current shelf — shelves rotate as new material clears the bar. The analysis below is unchanged. See what is featured now.

Why it earned a slot

Anima Anandkumar’s critique of foundation models for physics—especially weather—isn’t just a niche critique; it’s a fundamental challenge to the current AI hype cycle. Her work at Accelerated Understanding proves that physics simulations require *structured, interpretable* representations, not just scaling LLMs. The skepticism she faced isn’t just academic; it’s a warning that foundation models for domains beyond language are still in their infancy. This isn’t about ‘AI can’t do physics’—it’s about why we’re still treating physics as an afterthought in AI research. The episode’s tension—between the promise of general-purpose models and the reality of domain-specific constraints—mirrors the same debates we’ve seen in reinforcement learning (RL) vs. supervised learning. Anandkumar’s emphasis on *physics-informed neural networks* (PINNs) and hybrid architectures feels like a necessary corrective to the ‘bigger model = better’ narrative. If this discussion doesn’t land in the next wave of AI papers, we’re doomed to repeat the same mistakes we made with language models—only now with higher stakes (climate, engineering, etc.). One glaring omission: no direct comparison to recent work on *physics-guided diffusion models* (e.g., those from DeepMind or Meta). That’s a gap—because those projects *are* trying to bridge the gap she’s highlighting. Also, the ‘open-source weather model’ claim feels premature without a clear roadmap for deployment. Skepticism is fair, but so is the need for concrete benchmarks.

The short version

Foundation models for physics? Anima Anandkumar just dropped a reality check—here’s why it matters for everyone building AI systems.

Why it matters

Foundation models for language have dominated AI, but physics, chemistry, and engineering domains require *different* architectures—ones that respect causality, not just correlations. Anandkumar’s work exposes a critical flaw in the ‘one-size-fits-all’ AI paradigm, forcing us to ask: *When is a foundation model *not* a foundation model?* This is urgent because climate modeling, drug discovery, and robotics are all waiting for AI that understands *how* the world works, not just how to predict it. The stakes aren’t just academic. Climate models alone cost billions to run, and if we’re chasing LLMs for physics, we’re repeating the same mistakes we made with NLP—where ‘foundation models’ turned into black boxes that fail at nuance. Anandkumar’s critique is a call to arms for *domain-specific AI*, not just another scaling contest. Right now, the AI ecosystem is fixated on ‘AGI’ and ‘general intelligence,’ but Anandkumar’s focus on *structured reasoning* aligns with what agentic systems actually demand—interpretability and modularity are non-negotiable. The question isn’t whether AI can do physics; it’s whether we’re willing to build the right tools for it. The broader trend here is the shift from ‘data hunger’ to *knowledge-aware* models. Anandkumar’s work is a blueprint for how that might look—hybrid systems that combine deep learning with symbolic reasoning. That’s where the real breakthroughs will happen, not in another GPT-5.

My take

In agentic systems, interpretability isn’t optional—it’s a requirement for real-world deployment. Anandkumar’s critique resonates because ‘foundation models’ fail the moment you need to explain *why* a system made a decision, not just what it predicted. For example, in autonomous systems, you can’t just say ‘the model predicted a collision’—you need to know *why* it predicted it, and how to mitigate it. The real irony? The same teams pushing LLMs for physics are the ones who’ve already admitted their models don’t *understand* language. So why would we expect them to ‘understand’ physics? Anandkumar’s work is a wake-up call that we need *new* architectures—ones that combine the strengths of deep learning with the rigor of physics. That’s where the next generation of AI will come from, not another round of scaling. The other angle? This is a test case for *AI alignment*. If we can’t get physics right, how can we expect AI to align with human values in high-stakes domains? Anandkumar’s skepticism isn’t just about models—it’s about *capability*. And that’s a conversation we’re not having enough.

How it connects

• *Physics-informed neural networks (PINNs) are the missing link between deep learning and scientific modeling*—but they’re still treated as a niche. If we don’t invest in them now, we’ll be playing catch-up when climate models or drug discovery demand AI that *understands* causality.
• *The ‘foundation model’ label is a red herring*—it implies generality, but physics, chemistry, and engineering require *specialized* architectures. This is why we’re seeing a resurgence of hybrid models (e.g., neural-symbolic systems, graph-based approaches).
• *Anandkumar’s critique mirrors the debates in reinforcement learning*—where ‘general’ RL agents failed because they didn’t account for domain-specific constraints. The same will happen with physics.

Bottom line

If you’re building AI for domains beyond language, *ignore Anandkumar’s critique at your peril*—start designing architectures that respect physics, not just data.

Topics: physics-ai, weather-modeling, foundation-models, open-source, science