Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal

Latent Space · 2026-10-22

Why AI Compute Comes in Rectangles — Erik Bernhardsson, ModalLatent Space ▶ Play the video

This item is no longer on the current shelf. Its page remains available, and editorial corrections may be made. See what is featured now.

Why it earned a slot

Erik Bernhardsson dismantles the myth that GPU capacity can be bought in tidy three-year blocks, exposing a misalignment between market packaging and actual AI workload elasticity. Modal’s pooled compute model, as described, lets teams spin up exactly the horsepower they need and pay only for what they consume, which directly attacks the financial risk of over-provisioning. The discussion also surfaces how this rectangle mindset hampers rapid experimentation, especially in the current wave of foundation-model fine-tuning.

The short version

Rectangles are for art, not for AI compute.

Why it matters

AI projects now iterate at a pace where committing to multi-year hardware contracts is a liability, not a safety net. Modal’s on-demand pool gives teams the ability to match compute spend to experimental cycles, preserving cash while keeping performance on-track. Understanding this shift is crucial for any organization that wants to stay nimble in the face of exploding model sizes.

My take

I’d argue the real competitive edge comes from how you finance compute, not just how fast your GPUs run. Modal’s model flips the script, turning compute into a utility rather than a capital asset. The financial risk mitigation they describe—pay-as-you-go with pooled capacity—mirrors the SaaS transition we saw in storage and networking a decade ago. If you’re still buying fixed-size GPU clusters, you’re effectively over-engineering for a future that may never arrive.

How it connects

• Parallels the shift to serverless compute in cloud-native architectures.
• Supports the emerging practice of cost-aware model scaling in large-scale training pipelines.
• Enables more aggressive experimentation cycles for foundation-model adaptation.

Bottom line

Prototype your next model on a pooled compute platform before committing to any long-term GPU lease.