AI computing is so energy inefficient because it is built on a 80-year-old digital abstraction — floating point numbers and von Neumann architecture — designed for a completely different purpose on a completely different substrate. Today's AI chips move data endlessly in and out of memory to perform matrix math, and that constant data movement burns the vast majority of energy consumed. The result: a single large AI inference workload can consume megawatts of power, while a human brain running far more sophisticated intelligence uses just 20 watts.
That gap isn't a minor engineering problem. It's a civilizational constraint. And according to Naveen Rao — neuroscientist, former head of Mosaic ML, and architect of Databricks AI — we are about to hit a hard wall unless we fundamentally rethink how computers process information.
Why Is AI Computing So Energy Inefficient?
The core problem is that we never designed computers for intelligence. We designed them for calculation. The digital abstraction — ones and zeros, floating point arithmetic, clock-driven sequential processing — was invented in the 1940s for tasks like ballistics calculations and census tabulation. We have been iterating on that same foundation ever since.
When Nvidia builds a new GPU, it gets faster and cheaper partly because manufacturing improves, and partly because packaging improves. But the actual energy per floating point operation with memory access has barely moved. The physics of the von Neumann model — read state from memory, operate on it, write it back, repeat — is inherently wasteful, and we are approaching the limits of what incremental improvements can fix.
Rao puts it bluntly: we chose this path for a machine built from a different substrate, for a different purpose. Now we are trying to build machines for intelligence, and the old assumptions no longer hold.
How Much Energy Does AI Use vs. the Human Brain?
The numbers here are striking. There are approximately 8 billion humans on Earth. Each human brain consumes roughly 20 watts. That means the total cognitive energy of all of humanity is about 160 gigawatts.
The entire installed power generation capacity of the world today is around 9,000 gigawatts. The United States alone has about 1,000 gigawatts — and that powers everything: homes, factories, electric vehicles, data centers, all of it.
AI data centers are already consuming many gigawatts for inference and training combined. Rao's projection: within two to four years, not ten, we will face a genuine energy ceiling for AI. There simply will not be enough power generation capacity to keep scaling AI on the current paradigm without crowding out everything else.
To put biology's efficiency in even sharper relief: a squirrel jumping ten feet between branches in the wind — performing real-time physics calculations, motor control, and environmental sensing — is running on less than 10 milliwatts. That is one-hundredth the power of your phone sitting idle in your pocket. We cannot replicate that feat with a computer thousands of times larger and more power-hungry.
What Is Neuromorphic Computing and How Does It Work?
Neuromorphic computing is an approach to chip design that takes direct inspiration from how biological brains actually process information — not as an abstraction, but at the physics level.
The key insight is that brains do not do matrix math. They use nonlinear dynamics: time-varying interactions between neurons where the computation is embedded in the physics of the system itself, not in discrete read-operate-write cycles. Neurons are also stochastic — they do not operate in strict ones and zeros — and yet the system as a whole produces extraordinarily reliable and efficient intelligence.
Rao's company, Unconventional AI, is building chips that use oscillator-based circuits with trainable coupling between them. Instead of feeding a matrix operation to a chip and waiting for a numerical result, you set an initial state, apply a small perturbation — "kick it" — and let the physics of the system run. The computation happens in the time dimension of the physics itself, not in discrete clock cycles reading and writing memory.
The practical result: state and computation are no longer separated. There is no expensive memory bottleneck. Energy efficiency improves dramatically because you eliminate the single biggest source of waste in conventional computing.
What Is the Landauer Principle and Why Does It Matter for AI?
The Landauer principle establishes a thermodynamic minimum energy cost for irreversible computation. In plain terms, there is a physical floor below which no computing system — biological or silicon — can go. Physics sets the ultimate limit on intelligence per watt.
Here is where things get interesting. Biological brains are remarkably close to that thermodynamic limit — within perhaps two orders of magnitude. Current silicon AI chips, by contrast, are roughly three orders of magnitude away from the Landauer limit. That means conventional chips are about 1,000 times less efficient than what the laws of physics say is theoretically achievable.
Two-dimensional lithography — the process used to manufacture today's chips — has its own ceiling well below the Landauer limit. Even with perfect 2D chip manufacturing, we are still far from optimal. Neuromorphic and three-dimensional approaches could close much of that gap, pushing toward the physical asymptote rather than chasing incremental gains on a curve that has already begun to flatten.
What Is Non-Von Neumann Computing?
Every computer you have ever used is almost certainly a von Neumann machine. The von Neumann architecture, formalized in the 1940s, separates memory from processing. The CPU fetches data from memory, performs an operation, writes the result back to memory, and fetches the next instruction. It is sequential, explicit, and deterministic.
GPUs extended this by doing many of these fetch-operate-write cycles simultaneously — massive parallelism — but the fundamental pattern is the same. Even compute-in-memory approaches like those pioneered by Groq simply bring memory closer to the processor; they do not eliminate the separation between state and computation.
Non-von Neumann computing eliminates that separation entirely. In a dynamical system like the oscillator circuits Rao describes, the state of the system and the computation being performed are the same thing, expressed through the physics. There is no external loop. There is no explicit memory write-back. The system evolves, and the evolution is the answer.
This is not a software optimization or a new instruction set. It is a fundamental reimagining of what a computer is and how it works.
What Is Kuramoto Synchronization and Why Is It Key to Next-Gen AI?
One of the most compelling demonstrations Rao shares is the Kuramoto model — a mathematical framework describing how coupled oscillators synchronize over time. If you attach a set of oscillators to a shared mechanical system, they start in random states but gradually lock into a synchronized rhythm. The synchronization emerges purely from the coupling between them, with no central controller.
Generalize that coupling so it becomes trainable — so the strength and character of connections between oscillators can be adjusted through a learning process — and you have a system capable of remarkably rich dynamics. It can move through complex state spaces, represent many different classes of information, and morph between them in ways that matrix-based neural networks simply cannot replicate.
Rao's team demonstrated this by training such a system on image categories. Rather than storing pixel values and running matrix multiplications to classify them, the system learns to cluster its dynamics around meaningful regions of state space. Shown a random starting point and given a class label at time zero, it naturally converges toward that class and then continues to move fluidly between related representations — behavior that looks far more like biological perception than conventional deep learning inference.
Can AI Chips Ever Match the Efficiency of the Human Brain?
Possibly — and the gap is not as mystical as it might seem. Biology is not at the thermodynamic limit either, but it is dramatically closer than silicon. The path to brain-level efficiency requires abandoning the assumptions baked into our current stack: discrete time steps, floating point precision, separate memory and compute, deterministic operation.
What makes this moment historically unusual is that AI itself is now accelerating the hardware design process. Rao's team went from essentially no team in January to a full chip prototype heading to tape-out in six months — a timeline that would have been unthinkable for a traditional chip company. The absence of legacy infrastructure is not a weakness for a startup in this space; it is the competitive advantage.
The question of whether we reach human-level energy efficiency in AI hardware is less about whether the physics allows it — they do — and more about whether we have the will to question assumptions that have gone unquestioned for 80 years. If the energy wall Rao describes arrives in the next few years, that question may answer itself.
