Enlarge (credit: boonchai wedmakawand)

Reinforcement-learning algorithms in systems like ChatGPT or Google’s Gemini can work wonders, but they usually need hundreds of thousands of shots at a task before they get good at it. That’s why it’s always been hard to transfer this performance to robots. You can’t let a self-driving car crash 3,000 times just so it can learn crashing is bad.

But now a team of researchers at Northwestern University may have found a way around it. “That is what we think is going to be transformative in the development of the embodied AI in the real world,” says Thomas Berrueta who led the development of the Maximum Diffusion Reinforcement Learning (MaxDiff RL), an algorithm tailored specifically for robots.

Introducing chaos

The problem with deploying most reinforcement-learning algorithms in robots starts with the built-in assumption that the data they learn from is independent and identically distributed. The independence, in this context, means the value of one variable does not depend on the value of another variable in the dataset—when you flip a coin two times, getting tails on the second attempt does not depend on the result of your first flip. Identical distribution means that the probability of seeing any specific outcome is the same. In the coin-flipping example, the probability of getting heads is the same as getting tails: 50 percent for each.

Enlarge / The interior or the DIII-D tokamak. (credit: General Atomics)

Using nuclear fusion, the process that powers the stars, to produce electricity on Earth has famously been 30 years away for more than 70 years. But now, a breakthrough experiment done at the DIII-D National Fusion Facility in San Diego may finally push nuclear fusion power plants to be roughly 29 years away.

Nuclear fusion ceiling

The DIII-D facility is run by General Atomics for the Department of Energy. It includes an experimental tokamak, a donut-shaped nuclear fusion device that works by trapping astonishingly hot plasma in very strong, toroidal magnetic fields. Tokamaks, compared to other fusion reactor designs like stellarators, are the furthest along in their development; ITER, the world’s first power-plant-size fusion device now under construction in France, is scheduled to run its first tests with plasma in December 2025.

But tokamaks have always had some issues. Back in 1988, Martin Greenwald, a Massachusetts Institute of Technology expert on plasma physics, proposed an equation that described an apparent limit on how dense plasma could get in tokamaks. He argued that maximum attainable density is dictated by the minor radius of a tokamak and the current induced in the plasma to maintain magnetic stability. Going beyond that limit was supposed to make the magnets incapable of holding the plasma, heated up to north of 150 million degrees Celsius away from the walls of the machine.