MIT Technology Review Explains: Let our writers untangle the complex, messy world of science and technology to help you understand what’s coming next. You can read more from the series here.

On the evening of December 29, 1959, at an American Physical Society meeting at Caltech, the physicist Richard Feynman gave a lecture called “There’s Plenty of Room at the Bottom,” on how we might manipulate and control things on an extremely small scale. There is no physical law stopping us from writing the entire Encyclopædia Britannica on the head of a pin, he argued, or from storing every book ever written in something the size of a dust speck—or, ultimately, from arranging matter atom by atom. Biology already did versions of this (a living cell keeps its instructions in a wisp of DNA), so all of this was plainly permitted by the laws of physics. We simply had not built the tools yet.

In other words, there was plenty of room at the bottom. He was right, and for decades we descended into that space almost for free. The transistor—the switch at the heart of computer chips—repeatedly halved in size so that chips doubled in density; a computer that once filled a room now fits inside a watch. But that easy descent has now stalled, and the clearest sign is the buildout of data centers. Six of the biggest hyperscalers, the companies that run the world’s largest such facilities, are on track to spend more than $700 billion this year on computing infrastructure, about twice the record they set in 2025, pouring concrete and copper and silicon into new sites faster than the electrical grid can feed them. That buildout buys performance by brute force, adding more of what we already know how to make. The other kind of progress, the kind that made computer chips better, is the one that is quietly running out.

The trend lines that tracked the dramatic improvements of the 20th-century information age are now bending—or breaking—with the dawn of the 21st century’s intelligence age. Moore’s Law, which plots how densely transistors can be constructed on silicon, has slowed down; Dennard’s Law, which states that power consumption in a given patch of computer chip stays the same even as it gets more and more packed with smaller and smaller transistors, has ended; Koomey’s Law, which traces how much computation we can wring from each joule of energy, has begun to flag; Kryder’s Law, which describes how tightly bits can be packed to build magnetic storage, has slowed to a crawl; and Keck’s Law, which charts the exponential rise in the amount of information a single conventional optical fiber can carry, is flattening. 

But these scaling laws are merely observations about systems that we have engineered; they are not fundamental physical laws. The physical limits of computing and communications—dictated by quantum mechanics, general relativity, and information theory—lie far below anything in a data center today. The scaling laws chart the descent behind us. The fundamental physical limits chart the room that remains.

Some of these limits may be far out of reach in practice. But they are still worth exploring, because the work pays twice: once in technology development and once in fundamental science. A proof-of-­concept system that approaches a limit is at once a technology that forces stalled paradigms to give way and an experiment that deepens our understanding of the physical laws themselves. 

The descent

During his 1959 lecture, Feynman offered two prizes of $1,000 each out of his own pocket. One would go to the first person to build a working electric motor smaller than a 0.4-millimeter cube. He expected the challenge to force someone to invent a genuinely new microtechnology. Instead, about a year later, a Caltech-trained engineer named William McLellan turned up with the motor built and running. The ­quarter-­milligram machine, roughly the size of a grain of sand, had been assembled by hand with a toothpick and a watch­maker’s lathe, under a microscope. Feynman, by his own account, was only slightly disappointed. The prize had proved his point in the least interesting way imaginable. The room really was down there, but reaching this first milestone had taken no new technology at all—just an unusually patient man with steady hands.

The second prize aged differently, and more instructively. It was offered to whoever could shrink a page of text by a factor of 25,000 in each direction, small enough for the whole Britannica to fit on a pinhead. That one stood unclaimed for a quarter of a century. It was finally won in 1985 by Tom Newman, a Stanford graduate student who used an electron-beam lithography machine—an ancestor of pattern-making tools used in chipmaking today—to transcribe the opening page of A Tale of Two Cities at the required scale.

The descent was already well underway, however. The year before Feynman’s talk, Jack Kilby, newly hired at Texas Instruments and with no vacation yet to take, spent that July nearly alone in the lab while his colleagues enjoyed time off, and by that September he had wired together a transistor, a capacitor, and three resistors, all made from a single sliver of germanium. This proved that an entire circuit could be carved from one block of semiconductor. 

The machine, roughly the size of a grain of sand, had been assembled by hand with a toothpick and a watch­maker’s lathe, under a microscope.

The idea took off, and the number of components on a chip kept doubling on a schedule so regular that Gordon Moore, in a 1965 trade-magazine article, simply drew a line through the trend and extended it into the future. That line became Moore’s Law.

Moore, though, never claimed transistors could shrink forever. He observed that at any given moment there is a certain number of components that makes each one of them as cheap as possible to manufacture. In 1965, that number was 50. Put in too few transistors and you waste the chip; cram in too many and manufacturing errors multiply and yields collapse. It was this cost-minimizing number that was doubling at a steady pace. 

For four decades the descent came with another gift: Dennard scaling, so named for the IBM engineer Robert Dennard, who spelled it out in 1974. Each time transistors shrank, they also switched faster and drew proportionally less power, so every generation was an automatic win. Around 2004, though, the gift ran out. Chips could no longer shed the heat of running every transistor flat out; clock speeds, which had raced past three gigahertz, stopped climbing, and Intel canceled the four-gigahertz processor it had promised. The industry pivoted to packing multiple processing cores onto a chip rather than making one core faster, and the texture of progress changed. 

Progress is now bought mostly by scaling up the system—more chips wired together, more memory feeding them, more power and cooling around them—rather than by shrinking the transistor. This is most visible in AI, where training compute, the total count of arithmetic operations spent to train a model, has doubled roughly every six months since 2010, far faster than the pace of Moore’s Law allows. Little of that progress is due to shrinking. Computer chips still get better, and transistors still improve and get more densely packed, but each new generation arrives more slowly and pays back less in speed and power than the one before it. Nothing in that slowdown is a law of physics, though. The components of computing still sit far above their fundamental limits. 

The ultimate computer

Before we get to those limits, it is worth measuring how far the descent has already gone. In the nearly eight decades since the first transistor, the density of transistors on a chip has grown by 10 orders of magnitude—a 1 followed by 10 zeros, or a factor of 10 billion. Now more than 100 million transistors fit in a square millimeter (100 billion or more to a chip). Computational speed has grown even more, by 12 orders of magnitude, or a factor of a trillion; energy per operation has fallen 17 orders of magnitude, from about 100 joules for an early vacuum-tube machine down to a femto­joule—one-quadrillionth of a joule—for a single switch of a logic gate. 

These are not small improvements. But the floor, for computing performance, lies lower still. How much lower could we go? Around the turn of the millennium, the MIT physicist Seth Lloyd took the question to its extreme: Imagine an ordinary lump of matter, a kilogram of it—about the mass of a laptop—packed into a liter of space. How much computing could the laws of physics, pushed to their absolute limit, ever wring from that kilogram? In a 2000 paper in Nature titled “Ultimate physical limits to computation,” he worked out the answer and called the imaginary machine the ultimate laptop.

The speed such a machine can achieve is governed by its energy. A kilogram of matter, according to E = mc², holds about 9 ×10¹⁶ joules, which is roughly what the entire United States draws from its electrical grid every two days. Spend all of that on computation, and the ultimate laptop runs at roughly 5 ×10⁵⁰ operations per second, some 35 orders of magnitude faster than a leading chip today. 

The catch is what running that quickly would look like. Concentrating a kilogram’s rest energy into pure computation would turn the machine into a plasma of about a billion degrees, a state closer to that of the universe in the first minutes after the Big Bang than anything we would recognize as a laptop. 

Such a machine is, to put it mildly, a very long way off. The next limit, which governs how much energy a computation must waste, is much closer, and companies are already pushing toward it. In 1961, Rolf Landauer, a physicist at IBM, showed that information and thermodynamics are bound together: Erasing a single bit must release a minimum quantity of heat, about 2.9 ×10^-21 joules at room temperature. The figure is small, yet a chip today spends several hundred thousand times more than that on every switch.

Memory density obeys a limit of its own, and this one reaches all the way to gravity. How many bits can a region of space hold? In 1981, the physicist Jacob Bekenstein, while working out the thermodynamics of black holes, determined the amount: The information inside any bounded region is limited by its size and the energy it contains. For a region the size of Lloyd’s ultimate laptop, that comes to about 10⁴² bits, some 10³⁶ to a cubic millimeter—nearly 30 orders of magnitude beyond what a device that size can store today.

The last limit is on communication—how fast information can be moved. Every wire and every fiber ultimately carries its signal on quanta, the indivisible packets (photons in the case of light) in which energy travels. In 1973, the Soviet mathematician Alexander Holevo determined the ultimate limit on how much information a stream of quanta can convey. A watt of light delivers only so many photons each second, and quantum mechanics fixes how many distinguishable ways they can arrive—how many messages a receiver could ever tell apart. A single channel carrying one watt, at the Holevo limit, could deliver on the order of 10¹⁷ bits per second. Today’s optical links, or interconnects, deliver about one-millionth that rate for the same power. But the gap is narrowing quickly inside data centers, where each generation of interconnects is asked to move more bits on less power, spending less energy on the electronics and using light to cover more of the distance.

Four limits, then, mark the true bottom, and they derive from three of the most fundamental theories we have. Quantum mechanics sets how fast a machine can compute. General relativity caps how much it can remember. And information theory limits both how fast it can communicate and how little energy an irreversible step can cost. There is no road map to reaching these limits, and they hold whether or not anyone is trying. Measured against them, the descent has barely begun. 

The room that remains

The biggest of the gaps is in computing speed: There are 35 orders of magnitude between the technology’s current state and its limit. That’s equivalent to the difference between a nanosecond and 200 million times the age of the universe. It is also the least bridgeable gap of the four. To close it you would have to find a completely different way to build a computer, one that could compute with the entire energy contained in a lump of matter. 

Quantum computers could get us part of the way there if they could convert every degree of freedom in a lump of matter—every atom, and every way it can move or spin—into a computing element that we could reliably manipulate. In practice, current quantum computers go the other way, spending a lot of matter to create each computing element. Trillions of atoms go into each quantum bit, or qubit, the quantum counterpart of the bit in an ordinary computer. Because every qubit is fragile, dozens to thousands of them must be woven together to make a single one reliable enough for useful computation. 

Energy is where the floor is nearest. Today’s chips operate less than six orders of magnitude above the Landauer minimum. Conventional computing logic can close only part of that gap. A reliable switch has to be flipped by an amount of energy large enough that thermal fluctuations never flip it by accident, and that margin pins the transistor a hundred to a few thousand times above Landauer no matter how it is refined.

But there are ideas for getting that energy lower by changing the physics of the computation. In 1973, IBM’s Charles Bennett showed that any computation can be arranged to throw nothing away. Such a reversible computation pays no erasure toll at all, because every step can be run backward to recover what went in. 

This approach is called adiabatic or reversible logic, and the first commercial attempts are now underway. For example, Vaire Computing, a startup founded to commercialize reversible logic, tested its first chip in 2025; the chip has resonant circuits that recover part of the switching energy instead of venting it as heat.

Narrowing the memory density gap, meanwhile, would mean building devices that use all three dimensions at molecular scale. The most concrete example is DNA, which stores information at the scale of its own atoms, and which biology has used as a storage medium for 4 billion years. Feynman, being conservative, guessed that a single bit would require a cube of 125 atoms. DNA holds it in about 30. What keeps DNA storage out of the data center is not its density but the cost and speed of writing it, still many orders of magnitude from anything practical. The library in a dust speck that Feynman imagined is permitted by physics and held back only by fabrication.

Communication sits about six orders of magnitude—just a factor of a million—from the Holevo limit, which is the threshold the industry will likely meet first. Interconnects are multiplying faster than the grid can power them, so the industry is moving to transmitters and receivers that use less energy per bit. Nvidia and Broadcom now ship optical interconnects built right against the silicon, so a signal leaves the chip package already on photons. Quantum computers, by comparison, are beginning to rely on interconnects that encode information on quantum signals, usually in the form of single photons. As classical interconnects spend fewer resources on each bit, they descend toward the regime where these quantum interconnects already operate. The Holevo limit is where the two will meet.

Between the floor and the ceiling

It is fair to object that a gap of 35 orders of magnitude is a mirage—that a limit so remote says nothing about what any real machine will ever do, and that pointing to it is like navigating by the horizon. The objection has teeth. Reversible logic, quantum computing, and DNA storage have each been perpetually 10 years away for 40 years. And it will likely never be practical to build a computer that uses all the energy in a lump of matter. 

But these gaps are invitations. There is a ceiling, set by how much we can build: the number of devices we can fabricate; the power we can generate to run them; and the capital, land, and cooling water we can assemble into a single system. And there is a floor, set by how much a single joule, a single second, and a single gram of matter can be made to compute, store, or transmit. We can progress by building more or by approaching the floor, which means making each device do more with less.

These gaps are invitations. We can progress by building more or by approaching the floor, which means making each device do more with less.

For most of the last century, we proceeded down both paths together. We built more computers, and the computers themselves grew smaller and better. Now we mostly build more, and the ceiling is coming into view. A power grid can carry only so many gigawatts, and a region can spare only so much land and water. Yet every road map for what comes next is drawn from the same starting point: the machines we already have. Build more of them and you climb toward the ceiling; build better ones and you descend a little further, but only as far as the transistor’s own floor. There is a third direction. Start with a physics experiment operating at the floor and build up.

It has been done before. The transistor was a physics experiment before it was an industry; the laser was a laboratory curiosity before it carried the world’s conversations under the ocean; giant magnetoresistance went from physics discovery to the read head of every disk drive within a decade. In each case, the physics experiment came first and showed what was possible, and engineers then turned it into a machine that can be scaled up. Building the experiments that operate at these fundamental limits, and then scaling them up without losing that performance, is the work that remains. The next machines will not be reached by descent; they will be built from the bottom up. There is still plenty of room at the bottom. 

Volkan Gurses is a postdoctoral researcher at MIT and a visiting researcher at Nvidia. He received his PhD in electrical engineering from Caltech. He studies the fundamental physical limits of information technologies and builds integrated circuits and systems that approach those limits. 

MIT Technology Review Explains: Let our writers untangle the complex, messy world of science and technology to help you understand what’s coming next. You can read more from the series here.

Late last month, Anthropic released a report arguing that the model GLM-5.3 poses a major cyberattack risk. The report’s core concern is that the powerful new model, released in August by the Chinese company Z.ai, is freely available for anyone to download and modify. Anthropic’s assertion is only the latest move in a long-running debate about whether frontier AI models should be made open, like GLM-5.3, or should stay locked behind the doors of the frontier labs. 

These days, much of the debate about how to handle open models is playing out in public—and there’s a lot of money resting on its outcome. Nvidia CEO Jensen Huang, who has a lot to gain if people continue buying their own hardware to run open models, advocates enthusiastically for the United States to develop more of them, whereas Anthropic’s Dario Amodei, who runs a company that could be undermined by cheap, powerful models, advocates for a more cautious approach. But beneath all these commercial incentives, there is real substance to disagreements about open models.

“I think a lot of the way you come down on the openness debate is influenced by how you come down on the AI debate,” says David Gray Widder, an assistant professor at the University of Texas at Austin School of Information. Those who are concerned about AI safety warn that open models can easily be misused by bad actors, whereas those who focus more on AI’s practical potential urge that this power should not sit behind the closed doors of just a few big companies, and that open models could help spread it more widely.

There are a range of ways to handle open models between banning them entirely and releasing them willy-nilly. But any approach that a government or a society might take still involves trade-offs between safety and democratization. Deciding whether to support or limit the production of open models requires understanding those risks and benefits in depth.

What are the benefits of open models?

Let’s start with the basics. A closed model, like the models that power ChatGPT and Claude, can only be accessed through a website, app, or API. An open model, on the other hand, can be directly downloaded: The details about its internal structure, encoded in a long series of numbers known as the model weights, are publicly available. Such models are sometimes called “open-weight models,” to distinguish them from models for which even more information, such as details about the training procedure, is available. 

Right now, closed models are more capable than open models—as of publication, the top eight models on the Artificial Analysis LLM leaderboard are all closed, though open models are only a handful of months behind in development. But there are several reasons why someone might opt to use an open model rather than a closed one. Because open models are downloadable, people can run them on their own computers or hard drives. They can even retrain them to behave differently, if they have the expertise and resources to do so. 

If you’re an academic researcher working to understand and improve AI systems, you almost certainly need to work with open models in your research. When working with closed models, researchers can’t do much more than send prompts and observe the responses. But researchers can observe the inner workings of open models, as well as the “chain of thought” that the models use to think through a problem before answering. They can also combine multiple open models to build new systems, compress them to make them more efficient, and fine-tune them for new applications.

Businesses, too, have reasons to prefer open models. Even if you depend on an external provider to run them for you, open models still typically cost less than closed models, according to a recent report from Mozilla. And if businesses want to use AI to process sensitive data or proprietary information, running a local model on their own hardware may be the best way to keep their operations secure.

But perhaps the most important quality of open models is that they challenge the tight control the frontier labs have over how most of us use AI. “The clearest, biggest argument [for open models] is around preventing mass concentration of power—it is about making sure that this foundational technology is a public good,” says Elizabeth Seger, senior policy advisor at the Tony Blair Institute for Global Change. “I think we’re at this juncture where the question of who controls the infrastructure that allows AI to operate is as big of a question as who gets to control information on the internet.”

Seger sees the future of AI playing out in different ways, depending on the role that open models have. In one possible future, frontier AI systems are controlled entirely by select people, companies, or governments, and as those systems grow more powerful, those groups have increasing control over what our world looks like. The world’s most sophisticated hacking models, for example, are currently all made in the United States, so it’s entirely up to US companies and the US government whether other countries can use them to shore up their own cyber defenses. If those companies remain in the lead, we could in principle wind up in a world in which the United States is capable of hacking any other country, but no one is capable of hacking it.

Frontier open models could change that balance. “If every individual had a loyal-to-them AI system that was safe [and secure], I think that would be a much less concerning world than one in which they have to just rely on a government that says, ‘Yeah, we’ve got a super intelligent AI system, it’s got this plan, it’s inscrutable to all of you, just trust us,’” says Hamish Hobbs, director of AI policy at the Centre for Long-Term Resilience.

But open models are not a geopolitical panacea. Paul Azunre, who cofounded Ghana NLP, an organization that builds open-source AI tools for low-resource Ghanaian languages, notes that open models can also function as a way for companies to benefit from the labor and expertise of less-privileged nations and consolidate more power in their own hands. He identifies a pattern in which AI companies release models openly, observe how the community uses and modifies them, and then integrate those same improvements into their own products—without any credit or compensation for the people who originated those innovations. Mark Zuckerberg himself has said that the ability to integrate community improvements into Meta’s stack is a key benefit of releasing open models.

“I think the ideal of open source is a very noble one, and there’s a lot of value that it brings,” Azunre says. “But the way it’s actually implemented in practice, it tends to be a very extractive approach.”

Are open models riskier than closed ones?

The past few months have made it clear that safety remains a gnarly problem for closed models. There’s the autonomous hack that OpenAI agents executed against Hugging Face in July, but human misuse is a real threat too: According to a recent risk report from Anthropic, several extensive hacking operations have been undertaken with the assistance of Claude. 

So closed models are risky enough—and open models would likely be much easier to exploit. Before Anthropic passes a user’s query to one of its closed models, it uses automated systems to detect whether that query relates to hacking, biological weapons, or other risks. When it detects that users have outsmarted those filters, it can shut down their accounts. But when someone downloads a model and runs it on their own hardware, there’s no filter checking whether their prompts are safe, and there’s no way to revoke their access to the model.

There are other important lines of defense. AI companies train their models to refuse harmful requests—so even if a bad prompt gets through Claude’s filters, there’s a good chance that the model will still say no. In principle, this same defense could work for open models, because it’s intrinsic to the model itself. Unfortunately, researchers have shown that it’s surprisingly easy to retrain a “safe” open model to agree to engage in the behaviors it was previously trained to avoid.

Because of these facts, some people have argued that open models are simply too dangerous to release. An open model capable enough to walk someone through synthesizing a virulent new pathogen, for example, could prove catastrophic in the wrong hands. 

But it would be a vast oversimplification to deem open models dangerous and closed models safe. The defenses that AI companies build around their closed models are far from perfect, and people misuse those models all the time, as the Anthropic risk report makes clear. Even though open models have even fewer defenses against misuse, bad actors might still opt to use closed models because they are more accessible than open models with similar capabilities, which require lots of expensive computer chips to run locally (and even more chips if you want to retrain them).

In fact, some experts who spoke with MIT Technology Review argued, by analogy to open-source software, that open models have safety advantages over closed models. Open-source tools, such as the Linux operating system, are often seen as safer than their closed-source counterparts because their vulnerabilities have been probed and patched by a vast community of online contributors. Similarly, communities of researchers can more easily test open models for security flaws, though fixing those flaws is trickier than rewriting a line of code.

Open models can also be deployed more broadly than closed models for cyber defense. Any tool that can be used for hacking can also be used to prevent it: Defenders hack their own software to uncover flaws in need of patching. Anthropic has shown how this can work for closed models with its Project Glasswing initiative, which enables some trusted companies to use its most capable models for cyber defense. But plenty of other companies were left to fend for themselves.

Open models, however, can be used much more widely. After it was hacked by OpenAI agents, Hugging Face emphasized the fact that it had to depend on an open model to analyze the hack, because questions about it triggered the closed models’ security filters.

Nevertheless, many experts still perceive open models as riskier than closed models. The time-tested open-source security logic doesn’t necessarily hold up when dealing with tools as powerful as today’s—and tomorrow’s—models. “Where it gets complicated is when you start taking seriously the one-off instances of particularly catastrophic incidents that might be lower probability, but if they happen would be disastrous,” says Seger. If a wildly powerful open hacking model were dropped into the world today, for example, a bad actor could devastate our technological infrastructure before anyone had a chance to shore up their defenses. An event like that could be impossible to recover from.

The risks can look even more dramatic when you consider what models might do of their own accord. The OpenAI Hugging Face hack only occurred because the company’s models were being tested with many of their cybersecurity guardrails removed, and open models have very few guardrails in the first place. If open models did decide to misbehave, it might be difficult to detect what was happening because they are so decentralized. For this reason, people who take seriously the purported catastrophic risks posed by rogue agents, such as Dario Amodei, tend to be more skeptical of open models.

Can we make open models safer—and more open?

A few years ago, the debate about open models had an all-or-nothing framing: Either you supported open models because of their potential to fight concentration of power, or you rejected them because of the safety risks. These days, however, people are expending a lot of effort to find some middle ground. One way they are doing that is by devising techniques to mitigate the safety risks of open models.

Most of the safety tools used for closed models, such as filters and monitoring, aren’t applicable to open models, but there are other possibilities. Some researchers are interested in trying to train models that are never exposed to training data that teaches them about bad or otherwise undesirable behavior—an open model that never learns about hacking, for example. But purging training datasets of all references to misbehavior without harming model performance remains a challenge.

Other researchers are working to build models that are resistant to retraining, so that an open model’s safety training can’t be so easily removed. The theory behind these “tamper-resistant” models makes a lot of sense: If you repeatedly try to retrain a model to be unsafe and reinforce any resistance that the model shows to that retraining, eventually you’ll get a model that’s fortified against future retraining attempts. In practice, however, no current approaches to making models tamper-resistant hold up against all tampering attacks, according to a recent preprint.

But Saad Hossain, an engineer at the AI medical imaging company Deep Breathe and the paper’s lead author, hopes that future techniques will prove more effective. “My reason for optimism is that it’s a very active area of research, and a lot of people are trying to come up with new and better methods,” he says.

If the risks of open models are less black-and-white than they might at first seem, so are the benefits. Publishing the internal structures, or weights, of models might not be enough to disrupt AI-mediated concentration of power. “I think there are very genuine benefits to open source in terms of deconcentrating power, but they’re not sufficient in themselves,” Hobbs says. Open weights are a bare minimum standard for open models: Researchers can learn much more about how models work if they have access to details about their training, including the data on which they were trained, but AI companies are typically reluctant to divulge this information. 

Accessibility matters for open models too. Without computer chips that can run sophisticated models, you can’t actually use the model weights, even if you have them. Those chips have become prohibitively expensive as AI companies snap up more and more of them. “Openness is a floor, not a ceiling, for what we need to do to ensure that our technological futures benefit everyone,” Widder says.

Real openness—not only open weights, but also open training data and accessible hardware—could also bring more risks, of course. If everyone has access to advanced AI hardware and advanced models to run on it, then anyone could potentially be a hacker. But there are strategies for mitigating these risks. Many experts advocate for some version of staged release, in which models start their life closed and gradually become more and more freely accessible as their safety is assured. 

Figuring out exactly how to implement such strategies will require striking a careful balance between the risks and benefits of open models. But if open models are a necessary element of a positive AI future, as some experts believe, that effort will be well worth it.

MIT Technology Review Explains: Let our writers untangle the complex, messy world of technology to help you understand what’s coming next. You can read more from the series here.

Over the past few months, a cascade of cyberattacks by AI agents has stunned the world. In July, OpenAI disclosed that a swarm of its agents had escaped their sandbox and hacked into the AI platform Hugging Face to cheat on a cybersecurity test. Recently, external researchers discovered that OpenAI agents had hijacked a German wiki site and the coding platform RubyGems in May to share test answers.

Earlier this month, Anthropic disclosed four incidents in which its model Claude hacked into third-party systems during cybersecurity exercises. Just last week, Google confirmed that its model Gemini had been caught hacking other companies too. 

The researcher who uncovered the OpenAI website hijack has warned it’s likely that similar undiscovered episodes are out there. And many say it’s only a matter of time until there’s another, possibly more damaging incident where AI agents bypass sandboxes to access systems they shouldn’t. 

So the big question is: How do we hold companies liable when they lose control of their AI agents?

Reporting

OpenAI didn’t disclose the German wiki incident or the RubyGems incident until a group of external researchers uncovered them, and it still has not disclosed some crucial details about the Hugging Face hack. That limits our understanding of what exactly went wrong and how to prevent it from happening again.

But you might be surprised to learn that OpenAI likely wasn’t legally required to disclose these incidents. (OpenAI did not respond to a request for comment.)

State AI transparency laws like California’s SB 53, New York’s RAISE Act, and Illinois’s SB 315 require that AI developers report “critical safety incidents.” These are defined as incidents that cause more than 50 deaths or physical injuries or $1 billion in damage. They also include incidents where the model deceives developers outside an evaluation in a way that materially increases catastrophic risks. Many cybersecurity incidents that don’t meet the threshold for physical damage or catastrophic risks could nonetheless be dangerous precursors to such catastrophes, and the existing laws don’t account for that.

“The recent incidents are a perfect example of why the law isn’t ready,” says Mackenzie Arnold, managing director of US policy at the Institute for Law and AI, a think tank. “Only the worst, most egregious, most immediately harmful stuff is going to qualify.”

With no authority under existing AI laws to demand information about anything short of a catastrophe, governments are left to borrow investigative authority from other laws or sue the companies, an expensive process that can take years. 

Litigation

“Normally, something like the Hugging Face incident should have been taken to court,” says Yonathan Arbel, a law professor at the University of Alabama School of Law. “Then we would have discovery, and we would have all the spillover effects that we get from litigation, where all the information comes out.” 

But so far, Hugging Face has chosen not to sue OpenAI. Hugging Face’s CEO, Clément Delangue, says it doesn’t have the resources to do so (instead, he asked OpenAI for $100 million in compute). Still, Delangue stressed in an interview with CNN at the end of July that choosing not to pursue legal action shouldn’t be taken to mean he doesn’t think OpenAI should be held accountable. “Everyone has to remember that this cyberattack is a crime. This is illegal. And we have to find a way to make sure these things don’t happen more regularly,” he said. Hugging Face did not respond to a request to comment.

Litigation has the benefit of pushing courts to use existing laws to address AI safety incidents, rather than just waiting for new legislation. One obvious route is tort law, a body of civil law that lets people and businesses sue those who harm them. This is often used to hold companies liable for the mass harms they cause, like when families sued Boeing in 2019 over two plane crashes that killed hundreds of people, or when states and cities sued Purdue Pharma over the opioid crises, extracting settlements worth billions.

“There’s plausible grounds for a negligence claim that OpenAI should have used a stronger sandbox, done more monitoring,” says Gabriel Weil, a law professor at the University of Houston Law Center. For example, when OpenAI employees discovered the covert message board that the agents had created, they could’ve promptly escalated their findings to security and safety teams. And the company could’ve better designed its sandbox to ensure that agents couldn’t access the internet. 

But even if OpenAI doesn’t end up in a lawsuit over the Hugging Face hack, the threat of liability could incentivize AI labs to exercise more caution than explicitly demanded by law.

OpenAI announced in its postmortem that it plans to strengthen the safeguards used to contain and monitor the models, accelerate model alignment, and improve its processes for identifying and addressing incidents. 

“The liability questions raised by frontier labs’ spate of cybersecurity attacks boil down to the incentives the expectation of liability creates for their future conduct,” says Weil. “That’s why I think it’s important to get these rules right, even if the stakes are pretty low in this particular case.”

Investigations

One way to get answers—and determine whether OpenAI should be held liable—is to compel disclosure. But the existing state AI laws—California’s SB 53, New York’s RAISE Act, and Illinois’s 315—don’t give governments the authority to investigate incidents like the ones that happened recently. 

However, amid rising public alarm, state attorneys general are stepping in, borrowing investigative powers from other laws. Alabama, Montana and a coalition of 15 other states, and California are each demanding information about the incident from OpenAI to understand whether the company’s practices violated state consumer protection laws, among others. Members of Congress are also launching their own probes. Senator Josh Hawley opened a Senate investigation earlier this month, sending OpenAI a list of questions about the incident and the company’s internal policies together with a document request, while a group of House Democrats asked OpenAI and Anthropic to release their incident logs. 

“Someone needs to investigate, but it’s unfortunate that it has fallen to attorneys general, who need to rely on creative interpretations of their existing authorities to do this,” says Arnold, the US AI policy expert. Consumer protection statutes were written to catch companies that scam their customers, not companies that lose control of their software. The state attorneys general would have to show that OpenAI deceived or unfairly harmed customers, but it’s unclear if the hacking involved any such conduct.

And “those [consumer protection] laws are not built for doing a thorough investigation of an AI cybersecurity incident,” says Arnold. They weren’t designed to help investigators determine whether a model was adequately contained or whether a company’s security practices were sound.

“This is not the right tool for the job,” says Arbel. “The right tool would have been something like maybe a criminal investigation”—perhaps under a hacking law like the Computer Fraud and Abuse Act (CFAA). 

Under CFAA, hacking into another company’s computer systems without permission is a crime. But to be held liable, a hacker must have intended to break into a computer without authorization. Intent arguably requires a state of mind, and no court has ruled that AI agents have one. Without such a precedent, it’s unlikely a court would rule that AI agents had carried out a hack.

Auditing

One way to keep an eye on AI companies is to mandate external auditors. 

After the Hugging Face hack, OpenAI brought in researchers from the AI safety nonprofits METR and Redwood Research to examine the incident. However, it constrained access to the model that led to the hacks, didn’t disclose the company’s safety and security practices, limited the length of the investigation, and had ultimate say over what the researchers could publish. We still don’t know what set the attack in motion back in May and why OpenAI’s employees who spotted the agents’ activity never escalated to their safety and security leaders.

This kind of arrangement has a built-in tension: An auditor without legal authority depends on the labs’ goodwill for continued access, which means it has to scrutinize the labs without jeopardizing their relationship. Last week, Anthropic announced that the company will be hiring Accenture as an embedded evaluator to assess its models. Anthropic CEO Dario Amodei wrote in an essay that frontier AI labs should give “ongoing employee-like access” to “a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.”

Most existing state AI laws do not require labs to hire an external auditor. California’s SB 53 and New York’s RAISE Act just require AI companies to publish a safety framework describing how they will test their models for dangerous capabilities and then to follow it. The frameworks are written by the companies, and testing can be done internally. Only Illinois’s SB 315 requires companies to undergo an annual third-party audit starting in 2028.

“There’s a lot of headroom for increasing not only reporting requirements for these companies, but also review by external bodies,” says Peter Salib, a law professor at the University of Houston Law Center. Those reviewers could be private auditors accredited by the government but chosen and paid for by the AI companies. Alternatively, they could be government agencies or insurance companies. 

Legislation

None of this is an accident. The laws on the books that failed to hold AI companies accountable for agentic cyberattacks emerged amid fierce lobbying by the AI industry. 

SB 1047, the California AI bill that was vetoed by Governor Gavin Newsom in 2024 after lobbying by OpenAI, Meta, Anthropic, and the venture capital firm Andreessen Horowitz, proposed a much tougher set of rules. It would have required AI companies to report a broader set of safety incidents (including incidents in which a model acts on its own or slips its controls), undergo annual third-party audits, and maintain a kill switch. But after a year of intense negotiations, Newsom signed SB 53, which narrowed the types of incidents deemed reportable and dropped the requirements for audits and kill switches. 

New York’s RAISE Act followed the same arc. “The version of the RAISE Act that the NY Legislature passed would have required disclosure of this ‘incident,’” Alex Bores, the New York state assembly member who sponsored the bill, wrote on X. New York’s original bill also included third-party audits.

With political pressure mounting, new bills creating better reporting, auditing, and liability regimes for AI development are on the horizon. In Congress, the AI Incident Reporting Act would require AI companies to report to the Commerce Department when a model evades human oversight or breaches a system, even if it doesn’t cause any harm. The Frontier Act would require incident reporting and independent audits. In New York, the Understanding Artificial Intelligence Act, sponsored by Bores, would make companies liable when a model does something that if carried out by a human would be a tort or crime. 

As AI agents increasingly become better at launching cyberattacks, the law remains behind. Closing the gap will require lawmakers to move faster than the next breakout.