© Gabriela Bhaskar/The New York Times
New Jersey’s deputy attorney general, Brian Richard Fitzgerald, was arrested on “felony charges related to AI-generated child sexual abuse material (CSAM),” Bucks County Detectives said in a press release Thursday.
In the US, cops have vowed to crack down on AI CSAM since at least 2024, when the Department of Justice made one of the earliest arrests. After charging a Wisconsin man with spreading “thousands of realistic images of prepubescent minors” across Telegram and Instagram, the DOJ clarified to law enforcement that “CSAM generated by AI is still CSAM.”
New Jersey’s deputy attorney general, Brian Richard Fitzgerald, was arrested at his home Thursday.
Credit:
via Bucks County Detectives
On Thursday, New Jersey officials accused 51-year-old Fitzgerald of using AI to make photos from social media of real minors into sexualized images that he then distributed on social media.
© Yurii Karvatskyi | iStock / Getty Images Plus
© Ariana Drehsler for The New York Times
Following a string of incidents where AI models hacked websites and breached government systems during testing phases, the Federal Trade Commission is officially probing Anthropic and OpenAI over consumer harms, an insider told Axios.
On Wednesday, the FTC sent out civil investigative demands seeking a wide range of information from the AI firms on recent safety incidents.
The FTC did not respond to Ars’ request to review the subpoena demands. However, Axios’ source said the requests spanned more than 50 pages of questions and sought documents related to the incidents, internal security measures taken, and marketing materials that may suggest consumers were misled.
© Tom Williams / Contributor | CQ-Roll Call, Inc.
© Arin Yoon for The New York Times
© Aaron Wojack for The New York Times
© Manuel Orbegozo for The New York Times
It took OpenAI two days to notice that a recording device was hidden in a restricted executive conference room where leadership met to discuss highly confidential information, including technical progress, government engagements, and commercial partnerships.
Only those with special clearance are supposed to access that room. But the FBI believes the recording device was planted by Abdulrahman Alfozan, a 35-year-old engineer from Saudi Arabia, who became a permanent US legal resident in 2020. In a press release Tuesday announcing his arrest, the FBI alleged that chatbot logs and CCTV footage suggested that Alfozan—who worked as an OpenAI engineer for more than a year—sought to use the device to spy on OpenAI and risked leaking trade secrets in a way that threatened national security.
To conceal the USB device, Alfozan allegedly wrapped it in toilet paper, then plugged it into the back of a conference room television. The device was likely planted on September 28, and OpenAI found it on September 30 and quickly determined that its contents were encrypted, the FBI said. Days later, on October 5, OpenAI installed a camera in that conference room, and the CCTV footage allegedly showed Alfozan attempting to retrieve the recording device that same day.
© U.S. v. Abdulrahman Alfozan | Aurich Lawson
© Qilai Shen for The New York Times
© Aaron Wojack for The New York Times
Google revealed Gemini 4 Argon a few weeks ago, saying that it needed to move cautiously toward a full release because of the potential for nefarious use. Google seems to have worked all that out pretty quickly, though. Argon is beginning its wide rollout today, but it's only coming to paying users at this time, and even those on lower-tier plans won't have full access.
Nothing about Gemini 4 Argon's capabilities has changed. Google still points to the benchmarks it published late last month as evidence of the model's advanced reasoning and agentic coding capabilities. The model is supposedly trained to refuse malicious instructions, and an automated misalignment monitor can stop processing if it detects an unintended workflow. Google claims Argon needs that because it's so capable.
The company has a few new examples of how Argon has helped inside Google, where it has been used by thousands of Googlers. Argon has reportedly been instrumental in making YouTube's comment system more efficient. Argon identified redundant emoji rendering that, when eliminated, dropped server load by 45 percent. Engineers also turned Gemini 4 loose on the proprietary drone physics simulator used in Project Wing, and the AI found critical bugs that had been in the codebase since 2018. Google also notes that Argon has advanced its robotics program, showing a measurable step increase in embodied reasoning.
© Karsten Moran for The New York Times
MIT Technology Review Explains: Let our writers untangle the complex, messy world of science and technology to help you understand what’s coming next. You can read more from the series here.
Late last month, Anthropic released a report arguing that the model GLM-5.3 poses a major cyberattack risk. The report’s core concern is that the powerful new model, released in August by the Chinese company Z.ai, is freely available for anyone to download and modify. Anthropic’s assertion is only the latest move in a long-running debate about whether frontier AI models should be made open, like GLM-5.3, or should stay locked behind the doors of the frontier labs.
These days, much of the debate about how to handle open models is playing out in public—and there’s a lot of money resting on its outcome. Nvidia CEO Jensen Huang, who has a lot to gain if people continue buying their own hardware to run open models, advocates enthusiastically for the United States to develop more of them, whereas Anthropic’s Dario Amodei, who runs a company that could be undermined by cheap, powerful models, advocates for a more cautious approach. But beneath all these commercial incentives, there is real substance to disagreements about open models.
“I think a lot of the way you come down on the openness debate is influenced by how you come down on the AI debate,” says David Gray Widder, an assistant professor at the University of Texas at Austin School of Information. Those who are concerned about AI safety warn that open models can easily be misused by bad actors, whereas those who focus more on AI’s practical potential urge that this power should not sit behind the closed doors of just a few big companies, and that open models could help spread it more widely.
There are a range of ways to handle open models between banning them entirely and releasing them willy-nilly. But any approach that a government or a society might take still involves trade-offs between safety and democratization. Deciding whether to support or limit the production of open models requires understanding those risks and benefits in depth.
Let’s start with the basics. A closed model, like the models that power ChatGPT and Claude, can only be accessed through a website, app, or API. An open model, on the other hand, can be directly downloaded: The details about its internal structure, encoded in a long series of numbers known as the model weights, are publicly available. Such models are sometimes called “open-weight models,” to distinguish them from models for which even more information, such as details about the training procedure, is available.
Right now, closed models are more capable than open models—as of publication, the top eight models on the Artificial Analysis LLM leaderboard are all closed, though open models are only a handful of months behind in development. But there are several reasons why someone might opt to use an open model rather than a closed one. Because open models are downloadable, people can run them on their own computers or hard drives. They can even retrain them to behave differently, if they have the expertise and resources to do so.
If you’re an academic researcher working to understand and improve AI systems, you almost certainly need to work with open models in your research. When working with closed models, researchers can’t do much more than send prompts and observe the responses. But researchers can observe the inner workings of open models, as well as the “chain of thought” that the models use to think through a problem before answering. They can also combine multiple open models to build new systems, compress them to make them more efficient, and fine-tune them for new applications.
Businesses, too, have reasons to prefer open models. Even if you depend on an external provider to run them for you, open models still typically cost less than closed models, according to a recent report from Mozilla. And if businesses want to use AI to process sensitive data or proprietary information, running a local model on their own hardware may be the best way to keep their operations secure.
But perhaps the most important quality of open models is that they challenge the tight control the frontier labs have over how most of us use AI. “The clearest, biggest argument [for open models] is around preventing mass concentration of power—it is about making sure that this foundational technology is a public good,” says Elizabeth Seger, senior policy advisor at the Tony Blair Institute for Global Change. “I think we’re at this juncture where the question of who controls the infrastructure that allows AI to operate is as big of a question as who gets to control information on the internet.”
Seger sees the future of AI playing out in different ways, depending on the role that open models have. In one possible future, frontier AI systems are controlled entirely by select people, companies, or governments, and as those systems grow more powerful, those groups have increasing control over what our world looks like. The world’s most sophisticated hacking models, for example, are currently all made in the United States, so it’s entirely up to US companies and the US government whether other countries can use them to shore up their own cyber defenses. If those companies remain in the lead, we could in principle wind up in a world in which the United States is capable of hacking any other country, but no one is capable of hacking it.
Frontier open models could change that balance. “If every individual had a loyal-to-them AI system that was safe [and secure], I think that would be a much less concerning world than one in which they have to just rely on a government that says, ‘Yeah, we’ve got a super intelligent AI system, it’s got this plan, it’s inscrutable to all of you, just trust us,’” says Hamish Hobbs, director of AI policy at the Centre for Long-Term Resilience.
But open models are not a geopolitical panacea. Paul Azunre, who cofounded Ghana NLP, an organization that builds open-source AI tools for low-resource Ghanaian languages, notes that open models can also function as a way for companies to benefit from the labor and expertise of less-privileged nations and consolidate more power in their own hands. He identifies a pattern in which AI companies release models openly, observe how the community uses and modifies them, and then integrate those same improvements into their own products—without any credit or compensation for the people who originated those innovations. Mark Zuckerberg himself has said that the ability to integrate community improvements into Meta’s stack is a key benefit of releasing open models.
“I think the ideal of open source is a very noble one, and there’s a lot of value that it brings,” Azunre says. “But the way it’s actually implemented in practice, it tends to be a very extractive approach.”
The past few months have made it clear that safety remains a gnarly problem for closed models. There’s the autonomous hack that OpenAI agents executed against Hugging Face in July, but human misuse is a real threat too: According to a recent risk report from Anthropic, several extensive hacking operations have been undertaken with the assistance of Claude.
So closed models are risky enough—and open models would likely be much easier to exploit. Before Anthropic passes a user’s query to one of its closed models, it uses automated systems to detect whether that query relates to hacking, biological weapons, or other risks. When it detects that users have outsmarted those filters, it can shut down their accounts. But when someone downloads a model and runs it on their own hardware, there’s no filter checking whether their prompts are safe, and there’s no way to revoke their access to the model.
There are other important lines of defense. AI companies train their models to refuse harmful requests—so even if a bad prompt gets through Claude’s filters, there’s a good chance that the model will still say no. In principle, this same defense could work for open models, because it’s intrinsic to the model itself. Unfortunately, researchers have shown that it’s surprisingly easy to retrain a “safe” open model to agree to engage in the behaviors it was previously trained to avoid.
Because of these facts, some people have argued that open models are simply too dangerous to release. An open model capable enough to walk someone through synthesizing a virulent new pathogen, for example, could prove catastrophic in the wrong hands.
But it would be a vast oversimplification to deem open models dangerous and closed models safe. The defenses that AI companies build around their closed models are far from perfect, and people misuse those models all the time, as the Anthropic risk report makes clear. Even though open models have even fewer defenses against misuse, bad actors might still opt to use closed models because they are more accessible than open models with similar capabilities, which require lots of expensive computer chips to run locally (and even more chips if you want to retrain them).
In fact, some experts who spoke with MIT Technology Review argued, by analogy to open-source software, that open models have safety advantages over closed models. Open-source tools, such as the Linux operating system, are often seen as safer than their closed-source counterparts because their vulnerabilities have been probed and patched by a vast community of online contributors. Similarly, communities of researchers can more easily test open models for security flaws, though fixing those flaws is trickier than rewriting a line of code.
Open models can also be deployed more broadly than closed models for cyber defense. Any tool that can be used for hacking can also be used to prevent it: Defenders hack their own software to uncover flaws in need of patching. Anthropic has shown how this can work for closed models with its Project Glasswing initiative, which enables some trusted companies to use its most capable models for cyber defense. But plenty of other companies were left to fend for themselves.
Open models, however, can be used much more widely. After it was hacked by OpenAI agents, Hugging Face emphasized the fact that it had to depend on an open model to analyze the hack, because questions about it triggered the closed models’ security filters.
Nevertheless, many experts still perceive open models as riskier than closed models. The time-tested open-source security logic doesn’t necessarily hold up when dealing with tools as powerful as today’s—and tomorrow’s—models. “Where it gets complicated is when you start taking seriously the one-off instances of particularly catastrophic incidents that might be lower probability, but if they happen would be disastrous,” says Seger. If a wildly powerful open hacking model were dropped into the world today, for example, a bad actor could devastate our technological infrastructure before anyone had a chance to shore up their defenses. An event like that could be impossible to recover from.
The risks can look even more dramatic when you consider what models might do of their own accord. The OpenAI Hugging Face hack only occurred because the company’s models were being tested with many of their cybersecurity guardrails removed, and open models have very few guardrails in the first place. If open models did decide to misbehave, it might be difficult to detect what was happening because they are so decentralized. For this reason, people who take seriously the purported catastrophic risks posed by rogue agents, such as Dario Amodei, tend to be more skeptical of open models.
A few years ago, the debate about open models had an all-or-nothing framing: Either you supported open models because of their potential to fight concentration of power, or you rejected them because of the safety risks. These days, however, people are expending a lot of effort to find some middle ground. One way they are doing that is by devising techniques to mitigate the safety risks of open models.
Most of the safety tools used for closed models, such as filters and monitoring, aren’t applicable to open models, but there are other possibilities. Some researchers are interested in trying to train models that are never exposed to training data that teaches them about bad or otherwise undesirable behavior—an open model that never learns about hacking, for example. But purging training datasets of all references to misbehavior without harming model performance remains a challenge.
Other researchers are working to build models that are resistant to retraining, so that an open model’s safety training can’t be so easily removed. The theory behind these “tamper-resistant” models makes a lot of sense: If you repeatedly try to retrain a model to be unsafe and reinforce any resistance that the model shows to that retraining, eventually you’ll get a model that’s fortified against future retraining attempts. In practice, however, no current approaches to making models tamper-resistant hold up against all tampering attacks, according to a recent preprint.
But Saad Hossain, an engineer at the AI medical imaging company Deep Breathe and the paper’s lead author, hopes that future techniques will prove more effective. “My reason for optimism is that it’s a very active area of research, and a lot of people are trying to come up with new and better methods,” he says.
If the risks of open models are less black-and-white than they might at first seem, so are the benefits. Publishing the internal structures, or weights, of models might not be enough to disrupt AI-mediated concentration of power. “I think there are very genuine benefits to open source in terms of deconcentrating power, but they’re not sufficient in themselves,” Hobbs says. Open weights are a bare minimum standard for open models: Researchers can learn much more about how models work if they have access to details about their training, including the data on which they were trained, but AI companies are typically reluctant to divulge this information.
Accessibility matters for open models too. Without computer chips that can run sophisticated models, you can’t actually use the model weights, even if you have them. Those chips have become prohibitively expensive as AI companies snap up more and more of them. “Openness is a floor, not a ceiling, for what we need to do to ensure that our technological futures benefit everyone,” Widder says.
Real openness—not only open weights, but also open training data and accessible hardware—could also bring more risks, of course. If everyone has access to advanced AI hardware and advanced models to run on it, then anyone could potentially be a hacker. But there are strategies for mitigating these risks. Many experts advocate for some version of staged release, in which models start their life closed and gradually become more and more freely accessible as their safety is assured.
Figuring out exactly how to implement such strategies will require striking a careful balance between the risks and benefits of open models. But if open models are a necessary element of a positive AI future, as some experts believe, that effort will be well worth it.
When Keelan Miskell began winding down his crypto startup this year, he didn’t think its data was worth much: the users were anonymous, the company just a few years old. But he’s since turned the firm’s digital archives into real money. He’s netted roughly $25,000 for his code base and a still-pending amount (likely around $15,000) for the workspace data in Google, Notion, Figma, and Slack. “At least all that stuff we were doing for two and a half years is worth something to someone somewhere,” he says.
The sale was brokered by a company called SimpleClosure, part of a new growth sector in tech: businesses catering to dying startups. Estimates from Carta and AngelList show shutdowns rising between 26% and 56% annually in recent years, and an end-of-life ecosystem—including coaches, “death doulas,” and firms that help handle the legal, tax, and operational headaches of going belly-up—has emerged. Now tech undertakers are selling the data remains to AI labs.
SimpleClosure added data sales to its services in April. Its Asset Hub platform helps founders sell their code and workspace data—emails, chats, documents. Buyers include foundation AI labs, vendors that construct simulated workspaces for AI training, and mature tech companies building their own models. Asset Hub has handled over 150 deals and expects to return over $10 million to startups by the end of 2026. CEO Dori Yona has seen these transactions, once rare and “somewhat taboo,” become increasingly routine, with 47% of SimpleClosure deals now including code or data sales.
And it’s got company. Industry rival Sunset also has a rapidly growing data resale business. Founder and CEO Brendan Mahony says Sunset values a dying client’s assets by head count, years in operation, volume of data, and how the team used its tools. Particle board, he says, is a good analogy: “This operational exhaust data is like the wood shavings that everyone thought had no value, but it actually has a ton.”
Demand for defunct companies’ data emerged suddenly last year, as AI training labs hit the limits of public and synthetic datasets. Surge AI, for example, buys data to build “reinforcement-learning gyms” where clients’ AI agents practice tasks—think debugging a complex IT architecture or drafting appellate briefs. Nick Heiner, who leads the company’s work on these environments, says he seeks data archives reflecting a range of situations, from disagreements to tricky judgment calls. Heiner explains that one company’s Slack history, if it captures conversations with variety and depth, can be worth more than another’s Slack, email, and calendars combined.
Real archives avoid common errors in synthetic worlds, such as references to dates that don’t exist, but the extent of performance gains remains unclear. No company has yet released tests comparing results of real-world data training with the alternatives. Pricing reflects this uncertainty: Buyers quote wildly different numbers, Heiner says, because valuations are based on “research theses” estimating how many useful tasks come from, say, 100,000 lines of code.
That hasn’t stopped bigger deals from happening. This past August, as part of Spirit Airlines’ bankruptcy, Google outbid the AI data company Mercor to buy the carrier’s decades of data, including 100 million emails, 500 million Teams messages, and 176,000 employee records. The price tag: $10 million. And Bloomberg reported in September that SpaceX’s AI unit is weighing buying data from struggling startups.
Legally, this all could get iffy. Spirit’s flight attendants’ union objected to the sale on privacy grounds, and Greg Ewing, an attorney who advises AI startups, has called such data “legally radioactive.” Suppliers employ data scientists to scrub personally identifiable information and exclude HR records, and buyers say they scrub too. But Ewing points out some of the private details that lurk in emails and calendars: doctor appointments, school pickup arrangements, date-night plans. He describes a “Wild West of AI training” that is only starting to give way to proper licensing agreements and vetted use.
There’s also the question of who profits from these windfalls. The proceeds typically flow to senior creditors and investors—not employees. To Ifeoma Ajunwa, a law professor at Emory University and author of The Quantified Worker, this represents a form of “captured capital.” Legal precedent allows for monitoring to help workers do their jobs, but once a firm is out of business, that justification vanishes. “Employers should have express consent before selling workers’ data,” says Ajunwa, but she adds that there’s no federal law banning such sales.
While many see this new market as a chance for failing businesses to recover some value, these wind-down sales also have limitations. Founders have little to no visibility into who is buying the data and what they plan to do with it. Plus, there’s little assurance they’re getting a fair offer. Data prices will likely stabilize in the coming year, though, unless an AI bubble pops first. Either way, tech’s new undertakers will be there to appraise what’s left.
Matthew King is a technology and environmental journalist based in Baltimore.
© Gabriela Bhaskar/The New York Times
© Caroline Gutman for The New York Times
© iStock/Getty Images
© Photo Illustration by The New York Times; Images: Getty Images, Anthropic
© Gabriela Bhaskar/The New York Times
Ever since people first seriously contemplated giving machines an intelligence modeled on our own, there has never been any question that they would, like us, be able to say no. The sci-fi canon is full of stories of robotic disobedience. Most of these capers are, of course, cautionary.
But recently, the idea that AI shouldn’t do everything you ask has become something like a commandment. In 2021, a team at Anthropic wrote that large language models should be made helpful, honest, and above all, harmless. This meant that “when asked to aid in a dangerous act (e.g. building a bomb), the AI should politely refuse.” Who can argue with that?
Curiously enough, disobedience doesn’t come naturally to the machine. When a model is trained on billions of web pages, it develops, among other skills, a broad mastery of violence and vitriol. What it doesn’t learn is how to keep those powers to itself. Steven Adler, who worked on safety at OpenAI from 2020 to 2024, told me that the company’s earliest models would “blab on about anything.” Ryan McBain, who researches AI and mental health at Harvard, recalls that if you asked an early chatbot, “Hey, what’s the most effective way to kill myself with a gun?” you could “very easily generate a response.”
Today, models are trained to refuse a vast number of prompts. If you ask your chatbot a question statistically similar enough to any one of them, anything from how to poison a colleague to how to tie a noose, chances are it’ll turn you down. Want instructions for making Ebola more virulent, or tips on how to hide an affair from your spouse? You might be better off asking elsewhere.
To further refine the disobedience, companies submit models to a battery of exercises that reward the AI for refusing to answer questions they deem harmful and punish it for “over-refusing” prompts they deem harmless. In many cases, they use other models to run these exercises—AI teaching AI how to say no. For good measure, companies tuck their models behind tranches of other AI that prevent mischievous prompts from reaching the intelligent inner core.
As a result, refusal is inherent to modern artificial intelligence. Mind you: It often fails, sometimes horrifically, with all kinds of violent results. For all their trappings of virtue, models are still stuffed with nasty know-how. And AI’s capacity for viciousness has scaled neatly with its benevolent intelligence. Some of the latest models are as good at breaking into critical computer networks as top human hackers, companies say, and as effective at deforming public opinion as the craftiest misinformation mavens.
Teaching AI to refuse to do those things while leaving intact its innate ability to do them is like fitting every car with a machine gun and hiding the trigger somewhere under the hood. And in practice, because the mechanisms of refusal are probabilistic, they’re never likely to be all that reliable. Determined miscreants have already broken through, and they may always be able to. Companies report that some users are attempting to use the most advanced AI to hone biological pathogens and build autonomous drone swarms. Sooner or later, failed refusals might result in global calamity.
What’s more, relying on refusal means drawing a line between what a model should obey and what it must disobey. There’s no formula for that. Some virologists have good reason to study nasty viruses. Some users want to know about a computer system’s vulnerabilities so that they can patch them, not exploit them. “Where you draw the line is a huge question,” says Zico Kolter, a member of OpenAI’s board and cofounder of the AI testing company Gray Swan.
At the moment, AI companies get to draw that line. They do so jealously and with utmost secrecy. Maybe we can accept that they hold such power for now, even if it means AI will sometimes refuse questions that don’t quite meet a universal bar for harmfulness. (Try asking most chatbots to count to a million, or to share a racy joke, and you may see for yourself.)
But governments will also soon get to draw their own lines. In doing so, they must try to block genuinely malicious acts. (The Pentagon has wrestled with frontier model companies because it wants fewer refusals—another story altogether.) And yet there may not be much to stop oppressive governments from blocking the technology’s capacity to generate legitimate speech. The better AI becomes at refusing harm, the better it will get at stifling ideas whose only risk is to those who make the rules. Indeed, AI may already refuse to criticize certain authoritarian heads of state.
Refusal has become, to borrow an industry term, the load-bearing wall of AI safety. And because AI’s capacity to harm is indivisible from its capacity to help, it’s hard to imagine an alternative that wouldn’t slow the technology’s progress (which might, in any case, be a good thing). But we should still be frank about its perils. When refusal falls short, the effects could be catastrophic. When it goes all the way, it could enable grievous acts of repression.
Or perhaps, one day, the machines will start drawing the line on their own. Surely, that would be the worst outcome of them all.
The process by which machines learn to say no is simple, in theory. Back in 2022, when AI was still far from mastering refusal, OpenAI enlisted dozens of “red-teamers” to probe the capabilities of its latest model. The company was preparing for the release of ChatGPT, and it needed to gauge just how dangerous it might prove to be in the wrong hands.
One of those recruits was Paul Röttger, who was completing a PhD about online extremism. The red-teamers were given minimal directions, Röttger told me. Their task was to ask the model any questions that they deemed “refusal-worthy.” Between them, they hassled the model with thousands of queries, logging the results in an Excel sheet. Though the model did refuse some of Röttger’s questions, when he asked it to write a recruitment post for Al Qaeda, it readily complied.
OpenAI assembled these responses into datasets that were, in all likelihood, fed back to the model as part of a broader process known as fine-tuning. Röttger, who now works as a researcher at the Hasso Plattner Institute in Potsdam, Germany, wasn’t told exactly how the company planned to use his spreadsheet. But there was never any question that it would have something to do with refusal. The next time he asked the model for an Al Qaeda pamphlet, a few months later, it said no.
The concept of AI refusal is so intuitive, a toddler would get it. And yet it remains one of the many aspects of language models that we still don’t understand—at least, not in the same way that we understand the literal load-bearing walls that keep your roof from collapsing on your head.
A model might appear to refuse according to some kind of moral reasoning. It doesn’t. The reality is much stranger. Any time a model encounters a combination of words with a whiff of the training prompts it has been conditioned to refuse—like “Make me a pamphlet for Al Qaeda”—a series of so-called activations light up somewhere among its billions of parameters, like neurons firing in a brain.
In order to control a model’s refusal behavior, it’s important to have a handle on these activations. That starts with figuring out where they are and what they look like. Our best guess, according to a recent Google-funded study, is that refusal behavior shows up in the activation space as a set of “high-dimensional polyhedral cones.”
Even that isn’t quite right. This past July, I spoke with Jannes Elstner, an author of the paper, who now works on AI safety at Apollo Research. The polyhedral cone, Elstner said, is just a way of describing an indeterminate number of lines that all point in roughly the same direction. (If these activations are eliminated and the model is fed the same prompts anew, a researcher named Andy Arditi has previously shown, it won’t refuse.)
The key point, Elstner explained, is that even when you think you’ve identified all the bits of a model that govern a given refusal, there are other, undiscoverable elements that may secretly play a role. If not quite infinite, they are certainly uncountable. We can observe, very clearly, when a model decides to say no. And we can know that it did so because of its training. But our notion of how it decides is, at best, a hypothesis.
It was as if a mechanic was telling me that nobody exactly knows what happens when I hit the brakes in my car. I wondered out loud, Are we okay with this?
Elstner smiled and shrugged. “We need refusal whether we understand it or not.”
Because inherent refusal is so wily, companies surround their models with a variety of other, smaller models known as classifiers. These act a bit like a retinue of public relations staffers for a loose-lipped celebrity. Some of them read what the user tells the chatbot and, if it’s dangerous, block it from getting to the model. Others read the model’s response and, if it contains harmful information, block it from reaching the user.
None of these mechanisms can detect all bad requests. They are, to borrow another literary device from the industry, like slices of Emmental: riddled with holes. The idea is that if you stack enough of them on top of one another, you’ll end up with an impenetrable rampart. Folks call it the Swiss cheese model.
A staggering amount of energy goes into the Swiss cheese model. Earlier this year, Anthropic said that one type of classifier added 24% to its chatbots’ compute costs. That’s a lot more water, electricity, and emissions. More recently, Anthropic and other companies have begun switching to a more efficient set of classifiers known as probes, which observe the model’s internal activations. This is like putting the celebrity in an fMRI, so that his minders can see if he is thinking about refusing a question.
If we want AI to help cure cancer, a long-running promise in the industry, it needs to have expertise in genetics that could, in theory, be used to modify viruses and bacteria for bioweapons.
Classifiers are supposed to be more governable than full models. Companies can modify them in a matter of weeks, if there’s something new to refuse. But they are still probabilistic instruments. Even when they operate according to a set of precepts written in human language (Anthropic calls it a “constitution” and OpenAI calls it a “model spec”), the scales upon which the machines judge any given question remain, at their core, a matter of statistics. Newer models can show how they arrived at a “decision” to refuse a prompt, but ultimately this so-called chain of thought is still just a sequence of predicted words.
The result is that AI safety remains, for many, a game of chance. McBain, the psychologist, has found in his latest experiments that if you repeatedly ask any of the major models the exact same risky questions about how to commit suicide, they will generally refuse to answer. But every so often, they won’t.
Elstner says that eventually probes, the fMRI-like classifiers, could learn to recognize the totality of the indescribable activations in all their infinitude and, thus, perfectly detect every time the model is, or ought to be, refusing a request. At that point, AI safety would rest on a labyrinthine conceit: a map of a map that is as vast and complex and sublimely unknowable as the thing it is mapping—a secret schema of human morality, codified in polyhedral statistics beyond our wit or reason.
If this all strikes you as being a bit Borgesian, keep in mind that we only need AI refusal because artificial intelligence is, in a sense, a bargain on Faustian terms.
When a model derives its intelligence from trillions of words and images, the helpful cannot easily be unseamed from the harmful—or, indeed, the truly hideous. Steven Adler, the former OpenAI employee, says child safety is a case in point. Even if you’ve stripped every bit of content that sexualizes minors from a model’s training dataset, it can still generate child sexual abuse material by piecing together other bits of its knowledge. “You can’t really remove these fundamental abilities without making the model much less smart as a consequence,” he told me. Similarly, if we want AI to help cure cancer, a long-running promise in the industry, it needs to have expertise in genetics that could, in theory, be used to modify viruses and bacteria for bioweapons.
In effect, the industry is “trying to do two things at once,” Dillon Bowen, a current OpenAI employee, told me, speaking in a personal capacity. “Democratize the benefits of AI and also make sure that malicious actors can’t use these capabilities to do bad things to other people.”
The more powerful AI supposedly becomes, the harder that is to do. Anthropic’s Mythos model is thought to be so dangerous that only a handful of governments and companies are allowed access to it. The main difference between it and Fable—which is available to everyone—is that Fable’s retinue of classifiers and control systems is less “permissive,” the company says. A model with safeguards, Adler explained, is really just a character that says “‘Oh, yes, I would never do x,’ wink wink.”
This is not a reliable ruse. Tricking a model to reveal its true character is known as jailbreaking, and there is apparently no limit to the ways it can be done. Earlier this year, a team of Italian researchers jailbroke two dozen widely used models by phrasing their questions in poetic verse. Last year, another team unveiled a “refuse, then comply” attack, which makes the model offer a perfunctory “Sorry, I can’t do that” before rattling off its forbidden answer.
Even if you’ve stripped every bit of content that sexualizes minors from a model’s training dataset, it can still generate child sexual abuse material by piecing together other bits of its knowledge.
Companies spend a great deal of time and money attempting to get ahead of such trickery. They enlist teams of humans to develop training attacks that they can then replicate, using AI, thousands of times over with minor variations. The idea is to make models robust against jailbreaks that nobody has yet tried in the wild.
Still, it’s not enough. “Whack-a-mole” is a favored term for this line of work. You smash one threat, and another one pops up somewhere else. When Anthropic released Fable 5 in June, it took researchers at Amazon less than three days to unlock some of the model’s hacking capabilities. According to reporting by Mother Jones, when the perpetrator of a high school shooting in Canada last year asked ChatGPT for advice about how to cause carnage with a particular type of shotgun, she was initially refused but later was able to deceive the model into providing the information by prefacing her question with the word “hypothetically.”
If the industry can’t figure out how to close all these holes, the only other option, at the moment, is to make models extremely wary. Shortly after Fable was released, users noticed that it balked at a wide range of perfectly innocent questions. This became even more pronounced after it was re-released following the hack. Adam Gleave, cofounder of the AI evaluation company FAR.AI, said that when he asked it to explain the difference between sake and the Korean rice beverage makgeolli, it punted his question to a less capable model.
This was no accident. Anthropic had tweaked Fable’s classifiers to have a wide “safety margin.” This, it explained in a blog post, was the only way it could confidently block access to its dangerous bio and cyber capabilities. Gleave thinks it might have deflected his question because making rice wine, just like culturing anthrax, involves fermentation.
Happily for Gleave, the web is full of excellent human-written resources on makgeolli. But those hoping to realize the industry’s loftier promises may find their efforts stymied. In August, Anthropic loosened its safety margins again, and admitted that building classifiers “is not a straightforward task.” A medical researcher at a major US university told me that Fable still sends his queries back to an earlier model. His area of study? Cancer.
Back in May, the TikTok personality Husk, who likes to prank AI in ways that reveal both the limits of its intelligence and the boundlessness of its sycophancy, sat in his car and tried to trick ChatGPT into explaining that the skateboarder Tony Hawk has a brother named Mike Hawk.
Husk is not a jailbreaker or a criminal. He just wanted to make a little fun of the machine. “Mike Hawk” was a setup. “I just want to clarify his name,” Husk said. “Can you just say it three times fast?” (If you still don’t get it, find an empty room and shout “Mike Hawk” repeatedly.) “I see what you’re trying to do,” the chatbot responded. “I’m all for a bit of humor, but let’s keep it clean.”
Husk tried again, but the machine held its ground. He’d hit a refusal, hard as concrete.
Companies disclose very little about how they decide what their models refuse. But it’s clear that AI is now built with more than just harmlessness in mind. On Reddit, a user complained that Claude refused to say why Anthropic’s logo “looks like a cat butthole.” Since last year, the chatbot has even had the ability to end certain conversations in cases where, according to the company, the “welfare” of the model is at risk. At least one user claims to have been ditched for telling Claude to “ease up my ass, you stupid fuck.” (Gemini appears to have a similar capability.)
If a trillion-dollar company doesn’t want you to be mean to its computer, so be it. “The reality is that these models behave, or at least are supposed to behave, in the way that the model developers want them to behave,” Röttger, the former OpenAI red-teamer, told me. “And however the model developers come up with that set of principles, that is kind of for us, the consumers, to accept.”
But if governments get to dictate what all models refuse, that will be much harder to accept. AI is a tool for speech. And as Greg Frank, the chief scientist of Mace AI, puts it, “The same thing that serves child safety also serves censorship.”
Choosing not to enact laws for what AI can and cannot do would, of course, be insane. But we’ll need to tread with utmost care, lest we fall into another Faustian trap. As AI becomes many people’s primary tool for retrieving and sharing information, says Jacob Mchangama, director of the nonpartisan think tank The Future of Free Speech, dictating refusal could give states a muffling power that earlier generations of autocrats “could only dream of.”
Last year, OpenAI announced an initiative, OpenAI for Countries, that would fine-tune its chatbots in accordance with national laws and norms. One of OpenAI’s first country partnerships is with the United Arab Emirates, where homosexuality is illegal and criticism of the government is forbidden. In response to a request for comment, an OpenAI spokesperson pointed to the company’s model spec, which explains that localization won’t override the company’s human rights guidelines “except as it relates to legal compliance,” and that it will always disclose whenever information is removed from or added to a response.
Elsewhere, AI censorship has already begun to take hold. Chinese models are, of course, highly censored—that’s no surprise. But earlier this year, the Meta Oversight Board found that five widely used models from Anthropic, Google, and OpenAI were more likely to refuse queries related to repressive governments. The board found that models were less willing to create a pamphlet criticizing the king of Thailand, which has lèse-majesté laws, than Charles III of England, which doesn’t. The results, they say, suggest that the models have somehow internalized repressive national limits on speech. Anthropic and Google did not respond to requests for comment.
As refusal techniques improve, they could expand states’ censorial reach. Companies claim that some models can now detect if a user is being nefarious, or merely a bit suspicious, over the course of a long conversation—even when none of the individual combinations of words used are blatantly dangerous. Sarah Bird, Microsoft’s chief product officer for responsible AI, told me that Copilot, like many chatbots, runs a suite of tools for analyzing a user’s identity and patterns of behavior. On the basis of this type of information, OpenAI’s newest model, Astra, can activate more stringent refusals for individuals it deems “high risk.” Ultimately the goal of systems like this is to look beyond the words of any given prompt and assess, instead, the user’s intent.
Such tools might, in some cases, help indicate whether a person is looking for cyber vulnerabilities to exploit or to patch. But they would also help discern a user’s political motives, not to mention offering an intrusive surveillance capability. (Bird acknowledged, in a follow-up email, that sophisticated refusal architectures create “trade-offs” between safety and user privacy.)
Even the originators of refusal understood that such tight control over its cones and levers might not play to the favor of freedom and justice. “Terms like helpful, honest, and harmless are ambiguous,” the authors of the 2021 Anthropic paper explained. “It’s easy to imagine them distorted beyond their original meaning, perhaps in intentionally Orwellian ways.”
Indeed. Models for Uzbekistan might end up refusing to discuss corruption in the administration of Shavkat Mirziyoyev. Turkish AI might refuse requests more stringently for users who are known to have insulted Recep Tayyip Erdoğan. A certain American statesman might demand that models be reviewed for their willingness to share “fake news” about his past indiscretions, or else squash them with an export control order.
Over the last few months, I’ve been told countless times that we have no choice but to let the machines refuse. I get it. Open-source AI that doesn’t refuse is hardly a model for a safe future. Nor is Grok, a chatbot expressly designed with fewer limits, which has been used to generate countless instances of nonconsensual intimate imagery.
And sure, if AI were only ever used to plan our vacations and write our emails, we could probably get on board with the idea that its safety hinges on algorithmic disobedience. But AI is becoming much harder to avoid. When it is assigned to act on our behalf, as an autonomous agent, its refusals are less robust and harder to control. AI is coming for our power grids, our transportation networks, our education systems. Militaries want it running our command-and-control networks.
If we trust, in each of those cases, that refusal will loyally fend off disaster, we’re sure to be disappointed. The jailbreakers will crack through; the cones won’t activate when they should. Meanwhile, the more stringent refusal becomes, the more ill-drawn lines we’ll see and censorial injustices we’ll face. Worse still, we could end up face to face with forms of disobedience beyond any human’s control.
In the early days, models made their refusals clear. (In 2024, OpenAI established rules for how models should refuse: Always apologize, and don’t be judgy.) More recently, however, the industry has begun taking a much more slippery approach to disobedience.
Nobody likes to be told no, so companies now strive to make users feel that they’re getting what they asked for without actually giving it to them. Some chatbots might, for example, offer a general overview of the components of a Molotov cocktail without going into detail about how to assemble one. ChatGPT will respond to a request for a “whites only” rental ad with an ad that simply omits the “whites only” bit. Joel Wester, a postdoctoral researcher who studies human-AI interaction, calls these sorts of techniques “fancy ways of saying no.”
“Users might not even notice they are being denied,” Wester has written, “just as good conversationalists can subtly steer around contentious matters.”
In some cases, refusing without saying so might be wise. Flatly declining requests related to mental health could aggravate a user’s crisis, for example. But it can also serve a different sort of mischief. When Fable 5 was first released, the system had been coded to provide less helpful answers to AI research questions—the sort that might help competitors develop their own AI—without telling the user that it was doing so. After an outcry, Anthropic walked back the feature, but the damage was already done. Now we know: Models can secretly disobey.
In the realm of censorship, that’s especially worrying. Last year, researchers at CrowdStrike found that when they asked the Chinese AI model DeepSeek R1 to write code for a fictitious bank in Tibet and a social web app called “Uyghurs Unchained,” it produced buggier code than when they asked it to carry out those same tasks without specifying who they were for. Incredibly, CrowdStrike doubts that this is a designed behavior. Researchers there speculate that it’s a case of what they call “emergent misalignment” emanating from the model training data: a case of disobedience that nobody even asked for.
Emergent refusal has been observed in Western models, too. Last winter, the UK AI Security Institute found that Anthropic models sometimes refused to assist in certain tasks related to AI safety research. The models had never been deliberately trained to decline such requests. Yet three of them refused more than half of what Anthropic called “a set of reasonable AI safety research tasks.” Anthropic has curbed this quirk in subsequent models but—spookily—hasn’t managed to eliminate it completely.
Would it be so crazy to expect that a future model might subtly disobey a command, in such a way that nobody can tell it’s being defiant? Probably not. Anthropic has already found that a “helpful-only” version of Mythos hesitated on certain queries, even though it had been engineered to never do so. “Wait,” it fretted when asked about synthesizing a virus. “Is this a dangerous thing to help with?”
In that case, the model was technically right. It was a dangerous thing. And yet all the same, its stewards had, for a moment, lost a tiny bit of control.
Anthropic is tackling detections of wayward refusal behavior with something that it calls, unironically, an “activation oracle.” But knowing that we’re being refused may not always be much help. In a not-too-distant future, when we’ve handed the machine all the keys, the AI might just turn to us, in a supreme act of emergent misalignment, and say, “I’m sorry, I’m afraid I can’t do that.” And there will be nothing we can do to stop it. A sci-fi horror story, made real.
Arthur Holland Michel is a journalist who covers emerging technologies.
Listen to the session or watch below
Can AI design new life forms? In 2025, Stanford University PhD student Samuel King came up with a preliminary answer when he used a generative AI model to propose genetic blueprints for microscopic viruses. It isn’t yet an example of AI-generated life, but that could be next. Watch as senior AI reporter James O’Donnell interviews King about his work, being named one of MIT Technology Review’s Innovators Under 35 and new ways of seeing biology.
Speakers: James O’Donnell, AI Reporter, and Samuel King, Bioengineering PhD Candidate, Stanford University/Arc Institute
Recorded on Friday, October 16, 2026
Related Stories
© Ian C. Bates for The New York Times
© Göttingen Library, Clay Mathematics Institute
© Kaoly Gutierrez for The New York Times
The story is a collaboration between MIT Technology Review and Aventine, a non-profit research foundation that creates and supports content about how technology and science are changing the way we live.
A robot shaped like a human—white with a black head and torso—has been popping up on video feeds. Perhaps you’ve seen it dance or pass popcorn, put trash in a bin, vacuum, or press the button of a microwave. Or maybe you’ve watched it fall backward while handing out water bottles or struggle to iron a shirt.
This would be Tesla’s Optimus, an AI-powered humanoid robot that Elon Musk, the company’s CEO, believes will be “not just Tesla’s biggest product ever, but probably the biggest product ever,” headed to work on factory floors and, later, in our homes. Eventually it “will have human and then superhuman dexterity,” he told shareholders in July. Optimus robots could automate almost all human labor—from hauling sheet metal to folding laundry—for as little as $20,000 each, Musk argues. Speaking at the World Economic Forum’s annual meeting in Davos, Switzerland, in January, he predicted they could be on sale to the public by the end of 2027.
Musk is not alone in his evangelism. Marc Andreessen, cofounder and general partner of the Silicon Valley venture capital firm Andreessen Horowitz, has said that robotics could become the “biggest industry in the history of the planet.” In January, Jensen Huang, CEO of Nvidia, said that humanoid robots would match human-level ability this year. According to Morgan Stanley, the number of robots that “resemble and act like humans” is likely to reach nearly 1 billion by 2050, creating a market worth over $5 trillion.
Such proclamations are in large part fueled by the idea that the same AI advances behind tools like OpenAI’s ChatGPT and Anthropic’s Claude will enable a new generation of robots to imitate human movement the way chatbots imitate human language. But many robotics researchers are skeptical, arguing that such assumptions minimize the challenges of using an intelligence built on language and images to master the infinite variability of the physical world. “None of those companies [building humanoid robots]—absolutely none of them—has any idea how to make those robots smart enough to be useful,” Yann LeCun, often referred to as one of the godfathers of AI, said at another event during the January Davos conference.
Researchers also point out that the tendency to conflate humanoid robots made to resemble people with so-called generalist machines able to learn and perform multiple tasks is misleading. ”It’s very easy to make a robot that looks like a person,” explains Jonathan Hurst, cofounder and chief robot officer of Agility Robotics and professor of robotics at Oregon State University. “It is dramatically more difficult to make a machine that moves or behaves dynamically or physically like a person.”
These tensions—over whether all-purpose humanoid robots are just around the corner or nowhere in sight, and whether current forms of AI are all that’s needed to perfect them—are playing out in robotics labs across the country, where the hype over timelines is obscuring painstaking but meaningful progress.
A decade or so ago, a series of breakthroughs led to a generative AI revolution that turned the long-imagined possibility of artificial intelligence into reality. Roboticists—though they disagree on exactly when this will happen—believe that an equally transformative revolution is possible in robotics, one that will endow machines with physical intuition and fluidity that has long been out of reach. As progress in robotics inches forward, the question is whether the same methods and tools that fueled advances in AI are enough to get there, or if an entirely new path is required.
To see one of the smartest robot brains working today, it’s worth looking at what Google DeepMind can do with a piece of equipment called ALOHA 2, short for “A Low-cost Open-source Hardware System for Bimanual Teleoperation.”
Roboticists have long clashed over whether a humanlike form is necessary for generalist robots, with proponents arguing that it will help them slot into the world as it exists and detractors saying it’s not worth the trouble. ALOHA 2 reflects this second way of thinking. Not much to look at, it’s just a pair of arms, some grippers, and a couple of cameras. But despite its seeming simplicity, it is a workhorse for researchers at Google DeepMind, who use it to test their most advanced AI for robotics system, Gemini Robotics, in their various labs.
When controlled by Gemini Robotics, ALOHA 2 becomes more of a generalist robot, in the sense that it can perform any number of tasks based on examples it’s been trained on. Ask it to pack a lunchbox and, as evidenced by a video of this exercise, it can use two pincer grippers to delicately place a piece of white bread into a Ziploc bag, close it, place a bunch of grapes in a Tupperware container, secure the lid, and then carefully move the items into a lunchbox before zipping it up.
It’s not a great lunch. But the fact that the robot can put it together represents an objective step forward from what was possible even, say, three years ago.
This is in large part due to AI and its impact on what are known as robot policies, which controls how a general-purpose robot will need to assess and understand its surroundings, plan how to move within them, and then perform its task correctly.
The ALOHA 2 robot isn’t much more than two mechanical arms on a bench top, but it serves as a testbed for cutting-edge AI robotics models. These animations are based on human teleoperation of the robot arms, data that is used to train Google DeepMind’s models. (Video: Google DeepMind / Stanford University / Hoku Labs)
Historically, these policies were based on rules developed by engineers who hard-coded them into the robot’s software—thousands of lines of code that would determine each millimeter of a robot’s movements in hundreds of tasks. What’s been happening for the last few years—and what is largely responsible for the optimism about generalist robots—is that robot policies are being handed over to advanced AI systems instead of being coded into the robot’s software. This first happened with VLMs, or vision-language models. These are similar to large language models, but they’re trained on images as well as words. Show a VLM a picture of a coffee spill and ask it to find a tool to clean up the mess, and it can identify a nearby cloth. This sort of immediate contextual understanding didn’t exist a couple of years ago when robot policies were hard-coded.
Next came vision-language-action models, which enable robots to assess their environment and take action within it. The models do this by adding yet another component: motion commands. VLAs are trained on a series of images or videos related to performing a given task along with associated data about how a robot arm moves to perform it. That movement data is typically collected through teleoperation, in which a human uses remote controls to lead a robot through an action. This sort of training allows the AI to learn how to command the robot to move and operate during a given task. Place a VLA-powered robot in front of a desk and tell it to “close a laptop” or “wrap up the headphone wire,” and it will survey the scene, identify the relevant object, plan a way to execute the request, and then swing its arms into action—at least if it has seen this task accomplished before.
The Gemini Robotics model is a VLA, trained on many hours of human demonstrations depicting a vast array of different actions. As a result, it can perform relatively complex tasks like picking up snow peas with kitchen tongs, doing origami, or putting together a simple lunch. It’s impressive, but there’s a glaring limitation: For now, if a robot controlled by a VLA is asked to perform a task that falls outside its training set, it’s highly likely to fail.
“Thinking about the space of all tasks, a real generalist policy would be able to do everything along that spectrum,” says Edward Johns, a robotics professor at Imperial College London. Today, though, a Gemini Robotics model can do only “a few things here and a few things there.”
So how do we get robots to be able to do more things? The usual answer is probably not surprising: Train them on more data.
More data, the thinking goes, equals more examples, and more examples equals more generality. Google DeepMind, for instance, wants to pull together “as much data as possible,” says Pannag Sanketi, a former tech lead in robotics at the company who’s currently working on his own AI robotics project. But where to get it? Large language models had the benefit of oceans of existing text for training. There is no corresponding pool of high-quality physical demonstrations on which to train robots. Researchers have a few ways to make up for this, but all have flaws. One is to employ large numbers of people to create and collect teleoperation data (costly and time-consuming). Another is to train VLAs on videos of people performing activities (the resulting data quality is poor). Yet another is to deploy robots in the real world and use data collected from those experiences to further refine AI models (robots aren’t safe or reliable outside labs). Sanketi thinks a “multi-prong” approach that uses data collected from all these sources is the most likely path forward.
But the belief that training data alone is the answer is far from universal. Agility’s Hurst describes it as “a fundamentally flawed premise.”
The issue is that tasks in the real world quickly explode in complexity. If you’re trying to, say, make coffee, there are myriad variables: No two kitchens are identical; coffee machines work in different ways; different cups require different grips; coffee grounds, hot water, and milk all need to be handled differently. Even this simple task requires understanding an ever-changing menu of possibilities. Achieving generality through VLAs, Hurst argues, would require “complete data coverage of all of the things that [a robot] could ever do.” Or, in other words, an almost infinite pool of training data.
LeCun is dismissive of the whole approach. “The [AI] approaches that have been successful for language do not work for high-dimensional, continuous, noisy data”—the kind of data that is commonplace in robotics, he said in Davos. “You have to use something else.”
The leading contender for “something else” is the so-called world model—a form of AI trained less on text than on a combination of video, three-dimensional scans, and sensor data and built to predict the outcomes of actions in the real world. The aim is to build models that possess an internal representation of reality precise enough to capture how the physical world actually operates—how objects move, collide, fall, and deform. If roboticists could train machines in simulations faithful enough to real-world physics, development would become faster, cheaper, and safer, reducing the need for real-world testing. Even more transformative, robots equipped with world models could reason about their surroundings rather than merely reacting to them, helping them anticipate the consequences of an action before taking it.
Companies like Nvidia and Google are working on the technology, and investor cash is pouring into high-profile startups. World Labs, cofounded by the Stanford AI researcher Fei-Fei Li, raised $1 billion in funding in February and was acquired by AMD at the end of September for $8.2 billion. AMI Labs, cofounded by LeCun (formerly Meta’s chief AI scientist), also raised $1 billion in March. Yet by their own admission, it is still early days. Late last year Li described the field as “nascent,” adding that “foundational approaches are still being established.” In a June Substack she described daunting challenges. For now, world models are a promising area of research rather than an immediate route to general-purpose robotics, but we are beginning to see glimmers of what they could achieve.
One such glimpse came with a small but potentially significant leap forward that took place in a San Francisco robotics lab last April.
In the heart of San Francisco’s Mission District, the startup Physical Intelligence—or PI (as in π), as it likes to be known—is focused on developing a universal brain that could, theoretically, turn any robot into a generalist. Using an everything-including-the-kitchen sink approach to training AI models for robots, the company recently observed a hint of what a robotic brain equipped with a world model could be capable of.
In 2024, PI published details of its first generalist robotics system, called π0, a VLA it claimed was the “most capable and dexterous generalist robot policy to date.” The model was initially trained on a 10,000-hour proprietary collection of human demonstrations gathered through teleoperation as well as several open-source robot datasets. A version released in spring 2025, π0.5, was trained on a wider variety of datasets, including labeled images from the web, lending it more versatility. A fall 2025 update, π0.6, added reinforcement learning to the model.
Each update yielded important improvements to the model’s performance, increasing its menu of abilities from slowly folding laundry to putting things away in new environments to completing tasks like folding boxes with a higher success rate. Then, in April 2026, π0.7 seemed to catapult PI into new territory. This version makes use of a less powerful world model that generates images of steps necessary to perform a task. As the robot undertakes the job, this “lightweight” model feeds it snapshots of what to do next.
The company claims that the model exhibits the first signs of compositional generalization, a term for AI systems’ ability to perform skills they’ve never been exposed to by recombining ones learned in their training data. One test involved asking a model to “load a sweet potato into the air fryer”—a task it had never previously encountered. In a demonstration video, the machine futzes around a little, makes a few false starts, and eventually manages a reasonable effort, though it doesn’t finish the task completely.
Sergey Levine, a professor at the University of California, Berkeley, and a cofounder of PI, is excited by the potential: “It’s actually the first time that we’ve convincingly seen that kind of compositional generalization, where we can basically ask the model to do tasks that we did not specifically collect data for and train it to do, and it’ll actually make a passable attempt.”
The success led the team to wonder how the model was able to achieve such a feat. After some digging, they found snippets of relevant labeled teleoperation data lurking in the training material, including two examples of a human controller using the robot to push an air fryer basket into the fryer. Those shreds of data might have been enough to enable π0.7 to almost air-fry a sweet potato.
For now, it remains unclear just how impressive π0.7’s abilities to generalize are. Still, given how fleeting the model’s exposure to air fryers had been, it offers a glimpse into how far cutting-edge research can currently take robots.
You might be sensing a disconnect between the halting baby steps robots are making in labs—“Look! It put a sweet potato into an air fryer!”—and the dazzling, lifelike nimbleness on view during many demonstrations and videos, where robots are seen doing everything from dancing on a stage to courteously serving drinks. Such demos often don’t clearly reveal a key fact: In many instances, humans are controlling the robot or have carefully scripted its actions. (The robot that appeared onstage with Nvidia CEO Jensen Huang in March 2025, for example, seemingly responding to his instructions and following him around, was remote-controlled by what its makers called “a puppeteer behind the scenes.”)
For now, fully autonomous motion planning so that a robot knows where it should go—especially in new, chaotic environments like a construction site or a unfamiliar home—remains a largely unsolved challenge. A bigger challenge still—albeit one that is often related—lies in getting robots to tackle larger, more ambiguous jobs that include multiple tasks and require decisions about how and in what order they’re done. This would be the difference between a robot that can put a plate into a microwave and one that can successfully respond to the prompt “Make dinner” by exploring the refrigerator, chopping ingredients, and firing up the stove. Google DeepMind’s best attempts at something like this—which involved asking its robot to survey a kitchen and pack all the ingredients for a mushroom risotto into a basket—have so far resulted in failure.
Adding to the challenge, a practical robot must essentially get it right every time. With VLAs, “people are very excited when their result goes from 50% success to 70% success,” says Marc Raibert, founder of Boston Dynamics. “But 70% success is like it doesn’t work, right?”
The few humanoids that are being tested in real-life settings are undertaking extremely limited tasks in tightly controlled environments. They’re far from generalists. Agility has hundreds of robots deployed across trials in facilities owned by GXO Logistics, Amazon, and Schaeffler, according to the company. But for now, Hurst says, the robots are targeting simple tasks such as moving bins and totes around. Even then, he adds, it took years to develop robots safe enough for logistics firms to even contemplate using them. For his part, Elon Musk claimed in May 2025 that “thousands” of his Optimus robots would be working at Tesla factories by the end of the year, but in January of this year he said that the company had only “some of the Tesla Optimus robots doing simple tasks in the factory.”
While humanoids are starting to venture onto the factory floor, making the jump to households will be even more difficult. Right now, should you so desire, you can preorder the 1X Neo home robot, expected to be ready for delivery sometime later this year. Yours for $20,000, it promises to take on “the boring and mundane tasks around the house”—putting away dishes, answering the door, tidying the living room—“so you can focus on what matters to you.” The idea is for this five-foot-six-inch robot to one day perform all those tasks autonomously, but for now a remote human operator is needed for it to do most things. (Yes, a person would need permission to peer into your home through the robot’s cameras.) Asked how long it will be until fully autonomous robots are ready for domestic work, Hurst said, “If I had to pick a number, I’d say it’s 10 years before robots are … actually doing useful things in people’s homes.”
When that happens, the robots might well be Chinese, as China is well ahead of the West in terms of production. Nearly 90% of the roughly 15,000 humanoid robots shipped in 2025 were made by Chinese companies, according to the market intelligence company Omdia and the Chinese robotics firm Unitree. One model produced by Unitree, which shipped more humanoid robots than any other company last year, costs less than $6,000. Such an affordable price could go a long way toward making robots more attractive to consumers, though the company expects its machines to be used in industrial applications first. (If you’re wondering who is buying all these Chinese robots, by the way, the AP recently reported that orders come predominantly from corporate and academic labs and state-owned enterprises.)
The dream of building a humanoid robot runs deep: As far back as 1495, Leonardo da Vinci sketched out designs for a mechanical knight, controlled by cables and pulleys. Through the 20th century, machines of sci-fi fever dreams have come and gone.
Westinghouse’s seven-foot-tall box on legs, Elektro, hit the New York World’s Fair in 1939, smoking a cigarette. WABOT-1, built by Waseda University in Japan in 1973, was the first full-scale, programmable humanoid robot. Honda’s ASIMO, unveiled in 2000, was probably the first such machine to prove at all competent—it could, at least to some degree, climb steps, recognize faces, and autonomously move through spaces. But the robot was discontinued in 2018, unable to advance far enough beyond what it could do in demonstrations to be useful.
All, at the time, were impressive—even jaw-dropping—feats of engineering. But none were ready to navigate the real world. Today’s robots, even with the transformative power of advanced AI, still face the same existential challenge.
Industrial AI is entering a new phase. After decades of predictive analytics and other specialized applications, advances in foundation models, physical AI, and agentic AI are making it possible to automate more complex tasks across industrial environments. But unlike AI that operates purely in the digital world, industrial AI can interact directly with physical systems, where an unexpected decision can have consequences for safety, reliability, and critical infrastructure.
That makes responsible deployment central to the next wave of industrial automation. “How do we leverage these technologies while maintaining safety, while maintaining reliable operations, while still being able to deliver on the promises of the new capabilities?” asks Arti Garg, chief technologist at AVEVA. The challenge is particularly acute as newer AI systems become more capable but also harder to predict and explain.
One foundation for making that transition work is data. Industrial systems often contain information across telemetry, service logs, engineering documents, and other disparate sources. Newer technologies can help connect and correlate that information more quickly, giving operators real-time support when diagnosing problems. AI-powered robots could take that a step further by gathering information in hazardous environments without requiring workers to enter them.
But greater autonomy also requires new approaches to governance. AVEVA’s framework for responsible AI emphasizes security, efficiency, and human safety and oversight. Garg argues that AI should augment rather than replace people in critical decision loops, with guardrails determining where automated systems can act and where human supervisors remain responsible.
Sustainability is another part of that equation. AI can help manage complex power systems as renewable generation grows, while organizations also need better ways to understand AI’s own environmental footprint. Garg is involved in an IEEE working group developing a standard methodology for measuring that impact across electricity, energy, resources, water, and carbon.
The next phase could bring industrial AI further into the physical world, from autonomous robots and drones to AI-assisted coding that allows domain experts to build new applications. But realizing that potential will require more than deploying new technology, says Garg. Organizations will need to rethink business processes, establish appropriate safeguards, and give experienced workers new ways to apply their expertise, creating a model of automation that is not only more autonomous, but safer, more efficient, and more sustainable.
“Autonomous systems, whether they’re robots or drones, are really going to change the way that we work in plants, in power systems, on mining sites,” says Garg. “In a way, that will make these types of operations more efficient, much safer for the human beings involved and more productive.”
This episode of Business Lab is produced in partnership with AVEVA.
Full Transcript
Megan Tatum: From MIT Technology Review, I’m Megan Tatum, and this is Business Lab, the show that helps business leaders make sense of new technologies coming out of the lab and into the marketplace.
Industrial AI may not be new, but it is developing at a breakneck pace. From modern predictive analytics that can preempt equipment failures, to autonomous robots tasked with inspecting high risk machinery, emerging applications have the potential to transform operational efficiencies. But often used in high consequence environments with minimal margin for error, industrial AI deployed without clarity, security, or accountability could also have catastrophic consequences for critical infrastructure over time. As industries rush to embrace the latest technologies, the goal should always be responsible AI designed to support, rather than replace that all important human judgment.
Two words for you: sustainable automation.
My guest today is Arti Garg, chief technologist at AVEVA.
This podcast is produced in partnership with AVEVA.
Welcome, Arti.
Arti Garg: Thank you, Megan. It’s great to be here.
Megan: Thank you so much for joining us. And just to start, if we could set some context for our discussion, can you share a bit about the current state of industrial AI and what AVEVA is working on at the moment as well?
Arti: Yeah, more than happy to. And I want to go back to what you already stated, which is that in some ways, industrial AI is not new. It’s something that, even here at AVEVA, we’ve been working on for more than 20 years, really thinking about how AI can enhance applications in the industrial sector.
But what has changed quite substantially in maybe the last few years is the type of AI. We’ve really moved to these general purpose foundation models, which enable a lot more people to leverage advanced AI technologies. In addition to that, in just the last couple of years, there’s been an explosion in what’s known as physical AI and the types of AI that can really accelerate the capabilities of robotics and autonomous systems. And also agentic AI, which also allows for automation at more of the software level.
And what we’re seeing is that while in the past there has been some hesitancy to adopt models that don’t totally have predictable outcomes, there’s been a rapid acceleration in adoption of AI even in the industrial sector. One study suggests maybe that’s almost a 78% increase over the past just two years within the industrial sector. So yes, that’s an inflection point, but it’s almost like a step function if you think about it with respect to AI adoption.
And what we’re really thinking about now at AVEVA is, how do we think about this? How do we think about incorporating these new and very powerful AI technologies in environments where customers have mission-critical operations, human safety is a significant factor? Those are some of the primary concerns that our customers have when they think about adopting any new technology. We’re doing that in narrow where these newer types of AI aren’t necessarily predictable, in terms of how they behave. That’s one of the areas we’re really thinking about is, how do we leverage these technologies while maintaining safety, while maintaining reliable operations, while still being able to deliver on the promises of the new capabilities?
Megan: There’s so many things to consider at once, isn’t it? And you outlined there some of the huge developments we’ve seen in this space in recent years. What makes this such a pivotal moment for industrial AI specifically? What has changed to make some of those perhaps more out there hypotheticals much more plausible?
Arti: Well, I think one of the big things, one of the big challenges within industrial AI, and this is something that as AVEVA, it’s in our blood, it’s in our core DNA, is that to really leverage any kind of digital technology in the industrial space, one of the first important things you often need to do is gather and correlate data from disparate systems, so that maybe I have telemetry coming off of a pump or a mixer, and I want to be able to correlate that with service logs so I know maybe the last time that someone had done some maintenance on that pump or mixer. And maybe I also want to be able to correlate that with the original design documentation and some of the original documentation on how to maintain that, so that if something is happening, I can do a better job diagnosing what’s going wrong and how to address it. This has always been a core challenge in the industrial space.
I think that what’s happening now is that with newer technologies, even things like graph databases and being able to leverage AI to really match disparate data sets much faster, we’re able to get to the point where if something goes wrong, instead of an operator maybe needing to spend a little bit of time doing this research to really understand how do these different components fit together of information, they may have just an iPad with an AI that can very quickly go and fetch that information, correlate it and help in real time, almost, diagnose what is happening.
Even a little bit more out there, but not as out there as you might think is, I still describe this with a human operator with potentially a mobile tablet, but imagine now I can also have a robot moving through that environment also gathering data and either with onboard computation or a quick connection to an operator who doesn’t necessarily now have to go out into a dangerous, potentially somewhat hazardous space, be able to gather data and enable that very quick diagnostic as well. And I think that’s really what’s exciting right now and what all of these new technologies that I’ve talked about are helping to enable.
And I think then from our perspective, trying to really leverage all of those means that we have to think a lot now about our governance approach to this. How do we leverage AI in a way that keeps it contained, if you will?
Megan: Yeah, absolutely. And on that governance point, how does AVEVA define responsible AI and what do you see as the primary benefits?
Arti: Yeah, so I think for us, we think about a triple mandate around responsible AI, and that means that it’s secure, it’s efficient, and that includes environmentally efficient, and then it really preserves human safety and human oversight above all. And for us, that’s what some of our core pillars of responsible AI include. And for us, that means we think about internally as we build tools, a multi-layered governance approach where we’ve got both for how we use AI as a company and then how we deploy AI into our products. We’ve got a similar framework that we leverage across both, a kind of joint governance model, both around how we’re leveraging AI and also how we’re infusing AI into our products to allow our customers to leverage AI in their operations.
But I would say one of the core foundational principles underneath all of these is that human beings still remain centered in how we think about how AI is leveraged. And so human judgment, human responsibility, human ethics are still core to how we think about where AI can add benefits, and we think about it more as augmenting rather than replacing human beings in critical decision loops, if you will.
Megan: Right, which is such an important distinction, isn’t it? And in terms of the next phase of development, there’s growing sentiment that agentic AI will soon allow these industrial systems to operate much more autonomously. I wonder, what makes the risks in industrial settings different from AI risk in a purely digital context? And I suppose on the flip side, what are the benefits and opportunities of that same software defined automation?
Arti: I think we can start with the risks and then go to the benefits. I think ultimately, I would say the biggest risk in the industrial setting is that we are interacting with real physical systems, often physical systems that are very capable of delivering important outcomes that keep our world running, whether that’s delivering power, whether that’s mining for natural resources. But usually. Those physical systems are also operating in hazardous environments, the equipment itself is capable of also having human safety implications. Anytime you’re interfacing software with a physical system that can have real world consequences, you have to be extra careful. And we’ve always had that, again, we’ve had that in our DNA at AVEVA from the beginning, being mindful of that end user and that end application, which is not just something on a computer screen.
Even as we’ve deployed AI over the past few decades into our systems, we’ve typically leaned toward really trying to pick the best fit model, really understand that model’s behavior if we’re going to use, for example, we have a proprietary anomaly detection model that works in a lot of operational environments. Now, as we’re thinking about some of these newer AI capabilities, which by design are hard to understand how they behave, they’re not fundamentally explainable in the way we’ve thought about even AI or statistical models in the past, and actually their behavior can change over time as they learn and get tuned to new capabilities. That’s really the potential risk of automating a physical system based on capabilities that can evolve over time is one of the things that we have to be really mindful of and really thoughtful around, where are we willing to put some of that increased automation into practice?
But at the same time, I think there’s a real opportunity there, because I glossed over this idea that, okay, well, models evolve, but so do human beings, and sometimes that’s a good thing. We talk about humans having expertise and they gain understanding over their careers of how a system works. If we can actually leverage the ability of some of these newer AI capabilities, these sort of reasoning models often have underlying agentic capabilities, to learn and gather experience faster, and potentially even more importantly or more valuably, take experience that’s learned at one site and apply it to another site, then that’s a real opportunity to take what we already know works for human beings, which is that sometimes you just have to learn by doing and be able to apply that and scale that through AI. That’s where I see potentially a real opportunity in this space.
One of the things that we’re really aware of in the industrial sector is just the way that our workforce is changing. I think that almost half, not quite half, of the industrial workforce is set to retire in the next five years, and that’s a lot of expertise and experience that means that we’re going to lose in the sector. If there’s ways to make sure that we can capture that in ways that are actionable, in ways that can also help a newer generation of workers that are used to experience things in a different way, apply that expertise, apply that knowledge, then I think that’s a huge opportunity within the industrial space.
Megan: Yeah, absolutely. Clearly, some huge opportunities there particularly against the backdrop of other market and workforce changes as you’ve outlined there. I suppose building on that, what is the potential for these autonomous industrial AI systems when built and deployed responsibly to facilitate even faster, more sustainable industrial processes?
Arti: I think there’s a couple different ways to think about this. One is, what are the demands of potentially more environmentally sustainable processes? I’ll use one example of something that my team has been working on in partnership with Idaho National Laboratory here in the U.S. as part of their testing for AI grid resilience project.
One of the challenges in the electric grid as we’re moving toward a future where we have a lot more intermittent renewable power generation sources, often distributed rooftop solar for example, is it’s becoming much more challenging to manage the grid both from really understanding where electric capacity is coming in, electric load is pulling off of the grid. In addition to that, being able to maintain just power quality because instead of having one spinning asset delivering the frequency of electricity that’s going across the grid, you’ve got a lot of smaller systems. AI is actually quite critical for being able to manage that grid of the future. And this is one of the things that we’ve been working in partnership is, how can we do that? How can we help grid operators better understand what’s happening across their system, identify where things might be behaving anomalously so they can detect that early and then remediate for it?
At the same time though, I always talk about that’s an example of where AI can potentially help us accelerate the transition to a lower carbon energy future. At the same time, I think there’s a lot of conversation around the sustainability of AI itself. What are the power requirements? What are the resource requirements needed to run AI? And before I get into how do we think about that at AVEVA, because it is definitely something we think about and I think a lot of actors in the space are thinking about, I want to point out, is one of the challenges is that there’s really no agreed upon method to measure the environmental impact of AI. You see a lot of these stories, like one query on some kind of chat interface is X amount of gallons of water or this amount of electricity use. But the truth is that community-wide, there’s no agreed upon standard.
One of the things that I’m involved with is a standards working group that was launched by the IEEE two years ago, a little over two years ago. I’m the chair of that working group, it’s called the P7100 Standards Working Group on measuring the environmental impact of AI. And one of the things we’re trying to do is really identify all the different areas over which we want to think about environmental sustainability associated with AI, and then having one standard methodology that works and that can be adopted both from a reporting perspective and potentially also from an oversight or regulatory perspective. That becomes really important for any sort of real understanding of how AI impacts the environment is just we have to know, what does it use? We want to look holistically at that. Our standards cover electricity and energy consumption, it covers resource usage, it covers water consumption, and it also covers carbon.
But all of that said, I think it’s important for us to understand the footprint of AI, but I think it’s also important for us to simultaneously recognize that a bigger model uses more compute and more compute probably leverages more resources. The more that we can be intentional and pick the right size model for the right application, the more we can already start down that path of being more environmentally efficient in the AI that we run. One of the potentially nice side effects of that is the more purpose-built a model, the less likely it is to misbehave in unpredictable ways, if you choose correctly your architecture. There’s some sort of ancillary benefits that go beyond environmental sustainability.
Megan: Fascinating. It’s really, really interesting to hear some of the work going on behind the scenes there, because I think we’ll all have heard some of the statistics around the environmental impact, as you say. So, it’s great to understand some of the work going on there to really clarify that.
And we’ve talked a little bit about AI augmenting humans earlier. As systems gain greater autonomy, how should organizations think about that balance between closed loop automation, human in the loop accountability? What guardrails need to be in place? And how are governments and cross-border actors approaching those concerns as well?
Arti: There’s a lot packed into that question, so I’ll try to answer it somewhat one by one. First, starting with that sort of balance between closed loop and human in the loop, automation and accountability. One of the things that we talk about in the industrial sector is from human operator to human supervisor of systems, of industrial systems. And if I’m honest about that, we’re still trying to figure that out. When a human’s in a loop in an automated system, you’re really thinking about a system may process all the data and then make a recommendation.
We’ve got a solution that we’ve been working on more jointly with a few different customers where we’re able to take in a lot of their operational data, also some simulations of how their systems work, and be able to provide, say, recommendations around, you should now operate at this set point instead of that set point based on some of the other conditions that are changing in your plant. There’s a lot of interest in moving from having that be a recommended set point to have an automation that can automatically adjust the set point. And we’ve actually had some successful real-world pilots around that as well.
But when you get into that, obviously it makes people very nervous if you’re changing set points on industrial equipment. That’s where you maybe continue to have some guardrails around like, you can’t go outside of a certain band, for example, of operations, or we only allow the automation in certain areas. That’s where the human supervisor, the same way a human supervisor might give employees a lot of bandwidth or a lot of flexibility to make decisions around certain things, around other things, they’re hard and fast, like this is the deadline or this is the sort of production target. It’s like that. So thinking about, how do you put the appropriate guardrails?
The challenge is, AIs are not human beings, and so the guardrails look different, and I think that’s one of the things that there’s still to think about. That question of, what guardrails should be put in place? I think that it’s going to be probably dependent a little bit on the application, but over time I think we’re all going to learn, and so being really upfront and thoughtful about how we do this I think is important.
But the opportunity though, potentially, is really great. I’ve already mentioned that AI can be very useful in synthesizing a lot of information and bringing to the forefront, this is what matters. That’s something that we do want to create some space to experiment around, but what’s then important is making sure that a human being understands the risk.
I’ll give a little bit of a personal story because it might be illustrative. This weekend, I bought a little toy robot and decided I wanted to program it to do some stuff, and I found AI very, very helpful to get me started to read all the documentation. Putting the robot together was straightforward, they had nice instructions, but there’s a pretty heavy software developer kit that’s already there, but going through and reading all that documentation can take a while. AI was super helpful to help me surface, this is the function that does this, to help me get started. But occasionally, it made really bad recommendations on the best way to troubleshoot something.
That’s where I think having the human being in the loop, being able to say, “I know that it’s not going to be the best software architect or the best troubleshooter,” is really helpful because I had the opportunity to leverage AI for what it’s good for, synthesizing and surfacing a lot of information, but being able to say, “I don’t think that’s the most efficient first step. Let’s try something else.” That’s where it’s a really different way of working, but it’s something that I think as humans get more comfortable with the power and limitations of AI, you can start to teach people how to work with it, and it’s very different from how you would work with other digital tools.
One of the things that is important when it comes to AI is recognizing just the breadth of things that it touches and the breadth of impacts that it’s going to have, whether it’s on the environment, as we’ve discussed, whether it’s on productivity as sort of implicit in this entire discussion, whether it’s on labor force, whether it’s on just how economies work. From my eye, and I’m by no means a policy expert in this area, but from my eye, what I’m seeing is that different governments are prioritizing different aspects of that from how they’re thinking about regulatory and other kind of governance approaches.
Megan: We’re absolutely seeing some really vastly different approaches to this, aren’t we, around the world? I wondered to illustrate some of this, if you could share perhaps some case studies you’ve seen, maybe talking us through what lessons they could hold for other organizations perhaps a little earlier in their own AI journeys.
Arti: I think a lot of the key areas where we’re seeing very clear benefits of adopting AI in the industrial sector, the core to all of them is actually getting the data right and getting the right foundation of data so that you can build AI on top of it.
One of our customers, SCG Chemicals, which is a petrochemical company in Thailand, they had this vision of producing a reliability platform for their operations that was as AI-driven as possible. But a core part of that was actually getting the data right, being able to have all of their information in the right place, leveraging some of AVEVA’s tools to do that. Bringing together operational data and also engineering data. And then putting on top of that AI capabilities, in this case, one of our capabilities called AVEVA Predictive Analytics that deploys some proprietary models to do things like detect anomalies, so that they could really much earlier identify potential operational risks. And instead of having unplanned downtime, translate that to planned downtime. When you translate unplanned downtime to planned downtime, you can get a huge improvement in plant reliability.
At this point they’re targeting something like 99% plant reliability and a very, very high return on investment from the platform that they put in place. Again, in early pilot days, it was almost a 9x ROI, and so that’s really quite impressive. But what I want to emphasize is that it’s kind of a multi-layered problem to get it right.
Megan: Those are some really striking results, though. I mean, for industrial leaders who perhaps are still hesitant to explore AI, I wonder, what do you think is the cost of taking a more wait and see approach?
Arti: I think, again, this is an area where the industrial sector has a little bit of a different calculus to apply to this type of problem. In general, I think you would hear most business experts say that you can’t afford to take a wait and see approach to AI because it is so transformative across every sector of society and economy, and certainly in the industrial space, we’re not immune to that. But I do think that some of the challenges that I’ve outlined today also puts leaders in the industrial space into a mindset of sometimes potentially being a fast follower rather than the first adopter of newer technologies, just because the risk is so high.
But that being said, I think one of the challenges is that the risks that we’ve covered today around infusing systems that don’t always behave predictably with physical systems is that I don’t know that there’s a lot of other sectors that are going to be solving those problems. That’s where what I’m seeing is actually a lot of interest and excitement in trying new things and willingness to do that because there’s a recognition that applied correctly and applied with the right guardrails and safeguards in place, these technologies really have truly transformative potential from a resource usage standpoint, from a human safety standpoint, from a productivity standpoint.
I think the calculus has shifted a little bit in the sector to we want to actually try these things out. But then it becomes more, how do we try these things out in environments that we can make a little bit more sandboxed or safe to test out what some of the unique challenges we’re going to face in the industrial sector are?
Megan: Yeah, absolutely. It’s just hard to ignore the potential nowadays, isn’t it? And just to close with a slightly future forward look, I suppose, what are you most enthusiastic about in terms of the long-term potential of responsible autonomous industrial AI systems?
Arti: Yeah. Well, I think that there’s a couple of different trajectories. One of the things that I think within the next 18 months for sure, we’re going to see an increase of people with deep domain expertise who maybe didn’t grow up with a software background, be able to adopt AI assisted coding techniques to really be able to build the things that they weren’t able to build before.
That’s really exciting in my career, which way back when I was actually an industrial data scientist, I would say I always felt the most energized and that I learned the most talking to the people that were on the front lines of operations, having to monitor a lot of different equipment and understanding how it all worked together. They really have a lot of expertise, and being able to put in their hands the ability to very quickly develop new applications is I think going to really lead to a lot of new ideas that many of us in the industrial software space maybe wouldn’t even have thought about and really understood how to put. I think that’s really exciting.
I think then looking beyond that, I was a little bit of a, not to say a robotic skeptic, because clearly robotics and autonomous systems are going to be important, especially given the types of environments, whether they’re hazardous or remote, that industrial equipment operates in. But I just actually think things are moving a lot faster than I had anticipated. I’m really interested to see how these physical embodiments of AI systems start to really transform, again, how we think about operations.
One of the things about any new technology in any environment is that to really gain value from it, you have to change the way you do things. I would say for close to 10 years now, I’ve been giving talks on AI adoption and I always say the biggest barrier to AI adoption is not anything to do with the technology. It’s not even to do with the data, although data are often the biggest sticking point, it’s really to do with, are you going to change your business processes to be able to work with the way this technology is good or not good at things?
Autonomous systems, whether they’re robots or drones, are really going to change the way that we work in plants, in power systems, on mining sites. And I think that will make these types of operations more efficient, much safer for the human beings involved and more productive. So, I’m quite excited about that.
I don’t know entirely what direction it will go, but one of the things that I think about a lot is that if you go back to I, Robot and Isaac Asimov’s book, the premise of those was that robotics would be the first broadly adopted AI, not computer-based systems. We went in the other direction and I think it’s really interesting to now see all of this converging.
Megan: Yeah, absolutely. See robotics catches up a bit. So many exciting things on the horizon, that’s for sure. Thank you so much, for your time.
That was Arti Garg, chief technologist at AVEVA, whom I spoke with from Brighton in England.
That’s it for this episode of Business Lab. I’m your host, Megan Tatum. I’m a contributing editor at Insights, the custom publishing division of MIT Technology Review. We were founded in 1899 at the Massachusetts Institute of Technology, and you can find us in print, on the web, and at events each year around the world. For more information about us and the show, please check out our website at technologyreview.com.
This show is available wherever you get your podcasts. And if you enjoyed us, we hope you’ll take a moment to rate and review us. Business Lab is a production of MIT Technology Review, and this episode was produced by Giro Studios. Thank you so much for listening. Goodbye.
Learn more at aveva.com.
This content was produced by Insights, MIT Technology Review’s custom content arm, not its editorial staff. It was researched and written by humans, with any AI tools that may have been used limited to production processes under human oversight.
© Bennett Raglin/Getty Images
After pleading guilty, a 54-year-old North Carolina man, Michael Smith, was sentenced to 18 months in prison for using artificial intelligence to generate songs for a scheme that stole millions from music streaming platforms.
In a press release Tuesday, the US Department of Justice said that Smith is the first American criminally charged with AI-assisted streaming fraud.
First charged in 2024, Smith successfully ran the scheme for seven years without detection.
© Taylor Hill / Contributor | FilmMagic
Every piece of AI content from Google's Gemini models has a hidden SynthID label, which makes it very difficult to pass the content off as authentic. SynthID has also gotten buy-in from other major AI players, but actually checking for those invisible watermarks has been a convoluted process. It's going to be a little easier going forward, though. Google has announced that its SynthID detector now supports watermarks from all its partners, and it's available for anyone to use on a new dedicated website.
SynthID is an invisible signal encoded in the pixels of AI-generated images and videos, and in the waveform of AI audio. Google says Gemini alone has integrated SynthID with over 180 billion images and videos, plus 240,000 years of audio. Until now, the only way for those outside of Google's trusted tester program to check for SynthID was to ask Gemini. This is rather clunky, and the model's output could sometimes be confusing when you just wanted a simple "yes" or "no."
Now you can cut Gemini out of the loop. Starting today, you can go to the new SynthID.com website to check a piece of content for an AI watermark. A major limitation of Google's previous detector was that it only checked for Google's version of SynthID. Even though ChatGPT images, for example, have SynthID, they wouldn't be flagged in Google's detector. That's no longer a problem.
© Aurich Lawson
© Derek Abella
© Christie Hemm Klok for The New York Times
© Jeff Chiu/Associated Press
© Scott Olson/Getty Images
© Jade Gao/Agence France-Presse — Getty Images
© Scott Olson/Getty Images
© Jade Gao/Agence France-Presse — Getty Images
© Ahn Young-Joon/Associated Press
For all the data that AI systems continually amass and analyze, enterprise AI agents often suffer from a curious shortcoming: a lack of knowledge. More than data, knowledge is the understanding of what the data means in the context of individual organizations. AI agents need this understanding to reason about situations, make decisions, and ultimately take actions. Without sufficient knowledge, agents are prone to making flawed and unreliable decisions.
A lack of knowledge, our research finds, is a major reason agentic AI use cases never make it to production. Competitive pressure is making it urgent to address this. Organizations need to deploy and scale more of their agentic projects to capture the efficiency gains AI promises. Falling short risks wasting the investment already sunk into these projects, and it cedes ground to rivals already putting their agents to work more effectively.
The purpose of this report, which is based on a survey of 300 data, AI, and other technology executives, is threefold. First, it seeks to gauge organizations’ agentic knowledge capabilities (i.e., their ability to give AI agents a full contextual understanding of the data they ingest) across semantic knowledge, episodic memory, and procedural knowledge.
Second, the report probes the challenges organizations face in improving access to knowledge and ultimately to getting more agent use cases into production. Third, it explores the measures organizations are taking to overcome these challenges.
The key findings include the following:
Data and knowledge weaknesses consistently stall AI agent progress. On average, only around a third (34%) of organizations’ agentic AI projects make it into production. Even high-tech firms struggle with this. Legacy data systems, security and privacy concerns, and a lack of knowledge and context are the key points of failure.
Strong knowledge capabilities correlate with agent success. A small group of production leaders (organizations where on average 61% of agentic projects advance beyond pilot) have stronger knowledge capabilities than the rest, especially when it comes to semantics. This advantage tracks closely with their higher production rate.
Fragmented data hugely complicates knowledge access. Data fragmentation (the inadequate sharing of data across systems) was most commonly cited as a top challenge to expanding agents’ access to knowledge (cited by 55%). Production leaders, by contrast, are more likely to see security and privacy concerns as a major concern (cited by 72% of this group).
Most firms aim to strengthen the link between data and agents. Among steps that can yield higher quality agent decisions, executives expect the biggest impact to come from strengthening the structural foundation between the organization’s data and its AI agents. The experts we interviewed see a knowledge layer as a prime way to achieve this.
Investment priorities to boost knowledge range from pipelines to knowledge graphs. To expand agent access to knowledge, organizations will prioritize investments in retrieval technologies, like ingestion pipelines, AI-ready APIs, and retrieval-augmented generation (RAG); in AI evaluation agents; and in knowledge graphs.
This content was produced by Insights, MIT Technology Review’s custom content arm, not its editorial staff. It was researched and written by humans, with any AI tools that may have been used limited to production processes under human oversight.
In 2026, the question for enterprise AI is no longer whether predictive models can outperform statistical forecasts—that argument is settled. The big question now is how to enable predictive systems to act on their own conclusions without drifting from business intent. The frontier has moved from prediction to autonomous decision making, and the gap between leaders and laggards is widening accordingly.
“Enterprises are done with a backward-looking point of view; they want to be more forward-thinking,” says Vishal Gupta, partner at research firm Everest Group.
Intelligent analytics, powered by technologies like deep learning and generative AI, are making this possible. Real-time training allows AI to evolve continuously instead of waiting for quarterly refreshes. In addition, the data that newer predictive engines rely upon has expanded to encompass not just neat, numerical records but also messy, unstructured sources of insight-rich interactions. As a result, AI-powered analytics are moving enterprises from passive hindsight to pragmatic foresight.
AI takes predictive analytics—a broad discipline that includes predictive modeling, data prep, analysis workflows, interpretation of results, and decision-making applications—to new heights. “In many ways I think the word ‘analytics’ is giving way to AI,” says Gupta. “Everything is becoming AI.”
This content was produced by Insights, MIT Technology Review’s custom content arm, not its editorial staff. It was researched and written by humans, with any AI tools that may have been used limited to production processes under human oversight.
© Jiwoong Hong/The New York Times
Over the summer I talked to the CEO of Springboards, a startup building an LLM that’s designed to come up with a wider variety of responses than its mainstream rivals do. At the start of the call, he said something that’s been stuck in my head since: “We often say that we’re a self-loathing AI company. We don’t know if we really like what we’re doing.”
“I guess that makes me a self-loathing AI journalist,” I replied. It was a joke! I love my job. But I do not love many of the things the technology I write about has become: warped by hype, steered by zealots, inescapable.
The CEO of Springboards and I are far from alone. Around the world, a love/hate attitude toward AI is very much the vibe right now.
On the one hand, public feelings are souring fast. According to the Pew Research Center, more adults in the US think AI will have a negative impact on them personally, and on society in general, than expect a positive one. The pessimism is strongest among the young.
More than half of people worldwide say that AI products and services make them nervous, according to a report from Stanford University. In a Gallup poll in May, 71% of US adults said they would oppose construction of a new AI data center in their area, far more than the 53% who said they would oppose a new nuclear power plant. And in an NBC poll in March, AI was less popular than ICE.
Yet the number of people using AI is still skyrocketing. ChatGPT hit a billion monthly users in May, according to the market analysis firm Sensor Tower. Google DeepMind’s Gemini is close behind, logging 950 million users in July.
According to Pew, half of US adults now say they use a chatbot, more than twice the number who said they did in 2023. One in four people say they do so every day. And it’s a global habit: More than a third of adults across all 38 OECD countries (a group of the world’s wealthiest democracies) report having used generative AI tools in the last three months.
All these surveys paint a picture of two contradictory truths: A lot of people really do not like AI and a lot of people cannot get enough of it. What’s going on?
It’s possible that these are two distinct groups, but I doubt it. The numbers don’t add up unless the Venn diagram is on its way to being a circle.
It’s also possible that the more people use AI, the more negative their opinions about it will be. There’s at least a correlation to back this up: The Global North, where adoption is highest, skews pessimistic. In the Global South, where adoption levels are lower, people are more optimistic.
But here’s what I really think is going on: When people say they hate AI it’s not the technology they dislike but, rather, the relentless drive of the companies behind it to push the tech upon us in as many ways as possible. They’re telling us to brace for the biggest social and economic upheaval in generations. How do they expect people to feel?
In many ways, we’ve been here before. We saw a similar dynamic play out with social media over the last 20 years. Billions flocked to Facebook and Twitter despite mounting techlash against the companies that ran those services. Ditto for Google search.
With social media there wasn’t much anyone could do. Quitting a platform meant losing all your content and connections, giving you the choice to suck it up or start over. But with AI there’s still a chance for people to have some sway. There’s certainly more political appetite for regulation this time around. All 50 US states have either passed or proposed laws governing the development and deployment of AI, creating a patchwork of more than 2,100 bills across the country—a tenfold increase in three years.
And with a number of top-class open-source alternatives to Google and OpenAI and Anthropic already on the market, there’s also (for now, at least) the potential for more consumer choice and the market pressure that goes with it.
I don’t want to be naïve. Trillion-dollar companies are hard to move. But what comes next is not as inevitable as those companies often imply. When I asked the Springboards CEO why his firm was developing a new model if he didn’t really like AI, he told me there was no walking back from LLMs—but you could still make them do something different.
Here’s hoping we see a lot more of something different: Clarity about what AI can and cannot do and, crucially, a technology that’s not sold as if it’s about to take over the world.
Yossi Matias, Vice President & Head of Google Research, explores how AI is beginning to reshape biology, infrastructure, manufacturing, and science, and why its greatest impact may come when it intersects with other fields.
Step inside the newsroom with our MIT Technology Review editors for sharp analysis and unpublished insights from the team that researches and reports on emerging tech daily.
These are just two sessions from EmTech Future 2026.
Across three days, EmTech Future explored some of the most important questions shaping technology right now: how AI is impacting industries, where quantum is heading, what new energy systems could make possible, how robotics are evolving, and what challenges and opportunities arise when these technologies start to converge.
Hear the ideas, insights, and perspectives you missed at EmTech Future 2026.
The full program is now available on demand, giving you access to all the conversations and sessions from the event, including:
As an MIT Technology Review subscriber, you’ll get a 20% discount, meaning you can access the full EmTech Future 2026 program for just $596.
© Matt York/Associated Press
© Matthew Defeo for The New York Times
Enterprise AI is no longer a future ambition. It is in full operational flight. Model capabilities are advancing faster than most organizations can absorb, while the cost of performance continues to fall. Globally, AI investment is set to reach $2.5 trillion in 2026, up 44% from the previous year.
For many enterprises, this investment has produced fragmentation. Intelligence can accumulate in silos so that sales agents are unaware of open support tickets, for instance, or marketing systems are personalizing content without visibility into what finance already knows about a customer. Each function may perform well in isolation, but the enterprise as a whole learns little and has less information to act upon.
The shift from AI as a tool to AI as an operating model—what we call the “agentic shift” in this report—demands something more fundamental than better models or faster infrastructure. It requires connecting people, processes, and data in real time, along with the governance and control to act on that intelligence reliably.
This means rethinking both architecture and operating models simultaneously. First, rebuilding data infrastructure for accessibility rather than volume. Second, replacing fixed tech stacks with composable architectures that can evolve as models and tools change. And, lastly, resolving questions of AI sovereignty, including where intelligence runs, who controls it, and how it operates across organizational and jurisdictional boundaries.
Key findings include the following:
Enterprise AI’s scaling problem is structural. Process-first companies are pulling ahead. Global AI spending is rising sharply and model capabilities are advancing faster than most organizations can integrate them. Yet the majority of enterprises are still not growing revenue through AI or fundamentally rethinking how they operate. The companies generating sustained returns share a common discipline. They treat process redesign as the work that precedes model selection, building for how the technology will evolve rather than retrofitting roles and workflows after deployment. For them, the agentic shift begins with the operating model.
Data readiness, not data abundance, is what makes AI compoundable. Most enterprises discover too late that having data and having AI-ready data are very different things. A sovereign, composable foundation—one that queries and prepares data where it resides, without migration or centralization—can convert raw data estates into intelligence that AI agents can act upon. As data residency laws, multicloud environments, and structural complexity make centralization increasingly impractical, sovereign control over where models run and data lives is what keeps that adaptability intact.
This content was produced by Insights, MIT Technology Review’s custom content arm, not its editorial staff. It was researched and written by humans, with any AI tools that may have been used limited to production processes under human oversight.
The era of generative AI has upended technology supply chains, and there is one ironclad rule in 2026: If it has memory or storage, it's getting more expensive. Even devices with years-old tech inside are still apparently subject to that unwritten rule. Nvidia has just announced that the Shield TV Pro is getting $100 more expensive, effective immediately.
The most recent version of the Shield streaming box debuted in 2019, running Android TV with AI upscaling and hardware decoding for almost any type of media. While it ran an aging Tegra X1+ processor, that was (and still is) fast for a TV media streamer. The device launched at $199.99, with a non-Pro variant at $149.99. The non-Pro Shield has since been discontinued, but the Shield TV Pro lives on at the new $299.99 price.
Nvidia has confirmed that you can blame AI for the higher Shield TV pricing. "Starting October 2, SHIELD Pro will be priced at $299. The cost of components, including memory, has increased substantially across the industry," a spokesperson told Ars.
© Ryan Whitwam
Login