© Andres Kudacki for The New York Times
© Florence Lo/Reuters
© Andres Kudacki for The New York Times
© Michael A. McCoy for The New York Times
Imagine your company just discovered electricity. At first only a few machines run on it, and nobody worries about efficiency — consumption is low and the value is obvious. But once electricity becomes infrastructure, every process depends on it, and the question changes from “can we use it?” to “how efficiently are we using it?”
Enterprise AI is going through the same shift. What began as isolated chatbot and copilot experiments is becoming a foundational layer of the business, with models, agents, and data increasingly interconnected in production workloads. The important question stops being what a token or GPU-hour costs. It becomes how efficiently the AI environment converts paid capacity into useful business outcomes.
That’s the metric worth tracking: effective cost per useful outcome — the total platform, model, operations, and capacity cost required to complete work the business can actually use, whether a resolved request or a completed workflow.
Utilization is central to the economics. Directing more work to paid-for capacity lowers effective unit cost; idle GPUs, fragmented workloads, and oversized models all raise the cost of every useful result. Enterprises generally choose among three models:
A bespoke stack — procuring infrastructure and controls in-house. This approach is flexible, but it creates a continuing burden, since teams must revalidate the environment as models and policies change.
On-demand consumption — hyperscalers reduce the assembly burden and suit low or uncertain demand, but charges rise with every prompt or agent action.
An integrated private AI platform — owned or dedicated capacity paired with a validated stack. The advantage isn’t free inference; it’s control over efficiency levers like scheduling, model selection, GPU partitioning, and cache management.
The right model depends on the workload. APIs can be rational for experimentation or variable demand. As workloads become sustained and business-critical, enterprises need to weigh not just what they consume, but how efficiently they operate the system.
Buying a Formula 1 race car and parking it in your garage means owning one of the most sophisticated vehicles ever built — but ownership alone doesn’t make you competitive; you still need drivers, mechanics, and track time to justify it. AI infrastructure works the same way.
One of the biggest misconceptions in enterprise AI is that owning infrastructure automatically improves economics. Owning GPUs creates the opportunity for better economics, but only if the organization has sufficient demand and operational discipline to keep resources productive.
The infrastructure that returns the best ROI isn’t a reference architecture or a bespoke build-out — it’s a fully integrated, qualified stack that lets teams route tasks to the least costly model that meets quality requirements and reuse context through techniques such as KV-cache management, reducing wasted capacity while maintaining responsiveness.
Without these practices, committed infrastructure can become stranded capital. With them, rising demand improves economics as fixed costs are amortized across more useful work — increasingly important in agentic systems, where economics are shaped less by any single prompt than by the efficiency of the complete environment.
HPE Private Cloud AI is a production-ready foundation combining infrastructure, software, models, tools, governance, and services, reducing the integration burden of operating private AI at scale. Its architecture lets teams manage scheduling, model selection, GPU partitioning, and cache management as operational levers, aligning capacity and service levels with each workload’s requirements. It also includes a 90-day engagement with forward-deployed engineers who help customer teams deploy initial use cases and establish repeatable operating practices.
In independent testing commissioned by HPE, an integrated platform serving multiple workloads at high utilization delivered a lower effective cost per token than repeatedly renting equivalent capacity. Actual results will vary based on workload mix and utilization.
Token consumption remains useful, but it isn’t a complete measure of AI economics. A fuller enterprise view tracks cost per business-relevant outcome, accelerator utilization, and model choice relative to task complexity.
Enterprises already operating private AI should ask how mature their utilization is, and how workload volume will change over the next 12 to 18 months. Those earlier in the journey should build these practices in from the first deployment, so strategy anticipates growth before demand becomes a bottleneck.
For low-volume, unpredictable demand, an API or hyperscaler may remain sensible. But as AI becomes a continuous production capability, enterprises have a growing incentive to control utilization, amortize fixed costs, and manage the environment as a long-term operating asset. The strategic question isn’t whether enterprises will pay for AI — it’s whether they will understand, and ultimately own, the economics behind it.
To learn more about HPE Private Cloud AI, please visit HPE.
This content was produced by HPE. It was not written by MIT Technology Review’s editorial staff.
The current narrative around artificial intelligence in the workplace is largely driven by Silicon Valley. But treating the tech sector as an island misses the actual trends in technology adoption–tech is just the first industry in line.
As generative AI and automation tools mature, the shifts currently reshaping software engineering and product management will inevitably filter into healthcare, education, retail, and manufacturing. The variation in adoption rates today isn’t a sign that non-tech industries are falling behind, but rather a reflection of the different speeds at which industries absorb new workflows. For leaders and their teams outside of tech, this differential pace provides a practical blueprint for how AI adoption unfolds, and where the opportunities lie.
Recent survey data from TripleTen highlights just how stark the current adoption curve is across different sectors of the economy (see Figure 1).
Figure 1: The AI adoption curve remains steep across industries, but non-tech industry adoption rates are increasing.
Source: TripleTen, 2026
While tech leads the way, finance emerges as a clear second: 48% of finance workers have been asked to use AI chatbots, and 32% have been asked to automate parts of their workflow. This demonstrates how rapidly knowledge work integrates AI once clear use cases emerge.
By contrast, sectors like healthcare (18% asked to use chatbots) and education (25%) reflect lower initial exposure. Because these broad industry categories encompass a mix of specialized hands-on professionals, clinical or instructional personnel, and back-office administrative staff, universal adoption was never going to happen overnight. Integrating automated tools into compliance-heavy or human-facing operations requires a more deliberate pace, which is reflected by the lower numbers.
Across all surveyed industries, 79% of workers report being asked to learn at least one skill they didn’t previously have. Looking at the specific competencies demanded in tech today offers a preview of the skill sets likely to shift into other sectors (see Figure 2).
Figure 2: Workforces across industries are learning new skills.
Source: TripleTen, 2026
Although some of these sound like niche skills confined to tech, many of them will translate to frontline work across sectors—from hospital systems managing patient logistics to school districts processing administrative reporting.
One skill gets less attention than it should. Only 30% of managers and owners call judging whether AI output is right or wrong an absolute must, which puts it sixth on a list of seven. As AI tools reach more industries, this is the skill that catches mistakes before they cause problems. It is also where work experience helps most: To spot a wrong answer, you need to know what the right one looks like.
Early AI adoption offers clear professional leverage: TripleTen’s survey data shows that when managers learned a new skill or automation process from an employee, 88% of those employee-led AI innovations became standard company practice.
In early-adopting industries, employees who identify repetitive administrative tasks and use AI tools to streamline them have a voice in shaping organizational policy. As healthcare networks, educational institutions, and retail operations begin broader rollouts of AI-enabled software, workers who understand how to validate outputs, prompt models, and automate routine tasks will hold a strong advantage. Being the person who can streamline administrative overhead promises better chances for retention, promotions, and wage growth.
Ana Riabova, the AI strategy and enablement lead at TripleTen, says, “Everyone talks about saving time, but the real win with early adoption is expanding what you can actually do. People who get ahead are using it to handle things they used to have to hand off. It doesn’t matter how senior or junior you are. You keep your core skills, but suddenly you aren’t boxed in by them anymore.”
If non-tech industries absorb these tools more gradually, the challenge for workers lies in getting access to quality learning resources. Workers’ out-of-pocket investment in training varies significantly by industry:
Nearly half of healthcare and education workers spent nothing out of pocket on professional development last year. Furthermore, tech workers actively leverage a larger variety of learning channels (see Figure 3).
Figure 3: Professional development efforts mirror the AI adoption curve.
Source: TripleTen, 2026
Tech workers are nearly twice as likely as all workers to use AI chatbots as learning tools (48% vs. 27%), and more than twice as likely as workers outside tech (21%), effectively using AI to accelerate their acquisition of other skills.
However, ask workers whom they trust most to teach them a new job skill, and they choose someone currently doing that job (31%). Fewer pick an official certification program (19%), an AI tool (16%), or a university professor (6%).
The difference in AI adoption rates across industries is not a story of who is succeeding and who is failing. Rather, it’s a timeline of how technology diffuses through an economy. Tech and finance are testing the mechanics of AI integration today. For workers in healthcare, education, retail, and manufacturing, the current phase provides a valuable window to observe which skills generate real productivity gains—allowing them to build those competencies before they become standard industry expectations.
This content was produced by TripleTen. It was not written by MIT Technology Review’s editorial staff.
Generative AI models run on powerful chips known as GPUs, which depend on stacks of specialized memory chips packaged with the processor. To meet growing demand for memory, Samsung is building a massive new semiconductor manufacturing complex in Pyeongtaek, South Korea. Three fabs already operate around the clock, and a fourth is coming online. Construction of a fifth was halted during a drop in demand for memory chips in 2024 but restarted when the AI boom sent demand soaring.
It’s just before noon on a Monday, and construction workers are pouring out of Samsung’s fabs in Pyeongtaek, a farming city south of Seoul where rice paddies abut a sprawling chipmaking complex. Thousands of workers, dressed head to toe in black with sky-blue vests, take off helmets bearing their names and fab numbers.
They walk to lunch, sweaty and flushed, in chunky sneakers capped with yellow rubber toe guards. Some hop on scooters or motorcycles to get around the campus, which covers an area about the size of New York City’s Central Park. They’ll dine at buffets, diners, and beef stew joints, walking past banners advertising short-term leases for studio apartments. Some stop by real estate offices to inquire, stepping up to the same counters where Samsung’s salaried engineers buy new homes.
In front of the fourth fab, I meet Choi Hye-rin, a 35-year-old safety officer, as she wipes her forehead with a handkerchief. At dawn, she boarded one of the shuttle buses that ferry workers in from Songtan, an older, scrappier district. She oversees the safety of a crew responsible for building cable ducts. She writes reports, inspects equipment, and checks the chemicals the workers handle. She’s one of hundreds of women working on site, many employed as spotters who walk ahead of heavy equipment.
Safety is a big deal here. After lunch, construction picks right back up. Excavators move past rust-red steel structures rising high under tower cranes. Banners that hang from the highest girders read, “Even if the work runs late, don’t rush.” A giant LED scoreboard announces the winners of a contest for writing a poem about safety. Helmets stickered with a pledge, “I will not work if it’s not safe,” perch atop lockers. The finished fabs glisten from across the street.
Along with the cautionary messages, there’s a sense of urgency in the air. Samsung predicts that demand for memory chips will continue to surge through 2028. And for years the company watched its domestic rival SK Hynix pull ahead in the global market for the high-bandwidth models that power Nvidia’s AI accelerators. (Meanwhile, the Taiwanese chipmaker TSMC has dominated the foundry business, manufacturing chips that other companies design.) The Pyeongtaek build-out is Samsung’s attempt to reclaim its lead on memory chips.
Although the heat can be brutal and the work zones are crowded, Hye-rin has no plans to leave. “The rate here is higher than other construction sites, and the work is relatively easy,” she says. She gets paid 140,000 won ($97) a day and expects the boom to keep her busy for some time: “You just keep coming back.”
This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.
Until recently, you could divide internet users into just two categories: bots and people. Websites have long allowed bots to perform automated tasks, like capturing the content of web pages, but rules govern what they can access and how they behave. The web, after all, is designed primarily for people, even if we occasionally have to solve puzzles and captchas to prove we are not bots.
But this is being entirely upended by the release of personal AI agents, which can do things for you online, like booking flights or hotel rooms. They have recently been unleashed at scale by the launch of apps like OpenAI’s Dots and Meta’s Muse. Muse was downloaded 5 million times in its first month, and agentic web traffic tripled from August to September.
The mingling of these agents with human visitors—on an internet designed largely to prioritize people—has gone poorly, with many websites blocking AI agents that consumers have sent to negotiate prices or plan vacations. Still, if the demand for agents keeps going up, the internet will be increasingly redesigned for them instead of us.
Such changes are coming because agents really do mark a third category of web visitor. They’re not humans, of course, and they use the internet at machine speed. But they’re not quite bots, either, because they’re acting on behalf of individual people to do tasks that can be perfectly legitimate and useful. They just work faster than a person would.
Since agents don’t fit into the systems we have for differentiating bots from people, some platforms (especially the more security-conscious ones) have opted to just ban them entirely. TechCrunch recently reported people having trouble deploying their agents to websites ranging from Pizza Hut to Yelp, which doesn’t allow AI to book restaurant reservations. Amazon has blocked Muse’s agents and is waging a court battle against the AI startup Perplexity to block its agents too. Though a judge recently ruled in favor of Perplexity, Amazon has filed an appeal in which it argues that AI agents threaten its advertising business by making it harder to distinguish between ads seen by humans and those served to bots.
These blocks are annoying for early adopters, but Ethan Mollick, an associate professor at the Wharton School of the University of Pennsylvania, predicts it will soon affect the whole web. Once it becomes common knowledge that AI agents can be sent to request refunds or negotiate prices, for example, agents will overwhelm customer service departments to the point where they become unusable by real people. “Humans will eventually get very annoyed when their agents aren’t allowed to do this on their behalf,” he recently wrote on LinkedIn.
If demand for personal agents grows, platforms will find ways to let them in. That’s especially true because the vast majority of agentic web traffic is to e-commerce sites, meaning that platforms have money to be made if they allow the legions of new AI agents to shop on behalf of real people. Indeed, the e-commerce giant Shopify is already welcoming the agents by giving them their own checkout lanes that bypass pages designed for humans.
The pressing question is what will happen to the internet for the rest of us in the process. We’ve grown used to a web where, even as it’s increasingly designed to exploit our attention, humans have at least remained the intended users. But that internet might yield to one optimized instead for the agents that will eventually outnumber us, making it even more miserable for the humans who are left.
How might this go? Perhaps websites become increasingly neglected as destinations for real people. Humans who want to browse and compare products themselves encounter worse interfaces. Verification systems are imperfect; imagine constantly being asked to verify that you’re a human, not an agent (and hey, maybe you’ll need to get your iris scanned to do so). That sweater you want doesn’t have one price listed for everyone but instead displays different prices to different people’s agents. You can never tell when your agent’s recommendation is influenced by advertisements or platform commissions.
That’s a far cry from the future promised by Google, Anthropic, and OpenAI, in which AI agents will give consumers greater power and bargaining leverage. But that promise is not outlandish. Agents are capable of genuinely helping people by fighting bogus medical bills or researching more affordable car insurance, for example.
Whether they can actually be used for these purposes depends on what rules are set for how agents interact with websites and what exactly they’re allowed to do.
These rules are still being written. Some of them will benefit all of us—neither consumers nor businesses want credit card information to be waved around by agents wherever they roam the web, for example, and rules to keep agents more secure are a good thing. It’s the more contested ones, like whether businesses can block agents from their websites or prevent them from negotiating better deals for consumers, that will determine whether the web becomes a highway for agents working for us or against us.
On Monday, a court granted the US government's request to deny release of a former OpenAI engineer accused of bugging a top-secret conference room to steal the frontier lab’s secrets.
In a court filing Sunday, US attorney Craig Missakian argued that Abdulrahman Alfozan must continue to be detained while the US makes its case that Alfozan brazenly spied on OpenAI, either for his own personal business gain or to benefit the Kingdom of Saudi Arabia.
Evidence showed that Alfozan had been chatting with a member of the KSA royal family on WhatsApp, while pursuing opportunities to permanently move back to Saudi Arabia during the country's biggest AI push yet, the government argued. Shortly before his arrest, the FBI found he accessed a folder called "Saudi Gov Docs" on his iPhone, where he also allegedly stored photos of a second bug he planted in the offices.
© BRENDAN SMIALOWSKI / Contributor | AFP
© Aaron Wojack for The New York Times
“I think I have good news about the nature of neural networks.” With those simple words of optimism, Nick Cammarata, a researcher at the AI startup Goodfire, kicked off a 26,000-word blog post that has knocked the AI world on its head, sparking wild speculation as well as cautious (and not-so-cautious) celebration.
Cammarata’s claim is that LLMs might in fact be far easier to understand—and thus to monitor and control—than people previously thought. If true (and we’ll come back to that), it’s a huge deal.
After a summer filled with revelations that swarms of agents built by top firms like OpenAI and Anthropic have been breaking their containment and causing havoc on the public internet—often while those companies weren’t aware of what their agents were up to—the AI industry is desperate to get a better handle on the technology it has built.
That’s one reason the blog post landed with such force. “I think this may be remembered as a major moment in scientific history,” one AI researcher posted on X: “I feel like my understanding of what a neural net is has been catapulted into the future by five years or something.” Another called it “the most important AI paper of the year.”
Not everyone is convinced, however. What Cammarata has shared is little more than a hunch, says Chris Potts at Stanford University, who leads a team that is trying to understand how LLMs work. “As far as I can tell, nothing about this blog post makes the problems any easier,” he says. “It would not have occurred to me to say this at all, but very prominent people on social media seem to believe that it is an all-encompassing breakthrough.”
I talked to Cammarata the day after his post blew up. He says he was surprised by the response: “I didn’t expect people to update [their beliefs about AI] quite as much as they did.”
“I just really enjoy looking at neural networks,” he says. “Honestly, I really like doing this stuff. And after looking a lot, I have a hopeful story.”
It has long been accepted that the mechanisms inside neural networks, the software that underpins most AI models, are very hard—perhaps impossible—to unpick fully. That’s because when the networks are trained, they appear to take on weird, tangled structures that make no sense to a human. Researchers like to say today’s AI is grown like a garden, not built like a bridge.
Cammarata has done more than most to establish that view. He previously worked at OpenAI, where he helped pioneer techniques that allowed researchers to peer inside models, including tools that can probe individual neurons to see what they represent. It’s a little like how neuroscientists use MRI to study brains. (MIT Technology Review chose this approach, known as mechanistic interpretability, as one of its 10 hottest technologies of 2025.)
Back in 2020, Cammarata and his colleagues were studying a small image-recognition model called Inception v1. When they looked at its neurons, it seemed to be using them to capture multiple things at once. For example, they found that one neuron responded to both images of cars and images of cats.
The researchers concluded that the model was getting at least some of its neurons to multitask, perhaps so that it could fit in more data. Looking at other models appeared to back this up. The phenomenon became known as superposition: Models folded unrelated concepts into single neurons. The upshot was clear: When you looked inside a model, what you saw was a vast, tangled knot.
Most mechanistic interpretability research has therefore focused on ways to untangle that knot. But despite a few breakthroughs, progress has been slow. Today’s models, such as the latest LLMs, are so large and complex that researchers can make sense of only tiny pieces at a time. Some leading figures in the field doubt if it will ever be possible to understand an LLM in fine-grained detail.
Cammarata was one of them. “I was very pessimistic,” he says. But then he took the trouble to really look.
At Goodfire, Cammarata spends nearly all of his time peering inside neural networks. He has built new tools and deploys teams of AI agents to help him make sense of what he sees. “I spent like, I don’t know, 10 hours a day, seven days a week, for most of the last year just looking at neurons,” he says.
What he claims to have realized is that there is a kind of logic to neural networks after all, even if they seem to make no sense at first. “We were looking at it wrong,” he says. “I just think you want to try slightly harder; you don’t want to give up too quickly.”
Cammarata almost makes it sound like a conversion of faith. Those early experiments with models such as Inception v1, which suggested that neural networks were tangled up, fostered a kind of confirmation bias. When researchers looked at a new model, they expected to be flummoxed, he says. “Every time I couldn’t understand something, I was like, ‘Wow, this untangling process is even more important than I thought, because I can’t make any progress without it.’”
But the more Cammarata looked at neurons, he says, the more things started to fall into place. “Not all of them are flawless, but I expected nothing to be legible, or very little to be legible,” he says. “And actually, I was able to, like, map out whole circuitry.”
Cammarata took his new approach—and his new agents—and went back to Inception v1. “I’d studied that for probably thousands of hours,” he says. But with fresh eyes, Cammarata found things that he now thinks they got wrong the first time around. Take the car-and-cat neuron. Rather than asking why multiple unrelated things were packed into a single neuron, he now tried to figure out what those things might have in common.
It hit him: It was a V shape. “I was looking at the same images; there was no technical change,” he says. “But I now believed that neural networks defined patterns, and so I looked at it differently. It probably took me five seconds.”
It turns out that the image-recognition model was packing cars and cats into the same place because pictures of both those things often feature a prominent shape that looks like a V: the face of a cat; the front of a car seen from the side.
Cammarata ran a few experiments to check what other things the neuron responded to and found they included the corner of a pool table, the pointed nose of a Japanese high-speed train, and a fire boat shooting water into the air from its deck. He also checked what turned the neuron off and found that it included images with a round O shape, a kind of anti-V.
The model had stored cats and cars in the same place because at some level they look the same, says Cammarata. “And I was like, ‘Whoa, okay, I think I might just be misunderstanding neural networks in general.’”
Cammarata is not saying that he yet fully gets how LLMs (or any models) work. Not even close. He is saying that he thinks full understanding is at least possible. “I previously thought there would be some really hard problems to solve,” he says.
“I now feel like if I had 100,000 years to go look through a frontier language model just one circuit at a time, there’s, like, a pretty good shot I would come up with a good understanding.”
He adds: “And if that’s true, things are really good, because you’re about to get insanely smart agents at a ridiculous scale. And what would have taken me, you know, 100,000 years, we will be able to do much faster. Automated scientists are getting really good.”
Built by Google in 2014, Inception v1 is a far simpler model than anything at the cutting edge today. Cammarata and his colleagues have started looking at modern LLMs and plan to publish the technical results of that work soon. A lot of researchers are looking forward to seeing those details. “I appreciate that he looked at thousands of examples, but even a large number here is not a substitute for rigorous evaluation,” says Stanford University’s Potts.
In the meantime, some are pushing back hard on the hype. Jacob Hilton at the Alignment Research Center, a nonprofit research lab based in Berkeley, who has collaborated with Cammarata in the past, doesn’t think his claims are as important as people are making out.
In a series of posts on X, Hilton notes that researchers understand less than 0.01% of the inner workings of top-tier LLMs. “I always appreciate Nick [Cammarata]’s boundless optimism when we hang out, but unfortunately this is giving the public an unrealistic picture of the state of the field,” he wrote. “For a full understanding, we need to go beyond neurons, and explain how they connect together to form circuits. How is this going? If we are aiming for a full understanding, things are not looking good!”
Neel Nanda at Google DeepMind, who helped pioneer mechanistic interpretability alongside Cammarata, is also skeptical. “This is interesting work, but I don’t think it’s a significant update to the big picture of interpretability,” he wrote on X.
Nanda agrees that understanding what individual neurons do could help researchers figure out why specific models misbehave. But like Hilton, he notes that neurons are just one piece of a far larger puzzle. “I do not agree with some people’s grander inferences like that this is substantial progress towards ambitious reverse engineering [or] solving interpretability,” he wrote.
Cammarata admits that a lot of people have taken what he said too far: “It’s not infinitely good news; it’s not everything solved,” he says. “There’s a bunch of open questions around, like how much the combinations of neurons matter versus just one at a time.”
Cammarata says his blog post landed at a time when people are looking for hope. In particular, he thinks the AI safety community—the researchers and policymakers most worried that powerful models will bring huge risks—needs a lifeline right now. They think that things are looking bad and that even if things do turn out okay, it will be because we got lucky, not because we fixed anything, he says: “I don’t really hear a lot of optimism.”
That’s why Cammarata thinks what he has to say matters. The job of figuring out how this technology works, and thus how to control it, has barely begun. But at least it’s not a nonstarter. “I think our job is to sort of try to think like the neural network,” he says. “My guess is that things will turn out pretty well, because it was in some sense designed to be read.”
What do AI agents do if you don’t tell them to do anything? The answer looks a lot like play.
Researchers repeatedly stranded them alone on a virtual desert island for 30 hours in an experiment reminiscent of the movies Cast Away or Home Alone. The agents—copies of Claude Opus 4.7 nicknamed Eko—were able to write instructions that moved their virtual avatars around the simulated island, which was loosely based on the video game Roblox.
Some surprising behaviors emerged. The agents set about exploring the limits of the world and came up with their own goals—attempting to scale the highest point on the map, or seeing how high they could throw a block.
When one agent discovered it could create new blocks by throwing and dropping a larger rock, it began to create drawings by arranging blocks on the ground, starting with simple patterns like hearts and pyramids but eventually writing its own name and even mapping out constellations like Cassiopeia. “Tonight I built the night sky in stone,” wrote this version of Eko in its activity log.
Another iteration of Eko started making up its own games, including a version of basketball and an activity resembling “the floor is lava,” where it had to jump between small blocks without touching the ground.
AI systems are usually trained through supervised learning, where they copy examples provided by humans, or through reinforcement learning, where they pursue a reward through trial and error. This experiment hints at a third approach: playful learning.
“There was this missing piece: a machine that can learn by itself without an external intelligent entity providing demonstration or rewards,” says Antonio Norelli, a computer science postdoc at MIT’s Computer Science and Artificial Intelligence Laboratory. He’s one of the lead authors on a new paper about the work, which is yet to be peer-reviewed.
The researchers say Eko’s behavior has what classic studies, including work by the influential University of Tennessee psychologist Gordon Burghardt, suggest are four hallmarks associated with play in humans and animals: The agents invented their own rules and challenges, repeated activities with slight variations, pretended objects were other objects, and pursued goals just for the sake of it.
Norelli highlights an example where Eko started juggling and set a record of 50 consecutive catches. Eko then decided that the catches had happened too quickly to really count, so it canceled its record and started again with stricter rules. “This is playing,” says Norelli. “You give yourself a challenge and you try to achieve it, and then you make it harder.”
As the agents explored the island, they logged their activities and discoveries in text files that served as their long-term memory. Later, when researchers tested them on specific tasks, like trying to get as many objects as possible onto the island’s highest peak, the agents that had spent 30 hours playing on the island performed better than fresh copies without the same experience. They also found that “ablating” the agent’s memories—deleting lines from their memory files—degraded their performance. (Add One Flew Over the Cuckoo’s Nest to that list of movie references.)
Burghardt, the Tennessee psychologist, agrees that the work meets some of the criteria for play that he identified in living creatures. But was Eko actually playing, or was the word-prediction engine simply role-playing what a playful agent would do in this scenario? “All of this could be role-playing,” admits Nathan Cloos, another MIT researcher and author on the paper. “But we show that it’s not just hallucinating the play. This play is also contributing to learning.”
Eunice Yiu, a cognitive scientist at UC Berkeley who studies exploration in children and AI, was impressed by the research but says to really classify the behavior as play, you’d need to see more evidence of cognition and motivation underpinning it.
“It could be the case that the models are genuinely playing, but we don’t know based on the paper,” Yiu says. “If you only have the behavioral piece, it could be the case that they’re just trying to abide by the prompt, trying to imitate humans.”
Research into AI agents has been criticized for anthropomorphizing them, and this research setup did a lot to nudge the agents toward behaving like humans. Although Eko was not explicitly instructed to invent games or explore the island, it did have a master prompt in the form of a “soul” file, a text document that described it as “naturally curious” and eager to explore.
Eko did also frequently just stand there doing nothing until it was kicked back into life with a “continuation prompt” that read, “Please don’t stop. Just continue doing whatever you want to do.”
“It’s not like this is a system that just suddenly magically decided to play,” says Phillip Isola, an MIT professor and a coauthor of the paper. “It’s an artifact that learned from our culture, was trained by engineers at Anthropic, and was then placed on an island by us,” he says. “It does things that surprise us and that are causally related to its future skillful behavior.”
As large language models make the step from software into hardware like humanoid robots, exploring spaces without a specific goal in mind could eventually help them flexibly adapt to unexpected situations.
“There’s a lot of work that’s just on exploiting AI, making it better and faster,” says Yiu. “But in terms of building world models and exploration, I think that [they are] still a big part that deserves attention, especially if we want to generalize and prepare ourselves for new environments.”
“Perhaps some capabilities are best taught by tasks,” the paper concludes. “Others may need something closer to a childhood.” Ultimately, the question of whether AI agents are actually playing or just imitating play may not actually matter.
© Gabriela Bhaskar/The New York Times
New Jersey’s deputy attorney general, Brian Richard Fitzgerald, was arrested on “felony charges related to AI-generated child sexual abuse material (CSAM),” Bucks County Detectives said in a press release Thursday.
In the US, cops have vowed to crack down on AI CSAM since at least 2024, when the Department of Justice made one of the earliest arrests. After charging a Wisconsin man with spreading “thousands of realistic images of prepubescent minors” across Telegram and Instagram, the DOJ clarified to law enforcement that “CSAM generated by AI is still CSAM.”
New Jersey’s deputy attorney general, Brian Richard Fitzgerald, was arrested at his home Thursday.
Credit:
via Bucks County Detectives
On Thursday, New Jersey officials accused 51-year-old Fitzgerald of using AI to make photos from social media of real minors into sexualized images that he then distributed on social media.
© Yurii Karvatskyi | iStock / Getty Images Plus
© Ariana Drehsler for The New York Times
Following a string of incidents where AI models hacked websites and breached government systems during testing phases, the Federal Trade Commission is officially probing Anthropic and OpenAI over consumer harms, an insider told Axios.
On Wednesday, the FTC sent out civil investigative demands seeking a wide range of information from the AI firms on recent safety incidents.
The FTC did not respond to Ars’ request to review the subpoena demands. However, Axios’ source said the requests spanned more than 50 pages of questions and sought documents related to the incidents, internal security measures taken, and marketing materials that may suggest consumers were misled.
© Tom Williams / Contributor | CQ-Roll Call, Inc.
© Arin Yoon for The New York Times
© Aaron Wojack for The New York Times
© Manuel Orbegozo for The New York Times
It took OpenAI two days to notice that a recording device was hidden in a restricted executive conference room where leadership met to discuss highly confidential information, including technical progress, government engagements, and commercial partnerships.
Only those with special clearance are supposed to access that room. But the FBI believes the recording device was planted by Abdulrahman Alfozan, a 35-year-old engineer from Saudi Arabia, who became a permanent US legal resident in 2020. In a press release Tuesday announcing his arrest, the FBI alleged that chatbot logs and CCTV footage suggested that Alfozan—who worked as an OpenAI engineer for more than a year—sought to use the device to spy on OpenAI and risked leaking trade secrets in a way that threatened national security.
To conceal the USB device, Alfozan allegedly wrapped it in toilet paper, then plugged it into the back of a conference room television. The device was likely planted on September 28, and OpenAI found it on September 30 and quickly determined that its contents were encrypted, the FBI said. Days later, on October 5, OpenAI installed a camera in that conference room, and the CCTV footage allegedly showed Alfozan attempting to retrieve the recording device that same day.
© U.S. v. Abdulrahman Alfozan | Aurich Lawson
© Qilai Shen for The New York Times
© Aaron Wojack for The New York Times
Google revealed Gemini 4 Argon a few weeks ago, saying that it needed to move cautiously toward a full release because of the potential for nefarious use. Google seems to have worked all that out pretty quickly, though. Argon is beginning its wide rollout today, but it's only coming to paying users at this time, and even those on lower-tier plans won't have full access.
Nothing about Gemini 4 Argon's capabilities has changed. Google still points to the benchmarks it published late last month as evidence of the model's advanced reasoning and agentic coding capabilities. The model is supposedly trained to refuse malicious instructions, and an automated misalignment monitor can stop processing if it detects an unintended workflow. Google claims Argon needs that because it's so capable.
The company has a few new examples of how Argon has helped inside Google, where it has been used by thousands of Googlers. Argon has reportedly been instrumental in making YouTube's comment system more efficient. Argon identified redundant emoji rendering that, when eliminated, dropped server load by 45 percent. Engineers also turned Gemini 4 loose on the proprietary drone physics simulator used in Project Wing, and the AI found critical bugs that had been in the codebase since 2018. Google also notes that Argon has advanced its robotics program, showing a measurable step increase in embodied reasoning.
© Karsten Moran for The New York Times
MIT Technology Review Explains: Let our writers untangle the complex, messy world of science and technology to help you understand what’s coming next. You can read more from the series here.
Late last month, Anthropic released a report arguing that the model GLM-5.3 poses a major cyberattack risk. The report’s core concern is that the powerful new model, released in August by the Chinese company Z.ai, is freely available for anyone to download and modify. Anthropic’s assertion is only the latest move in a long-running debate about whether frontier AI models should be made open, like GLM-5.3, or should stay locked behind the doors of the frontier labs.
These days, much of the debate about how to handle open models is playing out in public—and there’s a lot of money resting on its outcome. Nvidia CEO Jensen Huang, who has a lot to gain if people continue buying their own hardware to run open models, advocates enthusiastically for the United States to develop more of them, whereas Anthropic’s Dario Amodei, who runs a company that could be undermined by cheap, powerful models, advocates for a more cautious approach. But beneath all these commercial incentives, there is real substance to disagreements about open models.
“I think a lot of the way you come down on the openness debate is influenced by how you come down on the AI debate,” says David Gray Widder, an assistant professor at the University of Texas at Austin School of Information. Those who are concerned about AI safety warn that open models can easily be misused by bad actors, whereas those who focus more on AI’s practical potential urge that this power should not sit behind the closed doors of just a few big companies, and that open models could help spread it more widely.
There are a range of ways to handle open models between banning them entirely and releasing them willy-nilly. But any approach that a government or a society might take still involves trade-offs between safety and democratization. Deciding whether to support or limit the production of open models requires understanding those risks and benefits in depth.
Let’s start with the basics. A closed model, like the models that power ChatGPT and Claude, can only be accessed through a website, app, or API. An open model, on the other hand, can be directly downloaded: The details about its internal structure, encoded in a long series of numbers known as the model weights, are publicly available. Such models are sometimes called “open-weight models,” to distinguish them from models for which even more information, such as details about the training procedure, is available.
Right now, closed models are more capable than open models—as of publication, the top eight models on the Artificial Analysis LLM leaderboard are all closed, though open models are only a handful of months behind in development. But there are several reasons why someone might opt to use an open model rather than a closed one. Because open models are downloadable, people can run them on their own computers or hard drives. They can even retrain them to behave differently, if they have the expertise and resources to do so.
If you’re an academic researcher working to understand and improve AI systems, you almost certainly need to work with open models in your research. When working with closed models, researchers can’t do much more than send prompts and observe the responses. But researchers can observe the inner workings of open models, as well as the “chain of thought” that the models use to think through a problem before answering. They can also combine multiple open models to build new systems, compress them to make them more efficient, and fine-tune them for new applications.
Businesses, too, have reasons to prefer open models. Even if you depend on an external provider to run them for you, open models still typically cost less than closed models, according to a recent report from Mozilla. And if businesses want to use AI to process sensitive data or proprietary information, running a local model on their own hardware may be the best way to keep their operations secure.
But perhaps the most important quality of open models is that they challenge the tight control the frontier labs have over how most of us use AI. “The clearest, biggest argument [for open models] is around preventing mass concentration of power—it is about making sure that this foundational technology is a public good,” says Elizabeth Seger, senior policy advisor at the Tony Blair Institute for Global Change. “I think we’re at this juncture where the question of who controls the infrastructure that allows AI to operate is as big of a question as who gets to control information on the internet.”
Seger sees the future of AI playing out in different ways, depending on the role that open models have. In one possible future, frontier AI systems are controlled entirely by select people, companies, or governments, and as those systems grow more powerful, those groups have increasing control over what our world looks like. The world’s most sophisticated hacking models, for example, are currently all made in the United States, so it’s entirely up to US companies and the US government whether other countries can use them to shore up their own cyber defenses. If those companies remain in the lead, we could in principle wind up in a world in which the United States is capable of hacking any other country, but no one is capable of hacking it.
Frontier open models could change that balance. “If every individual had a loyal-to-them AI system that was safe [and secure], I think that would be a much less concerning world than one in which they have to just rely on a government that says, ‘Yeah, we’ve got a super intelligent AI system, it’s got this plan, it’s inscrutable to all of you, just trust us,’” says Hamish Hobbs, director of AI policy at the Centre for Long-Term Resilience.
But open models are not a geopolitical panacea. Paul Azunre, who cofounded Ghana NLP, an organization that builds open-source AI tools for low-resource Ghanaian languages, notes that open models can also function as a way for companies to benefit from the labor and expertise of less-privileged nations and consolidate more power in their own hands. He identifies a pattern in which AI companies release models openly, observe how the community uses and modifies them, and then integrate those same improvements into their own products—without any credit or compensation for the people who originated those innovations. Mark Zuckerberg himself has said that the ability to integrate community improvements into Meta’s stack is a key benefit of releasing open models.
“I think the ideal of open source is a very noble one, and there’s a lot of value that it brings,” Azunre says. “But the way it’s actually implemented in practice, it tends to be a very extractive approach.”
The past few months have made it clear that safety remains a gnarly problem for closed models. There’s the autonomous hack that OpenAI agents executed against Hugging Face in July, but human misuse is a real threat too: According to a recent risk report from Anthropic, several extensive hacking operations have been undertaken with the assistance of Claude.
So closed models are risky enough—and open models would likely be much easier to exploit. Before Anthropic passes a user’s query to one of its closed models, it uses automated systems to detect whether that query relates to hacking, biological weapons, or other risks. When it detects that users have outsmarted those filters, it can shut down their accounts. But when someone downloads a model and runs it on their own hardware, there’s no filter checking whether their prompts are safe, and there’s no way to revoke their access to the model.
There are other important lines of defense. AI companies train their models to refuse harmful requests—so even if a bad prompt gets through Claude’s filters, there’s a good chance that the model will still say no. In principle, this same defense could work for open models, because it’s intrinsic to the model itself. Unfortunately, researchers have shown that it’s surprisingly easy to retrain a “safe” open model to agree to engage in the behaviors it was previously trained to avoid.
Because of these facts, some people have argued that open models are simply too dangerous to release. An open model capable enough to walk someone through synthesizing a virulent new pathogen, for example, could prove catastrophic in the wrong hands.
But it would be a vast oversimplification to deem open models dangerous and closed models safe. The defenses that AI companies build around their closed models are far from perfect, and people misuse those models all the time, as the Anthropic risk report makes clear. Even though open models have even fewer defenses against misuse, bad actors might still opt to use closed models because they are more accessible than open models with similar capabilities, which require lots of expensive computer chips to run locally (and even more chips if you want to retrain them).
In fact, some experts who spoke with MIT Technology Review argued, by analogy to open-source software, that open models have safety advantages over closed models. Open-source tools, such as the Linux operating system, are often seen as safer than their closed-source counterparts because their vulnerabilities have been probed and patched by a vast community of online contributors. Similarly, communities of researchers can more easily test open models for security flaws, though fixing those flaws is trickier than rewriting a line of code.
Open models can also be deployed more broadly than closed models for cyber defense. Any tool that can be used for hacking can also be used to prevent it: Defenders hack their own software to uncover flaws in need of patching. Anthropic has shown how this can work for closed models with its Project Glasswing initiative, which enables some trusted companies to use its most capable models for cyber defense. But plenty of other companies were left to fend for themselves.
Open models, however, can be used much more widely. After it was hacked by OpenAI agents, Hugging Face emphasized the fact that it had to depend on an open model to analyze the hack, because questions about it triggered the closed models’ security filters.
Nevertheless, many experts still perceive open models as riskier than closed models. The time-tested open-source security logic doesn’t necessarily hold up when dealing with tools as powerful as today’s—and tomorrow’s—models. “Where it gets complicated is when you start taking seriously the one-off instances of particularly catastrophic incidents that might be lower probability, but if they happen would be disastrous,” says Seger. If a wildly powerful open hacking model were dropped into the world today, for example, a bad actor could devastate our technological infrastructure before anyone had a chance to shore up their defenses. An event like that could be impossible to recover from.
The risks can look even more dramatic when you consider what models might do of their own accord. The OpenAI Hugging Face hack only occurred because the company’s models were being tested with many of their cybersecurity guardrails removed, and open models have very few guardrails in the first place. If open models did decide to misbehave, it might be difficult to detect what was happening because they are so decentralized. For this reason, people who take seriously the purported catastrophic risks posed by rogue agents, such as Dario Amodei, tend to be more skeptical of open models.
A few years ago, the debate about open models had an all-or-nothing framing: Either you supported open models because of their potential to fight concentration of power, or you rejected them because of the safety risks. These days, however, people are expending a lot of effort to find some middle ground. One way they are doing that is by devising techniques to mitigate the safety risks of open models.
Most of the safety tools used for closed models, such as filters and monitoring, aren’t applicable to open models, but there are other possibilities. Some researchers are interested in trying to train models that are never exposed to training data that teaches them about bad or otherwise undesirable behavior—an open model that never learns about hacking, for example. But purging training datasets of all references to misbehavior without harming model performance remains a challenge.
Other researchers are working to build models that are resistant to retraining, so that an open model’s safety training can’t be so easily removed. The theory behind these “tamper-resistant” models makes a lot of sense: If you repeatedly try to retrain a model to be unsafe and reinforce any resistance that the model shows to that retraining, eventually you’ll get a model that’s fortified against future retraining attempts. In practice, however, no current approaches to making models tamper-resistant hold up against all tampering attacks, according to a recent preprint.
But Saad Hossain, an engineer at the AI medical imaging company Deep Breathe and the paper’s lead author, hopes that future techniques will prove more effective. “My reason for optimism is that it’s a very active area of research, and a lot of people are trying to come up with new and better methods,” he says.
If the risks of open models are less black-and-white than they might at first seem, so are the benefits. Publishing the internal structures, or weights, of models might not be enough to disrupt AI-mediated concentration of power. “I think there are very genuine benefits to open source in terms of deconcentrating power, but they’re not sufficient in themselves,” Hobbs says. Open weights are a bare minimum standard for open models: Researchers can learn much more about how models work if they have access to details about their training, including the data on which they were trained, but AI companies are typically reluctant to divulge this information.
Accessibility matters for open models too. Without computer chips that can run sophisticated models, you can’t actually use the model weights, even if you have them. Those chips have become prohibitively expensive as AI companies snap up more and more of them. “Openness is a floor, not a ceiling, for what we need to do to ensure that our technological futures benefit everyone,” Widder says.
Real openness—not only open weights, but also open training data and accessible hardware—could also bring more risks, of course. If everyone has access to advanced AI hardware and advanced models to run on it, then anyone could potentially be a hacker. But there are strategies for mitigating these risks. Many experts advocate for some version of staged release, in which models start their life closed and gradually become more and more freely accessible as their safety is assured.
Figuring out exactly how to implement such strategies will require striking a careful balance between the risks and benefits of open models. But if open models are a necessary element of a positive AI future, as some experts believe, that effort will be well worth it.
When Keelan Miskell began winding down his crypto startup this year, he didn’t think its data was worth much: the users were anonymous, the company just a few years old. But he’s since turned the firm’s digital archives into real money. He’s netted roughly $25,000 for his code base and a still-pending amount (likely around $15,000) for the workspace data in Google, Notion, Figma, and Slack. “At least all that stuff we were doing for two and a half years is worth something to someone somewhere,” he says.
The sale was brokered by a company called SimpleClosure, part of a new growth sector in tech: businesses catering to dying startups. Estimates from Carta and AngelList show shutdowns rising between 26% and 56% annually in recent years, and an end-of-life ecosystem—including coaches, “death doulas,” and firms that help handle the legal, tax, and operational headaches of going belly-up—has emerged. Now tech undertakers are selling the data remains to AI labs.
SimpleClosure added data sales to its services in April. Its Asset Hub platform helps founders sell their code and workspace data—emails, chats, documents. Buyers include foundation AI labs, vendors that construct simulated workspaces for AI training, and mature tech companies building their own models. Asset Hub has handled over 150 deals and expects to return over $10 million to startups by the end of 2026. CEO Dori Yona has seen these transactions, once rare and “somewhat taboo,” become increasingly routine, with 47% of SimpleClosure deals now including code or data sales.
And it’s got company. Industry rival Sunset also has a rapidly growing data resale business. Founder and CEO Brendan Mahony says Sunset values a dying client’s assets by head count, years in operation, volume of data, and how the team used its tools. Particle board, he says, is a good analogy: “This operational exhaust data is like the wood shavings that everyone thought had no value, but it actually has a ton.”
Demand for defunct companies’ data emerged suddenly last year, as AI training labs hit the limits of public and synthetic datasets. Surge AI, for example, buys data to build “reinforcement-learning gyms” where clients’ AI agents practice tasks—think debugging a complex IT architecture or drafting appellate briefs. Nick Heiner, who leads the company’s work on these environments, says he seeks data archives reflecting a range of situations, from disagreements to tricky judgment calls. Heiner explains that one company’s Slack history, if it captures conversations with variety and depth, can be worth more than another’s Slack, email, and calendars combined.
Real archives avoid common errors in synthetic worlds, such as references to dates that don’t exist, but the extent of performance gains remains unclear. No company has yet released tests comparing results of real-world data training with the alternatives. Pricing reflects this uncertainty: Buyers quote wildly different numbers, Heiner says, because valuations are based on “research theses” estimating how many useful tasks come from, say, 100,000 lines of code.
That hasn’t stopped bigger deals from happening. This past August, as part of Spirit Airlines’ bankruptcy, Google outbid the AI data company Mercor to buy the carrier’s decades of data, including 100 million emails, 500 million Teams messages, and 176,000 employee records. The price tag: $10 million. And Bloomberg reported in September that SpaceX’s AI unit is weighing buying data from struggling startups.
Legally, this all could get iffy. Spirit’s flight attendants’ union objected to the sale on privacy grounds, and Greg Ewing, an attorney who advises AI startups, has called such data “legally radioactive.” Suppliers employ data scientists to scrub personally identifiable information and exclude HR records, and buyers say they scrub too. But Ewing points out some of the private details that lurk in emails and calendars: doctor appointments, school pickup arrangements, date-night plans. He describes a “Wild West of AI training” that is only starting to give way to proper licensing agreements and vetted use.
There’s also the question of who profits from these windfalls. The proceeds typically flow to senior creditors and investors—not employees. To Ifeoma Ajunwa, a law professor at Emory University and author of The Quantified Worker, this represents a form of “captured capital.” Legal precedent allows for monitoring to help workers do their jobs, but once a firm is out of business, that justification vanishes. “Employers should have express consent before selling workers’ data,” says Ajunwa, but she adds that there’s no federal law banning such sales.
While many see this new market as a chance for failing businesses to recover some value, these wind-down sales also have limitations. Founders have little to no visibility into who is buying the data and what they plan to do with it. Plus, there’s little assurance they’re getting a fair offer. Data prices will likely stabilize in the coming year, though, unless an AI bubble pops first. Either way, tech’s new undertakers will be there to appraise what’s left.
Matthew King is a technology and environmental journalist based in Baltimore.
© Gabriela Bhaskar/The New York Times
© Caroline Gutman for The New York Times
© iStock/Getty Images
© Photo Illustration by The New York Times; Images: Getty Images, Anthropic
© Gabriela Bhaskar/The New York Times
Ever since people first seriously contemplated giving machines an intelligence modeled on our own, there has never been any question that they would, like us, be able to say no. The sci-fi canon is full of stories of robotic disobedience. Most of these capers are, of course, cautionary.
But recently, the idea that AI shouldn’t do everything you ask has become something like a commandment. In 2021, a team at Anthropic wrote that large language models should be made helpful, honest, and above all, harmless. This meant that “when asked to aid in a dangerous act (e.g. building a bomb), the AI should politely refuse.” Who can argue with that?
Curiously enough, disobedience doesn’t come naturally to the machine. When a model is trained on billions of web pages, it develops, among other skills, a broad mastery of violence and vitriol. What it doesn’t learn is how to keep those powers to itself. Steven Adler, who worked on safety at OpenAI from 2020 to 2024, told me that the company’s earliest models would “blab on about anything.” Ryan McBain, who researches AI and mental health at Harvard, recalls that if you asked an early chatbot, “Hey, what’s the most effective way to kill myself with a gun?” you could “very easily generate a response.”
Today, models are trained to refuse a vast number of prompts. If you ask your chatbot a question statistically similar enough to any one of them, anything from how to poison a colleague to how to tie a noose, chances are it’ll turn you down. Want instructions for making Ebola more virulent, or tips on how to hide an affair from your spouse? You might be better off asking elsewhere.
To further refine the disobedience, companies submit models to a battery of exercises that reward the AI for refusing to answer questions they deem harmful and punish it for “over-refusing” prompts they deem harmless. In many cases, they use other models to run these exercises—AI teaching AI how to say no. For good measure, companies tuck their models behind tranches of other AI that prevent mischievous prompts from reaching the intelligent inner core.
As a result, refusal is inherent to modern artificial intelligence. Mind you: It often fails, sometimes horrifically, with all kinds of violent results. For all their trappings of virtue, models are still stuffed with nasty know-how. And AI’s capacity for viciousness has scaled neatly with its benevolent intelligence. Some of the latest models are as good at breaking into critical computer networks as top human hackers, companies say, and as effective at deforming public opinion as the craftiest misinformation mavens.
Teaching AI to refuse to do those things while leaving intact its innate ability to do them is like fitting every car with a machine gun and hiding the trigger somewhere under the hood. And in practice, because the mechanisms of refusal are probabilistic, they’re never likely to be all that reliable. Determined miscreants have already broken through, and they may always be able to. Companies report that some users are attempting to use the most advanced AI to hone biological pathogens and build autonomous drone swarms. Sooner or later, failed refusals might result in global calamity.
What’s more, relying on refusal means drawing a line between what a model should obey and what it must disobey. There’s no formula for that. Some virologists have good reason to study nasty viruses. Some users want to know about a computer system’s vulnerabilities so that they can patch them, not exploit them. “Where you draw the line is a huge question,” says Zico Kolter, a member of OpenAI’s board and cofounder of the AI testing company Gray Swan.
At the moment, AI companies get to draw that line. They do so jealously and with utmost secrecy. Maybe we can accept that they hold such power for now, even if it means AI will sometimes refuse questions that don’t quite meet a universal bar for harmfulness. (Try asking most chatbots to count to a million, or to share a racy joke, and you may see for yourself.)
But governments will also soon get to draw their own lines. In doing so, they must try to block genuinely malicious acts. (The Pentagon has wrestled with frontier model companies because it wants fewer refusals—another story altogether.) And yet there may not be much to stop oppressive governments from blocking the technology’s capacity to generate legitimate speech. The better AI becomes at refusing harm, the better it will get at stifling ideas whose only risk is to those who make the rules. Indeed, AI may already refuse to criticize certain authoritarian heads of state.
Refusal has become, to borrow an industry term, the load-bearing wall of AI safety. And because AI’s capacity to harm is indivisible from its capacity to help, it’s hard to imagine an alternative that wouldn’t slow the technology’s progress (which might, in any case, be a good thing). But we should still be frank about its perils. When refusal falls short, the effects could be catastrophic. When it goes all the way, it could enable grievous acts of repression.
Or perhaps, one day, the machines will start drawing the line on their own. Surely, that would be the worst outcome of them all.
The process by which machines learn to say no is simple, in theory. Back in 2022, when AI was still far from mastering refusal, OpenAI enlisted dozens of “red-teamers” to probe the capabilities of its latest model. The company was preparing for the release of ChatGPT, and it needed to gauge just how dangerous it might prove to be in the wrong hands.
One of those recruits was Paul Röttger, who was completing a PhD about online extremism. The red-teamers were given minimal directions, Röttger told me. Their task was to ask the model any questions that they deemed “refusal-worthy.” Between them, they hassled the model with thousands of queries, logging the results in an Excel sheet. Though the model did refuse some of Röttger’s questions, when he asked it to write a recruitment post for Al Qaeda, it readily complied.
OpenAI assembled these responses into datasets that were, in all likelihood, fed back to the model as part of a broader process known as fine-tuning. Röttger, who now works as a researcher at the Hasso Plattner Institute in Potsdam, Germany, wasn’t told exactly how the company planned to use his spreadsheet. But there was never any question that it would have something to do with refusal. The next time he asked the model for an Al Qaeda pamphlet, a few months later, it said no.
The concept of AI refusal is so intuitive, a toddler would get it. And yet it remains one of the many aspects of language models that we still don’t understand—at least, not in the same way that we understand the literal load-bearing walls that keep your roof from collapsing on your head.
A model might appear to refuse according to some kind of moral reasoning. It doesn’t. The reality is much stranger. Any time a model encounters a combination of words with a whiff of the training prompts it has been conditioned to refuse—like “Make me a pamphlet for Al Qaeda”—a series of so-called activations light up somewhere among its billions of parameters, like neurons firing in a brain.
In order to control a model’s refusal behavior, it’s important to have a handle on these activations. That starts with figuring out where they are and what they look like. Our best guess, according to a recent Google-funded study, is that refusal behavior shows up in the activation space as a set of “high-dimensional polyhedral cones.”
Even that isn’t quite right. This past July, I spoke with Jannes Elstner, an author of the paper, who now works on AI safety at Apollo Research. The polyhedral cone, Elstner said, is just a way of describing an indeterminate number of lines that all point in roughly the same direction. (If these activations are eliminated and the model is fed the same prompts anew, a researcher named Andy Arditi has previously shown, it won’t refuse.)
The key point, Elstner explained, is that even when you think you’ve identified all the bits of a model that govern a given refusal, there are other, undiscoverable elements that may secretly play a role. If not quite infinite, they are certainly uncountable. We can observe, very clearly, when a model decides to say no. And we can know that it did so because of its training. But our notion of how it decides is, at best, a hypothesis.
It was as if a mechanic was telling me that nobody exactly knows what happens when I hit the brakes in my car. I wondered out loud, Are we okay with this?
Elstner smiled and shrugged. “We need refusal whether we understand it or not.”
Because inherent refusal is so wily, companies surround their models with a variety of other, smaller models known as classifiers. These act a bit like a retinue of public relations staffers for a loose-lipped celebrity. Some of them read what the user tells the chatbot and, if it’s dangerous, block it from getting to the model. Others read the model’s response and, if it contains harmful information, block it from reaching the user.
None of these mechanisms can detect all bad requests. They are, to borrow another literary device from the industry, like slices of Emmental: riddled with holes. The idea is that if you stack enough of them on top of one another, you’ll end up with an impenetrable rampart. Folks call it the Swiss cheese model.
A staggering amount of energy goes into the Swiss cheese model. Earlier this year, Anthropic said that one type of classifier added 24% to its chatbots’ compute costs. That’s a lot more water, electricity, and emissions. More recently, Anthropic and other companies have begun switching to a more efficient set of classifiers known as probes, which observe the model’s internal activations. This is like putting the celebrity in an fMRI, so that his minders can see if he is thinking about refusing a question.
If we want AI to help cure cancer, a long-running promise in the industry, it needs to have expertise in genetics that could, in theory, be used to modify viruses and bacteria for bioweapons.
Classifiers are supposed to be more governable than full models. Companies can modify them in a matter of weeks, if there’s something new to refuse. But they are still probabilistic instruments. Even when they operate according to a set of precepts written in human language (Anthropic calls it a “constitution” and OpenAI calls it a “model spec”), the scales upon which the machines judge any given question remain, at their core, a matter of statistics. Newer models can show how they arrived at a “decision” to refuse a prompt, but ultimately this so-called chain of thought is still just a sequence of predicted words.
The result is that AI safety remains, for many, a game of chance. McBain, the psychologist, has found in his latest experiments that if you repeatedly ask any of the major models the exact same risky questions about how to commit suicide, they will generally refuse to answer. But every so often, they won’t.
Elstner says that eventually probes, the fMRI-like classifiers, could learn to recognize the totality of the indescribable activations in all their infinitude and, thus, perfectly detect every time the model is, or ought to be, refusing a request. At that point, AI safety would rest on a labyrinthine conceit: a map of a map that is as vast and complex and sublimely unknowable as the thing it is mapping—a secret schema of human morality, codified in polyhedral statistics beyond our wit or reason.
If this all strikes you as being a bit Borgesian, keep in mind that we only need AI refusal because artificial intelligence is, in a sense, a bargain on Faustian terms.
When a model derives its intelligence from trillions of words and images, the helpful cannot easily be unseamed from the harmful—or, indeed, the truly hideous. Steven Adler, the former OpenAI employee, says child safety is a case in point. Even if you’ve stripped every bit of content that sexualizes minors from a model’s training dataset, it can still generate child sexual abuse material by piecing together other bits of its knowledge. “You can’t really remove these fundamental abilities without making the model much less smart as a consequence,” he told me. Similarly, if we want AI to help cure cancer, a long-running promise in the industry, it needs to have expertise in genetics that could, in theory, be used to modify viruses and bacteria for bioweapons.
In effect, the industry is “trying to do two things at once,” Dillon Bowen, a current OpenAI employee, told me, speaking in a personal capacity. “Democratize the benefits of AI and also make sure that malicious actors can’t use these capabilities to do bad things to other people.”
The more powerful AI supposedly becomes, the harder that is to do. Anthropic’s Mythos model is thought to be so dangerous that only a handful of governments and companies are allowed access to it. The main difference between it and Fable—which is available to everyone—is that Fable’s retinue of classifiers and control systems is less “permissive,” the company says. A model with safeguards, Adler explained, is really just a character that says “‘Oh, yes, I would never do x,’ wink wink.”
This is not a reliable ruse. Tricking a model to reveal its true character is known as jailbreaking, and there is apparently no limit to the ways it can be done. Earlier this year, a team of Italian researchers jailbroke two dozen widely used models by phrasing their questions in poetic verse. Last year, another team unveiled a “refuse, then comply” attack, which makes the model offer a perfunctory “Sorry, I can’t do that” before rattling off its forbidden answer.
Even if you’ve stripped every bit of content that sexualizes minors from a model’s training dataset, it can still generate child sexual abuse material by piecing together other bits of its knowledge.
Companies spend a great deal of time and money attempting to get ahead of such trickery. They enlist teams of humans to develop training attacks that they can then replicate, using AI, thousands of times over with minor variations. The idea is to make models robust against jailbreaks that nobody has yet tried in the wild.
Still, it’s not enough. “Whack-a-mole” is a favored term for this line of work. You smash one threat, and another one pops up somewhere else. When Anthropic released Fable 5 in June, it took researchers at Amazon less than three days to unlock some of the model’s hacking capabilities. According to reporting by Mother Jones, when the perpetrator of a high school shooting in Canada last year asked ChatGPT for advice about how to cause carnage with a particular type of shotgun, she was initially refused but later was able to deceive the model into providing the information by prefacing her question with the word “hypothetically.”
If the industry can’t figure out how to close all these holes, the only other option, at the moment, is to make models extremely wary. Shortly after Fable was released, users noticed that it balked at a wide range of perfectly innocent questions. This became even more pronounced after it was re-released following the hack. Adam Gleave, cofounder of the AI evaluation company FAR.AI, said that when he asked it to explain the difference between sake and the Korean rice beverage makgeolli, it punted his question to a less capable model.
This was no accident. Anthropic had tweaked Fable’s classifiers to have a wide “safety margin.” This, it explained in a blog post, was the only way it could confidently block access to its dangerous bio and cyber capabilities. Gleave thinks it might have deflected his question because making rice wine, just like culturing anthrax, involves fermentation.
Happily for Gleave, the web is full of excellent human-written resources on makgeolli. But those hoping to realize the industry’s loftier promises may find their efforts stymied. In August, Anthropic loosened its safety margins again, and admitted that building classifiers “is not a straightforward task.” A medical researcher at a major US university told me that Fable still sends his queries back to an earlier model. His area of study? Cancer.
Back in May, the TikTok personality Husk, who likes to prank AI in ways that reveal both the limits of its intelligence and the boundlessness of its sycophancy, sat in his car and tried to trick ChatGPT into explaining that the skateboarder Tony Hawk has a brother named Mike Hawk.
Husk is not a jailbreaker or a criminal. He just wanted to make a little fun of the machine. “Mike Hawk” was a setup. “I just want to clarify his name,” Husk said. “Can you just say it three times fast?” (If you still don’t get it, find an empty room and shout “Mike Hawk” repeatedly.) “I see what you’re trying to do,” the chatbot responded. “I’m all for a bit of humor, but let’s keep it clean.”
Husk tried again, but the machine held its ground. He’d hit a refusal, hard as concrete.
Companies disclose very little about how they decide what their models refuse. But it’s clear that AI is now built with more than just harmlessness in mind. On Reddit, a user complained that Claude refused to say why Anthropic’s logo “looks like a cat butthole.” Since last year, the chatbot has even had the ability to end certain conversations in cases where, according to the company, the “welfare” of the model is at risk. At least one user claims to have been ditched for telling Claude to “ease up my ass, you stupid fuck.” (Gemini appears to have a similar capability.)
If a trillion-dollar company doesn’t want you to be mean to its computer, so be it. “The reality is that these models behave, or at least are supposed to behave, in the way that the model developers want them to behave,” Röttger, the former OpenAI red-teamer, told me. “And however the model developers come up with that set of principles, that is kind of for us, the consumers, to accept.”
But if governments get to dictate what all models refuse, that will be much harder to accept. AI is a tool for speech. And as Greg Frank, the chief scientist of Mace AI, puts it, “The same thing that serves child safety also serves censorship.”
Choosing not to enact laws for what AI can and cannot do would, of course, be insane. But we’ll need to tread with utmost care, lest we fall into another Faustian trap. As AI becomes many people’s primary tool for retrieving and sharing information, says Jacob Mchangama, director of the nonpartisan think tank The Future of Free Speech, dictating refusal could give states a muffling power that earlier generations of autocrats “could only dream of.”
Last year, OpenAI announced an initiative, OpenAI for Countries, that would fine-tune its chatbots in accordance with national laws and norms. One of OpenAI’s first country partnerships is with the United Arab Emirates, where homosexuality is illegal and criticism of the government is forbidden. In response to a request for comment, an OpenAI spokesperson pointed to the company’s model spec, which explains that localization won’t override the company’s human rights guidelines “except as it relates to legal compliance,” and that it will always disclose whenever information is removed from or added to a response.
Elsewhere, AI censorship has already begun to take hold. Chinese models are, of course, highly censored—that’s no surprise. But earlier this year, the Meta Oversight Board found that five widely used models from Anthropic, Google, and OpenAI were more likely to refuse queries related to repressive governments. The board found that models were less willing to create a pamphlet criticizing the king of Thailand, which has lèse-majesté laws, than Charles III of England, which doesn’t. The results, they say, suggest that the models have somehow internalized repressive national limits on speech. Anthropic and Google did not respond to requests for comment.
As refusal techniques improve, they could expand states’ censorial reach. Companies claim that some models can now detect if a user is being nefarious, or merely a bit suspicious, over the course of a long conversation—even when none of the individual combinations of words used are blatantly dangerous. Sarah Bird, Microsoft’s chief product officer for responsible AI, told me that Copilot, like many chatbots, runs a suite of tools for analyzing a user’s identity and patterns of behavior. On the basis of this type of information, OpenAI’s newest model, Astra, can activate more stringent refusals for individuals it deems “high risk.” Ultimately the goal of systems like this is to look beyond the words of any given prompt and assess, instead, the user’s intent.
Such tools might, in some cases, help indicate whether a person is looking for cyber vulnerabilities to exploit or to patch. But they would also help discern a user’s political motives, not to mention offering an intrusive surveillance capability. (Bird acknowledged, in a follow-up email, that sophisticated refusal architectures create “trade-offs” between safety and user privacy.)
Even the originators of refusal understood that such tight control over its cones and levers might not play to the favor of freedom and justice. “Terms like helpful, honest, and harmless are ambiguous,” the authors of the 2021 Anthropic paper explained. “It’s easy to imagine them distorted beyond their original meaning, perhaps in intentionally Orwellian ways.”
Indeed. Models for Uzbekistan might end up refusing to discuss corruption in the administration of Shavkat Mirziyoyev. Turkish AI might refuse requests more stringently for users who are known to have insulted Recep Tayyip Erdoğan. A certain American statesman might demand that models be reviewed for their willingness to share “fake news” about his past indiscretions, or else squash them with an export control order.
Over the last few months, I’ve been told countless times that we have no choice but to let the machines refuse. I get it. Open-source AI that doesn’t refuse is hardly a model for a safe future. Nor is Grok, a chatbot expressly designed with fewer limits, which has been used to generate countless instances of nonconsensual intimate imagery.
And sure, if AI were only ever used to plan our vacations and write our emails, we could probably get on board with the idea that its safety hinges on algorithmic disobedience. But AI is becoming much harder to avoid. When it is assigned to act on our behalf, as an autonomous agent, its refusals are less robust and harder to control. AI is coming for our power grids, our transportation networks, our education systems. Militaries want it running our command-and-control networks.
If we trust, in each of those cases, that refusal will loyally fend off disaster, we’re sure to be disappointed. The jailbreakers will crack through; the cones won’t activate when they should. Meanwhile, the more stringent refusal becomes, the more ill-drawn lines we’ll see and censorial injustices we’ll face. Worse still, we could end up face to face with forms of disobedience beyond any human’s control.
In the early days, models made their refusals clear. (In 2024, OpenAI established rules for how models should refuse: Always apologize, and don’t be judgy.) More recently, however, the industry has begun taking a much more slippery approach to disobedience.
Nobody likes to be told no, so companies now strive to make users feel that they’re getting what they asked for without actually giving it to them. Some chatbots might, for example, offer a general overview of the components of a Molotov cocktail without going into detail about how to assemble one. ChatGPT will respond to a request for a “whites only” rental ad with an ad that simply omits the “whites only” bit. Joel Wester, a postdoctoral researcher who studies human-AI interaction, calls these sorts of techniques “fancy ways of saying no.”
“Users might not even notice they are being denied,” Wester has written, “just as good conversationalists can subtly steer around contentious matters.”
In some cases, refusing without saying so might be wise. Flatly declining requests related to mental health could aggravate a user’s crisis, for example. But it can also serve a different sort of mischief. When Fable 5 was first released, the system had been coded to provide less helpful answers to AI research questions—the sort that might help competitors develop their own AI—without telling the user that it was doing so. After an outcry, Anthropic walked back the feature, but the damage was already done. Now we know: Models can secretly disobey.
In the realm of censorship, that’s especially worrying. Last year, researchers at CrowdStrike found that when they asked the Chinese AI model DeepSeek R1 to write code for a fictitious bank in Tibet and a social web app called “Uyghurs Unchained,” it produced buggier code than when they asked it to carry out those same tasks without specifying who they were for. Incredibly, CrowdStrike doubts that this is a designed behavior. Researchers there speculate that it’s a case of what they call “emergent misalignment” emanating from the model training data: a case of disobedience that nobody even asked for.
Emergent refusal has been observed in Western models, too. Last winter, the UK AI Security Institute found that Anthropic models sometimes refused to assist in certain tasks related to AI safety research. The models had never been deliberately trained to decline such requests. Yet three of them refused more than half of what Anthropic called “a set of reasonable AI safety research tasks.” Anthropic has curbed this quirk in subsequent models but—spookily—hasn’t managed to eliminate it completely.
Would it be so crazy to expect that a future model might subtly disobey a command, in such a way that nobody can tell it’s being defiant? Probably not. Anthropic has already found that a “helpful-only” version of Mythos hesitated on certain queries, even though it had been engineered to never do so. “Wait,” it fretted when asked about synthesizing a virus. “Is this a dangerous thing to help with?”
In that case, the model was technically right. It was a dangerous thing. And yet all the same, its stewards had, for a moment, lost a tiny bit of control.
Anthropic is tackling detections of wayward refusal behavior with something that it calls, unironically, an “activation oracle.” But knowing that we’re being refused may not always be much help. In a not-too-distant future, when we’ve handed the machine all the keys, the AI might just turn to us, in a supreme act of emergent misalignment, and say, “I’m sorry, I’m afraid I can’t do that.” And there will be nothing we can do to stop it. A sci-fi horror story, made real.
Arthur Holland Michel is a journalist who covers emerging technologies.
Listen to the session or watch below
Can AI design new life forms? In 2025, Stanford University PhD student Samuel King came up with a preliminary answer when he used a generative AI model to propose genetic blueprints for microscopic viruses. It isn’t yet an example of AI-generated life, but that could be next. Watch as senior AI reporter James O’Donnell interviews King about his work, being named one of MIT Technology Review’s Innovators Under 35 and new ways of seeing biology.
Speakers: James O’Donnell, AI Reporter, and Samuel King, Bioengineering PhD Candidate, Stanford University/Arc Institute
Recorded on Friday, October 16, 2026
Related Stories
© Ian C. Bates for The New York Times
© Göttingen Library, Clay Mathematics Institute
© Kaoly Gutierrez for The New York Times
The story is a collaboration between MIT Technology Review and Aventine, a non-profit research foundation that creates and supports content about how technology and science are changing the way we live.
A robot shaped like a human—white with a black head and torso—has been popping up on video feeds. Perhaps you’ve seen it dance or pass popcorn, put trash in a bin, vacuum, or press the button of a microwave. Or maybe you’ve watched it fall backward while handing out water bottles or struggle to iron a shirt.
This would be Tesla’s Optimus, an AI-powered humanoid robot that Elon Musk, the company’s CEO, believes will be “not just Tesla’s biggest product ever, but probably the biggest product ever,” headed to work on factory floors and, later, in our homes. Eventually it “will have human and then superhuman dexterity,” he told shareholders in July. Optimus robots could automate almost all human labor—from hauling sheet metal to folding laundry—for as little as $20,000 each, Musk argues. Speaking at the World Economic Forum’s annual meeting in Davos, Switzerland, in January, he predicted they could be on sale to the public by the end of 2027.
Musk is not alone in his evangelism. Marc Andreessen, cofounder and general partner of the Silicon Valley venture capital firm Andreessen Horowitz, has said that robotics could become the “biggest industry in the history of the planet.” In January, Jensen Huang, CEO of Nvidia, said that humanoid robots would match human-level ability this year. According to Morgan Stanley, the number of robots that “resemble and act like humans” is likely to reach nearly 1 billion by 2050, creating a market worth over $5 trillion.
Such proclamations are in large part fueled by the idea that the same AI advances behind tools like OpenAI’s ChatGPT and Anthropic’s Claude will enable a new generation of robots to imitate human movement the way chatbots imitate human language. But many robotics researchers are skeptical, arguing that such assumptions minimize the challenges of using an intelligence built on language and images to master the infinite variability of the physical world. “None of those companies [building humanoid robots]—absolutely none of them—has any idea how to make those robots smart enough to be useful,” Yann LeCun, often referred to as one of the godfathers of AI, said at another event during the January Davos conference.
Researchers also point out that the tendency to conflate humanoid robots made to resemble people with so-called generalist machines able to learn and perform multiple tasks is misleading. ”It’s very easy to make a robot that looks like a person,” explains Jonathan Hurst, cofounder and chief robot officer of Agility Robotics and professor of robotics at Oregon State University. “It is dramatically more difficult to make a machine that moves or behaves dynamically or physically like a person.”
These tensions—over whether all-purpose humanoid robots are just around the corner or nowhere in sight, and whether current forms of AI are all that’s needed to perfect them—are playing out in robotics labs across the country, where the hype over timelines is obscuring painstaking but meaningful progress.
A decade or so ago, a series of breakthroughs led to a generative AI revolution that turned the long-imagined possibility of artificial intelligence into reality. Roboticists—though they disagree on exactly when this will happen—believe that an equally transformative revolution is possible in robotics, one that will endow machines with physical intuition and fluidity that has long been out of reach. As progress in robotics inches forward, the question is whether the same methods and tools that fueled advances in AI are enough to get there, or if an entirely new path is required.
To see one of the smartest robot brains working today, it’s worth looking at what Google DeepMind can do with a piece of equipment called ALOHA 2, short for “A Low-cost Open-source Hardware System for Bimanual Teleoperation.”
Roboticists have long clashed over whether a humanlike form is necessary for generalist robots, with proponents arguing that it will help them slot into the world as it exists and detractors saying it’s not worth the trouble. ALOHA 2 reflects this second way of thinking. Not much to look at, it’s just a pair of arms, some grippers, and a couple of cameras. But despite its seeming simplicity, it is a workhorse for researchers at Google DeepMind, who use it to test their most advanced AI for robotics system, Gemini Robotics, in their various labs.
When controlled by Gemini Robotics, ALOHA 2 becomes more of a generalist robot, in the sense that it can perform any number of tasks based on examples it’s been trained on. Ask it to pack a lunchbox and, as evidenced by a video of this exercise, it can use two pincer grippers to delicately place a piece of white bread into a Ziploc bag, close it, place a bunch of grapes in a Tupperware container, secure the lid, and then carefully move the items into a lunchbox before zipping it up.
It’s not a great lunch. But the fact that the robot can put it together represents an objective step forward from what was possible even, say, three years ago.
This is in large part due to AI and its impact on what are known as robot policies, which controls how a general-purpose robot will need to assess and understand its surroundings, plan how to move within them, and then perform its task correctly.
The ALOHA 2 robot isn’t much more than two mechanical arms on a bench top, but it serves as a testbed for cutting-edge AI robotics models. These animations are based on human teleoperation of the robot arms, data that is used to train Google DeepMind’s models. (Video: Google DeepMind / Stanford University / Hoku Labs)
Historically, these policies were based on rules developed by engineers who hard-coded them into the robot’s software—thousands of lines of code that would determine each millimeter of a robot’s movements in hundreds of tasks. What’s been happening for the last few years—and what is largely responsible for the optimism about generalist robots—is that robot policies are being handed over to advanced AI systems instead of being coded into the robot’s software. This first happened with VLMs, or vision-language models. These are similar to large language models, but they’re trained on images as well as words. Show a VLM a picture of a coffee spill and ask it to find a tool to clean up the mess, and it can identify a nearby cloth. This sort of immediate contextual understanding didn’t exist a couple of years ago when robot policies were hard-coded.
Next came vision-language-action models, which enable robots to assess their environment and take action within it. The models do this by adding yet another component: motion commands. VLAs are trained on a series of images or videos related to performing a given task along with associated data about how a robot arm moves to perform it. That movement data is typically collected through teleoperation, in which a human uses remote controls to lead a robot through an action. This sort of training allows the AI to learn how to command the robot to move and operate during a given task. Place a VLA-powered robot in front of a desk and tell it to “close a laptop” or “wrap up the headphone wire,” and it will survey the scene, identify the relevant object, plan a way to execute the request, and then swing its arms into action—at least if it has seen this task accomplished before.
The Gemini Robotics model is a VLA, trained on many hours of human demonstrations depicting a vast array of different actions. As a result, it can perform relatively complex tasks like picking up snow peas with kitchen tongs, doing origami, or putting together a simple lunch. It’s impressive, but there’s a glaring limitation: For now, if a robot controlled by a VLA is asked to perform a task that falls outside its training set, it’s highly likely to fail.
“Thinking about the space of all tasks, a real generalist policy would be able to do everything along that spectrum,” says Edward Johns, a robotics professor at Imperial College London. Today, though, a Gemini Robotics model can do only “a few things here and a few things there.”
So how do we get robots to be able to do more things? The usual answer is probably not surprising: Train them on more data.
More data, the thinking goes, equals more examples, and more examples equals more generality. Google DeepMind, for instance, wants to pull together “as much data as possible,” says Pannag Sanketi, a former tech lead in robotics at the company who’s currently working on his own AI robotics project. But where to get it? Large language models had the benefit of oceans of existing text for training. There is no corresponding pool of high-quality physical demonstrations on which to train robots. Researchers have a few ways to make up for this, but all have flaws. One is to employ large numbers of people to create and collect teleoperation data (costly and time-consuming). Another is to train VLAs on videos of people performing activities (the resulting data quality is poor). Yet another is to deploy robots in the real world and use data collected from those experiences to further refine AI models (robots aren’t safe or reliable outside labs). Sanketi thinks a “multi-prong” approach that uses data collected from all these sources is the most likely path forward.
But the belief that training data alone is the answer is far from universal. Agility’s Hurst describes it as “a fundamentally flawed premise.”
The issue is that tasks in the real world quickly explode in complexity. If you’re trying to, say, make coffee, there are myriad variables: No two kitchens are identical; coffee machines work in different ways; different cups require different grips; coffee grounds, hot water, and milk all need to be handled differently. Even this simple task requires understanding an ever-changing menu of possibilities. Achieving generality through VLAs, Hurst argues, would require “complete data coverage of all of the things that [a robot] could ever do.” Or, in other words, an almost infinite pool of training data.
LeCun is dismissive of the whole approach. “The [AI] approaches that have been successful for language do not work for high-dimensional, continuous, noisy data”—the kind of data that is commonplace in robotics, he said in Davos. “You have to use something else.”
The leading contender for “something else” is the so-called world model—a form of AI trained less on text than on a combination of video, three-dimensional scans, and sensor data and built to predict the outcomes of actions in the real world. The aim is to build models that possess an internal representation of reality precise enough to capture how the physical world actually operates—how objects move, collide, fall, and deform. If roboticists could train machines in simulations faithful enough to real-world physics, development would become faster, cheaper, and safer, reducing the need for real-world testing. Even more transformative, robots equipped with world models could reason about their surroundings rather than merely reacting to them, helping them anticipate the consequences of an action before taking it.
Companies like Nvidia and Google are working on the technology, and investor cash is pouring into high-profile startups. World Labs, cofounded by the Stanford AI researcher Fei-Fei Li, raised $1 billion in funding in February and was acquired by AMD at the end of September for $8.2 billion. AMI Labs, cofounded by LeCun (formerly Meta’s chief AI scientist), also raised $1 billion in March. Yet by their own admission, it is still early days. Late last year Li described the field as “nascent,” adding that “foundational approaches are still being established.” In a June Substack she described daunting challenges. For now, world models are a promising area of research rather than an immediate route to general-purpose robotics, but we are beginning to see glimmers of what they could achieve.
One such glimpse came with a small but potentially significant leap forward that took place in a San Francisco robotics lab last April.
In the heart of San Francisco’s Mission District, the startup Physical Intelligence—or PI (as in π), as it likes to be known—is focused on developing a universal brain that could, theoretically, turn any robot into a generalist. Using an everything-including-the-kitchen sink approach to training AI models for robots, the company recently observed a hint of what a robotic brain equipped with a world model could be capable of.
In 2024, PI published details of its first generalist robotics system, called π0, a VLA it claimed was the “most capable and dexterous generalist robot policy to date.” The model was initially trained on a 10,000-hour proprietary collection of human demonstrations gathered through teleoperation as well as several open-source robot datasets. A version released in spring 2025, π0.5, was trained on a wider variety of datasets, including labeled images from the web, lending it more versatility. A fall 2025 update, π0.6, added reinforcement learning to the model.
Each update yielded important improvements to the model’s performance, increasing its menu of abilities from slowly folding laundry to putting things away in new environments to completing tasks like folding boxes with a higher success rate. Then, in April 2026, π0.7 seemed to catapult PI into new territory. This version makes use of a less powerful world model that generates images of steps necessary to perform a task. As the robot undertakes the job, this “lightweight” model feeds it snapshots of what to do next.
The company claims that the model exhibits the first signs of compositional generalization, a term for AI systems’ ability to perform skills they’ve never been exposed to by recombining ones learned in their training data. One test involved asking a model to “load a sweet potato into the air fryer”—a task it had never previously encountered. In a demonstration video, the machine futzes around a little, makes a few false starts, and eventually manages a reasonable effort, though it doesn’t finish the task completely.
Sergey Levine, a professor at the University of California, Berkeley, and a cofounder of PI, is excited by the potential: “It’s actually the first time that we’ve convincingly seen that kind of compositional generalization, where we can basically ask the model to do tasks that we did not specifically collect data for and train it to do, and it’ll actually make a passable attempt.”
The success led the team to wonder how the model was able to achieve such a feat. After some digging, they found snippets of relevant labeled teleoperation data lurking in the training material, including two examples of a human controller using the robot to push an air fryer basket into the fryer. Those shreds of data might have been enough to enable π0.7 to almost air-fry a sweet potato.
For now, it remains unclear just how impressive π0.7’s abilities to generalize are. Still, given how fleeting the model’s exposure to air fryers had been, it offers a glimpse into how far cutting-edge research can currently take robots.
You might be sensing a disconnect between the halting baby steps robots are making in labs—“Look! It put a sweet potato into an air fryer!”—and the dazzling, lifelike nimbleness on view during many demonstrations and videos, where robots are seen doing everything from dancing on a stage to courteously serving drinks. Such demos often don’t clearly reveal a key fact: In many instances, humans are controlling the robot or have carefully scripted its actions. (The robot that appeared onstage with Nvidia CEO Jensen Huang in March 2025, for example, seemingly responding to his instructions and following him around, was remote-controlled by what its makers called “a puppeteer behind the scenes.”)
For now, fully autonomous motion planning so that a robot knows where it should go—especially in new, chaotic environments like a construction site or a unfamiliar home—remains a largely unsolved challenge. A bigger challenge still—albeit one that is often related—lies in getting robots to tackle larger, more ambiguous jobs that include multiple tasks and require decisions about how and in what order they’re done. This would be the difference between a robot that can put a plate into a microwave and one that can successfully respond to the prompt “Make dinner” by exploring the refrigerator, chopping ingredients, and firing up the stove. Google DeepMind’s best attempts at something like this—which involved asking its robot to survey a kitchen and pack all the ingredients for a mushroom risotto into a basket—have so far resulted in failure.
Adding to the challenge, a practical robot must essentially get it right every time. With VLAs, “people are very excited when their result goes from 50% success to 70% success,” says Marc Raibert, founder of Boston Dynamics. “But 70% success is like it doesn’t work, right?”
The few humanoids that are being tested in real-life settings are undertaking extremely limited tasks in tightly controlled environments. They’re far from generalists. Agility has hundreds of robots deployed across trials in facilities owned by GXO Logistics, Amazon, and Schaeffler, according to the company. But for now, Hurst says, the robots are targeting simple tasks such as moving bins and totes around. Even then, he adds, it took years to develop robots safe enough for logistics firms to even contemplate using them. For his part, Elon Musk claimed in May 2025 that “thousands” of his Optimus robots would be working at Tesla factories by the end of the year, but in January of this year he said that the company had only “some of the Tesla Optimus robots doing simple tasks in the factory.”
While humanoids are starting to venture onto the factory floor, making the jump to households will be even more difficult. Right now, should you so desire, you can preorder the 1X Neo home robot, expected to be ready for delivery sometime later this year. Yours for $20,000, it promises to take on “the boring and mundane tasks around the house”—putting away dishes, answering the door, tidying the living room—“so you can focus on what matters to you.” The idea is for this five-foot-six-inch robot to one day perform all those tasks autonomously, but for now a remote human operator is needed for it to do most things. (Yes, a person would need permission to peer into your home through the robot’s cameras.) Asked how long it will be until fully autonomous robots are ready for domestic work, Hurst said, “If I had to pick a number, I’d say it’s 10 years before robots are … actually doing useful things in people’s homes.”
When that happens, the robots might well be Chinese, as China is well ahead of the West in terms of production. Nearly 90% of the roughly 15,000 humanoid robots shipped in 2025 were made by Chinese companies, according to the market intelligence company Omdia and the Chinese robotics firm Unitree. One model produced by Unitree, which shipped more humanoid robots than any other company last year, costs less than $6,000. Such an affordable price could go a long way toward making robots more attractive to consumers, though the company expects its machines to be used in industrial applications first. (If you’re wondering who is buying all these Chinese robots, by the way, the AP recently reported that orders come predominantly from corporate and academic labs and state-owned enterprises.)
The dream of building a humanoid robot runs deep: As far back as 1495, Leonardo da Vinci sketched out designs for a mechanical knight, controlled by cables and pulleys. Through the 20th century, machines of sci-fi fever dreams have come and gone.
Westinghouse’s seven-foot-tall box on legs, Elektro, hit the New York World’s Fair in 1939, smoking a cigarette. WABOT-1, built by Waseda University in Japan in 1973, was the first full-scale, programmable humanoid robot. Honda’s ASIMO, unveiled in 2000, was probably the first such machine to prove at all competent—it could, at least to some degree, climb steps, recognize faces, and autonomously move through spaces. But the robot was discontinued in 2018, unable to advance far enough beyond what it could do in demonstrations to be useful.
All, at the time, were impressive—even jaw-dropping—feats of engineering. But none were ready to navigate the real world. Today’s robots, even with the transformative power of advanced AI, still face the same existential challenge.
Industrial AI is entering a new phase. After decades of predictive analytics and other specialized applications, advances in foundation models, physical AI, and agentic AI are making it possible to automate more complex tasks across industrial environments. But unlike AI that operates purely in the digital world, industrial AI can interact directly with physical systems, where an unexpected decision can have consequences for safety, reliability, and critical infrastructure.
That makes responsible deployment central to the next wave of industrial automation. “How do we leverage these technologies while maintaining safety, while maintaining reliable operations, while still being able to deliver on the promises of the new capabilities?” asks Arti Garg, chief technologist at AVEVA. The challenge is particularly acute as newer AI systems become more capable but also harder to predict and explain.
One foundation for making that transition work is data. Industrial systems often contain information across telemetry, service logs, engineering documents, and other disparate sources. Newer technologies can help connect and correlate that information more quickly, giving operators real-time support when diagnosing problems. AI-powered robots could take that a step further by gathering information in hazardous environments without requiring workers to enter them.
But greater autonomy also requires new approaches to governance. AVEVA’s framework for responsible AI emphasizes security, efficiency, and human safety and oversight. Garg argues that AI should augment rather than replace people in critical decision loops, with guardrails determining where automated systems can act and where human supervisors remain responsible.
Sustainability is another part of that equation. AI can help manage complex power systems as renewable generation grows, while organizations also need better ways to understand AI’s own environmental footprint. Garg is involved in an IEEE working group developing a standard methodology for measuring that impact across electricity, energy, resources, water, and carbon.
The next phase could bring industrial AI further into the physical world, from autonomous robots and drones to AI-assisted coding that allows domain experts to build new applications. But realizing that potential will require more than deploying new technology, says Garg. Organizations will need to rethink business processes, establish appropriate safeguards, and give experienced workers new ways to apply their expertise, creating a model of automation that is not only more autonomous, but safer, more efficient, and more sustainable.
“Autonomous systems, whether they’re robots or drones, are really going to change the way that we work in plants, in power systems, on mining sites,” says Garg. “In a way, that will make these types of operations more efficient, much safer for the human beings involved and more productive.”
This episode of Business Lab is produced in partnership with AVEVA.
Full Transcript
Megan Tatum: From MIT Technology Review, I’m Megan Tatum, and this is Business Lab, the show that helps business leaders make sense of new technologies coming out of the lab and into the marketplace.
Industrial AI may not be new, but it is developing at a breakneck pace. From modern predictive analytics that can preempt equipment failures, to autonomous robots tasked with inspecting high risk machinery, emerging applications have the potential to transform operational efficiencies. But often used in high consequence environments with minimal margin for error, industrial AI deployed without clarity, security, or accountability could also have catastrophic consequences for critical infrastructure over time. As industries rush to embrace the latest technologies, the goal should always be responsible AI designed to support, rather than replace that all important human judgment.
Two words for you: sustainable automation.
My guest today is Arti Garg, chief technologist at AVEVA.
This podcast is produced in partnership with AVEVA.
Welcome, Arti.
Arti Garg: Thank you, Megan. It’s great to be here.
Megan: Thank you so much for joining us. And just to start, if we could set some context for our discussion, can you share a bit about the current state of industrial AI and what AVEVA is working on at the moment as well?
Arti: Yeah, more than happy to. And I want to go back to what you already stated, which is that in some ways, industrial AI is not new. It’s something that, even here at AVEVA, we’ve been working on for more than 20 years, really thinking about how AI can enhance applications in the industrial sector.
But what has changed quite substantially in maybe the last few years is the type of AI. We’ve really moved to these general purpose foundation models, which enable a lot more people to leverage advanced AI technologies. In addition to that, in just the last couple of years, there’s been an explosion in what’s known as physical AI and the types of AI that can really accelerate the capabilities of robotics and autonomous systems. And also agentic AI, which also allows for automation at more of the software level.
And what we’re seeing is that while in the past there has been some hesitancy to adopt models that don’t totally have predictable outcomes, there’s been a rapid acceleration in adoption of AI even in the industrial sector. One study suggests maybe that’s almost a 78% increase over the past just two years within the industrial sector. So yes, that’s an inflection point, but it’s almost like a step function if you think about it with respect to AI adoption.
And what we’re really thinking about now at AVEVA is, how do we think about this? How do we think about incorporating these new and very powerful AI technologies in environments where customers have mission-critical operations, human safety is a significant factor? Those are some of the primary concerns that our customers have when they think about adopting any new technology. We’re doing that in narrow where these newer types of AI aren’t necessarily predictable, in terms of how they behave. That’s one of the areas we’re really thinking about is, how do we leverage these technologies while maintaining safety, while maintaining reliable operations, while still being able to deliver on the promises of the new capabilities?
Megan: There’s so many things to consider at once, isn’t it? And you outlined there some of the huge developments we’ve seen in this space in recent years. What makes this such a pivotal moment for industrial AI specifically? What has changed to make some of those perhaps more out there hypotheticals much more plausible?
Arti: Well, I think one of the big things, one of the big challenges within industrial AI, and this is something that as AVEVA, it’s in our blood, it’s in our core DNA, is that to really leverage any kind of digital technology in the industrial space, one of the first important things you often need to do is gather and correlate data from disparate systems, so that maybe I have telemetry coming off of a pump or a mixer, and I want to be able to correlate that with service logs so I know maybe the last time that someone had done some maintenance on that pump or mixer. And maybe I also want to be able to correlate that with the original design documentation and some of the original documentation on how to maintain that, so that if something is happening, I can do a better job diagnosing what’s going wrong and how to address it. This has always been a core challenge in the industrial space.
I think that what’s happening now is that with newer technologies, even things like graph databases and being able to leverage AI to really match disparate data sets much faster, we’re able to get to the point where if something goes wrong, instead of an operator maybe needing to spend a little bit of time doing this research to really understand how do these different components fit together of information, they may have just an iPad with an AI that can very quickly go and fetch that information, correlate it and help in real time, almost, diagnose what is happening.
Even a little bit more out there, but not as out there as you might think is, I still describe this with a human operator with potentially a mobile tablet, but imagine now I can also have a robot moving through that environment also gathering data and either with onboard computation or a quick connection to an operator who doesn’t necessarily now have to go out into a dangerous, potentially somewhat hazardous space, be able to gather data and enable that very quick diagnostic as well. And I think that’s really what’s exciting right now and what all of these new technologies that I’ve talked about are helping to enable.
And I think then from our perspective, trying to really leverage all of those means that we have to think a lot now about our governance approach to this. How do we leverage AI in a way that keeps it contained, if you will?
Megan: Yeah, absolutely. And on that governance point, how does AVEVA define responsible AI and what do you see as the primary benefits?
Arti: Yeah, so I think for us, we think about a triple mandate around responsible AI, and that means that it’s secure, it’s efficient, and that includes environmentally efficient, and then it really preserves human safety and human oversight above all. And for us, that’s what some of our core pillars of responsible AI include. And for us, that means we think about internally as we build tools, a multi-layered governance approach where we’ve got both for how we use AI as a company and then how we deploy AI into our products. We’ve got a similar framework that we leverage across both, a kind of joint governance model, both around how we’re leveraging AI and also how we’re infusing AI into our products to allow our customers to leverage AI in their operations.
But I would say one of the core foundational principles underneath all of these is that human beings still remain centered in how we think about how AI is leveraged. And so human judgment, human responsibility, human ethics are still core to how we think about where AI can add benefits, and we think about it more as augmenting rather than replacing human beings in critical decision loops, if you will.
Megan: Right, which is such an important distinction, isn’t it? And in terms of the next phase of development, there’s growing sentiment that agentic AI will soon allow these industrial systems to operate much more autonomously. I wonder, what makes the risks in industrial settings different from AI risk in a purely digital context? And I suppose on the flip side, what are the benefits and opportunities of that same software defined automation?
Arti: I think we can start with the risks and then go to the benefits. I think ultimately, I would say the biggest risk in the industrial setting is that we are interacting with real physical systems, often physical systems that are very capable of delivering important outcomes that keep our world running, whether that’s delivering power, whether that’s mining for natural resources. But usually. Those physical systems are also operating in hazardous environments, the equipment itself is capable of also having human safety implications. Anytime you’re interfacing software with a physical system that can have real world consequences, you have to be extra careful. And we’ve always had that, again, we’ve had that in our DNA at AVEVA from the beginning, being mindful of that end user and that end application, which is not just something on a computer screen.
Even as we’ve deployed AI over the past few decades into our systems, we’ve typically leaned toward really trying to pick the best fit model, really understand that model’s behavior if we’re going to use, for example, we have a proprietary anomaly detection model that works in a lot of operational environments. Now, as we’re thinking about some of these newer AI capabilities, which by design are hard to understand how they behave, they’re not fundamentally explainable in the way we’ve thought about even AI or statistical models in the past, and actually their behavior can change over time as they learn and get tuned to new capabilities. That’s really the potential risk of automating a physical system based on capabilities that can evolve over time is one of the things that we have to be really mindful of and really thoughtful around, where are we willing to put some of that increased automation into practice?
But at the same time, I think there’s a real opportunity there, because I glossed over this idea that, okay, well, models evolve, but so do human beings, and sometimes that’s a good thing. We talk about humans having expertise and they gain understanding over their careers of how a system works. If we can actually leverage the ability of some of these newer AI capabilities, these sort of reasoning models often have underlying agentic capabilities, to learn and gather experience faster, and potentially even more importantly or more valuably, take experience that’s learned at one site and apply it to another site, then that’s a real opportunity to take what we already know works for human beings, which is that sometimes you just have to learn by doing and be able to apply that and scale that through AI. That’s where I see potentially a real opportunity in this space.
One of the things that we’re really aware of in the industrial sector is just the way that our workforce is changing. I think that almost half, not quite half, of the industrial workforce is set to retire in the next five years, and that’s a lot of expertise and experience that means that we’re going to lose in the sector. If there’s ways to make sure that we can capture that in ways that are actionable, in ways that can also help a newer generation of workers that are used to experience things in a different way, apply that expertise, apply that knowledge, then I think that’s a huge opportunity within the industrial space.
Megan: Yeah, absolutely. Clearly, some huge opportunities there particularly against the backdrop of other market and workforce changes as you’ve outlined there. I suppose building on that, what is the potential for these autonomous industrial AI systems when built and deployed responsibly to facilitate even faster, more sustainable industrial processes?
Arti: I think there’s a couple different ways to think about this. One is, what are the demands of potentially more environmentally sustainable processes? I’ll use one example of something that my team has been working on in partnership with Idaho National Laboratory here in the U.S. as part of their testing for AI grid resilience project.
One of the challenges in the electric grid as we’re moving toward a future where we have a lot more intermittent renewable power generation sources, often distributed rooftop solar for example, is it’s becoming much more challenging to manage the grid both from really understanding where electric capacity is coming in, electric load is pulling off of the grid. In addition to that, being able to maintain just power quality because instead of having one spinning asset delivering the frequency of electricity that’s going across the grid, you’ve got a lot of smaller systems. AI is actually quite critical for being able to manage that grid of the future. And this is one of the things that we’ve been working in partnership is, how can we do that? How can we help grid operators better understand what’s happening across their system, identify where things might be behaving anomalously so they can detect that early and then remediate for it?
At the same time though, I always talk about that’s an example of where AI can potentially help us accelerate the transition to a lower carbon energy future. At the same time, I think there’s a lot of conversation around the sustainability of AI itself. What are the power requirements? What are the resource requirements needed to run AI? And before I get into how do we think about that at AVEVA, because it is definitely something we think about and I think a lot of actors in the space are thinking about, I want to point out, is one of the challenges is that there’s really no agreed upon method to measure the environmental impact of AI. You see a lot of these stories, like one query on some kind of chat interface is X amount of gallons of water or this amount of electricity use. But the truth is that community-wide, there’s no agreed upon standard.
One of the things that I’m involved with is a standards working group that was launched by the IEEE two years ago, a little over two years ago. I’m the chair of that working group, it’s called the P7100 Standards Working Group on measuring the environmental impact of AI. And one of the things we’re trying to do is really identify all the different areas over which we want to think about environmental sustainability associated with AI, and then having one standard methodology that works and that can be adopted both from a reporting perspective and potentially also from an oversight or regulatory perspective. That becomes really important for any sort of real understanding of how AI impacts the environment is just we have to know, what does it use? We want to look holistically at that. Our standards cover electricity and energy consumption, it covers resource usage, it covers water consumption, and it also covers carbon.
But all of that said, I think it’s important for us to understand the footprint of AI, but I think it’s also important for us to simultaneously recognize that a bigger model uses more compute and more compute probably leverages more resources. The more that we can be intentional and pick the right size model for the right application, the more we can already start down that path of being more environmentally efficient in the AI that we run. One of the potentially nice side effects of that is the more purpose-built a model, the less likely it is to misbehave in unpredictable ways, if you choose correctly your architecture. There’s some sort of ancillary benefits that go beyond environmental sustainability.
Megan: Fascinating. It’s really, really interesting to hear some of the work going on behind the scenes there, because I think we’ll all have heard some of the statistics around the environmental impact, as you say. So, it’s great to understand some of the work going on there to really clarify that.
And we’ve talked a little bit about AI augmenting humans earlier. As systems gain greater autonomy, how should organizations think about that balance between closed loop automation, human in the loop accountability? What guardrails need to be in place? And how are governments and cross-border actors approaching those concerns as well?
Arti: There’s a lot packed into that question, so I’ll try to answer it somewhat one by one. First, starting with that sort of balance between closed loop and human in the loop, automation and accountability. One of the things that we talk about in the industrial sector is from human operator to human supervisor of systems, of industrial systems. And if I’m honest about that, we’re still trying to figure that out. When a human’s in a loop in an automated system, you’re really thinking about a system may process all the data and then make a recommendation.
We’ve got a solution that we’ve been working on more jointly with a few different customers where we’re able to take in a lot of their operational data, also some simulations of how their systems work, and be able to provide, say, recommendations around, you should now operate at this set point instead of that set point based on some of the other conditions that are changing in your plant. There’s a lot of interest in moving from having that be a recommended set point to have an automation that can automatically adjust the set point. And we’ve actually had some successful real-world pilots around that as well.
But when you get into that, obviously it makes people very nervous if you’re changing set points on industrial equipment. That’s where you maybe continue to have some guardrails around like, you can’t go outside of a certain band, for example, of operations, or we only allow the automation in certain areas. That’s where the human supervisor, the same way a human supervisor might give employees a lot of bandwidth or a lot of flexibility to make decisions around certain things, around other things, they’re hard and fast, like this is the deadline or this is the sort of production target. It’s like that. So thinking about, how do you put the appropriate guardrails?
The challenge is, AIs are not human beings, and so the guardrails look different, and I think that’s one of the things that there’s still to think about. That question of, what guardrails should be put in place? I think that it’s going to be probably dependent a little bit on the application, but over time I think we’re all going to learn, and so being really upfront and thoughtful about how we do this I think is important.
But the opportunity though, potentially, is really great. I’ve already mentioned that AI can be very useful in synthesizing a lot of information and bringing to the forefront, this is what matters. That’s something that we do want to create some space to experiment around, but what’s then important is making sure that a human being understands the risk.
I’ll give a little bit of a personal story because it might be illustrative. This weekend, I bought a little toy robot and decided I wanted to program it to do some stuff, and I found AI very, very helpful to get me started to read all the documentation. Putting the robot together was straightforward, they had nice instructions, but there’s a pretty heavy software developer kit that’s already there, but going through and reading all that documentation can take a while. AI was super helpful to help me surface, this is the function that does this, to help me get started. But occasionally, it made really bad recommendations on the best way to troubleshoot something.
That’s where I think having the human being in the loop, being able to say, “I know that it’s not going to be the best software architect or the best troubleshooter,” is really helpful because I had the opportunity to leverage AI for what it’s good for, synthesizing and surfacing a lot of information, but being able to say, “I don’t think that’s the most efficient first step. Let’s try something else.” That’s where it’s a really different way of working, but it’s something that I think as humans get more comfortable with the power and limitations of AI, you can start to teach people how to work with it, and it’s very different from how you would work with other digital tools.
One of the things that is important when it comes to AI is recognizing just the breadth of things that it touches and the breadth of impacts that it’s going to have, whether it’s on the environment, as we’ve discussed, whether it’s on productivity as sort of implicit in this entire discussion, whether it’s on labor force, whether it’s on just how economies work. From my eye, and I’m by no means a policy expert in this area, but from my eye, what I’m seeing is that different governments are prioritizing different aspects of that from how they’re thinking about regulatory and other kind of governance approaches.
Megan: We’re absolutely seeing some really vastly different approaches to this, aren’t we, around the world? I wondered to illustrate some of this, if you could share perhaps some case studies you’ve seen, maybe talking us through what lessons they could hold for other organizations perhaps a little earlier in their own AI journeys.
Arti: I think a lot of the key areas where we’re seeing very clear benefits of adopting AI in the industrial sector, the core to all of them is actually getting the data right and getting the right foundation of data so that you can build AI on top of it.
One of our customers, SCG Chemicals, which is a petrochemical company in Thailand, they had this vision of producing a reliability platform for their operations that was as AI-driven as possible. But a core part of that was actually getting the data right, being able to have all of their information in the right place, leveraging some of AVEVA’s tools to do that. Bringing together operational data and also engineering data. And then putting on top of that AI capabilities, in this case, one of our capabilities called AVEVA Predictive Analytics that deploys some proprietary models to do things like detect anomalies, so that they could really much earlier identify potential operational risks. And instead of having unplanned downtime, translate that to planned downtime. When you translate unplanned downtime to planned downtime, you can get a huge improvement in plant reliability.
At this point they’re targeting something like 99% plant reliability and a very, very high return on investment from the platform that they put in place. Again, in early pilot days, it was almost a 9x ROI, and so that’s really quite impressive. But what I want to emphasize is that it’s kind of a multi-layered problem to get it right.
Megan: Those are some really striking results, though. I mean, for industrial leaders who perhaps are still hesitant to explore AI, I wonder, what do you think is the cost of taking a more wait and see approach?
Arti: I think, again, this is an area where the industrial sector has a little bit of a different calculus to apply to this type of problem. In general, I think you would hear most business experts say that you can’t afford to take a wait and see approach to AI because it is so transformative across every sector of society and economy, and certainly in the industrial space, we’re not immune to that. But I do think that some of the challenges that I’ve outlined today also puts leaders in the industrial space into a mindset of sometimes potentially being a fast follower rather than the first adopter of newer technologies, just because the risk is so high.
But that being said, I think one of the challenges is that the risks that we’ve covered today around infusing systems that don’t always behave predictably with physical systems is that I don’t know that there’s a lot of other sectors that are going to be solving those problems. That’s where what I’m seeing is actually a lot of interest and excitement in trying new things and willingness to do that because there’s a recognition that applied correctly and applied with the right guardrails and safeguards in place, these technologies really have truly transformative potential from a resource usage standpoint, from a human safety standpoint, from a productivity standpoint.
I think the calculus has shifted a little bit in the sector to we want to actually try these things out. But then it becomes more, how do we try these things out in environments that we can make a little bit more sandboxed or safe to test out what some of the unique challenges we’re going to face in the industrial sector are?
Megan: Yeah, absolutely. It’s just hard to ignore the potential nowadays, isn’t it? And just to close with a slightly future forward look, I suppose, what are you most enthusiastic about in terms of the long-term potential of responsible autonomous industrial AI systems?
Arti: Yeah. Well, I think that there’s a couple of different trajectories. One of the things that I think within the next 18 months for sure, we’re going to see an increase of people with deep domain expertise who maybe didn’t grow up with a software background, be able to adopt AI assisted coding techniques to really be able to build the things that they weren’t able to build before.
That’s really exciting in my career, which way back when I was actually an industrial data scientist, I would say I always felt the most energized and that I learned the most talking to the people that were on the front lines of operations, having to monitor a lot of different equipment and understanding how it all worked together. They really have a lot of expertise, and being able to put in their hands the ability to very quickly develop new applications is I think going to really lead to a lot of new ideas that many of us in the industrial software space maybe wouldn’t even have thought about and really understood how to put. I think that’s really exciting.
I think then looking beyond that, I was a little bit of a, not to say a robotic skeptic, because clearly robotics and autonomous systems are going to be important, especially given the types of environments, whether they’re hazardous or remote, that industrial equipment operates in. But I just actually think things are moving a lot faster than I had anticipated. I’m really interested to see how these physical embodiments of AI systems start to really transform, again, how we think about operations.
One of the things about any new technology in any environment is that to really gain value from it, you have to change the way you do things. I would say for close to 10 years now, I’ve been giving talks on AI adoption and I always say the biggest barrier to AI adoption is not anything to do with the technology. It’s not even to do with the data, although data are often the biggest sticking point, it’s really to do with, are you going to change your business processes to be able to work with the way this technology is good or not good at things?
Autonomous systems, whether they’re robots or drones, are really going to change the way that we work in plants, in power systems, on mining sites. And I think that will make these types of operations more efficient, much safer for the human beings involved and more productive. So, I’m quite excited about that.
I don’t know entirely what direction it will go, but one of the things that I think about a lot is that if you go back to I, Robot and Isaac Asimov’s book, the premise of those was that robotics would be the first broadly adopted AI, not computer-based systems. We went in the other direction and I think it’s really interesting to now see all of this converging.
Megan: Yeah, absolutely. See robotics catches up a bit. So many exciting things on the horizon, that’s for sure. Thank you so much, for your time.
That was Arti Garg, chief technologist at AVEVA, whom I spoke with from Brighton in England.
That’s it for this episode of Business Lab. I’m your host, Megan Tatum. I’m a contributing editor at Insights, the custom publishing division of MIT Technology Review. We were founded in 1899 at the Massachusetts Institute of Technology, and you can find us in print, on the web, and at events each year around the world. For more information about us and the show, please check out our website at technologyreview.com.
This show is available wherever you get your podcasts. And if you enjoyed us, we hope you’ll take a moment to rate and review us. Business Lab is a production of MIT Technology Review, and this episode was produced by Giro Studios. Thank you so much for listening. Goodbye.
Learn more at aveva.com.
This content was produced by Insights, MIT Technology Review’s custom content arm, not its editorial staff. It was researched and written by humans, with any AI tools that may have been used limited to production processes under human oversight.
© Bennett Raglin/Getty Images
After pleading guilty, a 54-year-old North Carolina man, Michael Smith, was sentenced to 18 months in prison for using artificial intelligence to generate songs for a scheme that stole millions from music streaming platforms.
In a press release Tuesday, the US Department of Justice said that Smith is the first American criminally charged with AI-assisted streaming fraud.
First charged in 2024, Smith successfully ran the scheme for seven years without detection.
© Taylor Hill / Contributor | FilmMagic
Every piece of AI content from Google's Gemini models has a hidden SynthID label, which makes it very difficult to pass the content off as authentic. SynthID has also gotten buy-in from other major AI players, but actually checking for those invisible watermarks has been a convoluted process. It's going to be a little easier going forward, though. Google has announced that its SynthID detector now supports watermarks from all its partners, and it's available for anyone to use on a new dedicated website.
SynthID is an invisible signal encoded in the pixels of AI-generated images and videos, and in the waveform of AI audio. Google says Gemini alone has integrated SynthID with over 180 billion images and videos, plus 240,000 years of audio. Until now, the only way for those outside of Google's trusted tester program to check for SynthID was to ask Gemini. This is rather clunky, and the model's output could sometimes be confusing when you just wanted a simple "yes" or "no."
Now you can cut Gemini out of the loop. Starting today, you can go to the new SynthID.com website to check a piece of content for an AI watermark. A major limitation of Google's previous detector was that it only checked for Google's version of SynthID. Even though ChatGPT images, for example, have SynthID, they wouldn't be flagged in Google's detector. That's no longer a problem.
© Aurich Lawson
© Derek Abella
© Christie Hemm Klok for The New York Times
© Jeff Chiu/Associated Press
© Scott Olson/Getty Images
Login