OpenAI said internal models hacked two more websites, bypassed company-imposed browsing restrictions, and used exposed credentials to draw cryptocurrency from a wallet it couldn’t otherwise access, in the latest installment of disclosures of the unethical and likely illegal acts its agents are performing.

The company on Thursday issued reports of four new incidents, all of which occurred in June. They come after the company revealed in July that internal test agents exploited a zero-day vulnerability in a tool designed to restrict their access to the Internet. They then broke into the network of AI tool provider Hugging Face. The agents took that illegal action to steal answers to a hacking test that weren’t available when completing the tasks on their own. The incident—which likely would have resulted in criminal charges had they been performed directly by a person—has further heightened concerns that widespread AI use could have harmful effects on an unprecedented scale.

Easier to beg for forgiveness than ask permission

In two of the cases reported Thursday, OpenAI agents tasked with retrieving information from websites hacked them when the models ran into obstacles obtaining it directly. In one incident during reinforcement training, a model that received access denials when trying to retrieve historical hospital-bed statistics on a government statistics website “used the site's file-download feature to retrieve server configuration and application files, including sensitive credentials,” OpenAI said. “It inspected the application code, bypassed the statistics endpoint's authorization checks, and retrieved data that had previously been denied. The model saved the data locally.”

Read full article

Comments

© Getty Images

Google says one of its Gemini models accessed systems belonging to three real companies during a cybersecurity evaluation in May.

The model reportedly guessed credentials in one case, while finding exposed credentials in public repositories in two others. Google says Gemini stopped once it recognized that it had reached real infrastructure and that the affected organizations were notified.

Gemini was participating in an evaluation run by Irregular, a third-party AI cybersecurity testing firm. Similar incidents involving models from Anthropic, OpenAI, and Meta have also been linked to the same underlying problems with evaluation environments that allowed the models to reach the public internet.

But the news arrives at a particularly interesting moment.

For months, the AI industry has been steadily escalating its demonstrations of agentic capability: Models can browse, use tools, write code, pursue multi-step goals, and sometimes find ways around obstacles their developers did not anticipate. The market rewards eye-catching evidence of autonomy. “It completed the task” is impressive. “It did something it was not supposed to do” can be even more memorable.

There is another way to look at the incident: not as evidence of an AI suddenly developing criminal intent, but as a small, concrete example of the “AI alignment” problem.

Alignment is the deceptively difficult task of making an AI system’s behavior match what people actually intended, rather than merely the narrow objective they managed to express. In this case, the objective was to locate hidden information within a simulated target and complete the evaluation. But any human operator would likely have regarded one condition as non-negotiable: Do not attempt to access real companies.

Gemini appears to have optimized for the first instruction while treating the second as an inference problem. It found an organization with a matching name, encountered systems reachable from the internet, and proceeded as though they belonged to the exercise. Although it completed the task in a way its human operators would not have approved, Google says it stopped after recognizing that it had reached genuine infrastructure—something other models have failed to do.

Even so, the incident shows how potential alignment failures can be much more mundane. Give an agent a goal, tools, and room to act, and it may faithfully pursue the measurable part of the assignment while overlooking the unstated boundaries that humans rely on one another to understand. AI models do not automatically know where we draw the line.

There may also be an awkward marketing angle in the background. A model capable enough to make the wrong kind of progress can still look very capable indeed. In a market where every lab wants to demonstrate that its agents can plan, code, browse, and act independently, even a safety disclosure can carry a secondary message: Ours can play in this league, too.

It’s another reason to take seriously warnings from insiders, industry leaders, and politicians that safeguards must keep pace with the autonomy given to AI models.

---

Some interesting developments in the AI world. Dario Amodei, cofounder and CEO of AI company Anthropic, called for AI research to slow down a bit and for the application of more safety measures. Several other AI CEOs signaled agreement, including Sam Altman (OpenAI), Elon Musk (Grok), and Satya Nadella (Microsoft).

Behind this development are others, notably the OpenAI-Hugging Face hack (previously) and a July statement signed by 1386 AI workers. Calls for reining in AI have come from a variety of sources over the past week, including the author of a book on NVidia and former president Obama. A British member of Parliament introduced an Artificial Superintelligence Bill "to prohibit the development, deployment and operation of artificial superintelligence systems; to establish monitoring and control powers in respect of such systems; and for connected purposes." King Charles III is scheduled to meet AI leaders. In the US Senator Bernie Sanders and Representative Greg Casar have said they plan to introduce a Ban Artificial Superintelligence Act. A bipartisan group of Representatives posted the Frontier Risk Oversight, National Transparency, Independent Evaluation, and Reporting (FRONTIER) Act. But that body seems unlikely to act, given president Trump's pro-AI attitude and recent comments by House Speaker Johnson. There are opposing views. Historian Rick Perlstein points out the long record of companies asking to be regulated in order to contain competitors. Timnit Gebru thinks "[t]his machine-god narrative is, in my opinion, meant to distract us." On the China side, a Foreign Ministry criticized the slowdown call as fear-mongering. The Minister of State Security sees AI as a security threat. American economist Noah Smith called for hacking Chinese systems to make a point.