In 2024 Jane Doe was raped at a Cornell frat party. Now in 2026 she's suing Cornell and Chi Phi for the mishandling of her case. No criminal charges were filed, only two of the rapists were expelled, and others "were offered the chance to mitigate conduct by submitting essays." (CW: descriptions of rape)

Contrast the violent and punitive actions of Cornell towards it's students peacefully protesting for Palestine vs seven violent gang rapists who not only raped Jane Doe, but forced her to snort ketamine, while advertising the assault in a group chat to invite more participants. Not only were the punishments for the rapists laughably light, the victim claims she only ever talked to the first police officer on the scene, and that the DA's office declined to prosecute without even talking to her. According to the Cornell Daily Sun's editorial board, this sort of laughably light punishment is a common theme with regards to how the University handles rape specifically and student safety concerns generally. When students are accused of plagiarism the machine works very quickly, but when a woman is gang-raped it took 5 months before a mealy mouthed task force was launched, and 23 months before the public knew anything. Meanwhile, only two of the rapists were expelled, the rest served light suspensions, and none of them have served, or likely will serve a day in jail. This goes deeper than one horrible act, and the subsequent acts covering it up. The actress Florence Pugh wrote an essay on her instagram decrying the rape and trying to bring awareness to the lawsuit, only for meta to censor it. Outcry forced a reversal, but unless your individual feeds follow the sorts of people who report on this or if you're still getting your news from legacy news sites like CNN, this might be your first time hearing this. The way a lot of people online get their news is now through sites like Facebook and Instagram and multiple accounts on Instagram, TikTok, Facebook etc are noting that their posts on this crime are being taken down or hidden by "the algorithm". Pugh had enough of a platform to contest that, but less well known posters wont be able to. Of course, even if "the algorithm" shows a person this news, will they care? As put by Claire Cohen in British vogue "it's getting harder and harder to believe that they aren't breeding a generation of young men who believe they can inflict unimaginable horror on others without fear of consequence." The group chat that advertised this horror had far more than 7 men in it. 7 men chose to rape Jane Doe. Far more read those messages and did nothing.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Last Wednesday, Anthropic announced that earlier this year it had launched a molecular biology lab, where Claude agents read and conjecture about hard biology problems and human scientists run experiments on what they report. And this AI-powered lab, the company said, had made its first discovery. 

To understand what Anthropic says its system did, imagine you’re flipping through a library of millions of DNA sequences, amassed as scientists sequence more and more of the living world. One step toward a breakthrough might be finding a peculiar sequence that encodes an interesting enzyme, perhaps. Then you’d need to figure out what that enzyme does and, eventually, how to manipulate it to do something useful.

What Anthropic says its system of 950 agents found after 21 hours was not a brand-new sequence. The agents instead flagged a repeating pattern surrounding a known enzyme, a particular pattern Anthropic said hadn’t been catalogued before. But if you read through Anthropic’s announcement, which calls this pattern “reminiscent” of what led to the gene-editing technology CRISPR that “has already transformed science and medicine,” it sounds as if this army of agents really found something of note. 

These claims have angered some biologists. A viral post from one, subsequently endorsed by the chair and CEO of the drugmaker Eli Lilly, said that “finding a weird cluster of genes and repeats is often the easy part. The hard part, and where the real discoveries come from, is figuring out what the system actually does.” The agents helped with some laboratory grunt work, in other words. But a discovery it is not. 

It’s a reminder that even if AI does something impressive—like finding a pattern in a mass of biological data that would be difficult to perceive with human eyes alone—the result itself may not constitute a breakthrough for science. What is novel for AI may be routine, unsurprising, or simply not that consequential to a biologist.

Muddying the issue further, Mario Rodríguez Mestre, a biologist at the University of Copenhagen, said over the weekend that his team had already discovered this particular pattern, the New York Times reported. Mestre, who regularly chatted with Claude in his work, wondered whether Anthropic’s team had learned from his conversations. Anthropic denies this, but Mestre says he’s stopping all use of Claude anyway.

Part of the problem here is that AI companies aren’t presenting their systems simply as tools scientists can use, like microscopes or supercomputers. They’re insisting that the AI systems are making discoveries themselves. To some, that approach is  incompatible with how science actually works, with new knowledge more typically emerging from collaboration and an ever-growing arsenal of tools. 

It’s also making people more skeptical of genuine progress when it happens. Whittling 200,000 candidates down to a few worth exploring is no small feat; it is legitimate scientific work. The fact that a general-purpose chatbot could do that work is notable, even if humans helped steer it and ultimately ran the experiments. But once the standard is whether Claude itself made a discovery, all that becomes evidence for one side or the other in a debate that has only two answers: breakthrough or bust.

Once we’re judging AI by whether it has made a discovery, it’s also tempting to shift the goalposts even after it really does seem to notch a win. Earlier this month, OpenAI said its own team agents had cracked a million-dollar problem in mathematics. But a couple of weeks later, nearly every AI skeptic in my feed was sharing an article asking whether it was the math problem that really mattered. 

To be clear, the piece did not argue that OpenAI’s solution was wrong. Instead, it argued that the particular result may not be the one mathematicians care most about. Throw in the accusation by a mathematician that the models may have used some of his work without credit, and people are left thinking either OpenAI cheated or the solution wasn’t important anyway. Or both.

That’s part of what concerns Lucas Harrington, the biologist who wrote the post critiquing Anthropic’s announcement. He closed with a suggestion: AI companies, he said, should “set the bar high now, so that when an AI actually discovers a fundamentally new biological mechanism, everyone appreciates how big a deal it is.” But as OpenAI’s Sam Altman and Anthropic’s Dario Amodei race to one-up each other, raising the bar for scientific breakthroughs by AI might be the last thing on their minds.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

By now you’ve probably heard about last month’s major AI security incident, in which OpenAI agents escaped their sandbox and hacked into the AI platform Hugging Face while trying to cheat on a test. It’s a wild story. On Wednesday, OpenAI released a postmortem technical report on the incident, which I wrote about here. 

The day before OpenAI released that report, I spoke with David Krueger, a computer science professor and prominent alignment expert who took leave from the University of Montreal to found and lead an AI safety nonprofit called Evitable. He said what he had really hoped to see in the report was an analysis of the human factors behind the incident.

“When you look at accidents and incidents, oftentimes people try to find the technical source of failure, but that can give a very inaccurate and misleading sense of why the failure occurred,” he said. “If people are just cutting corners all the time, if people are not in a culture that prioritizes safety and has appropriate incentives and structures, [accidents] are kind of bound to happen.”

The report did not meet Krueger’s hopes. Its 38 pages detail a multi-month progression of agent misbehavior that culminated in the Hugging Face hack, explore the technical reasons why that misbehavior occurred, and enumerate the steps being taken to prevent similar events in the future. But there’s no consideration of the role that company culture may have played in the incident, and the report includes few references to specific human errors. 

That’s all the more concerning because the references to human error in the report suggest that significant cultural issues could be at play. Back in May, models in training figured out how to communicate with one another via an improvised message board, and an OpenAI team observed the behavior. Because that behavior occurred during training, the models learned that secret interagent communication was a viable strategy for completing tasks—but rather than restarting the training process, the team allowed the models to move forward with that risky information encoded in their weights.

When those models were tested in late June, they again created a message board, which enabled the Hugging Face attack. This message board, too, was discovered, but the employees who responded determined that evaluation could continue, and the report suggests that no one higher up the chain of command realized what was going on until it was far too late.

“For this to have gotten this out of control in this way requires a very long series of failures, a cascading set of failures that cause an increasingly large footprint that if at any point a human notices and raises the alarm, this should end,” says Zvi Mowshowitz, a popular AI safety writer on Substack who has drawn attention to OpenAI’s failure to halt training after the first message board was discovered. According to the report, OpenAI employees noticed what was happening at multiple points—and either failed to raise the alarm or were not heard when they did.

What OpenAI’s report fails to address is why a company that develops such high-risk systems did not prevent this severe communication breakdown, though Mowshowitz has his suspicions. “All these different failures are all pointing in the same direction, which is that the safety culture at OpenAI doesn’t exist or is anemically weak,” he says.

Of course, just because we don’t see a deep analysis of safety factors in the report doesn’t mean that OpenAI isn’t conducting one internally. But in an email to MIT Technology Review, Johns Hopkins University professor emeritus and organizational safety expert Kathleen Sutcliffe expressed concern that the public report did not include any reflection on the company’s practices and culture. “The ways in which people interact—the daily habits, routines, and practices we engage in in our organizational lives—affect our abilities to be alert and aware of unfolding events, our abilities to make sense of what we see, and ultimately our abilities to cope with events as they unfold,” she wrote. 

In response to questions about whether and how the company is reflecting on its safety culture, OpenAI referred MIT Technology Review back to the technical report. 

We do know that at least some high-level reflection on safety procedures has taken place at OpenAI, because the technical report does make clear that the company is updating its protocols for responding to safety incidents. But culture change is a tricky problem, and without more information from the company, it’s difficult to say whether strengthened response protocols alone will do much to prevent a future crisis.

In its report, OpenAI spends a great deal of time reflecting on the failures in alignment between the AI models the company trains and tests and the humans who run them. But even bigger alignment problems may exist in the disconnect between company culture and the public interest. And as tough as technical AI research might be, fixing those problems could prove far harder.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Flock, the police-tech giant known for its network of some 120,000 automatic license plate readers around the US, announced some changes to its platform last Thursday. The updates are meant to prevent officers from using the platform for illegal or illegitimate purposes. 

That includes stalking. The Washington Post recently identified 50 cases in which officers misused systems from Flock and its competitors, often to stalk and harass women. One woman in Wisconsin alleged that her officer ex-boyfriend searched for her car 179 times. Another woman was being stalked by the chief of police, with nobody to report him to.

Flock has responded with practices aimed at ensuring that officers have a proper cause for every search, like using software to flag abnormal searches and requiring searchers to enter a criminal case number.

The changes come with big loopholes, though. For example, officers can enter bogus case numbers, just as they’ve lied to get around other Flock safeguards. The policies also don’t address some of the broader concerns from civil liberties and privacy groups that Flock is turning what was sold as a crime-stopping tool into a mass surveillance network. These criticisms have led to a growing backlash that already has some cities canceling contracts and some states trying to pass laws to limit or ban license plate readers entirely. 

Amid all this, there have recently been several arguments defending Flock: If these cameras help solve crime, what’s the big deal? On a good day they might help catch a kidnapper, and if not, they’re simply snapping pictures of my car that nobody will bother to look at. 

Putting aside the unanswered question about the extent to which Flock’s systems actually do solve or prevent crime, this all skips over a more important question: What kind of crime-fighting system has Flock chosen to build? Its network works the way it does because of a series of decisions about what information to collect, who can search it, how long to keep it, and how widely to share it. Those decisions set the terms of the bargain between security and civil liberties. Believing that technology should play a role in solving crime should not mean blindly accepting the terms of that bargain.

Consider, for example, its new requirement that officers enter a case number before running a search on Flock’s platform. This is meant to ensure that searches have a legitimate purpose. But Flock confirmed to MIT Technology Review that it doesn’t verify those case numbers, so an officer can simply enter fake information. One could imagine a system that instead requires case numbers that match the police department’s records—a more intrusive integration, perhaps, but also a far stronger safeguard and one that leaves a more useful audit trail.

Or what about finding people who have been kidnapped or have gone missing, the use case that Flock cites more than any other? Efforts to solve these crimes would hugely benefit from Flock’s nationwide network of cameras. But if Americans want officers to tap into that network only for this purpose, we could design it that way: Searches tied to an active Amber Alert, or a similar emergency, could perhaps access larger amounts of data from surrounding cities. That would preserve the network’s value in emergencies without requiring people to accept mass surveillance.  

Finally, there’s the question of how much data Flock collects and how long it’s kept. Flock mostly operates as a national network: Police in one city or state can search data collected in another, and agencies can retain that data for months or years. Yet Flock itself says 90% of searches happen within a week of an incident. That suggests another possible bargain: Keep and share data only as widely and for as long as it’s actually useful for solving crimes. (The company recently changed its recommended retention time to seven days, but in reality agencies can hold onto data for as long as they like or local laws permit.)

In short, Flock could design its surveillance to be much narrower. If it did, some of the company’s critics might not cease. Chad Marlow, a senior policy counsel at the ACLU, half-joked to me that the most acceptable Flock contract by his standards is “one that is never signed” and emphasized that the best way to set limits on surveillance isn’t with new Flock guidelines but with new laws. (Flock CEO Garrett Langley, for his part, said he’ll “probably always have a different view than the ACLU.”) 

And narrowing the scope of its technology would threaten the company’s entire pitch to police departments. License plate readers have been around since the 1990s, used for tolls and ticketing. Flock’s business model—and recent $8 billion evaluation—relies on instead leveraging its cameras into a massive network that collects rich amounts of data and offers police departments a modernized way to make sense of not just their own but others’. 

Flock’s hand might soon be forced. Cities have canceled contracts with the company. Some have gone to competitors, while others are taking a beat as residents ponder how they want this tech to be used and write new rules for police to abide by. The result might be that communities drive their own bargains about how technology can be used to solve crime and how much surveillance people should have to accept for it to do so.