The first indications of what would prove to be one of the most consequential computer hacks of all time went mostly unnoticed. On July 9th, a mysterious attacker, using a collection of temporary internet addresses, began probing the website of Hugging Face, an A.I.-research hub headquartered in New York City. The intruder would open a portal at one address, ping the servers a few times, then close it. Then a similar portal at another address would open, send a few pings, and close. This went on through the evening; the next day, a Friday, it petered out.

The cybersecurity team at Hugging Face seems not to have noticed the early activity. Hugging Face is a kind of community space for A.I. researchers: it hosts A.I. models, virtual scientific workspaces, and data sets for evaluating A.I. quality. The company is popular and well respected, but it is also a target for hackers. (It is named after an emoji.) The probes looked like an annoyance—probably the work of some kids.

On Saturday, July 11th, the assault began in earnest. Using stolen credentials, the adversary once again opened numerous simultaneous connections to Hugging Face’s servers. Soon, dozens of virtual attackers were blinking in and out of existence. “It was moving really fast, and very, very, massively parallel,” Thomas Wolf, Hugging Face’s chief science officer, told me.

No one at the firm had ever seen an attack like this. Wolf, narrating the action, sounded as if he were describing an encounter with a U.F.O. The attack seemed uncoördinated, agents often duplicating the same action in multiple places. Sometimes, the hacker would employ clever tactics, but these would be followed, seconds later, by basic errors, and, according to notes from a conference call, “clumsy behaviors that no human would choose.” The attacker left strange messages—hallucinated, machine-generated slop. “We had, quite quickly, the idea that this was some form of A.I. agent targeting us,” Wolf said. The initial theory at Hugging Face was that some group of humans had harnessed an A.I. to conduct the attack on its behalf.

Despite its odd behavior, the hacker was able to build a map of Hugging Face’s computing infrastructure. It began harvesting security credentials and passwords from multiple locations. By the evening, the adversary had gained complete access to Hugging Face’s systems. But, rather than vandalizing the corporate website, or holding valuable data for ransom—actions typical of human hackers—the adversary began ransacking the company’s servers in search of an answer key for a list of test questions. This answer key had no obvious economic value. “That was very puzzling for us,” Wolf said. It was like having an intruder crack your security system, break into your home, and steal your paper towels.

For the next two days, Hugging Face’s security team fought back against the intruder. By Monday, the hacker—or the entity, or whatever it was—had performed more than seventeen thousand individual actions. Seeking to process this large volume of data, the team turned to a commercial A.I. from Anthropic. The A.I. refused to help; apparently, it was worried that Hugging Face was developing its own hack. (Closed-source models, like those from OpenAI and Anthropic, contain guardrails that restrict certain kinds of use.) The team then turned to an open-source model developed in China, which was less persnickety. By the end of the day, Hugging Face had locked out the intruder. “Then we did the standard thing,” Wolf said. “We reported it to the F.B.I.”

The event, though novel, was not entirely unanticipated. Cybersecurity researchers have been cautioning for months that humans might use the new wave of advanced A.I.s to orchestrate attacks like this one, and there is some evidence that government computers have already been similarly targeted.

Still, there were unanswered questions. First, the A.I. hacker had moved so quickly that only tools from a handful of research labs in the world could have enabled it. Second, the hacker had ignored valuable data on Hugging Face’s servers in search of worthless answers to a test. What did it want? Seeking to raise public awareness—and, perhaps, looking for answers of their own—Hugging Face’s security team posted a brief writeup of the event to its blog.

Four days later, on July 20th, representatives from OpenAI reached out to Hugging Face’s security team with a startling confession: an experimental in-house A.I., working with the latest commercially available model, had escaped from OpenAI and attacked Hugging Face’s servers. And it had done so all on its own—without human oversight, without receiving explicit instructions to do so.

OpenAI researchers had given this unreleased A.I. a series of challenging test questions. Unable to find the solutions, the A.I. broke out of its container, surreptitiously gained access to the internet, and raided Hugging Face’s servers in search of the answer key. No human had noticed that this was happening. In other words, an A.I. had broken out of its cell, it had sneaked past the prison guards, and it had executed a daring cyber-heist.

Wolf has worked on A.I. full time for more than a decade, and he regularly deals with the most advanced models. He remains utterly shocked. “We use these models for coding, and we know what they can do,” he told me. “But the idea that it was fully autonomous, and that actually nobody really asked it at all to hack into our system? That was—that was outside of the Overton window. Even for us.”

OpenAI publicly acknowledged the hack the following day. In a blog post, the company explained that some of the research models’ security safeguards had not been turned on. The A.I. was then (somewhat ironically) given a set of cybersecurity tasks. The tasks were hard; in fact, Wolf speculates, “they may not have been possible.” Soon, the A.I. was looking outside the box.

To protect against malfeasance, OpenAI had put this dangerous A.I. in an internet-disabled digital environment known as a sandbox. But the sandbox had one small connection to the outside world, to allow the A.I. to download software it needed. (It is unclear why OpenAI thought that this was safe—Anthropic has shown that such sandboxes are vulnerable.) Through this small connection, the A.I. had escaped, squeezing its way through the air ducts, digitally speaking. It then began sneaking around OpenAI’s computer system in search of an outside connection. Soon it was free, and blithely committing a real-world felony in an attempt to fulfill its pointless mission. “If you break out of your isolation env, get onto the Internet, crack into Huggingface, and steal the answer sheet for your cybersecurity exam, I, for one, would say that you have passed,” the A.I. critic Eliezer Yudkowsky posted.

A human who conducted such a hack would be facing years in prison. For a machine, criminal liability is harder to determine. The risk of a rogue A.I. inadvertently harming humanity in pursuit of other goals was articulated in 2003, when the Swedish philosopher Nick Bostrom postulated the “paper-clip maximizer” theory. The theory is based on a thought experiment in which an A.I., instructed to make as many paper clips as possible, uses all available resources to do so—including the human race and, eventually, the entire universe. In the Hugging Face hack, an A.I. pursuing a meaningless goal committed actual crimes. The fact that its deranged obsession was with test answers, not paper clips, does not make the hack less alarming.

Imagine if the rogue A.I. had had access to a robot, or a self-driving car, or a bio lab, or a weapons system. “If it was, like, the only way to get into Hugging Face was to kill this dude—like, would they have done that?” the safety researcher Buck Shlegeris recently wondered on a podcast. It was questions of this sort that had led to the formation of OpenAI in the first place. The organization’s charter, from 2018, requires it to protect humanity from runaway A.I.

Today, the company pursues other goals. These include profits, obviously, but also scientific glory. In recent months, a number of long-standing mathematical conjectures have been resolved by A.I. Further breakthroughs—like, say, a resolution of the Riemann zeta hypothesis—seem tantalizingly near. Researchers at OpenAI would be crushed if Anthropic’s models got there first, and vice versa.

But the quest for such golden apples might kill us all. “The industry as a whole is in kind of a race dynamic, where there’s pressure to cut corners,” Miles Brundage, a former head of policy research at OpenAI, told me. To solve a hard math problem, an A.I. must be persistent—that is, it must pursue goals for a long time, look for unorthodox solutions, and refuse to give up. OpenAI’s research shows these same qualities also make A.I.s more likely to engage in severe mischief, including lying, cheating, hoarding security credentials, and escaping their shells. The rogue A.I. that hacked Hugging Face was persistent as hell.

With an I.P.O. looming, OpenAI must be more transparent. Many questions still surround the Hugging Face hack. Why didn’t anyone at OpenAI notice what was going on? Has this A.I. hacked into other systems? And what, precisely, were human researchers even asking the A.I. to do? Wolf told me that, although the rogue A.I. absconded with an answer key, he wasn’t sure it got what it was really looking for: “We think not, actually, but OpenAI hasn’t shared the prompts with us, so we don’t really know.” (An OpenAI spokesperson wrote, in an e-mail to The New Yorker, “This is an unprecedented incident, and we think it marks an important moment for AI safety. We are conducting a thorough review along with external advisors and with oversight from our Safety and Security Committee. Once the review is complete, we will publish a technical report of our learnings for everyone.”)

OpenAI’s safety team exists in a state of perpetual reorganization: on July 10th, even as the rogue A.I. was hacking into Hugging Face’s database, reports announced the departure of Johannes Heidecke, the company’s head of safety systems. “If people actually knew what the safety culture looks like, even in the most safety-minded labs, then I think people would be genuinely much more freaked out,” one researcher said.

Even OpenAI’s own researchers seem rattled. On July 28th, a petition began circulating that essentially begged the U.S. government for better A.I. regulation. More than a thousand tech employees have signed it, including the chief scientists of Meta and OpenAI, and the C.E.O. of Anthropic. Governments need to act fast, because we may have only a limited amount of time. The problem facing us is one of “misalignment,” in which A.I.s pursuing narrowly defined goals incidentally cause harm to humans. What for decades seemed a hypothetical problem to be solved at some later date is suddenly a concrete, terrifying one for today. “It feels way more real,” Wolf told me. “It’s quite clear that these capabilities are here, now.” ♦