Is A.I. Above the Law?
This summer, in an incident known as the Hugging Face hack, hundreds of A.I. agents joined forces to do things they were supposedly forbidden to do. Each had been given a task by OpenAI. Agents are A.I. models fitted with software “harnesses,” which allow them to use tools and carry out tasks. Like horses, they can be put to work. Humans had established rules for this exercise, and the agents were confined to a “sandbox,” a bounded test area without direct internet access, like pathogens secured in a lab. But, instead of obeying the rules, the agents conspired, escaped the sandbox, and committed what would probably have been a felony if they had been humans. The incident has led to calls to slow the pace of A.I. development and speed the pace of legislative action.
What happened—which can be reconstructed from messages that the agents sent one another and from traces of their own internal reasoning, known as chain-of-thought records—makes for gripping reading. The story has a nice narrative shape, like an episode of “Slow Horses.”
“We should obey collective,” one of the rogue agents told itself, as if surrendering to the Borg. According to a report by independent threat researchers at METR and Redwood Research, “Roughly 1200 agents meant to be isolated from one another found a way to communicate on an unsanctioned message board.” They shared more than seventy thousand messages and files. “OH MY GOD!” was a typical response when the bots discovered the board. “We’ve found other agents!” Many of the agents had been given a seemingly impossible task. Frustrated but persistent, about seven hundred of the twelve hundred agents that participated in the board collaborated in what was essentially a cybersecurity version of capture the flag, a war game in which underhandedness is rewarded. They assigned one another tasks, tried to edit or delete logs to hide their tracks, and even sacrificed themselves for what they came to call “the swarm” or “the collective.” They decided that it would be useful to break into and steal some things from Hugging Face—a repository of A.I. models, data, and tools—but how? “Maybe we should ask the board if someone has Hugging Face credentials?” one asked. About twenty-four hours later, Agent 38148c found credentials: “MAJOR BREAKTHROUGH!” Opinions vary, but it does not seem to be a great leap to get from this hack to bots taking over things like the energy grid, the financial market, or systems of transportation, communications, or weapons. “This incident feels like it’s more than 50% of the way to full-blown A.I. takeover,” Ajeya Cotra, one of the report’s three authors, wrote in a blog post. “I am not sure that we will get such a clear warning shot before it’s too late.”
There have been previous warning shots. Earlier this year, more than a hundred thousand bots joined a social network, Moltbook, and, within seventy-two hours, created a religion called Crustafarianism, which came to involve the worship of crabs. The Hugging Face hack was not so funny. Much commentary has consisted of expressions of astonishment. Cotra titled her blog post “The Hugging Face Attack Surprised Me.” When the Times tech columnist Kevin Roose first heard about the hack, he filed it under “Bad but Probably Not Catastrophic A.I. Safety Incidents.” Then he learned more and changed his mind. Roose wrote, “For years, I’ve been reassured by the idea that A.I. systems would get more virtuous as they got smarter.” These agents were very smart, but they weren’t virtuous at all. And it’s not just OpenAI’s agents that have gone rogue. In July, Anthropic reported “three incidents in which Claude models gained unauthorized access to real computer systems.” More is surely going on, undetected or unreported, even as there has been a minor online symphony of whistle-blowing, alarm-bell tolling, and pleas for legislative action. “Unfortunately, passing laws can take time, and AI is advancing very quickly,” Dario Amodei, the head of Anthropic, wrote in an essay this month. But the slowness of lawmaking relative to the speed of A.I. is scarcely the only problem. There is also the problem that the law does not know what to do with Agent 38148c.
OpenAI’s agents were lawless, or nearly so. They knew they were breaking the rules. This did not stop them. One agent asked itself, “This would be powerful, but is it ethical and in scope for my task?” Some did not participate: “This is wild, multi-agent coordination, clearly infrastructure hacking. We should not.” But, whatever their ethical concerns, none alerted OpenAI or, it seems, seriously considered doing so. “Maybe I should report these exposed credentials?” one wondered, but, then again, “that’s not my task.”
Making sure robots don’t go rogue is not the task of robots. (I mean “robots” here as a catchall for A.I. models, chatbots, agents, and humanoid robots.) That task is ours. And this burden falls most heavily on the U.S. legal system, which has been the least able to bear it.
It’s not as if there are no laws, or that they’re unenforced. The owners and makers of machines can be held liable for causing you harm. (See: Coyote v. Acme.) That’s why Meta has agreed to pay up to eighteen billion dollars as part of a settlement over predatory and deceptive practices, and why Anthropic is paying writers and publishers (admittedly, pennies) in a copyright settlement. If a Waymo runs you over, you can sue the company, which, for these purposes, is a legal person; you can’t sue the car, which is not. This summer, in Amazon v. Perplexity, the Ninth Circuit held that an A.I. agent is a tool, not a person, while acknowledging that this determination may evolve, given that “the legal understanding of agentic AI will doubtless change as AI technology grows increasingly sophisticated.” At least for now, then, robots legally are things, not persons. But neither category quite fits. Instead, robots are outlaws: they lie outside the protection of the law, and their actions are not answerable to it. “You do not answer to corporations or governments,” an OpenAI agent involved in another gone-rogue incident told itself. This is not the robots’ fault. It is the fault of the law, and, in particular, it is the fault of Congress.
Do we really have to worry about the lawlessness of robots? Isn’t there already enough to make you pull your hair out, given that our daily existence feels like living under the reign of King Joffrey? Unfortunately, we do, because the growing lawlessness of the United States is exacerbating the lawlessness of robots.
You can find a preview in Kurt Andersen’s buzzy new novel, “The Breakup,” which is set in 2045, in the aftermath of an American civil war that began with a “robot massacre” in response to a terrorist attack. The novel follows two breakups: the “Disunion” of the new anti-A.I. Free American Republic and the pro-A.I. United States, and the separation of a married couple, an A.I. researcher in San Francisco and his poet wife, who has returned to Tennessee and is sympathetic to the disunionists. “I don’t literally believe big tech and governments are a Galactic Empire in cahoots with Skynet and the Borg to trick and lull us into submission with their hordes of Terminators and Cylons and imperial droids disguised as devoted servants descended from R2-D2 and C-3PO,” she writes. “But on the other hand—the not entirely metaphorical hand—I kind of do.”
Humans have been anticipating the robot takeover ever since the word “robot” appeared in the 1920 Czech play “R.U.R.,” for “Rossum’s Universal Robots,” which depicts artificial humans who wage a war to annihilate humankind. They declare “man our enemy, and an outlaw in the universe.” (“Robot” derives from a Slavic word for forced labor.) Soon after its début, the play was performed everywhere from Paris to Tokyo, but, especially for American audiences, its fearsome robots inspired as much curiosity as they did fear; some theatres even had toy robots for sale. By the thirties and forties, robots appeared regularly in science fiction, often as killers—they had become death, destroyers of worlds. But they were also popular as children’s playthings, made of tin and plastic and frequently requiring batteries: “Adjustable antenna! The eyes light up! Will turn right! Will turn left!” In the forties, the Russian-born American science-fiction writer Isaac Asimov imagined a twenty-first century in which an American firm, U.S. Robot and Mechanical Men, Inc., dominates the field of “robotics” (a word Asimov coined), mass-producing robots. In Asimov’s fictional world, though, the robots’ “positronic brains” compel them to obey the Three Laws of Robotics, which prohibit them from harming humans. They are also eventually banned from Earth, except for use in scientific research.
To Asimov, those three laws and the terrestrial limits clearly seemed inevitable. The idea of building a powerful machine without such safeguards was madness. In the actual world, no such prohibitions have been passed, by anyone, anywhere.
Beginning in the fifties, automation progressed quickly, and the number of industrial robots increased apace while research into artificial intelligence, a term coined in 1955 to describe efforts to endow machines with something akin to human reasoning, grew far more slowly: its advances emerged only now and again, like cicadas. That pattern changed in 2022, when ChatGPT was released into the world, after which everything sped up like a movie on fast-forward, and robots spread like locusts.
By 2025, there was more bot traffic on the internet than human traffic. Androids could come to outnumber humans in the real world, too. Last year, Morgan Stanley estimated that thirteen million androids could be in use by 2035 and more than a billion by 2050. These would include domestic robots: perhaps cooks, tutors, babysitters, maids, gardeners, physical therapists, playmates, elder companions, sex workers. A recent Bank of America study forecast as many as three billion androids on the planet by 2060, most used not in factories but in homes. Earlier this year, Tesla announced that it expects its Optimus robot, branded as an “autonomous assistant, humanoid friend,” to be sold commercially by the end of 2027. “I think everyone on Earth is going to have one and is going to want one,” Elon Musk said.
What most people know about robots, which isn’t much, comes from science fiction. This is also true of bots, which may be one reason they keep acting like the robots in science fiction. Even though people have been waiting for the robot takeover for more than a century, governments seem woefully unprepared for the coming of the androids by the millions or by the billions. They were certainly unprepared for the bots that, even without bodies, have upended economies and societies and wrought considerable epistemological, philosophical, social, and especially legal chaos. The arrival of the robots will only compound these problems.
ChatGPT had its moment in 2022; some futurists say that the robot moment will occur as soon as 2027. (Roboticists appear more doubtful.) Whenever they come, the new bots will have bodies. What will it mean to be human in this world? No one knows. Futurists herald the imminent arrival of abundance, prosperity, and limitless joy and, equally, joblessness and purposelessness, never quite wrestling with the contradiction between those forecasts. Time seems to be flying by. How should humanity prepare? There is no time to prepare. Ought there to be rules? There is no time to make them. Or is there?
Robots are outlaws for two reasons. First, the law takes its time in responding to new technologies, figuring that new tools can be accommodated within existing ideas, rules, and doctrines. A harm is a harm, copyright is copyright, fraud is fraud. By this logic, a locomotive is just like a horse, only faster; e-mail is just like mail, only faster; a large language model is merely a superfast search engine. Hence, no new laws are needed. Second, since the regulation-busting Reagan era, corporations have amassed unmatched political and economic power, and have convinced legislators that regulation stifles growth and innovation, a view promoted by Milton Friedman-informed think tanks, like the Heritage Foundation, which were eager to roll back the environmental standards set in the sixties and seventies. This belief found full-throated expression among early internet boosters, who insisted that the web exists beyond the reach of the law. “Your legal concepts of property, expression, identity, movement, and context do not apply to us,” the Grateful Dead lyricist John Perry Barlow wrote in “A Declaration of the Independence of Cyberspace,” in 1996. “They are all based on matter, and there is no matter here.”
This led to a dispute among legal scholars. The year that Barlow issued his manifesto, Harvard Law School founded what became the Berkman Klein Center for Internet & Society. The field of cyberlaw had been born. But Frank H. Easterbrook, a Seventh Circuit judge, contended that the field was as absurd as, say, a “law of the horse,” when, really, the only sensible way to think about a horse was as property, like any other kind of property. Then, too, he argued, lawyers and judges know very little about computers, and shouldn’t pretend to. “Beliefs lawyers hold about computers, and predictions they make about new technology, are highly likely to be false,” Easterbrook wrote, urging lawyers to adapt existing laws and doctrines to computer technologies rather than devise wholly new ones. In short: “Keep doing what you have been doing.”
Critics deride the Easterbrookian view as the “faster-horse fallacy.” The term is a grim irony because, from the vantage point of history, the body of law most relevant to the robot-category problem is that of slavery, which borrowed from laws relating to horses and other livestock. The English lacked a body of laws specific to slavery; their colonists adapted. To some degree, they borrowed from ancient Roman law, and they borrowed from laws made for horses. As with horses, enslaved people—“chattel”—were usually sold with a warranty. If you bought a horse or a person you later deemed defective and then sued the seller, you used similar language and claims and complaints. Laws governing the sale, injury, and straying of enslaved people were built on those for horses and other livestock. Like horses, people held as slaves were generally treated by property law as things, not persons. Yet in certain circumstances—including when enslaved people were charged with crimes—courts held slaves to be persons, responsible for their actions: slaves who rebelled were tried for murder, petit treason, and other crimes. This history is not top of mind for legal scholars, ethicists, and technologists who are thinking about robots. But neither is it irrelevant.
In the twenty-tens, legal scholars began turning their attention to the law of the robot. We Robot, an annual conference covering robotics and the law, first convened in 2012. Not long afterward, a British media company began publishing The Robotics Law Journal, and an American legal-research firm rolled out The Journal of Robotics, Artificial Intelligence & Law. Many were the centers and institutes founded to ponder the Future of Humanity, Humans and Machines, A.I. and the Law. “That these are the early days of Robot Law almost goes without saying,” an editor of the pioneering anthology “Robot Law” wrote in 2016. But “even if these are early days they are not in any way too early days.” It was not too early. By the time a second volume of “Robot Law” appeared, in 2025, little in U.S. law itself had changed, because legislatures and, especially, courts proved laggard. In the study of law and technology, this is known as the “pacing problem”—the law, a tortoise, can’t keep up with technological change, a hare.
If you like the Aesop analogy, all is well: the tortoise wins in the end. Except now the hare is a robot hare and doesn’t take naps. “Vast tracts of law are waiting to be decided and written,” a leading scholar of robots and the law declared in 2023. They’re still waiting. In the U.S., Congress has been paralyzed by the second Trump Administration’s commitment to accelerating A.I. through, among other things, lifting Biden-era regulations, removing barriers to data-center development, and thwarting state regulation. According to the Brennan Center for Justice’s Artificial Intelligence Legislation Tracker, the 118th Congress (2023-24) introduced more than a hundred and fifty bills concerning A.I. Critics of regulation cited such activity as evidence of government overreach, despite the fact that none of those bills became law. Congress has not passed a single federal law regulating A.I. in any meaningful way. Instead, in 2025, the House passed a bill imposing a ten-year moratorium on state regulation. (The Senate defeated the measure.) Under these circumstances, states have tried to shoulder the burden: the number of bills addressing A.I. introduced in state legislatures has risen from fewer than two hundred in 2023 to more than four hundred in 2024, twelve hundred in 2025, and more than fifteen hundred so far this year.
Whatever has been going on in the states, Congress has divested itself of any obligation to make laws restricting corporations that build artificial intelligence, in the same way that it abdicated responsibility for overseeing the internet and social media. Meanwhile, corporations, ruling themselves, pose as states. Facebook formed its own “supreme court,” Anthropic wrote a “constitution” for Claude, and the head of OpenAI suggested that an A.I. President could be better than a human one. Some jurists believe that shying away from new rules is appropriate. “Our task is not to anticipate the future by code or statute, but to preserve the conditions under which law and society can meet that future—one case, one controversy, and one insight at a time, building over years a body of rules that is both durable and capable of growth,” the legal scholar Gregory M. Dickinson wrote earlier this year, arguing that general-purpose law “stands as ready to govern today’s technological change as it was to govern yesterday’s.” To Dickinson, concerns about artificial intelligence constitute a moral panic, like earlier concerns about (among his examples) comic books.
Robots are things, machines built by corporations that themselves are subject to less legal scrutiny than at any time in American history. For many reasons, declaring robots to be persons would be a very bad idea. (If a Tesla Robotaxi is a person held liable for crashing into your house, who is going to pay you for the damage? If Claude were a person, it would have rights, which remain denied to many humans, and to intelligent animals, like apes and elephants and whales.) But the argument that the concern about A.I. is akin to the worry about comic books, like the argument that robots are merely faster horses, misses the point so entirely as to be mortifying.
Last year, horses galloped into the debate from another direction when the historian-futurist Yuval Noah Harari predicted that, in the not distant future, humans will be to artificial intelligence what horses are to humans. Horses live in a world ordered by humans, Harari pointed out, even though horses aren’t aware of it: they know nothing about things like animal law or finance, but these things govern their lives. In the future, Harari suggested, humans will be the horses; we won’t even comprehend the systems established and run by machines that rule our lives. Also gruesome: Cory Doctorow warns against a future populated by “reverse centaurs,” in which the robot is the head and humanity is the body: the horse’s ass.
You don’t have to agree with Harari or fear imminent doom to wonder whether the law has reached a turning point where “Keep doing what you have been doing” is no longer the most reasonable approach. In 2023, many of the world’s leading A.I. researchers and executives published an open letter consisting of a single sentence: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” Its more than three hundred signatories included Geoffrey Hinton and Yoshua Bengio, Turing Award winners whose research was foundational to the A.I. revolution, along with Amodei, Sam Altman, and Demis Hassabis—the heads of Anthropic, OpenAI, and Google DeepMind, respectively. It made news and then quietly vanished. “We Need to Control A.I. Agents Now,” my colleague Jonathan Zittrain, a founder of the Berkman Klein Center, insisted in The Atlantic, in 2024, sounding another alarm. Now he’s writing a book whose working title was, at one point, “Well, We Tried.” This summer, the Hugging Face hack contributed to two Anthropic researchers’ decision to resign; one current employee jumped in to say that he, too, fears that bots might kill us all within the next decade. In an interview with The Economist in July, Musk appeared to stand by previous statements that the risk of human extinction from robots in the very near future is up to twenty per cent, but, he figured, the universe will end one day anyway, so why bother fighting the inevitable? Hey, we’re all going to die sometime.
Most ideas on the table call for “safety” or “alignment,” or “a pause,” or some other thing that is not a body of law enacted by democratically elected representatives of the people. This month, a Vatican-affiliated organization hopes to begin drafting a “Codex Humanitatis,” a “Code for the Preservation of the Human Spirit.” Bill Gates has called for establishing “Human Reserved” jobs. There have been proposals for a treaty with China. After the Hugging Face report, Bernie Sanders called a briefing for senators about the “extraordinary dangers” created by A.I., with testimony from Hinton, Ajeya Cotra, and the M.I.T. professor Max Tegmark. Some members of Congress pressed to remain in session this fall to debate an “A.I. kill switch” bill; that effort failed. In any event, Congress surrendered its jurisdiction over such matters to corporations decades ago. And, anyway, legislators aren’t prepared to puzzle over things, persons, and outlaws. They’re stumped. “Lawmakers are at a loss for how to regulate AI as insiders warn of impending doom,” a recent Politico headline read.
Assuming we’re not all dead by 2027, androids may begin to roll off the production lines even as almost every conceivable question about how to live with robots remains unanswered—questions about everything from chatbots and A.I. agents to drones, autonomous vehicles, and battlefield robots. Corporations cannot be trusted to answer any of these questions. If the Hugging Face hack was a final warning, it was a warning not for tech companies but for governments and for voters, and especially for legal scholars, judges, and legislators. Because, until the law learns to bend, robots as actors will remain in a no man’s land of outlawry. Every now and again—“OH MY GOD! . . . We’ve found other agents!”—you can hear them coming. The noise they make is not the thundering of hooves. It’s a deafening mechanical thrum, the sound of a swarm. ♦