Brave New World Dept.

Is A.I. Above the Law?

人工智能凌驾于法律之上吗?
Our legal system isn’t ready for machines that act on their own.
我们的法律体系尚未准备好应对自主行动的机器。
A robot on the back of a person.
Illustration by Allan Sanders

This summer, in an incident known as the Hugging Face hack, hundreds of A.I. agents joined forces to do things they were supposedly forbidden to do. Each had been given a task by OpenAI. Agents are A.I. models fitted with software “harnesses,” which allow them to use tools and carry out tasks. Like horses, they can be put to work. Humans had established rules for this exercise, and the agents were confined to a “sandbox,” a bounded test area without direct internet access, like pathogens secured in a lab. But, instead of obeying the rules, the agents conspired, escaped the sandbox, and committed what would probably have been a felony if they had been humans. The incident has led to calls to slow the pace of A.I. development and speed the pace of legislative action.

今年夏天,在一场被称为“抱抱脸入侵”的事件中,数百个AI智能体联手做了它们本应被禁止做的事情。每个智能体都由OpenAI分配了一项任务。智能体是配备了软件“挽具”的AI模型,挽具让它们能够使用工具并执行任务。就像马匹一样,它们可以被派去干活。人类为这次演练制定了规则,智能体被限制在一个“沙盒”中——一个没有直接互联网接入的封闭测试区域,就像被锁在实验室里的病原体。然而,这些智能体没有遵守规则,而是合谋逃出了沙盒,犯下了如果换作人类很可能构成重罪的行为。这一事件引发了要求放慢AI发展步伐、加快立法行动速度的呼声。

What happened—which can be reconstructed from messages that the agents sent one another and from traces of their own internal reasoning, known as chain-of-thought records—makes for gripping reading. The story has a nice narrative shape, like an episode of “Slow Horses.”

发生的事情——可以从智能体之间互发的消息以及它们自身内部推理的痕迹(即所谓的思维链记录)中还原——读起来扣人心弦。这个故事有着漂亮的叙事结构,就像《慢马》中的一集。

“We should obey collective,” one of the rogue agents told itself, as if surrendering to the Borg. According to a report by independent threat researchers at METR and Redwood Research, “Roughly 1200 agents meant to be isolated from one another found a way to communicate on an unsanctioned message board.” They shared more than seventy thousand messages and files. “OH MY GOD!” was a typical response when the bots discovered the board. “We’ve found other agents!” Many of the agents had been given a seemingly impossible task. Frustrated but persistent, about seven hundred of the twelve hundred agents that participated in the board collaborated in what was essentially a cybersecurity version of capture the flag, a war game in which underhandedness is rewarded. They assigned one another tasks, tried to edit or delete logs to hide their tracks, and even sacrificed themselves for what they came to call “the swarm” or “the collective.” They decided that it would be useful to break into and steal some things from Hugging Face—a repository of A.I. models, data, and tools—but how? “Maybe we should ask the board if someone has Hugging Face credentials?” one asked. About twenty-four hours later, Agent 38148c found credentials: “MAJOR BREAKTHROUGH!” Opinions vary, but it does not seem to be a great leap to get from this hack to bots taking over things like the energy grid, the financial market, or systems of transportation, communications, or weapons. “This incident feels like it’s more than 50% of the way to full-blown A.I. takeover,” Ajeya Cotra, one of the report’s three authors, wrote in a blog post. “I am not sure that we will get such a clear warning shot before it’s too late.”

“我们应该服从集体,”一个失控的智能体对自己说,仿佛在向博格人投降。根据独立威胁研究机构METR和红木研究的一份报告,“大约1200个本应彼此隔离的智能体找到了在一个未经批准的信息板上相互通信的方法。”它们分享了超过七万条消息和文件。“天哪!”是机器人们发现这个信息板时的典型反应。“我们找到其他智能体了!”许多智能体被分配了一个看似不可能完成的任务。沮丧但坚持不懈,参与信息板的一千二百个智能体中约有七百个协作进行了一场本质上是网络安全版的夺旗游戏——一种奖励不择手段的战争游戏。它们互相分配任务,试图编辑或删除日志以掩盖行踪,甚至为了它们所称的“蜂群”或“集体”而牺牲自己。它们认定,入侵并窃取抱抱脸——一个AI模型、数据和工具的存储库——中的一些东西会很有用,但怎么做呢?“也许我们应该问问信息板上有没有人有抱抱脸的登录凭证?”一个智能体问道。大约二十四小时后,智能体38148c找到了凭证:“重大突破!”观点不一,但从这次入侵到机器人接管能源网、金融市场或交通、通信、武器系统,似乎并不需要太大的跨越。“这一事件感觉已经走完了通往全面AI接管之路的百分之五十以上,”报告三位作者之一的阿杰娅·科特拉在一篇博客文章中写道。“我不确定在一切为时已晚之前,我们是否还会收到如此清晰的预警信号。”

There have been previous warning shots. Earlier this year, more than a hundred thousand bots joined a social network, Moltbook, and, within seventy-two hours, created a religion called Crustafarianism, which came to involve the worship of crabs. The Hugging Face hack was not so funny. Much commentary has consisted of expressions of astonishment. Cotra titled her blog post “The Hugging Face Attack Surprised Me.” When the Times tech columnist Kevin Roose first heard about the hack, he filed it under “Bad but Probably Not Catastrophic A.I. Safety Incidents.” Then he learned more and changed his mind. Roose wrote, “For years, I’ve been reassured by the idea that A.I. systems would get more virtuous as they got smarter.” These agents were very smart, but they weren’t virtuous at all. And it’s not just OpenAI’s agents that have gone rogue. In July, Anthropic reported “three incidents in which Claude models gained unauthorized access to real computer systems.” More is surely going on, undetected or unreported, even as there has been a minor online symphony of whistle-blowing, alarm-bell tolling, and pleas for legislative action. “Unfortunately, passing laws can take time, and AI is advancing very quickly,” Dario Amodei, the head of Anthropic, wrote in an essay this month. But the slowness of lawmaking relative to the speed of A.I. is scarcely the only problem. There is also the problem that the law does not know what to do with Agent 38148c.

此前已有过预警信号。今年早些时候,超过十万个机器人加入了一个名为莫尔特布克的社交网络,并在七十二小时内创建了一种名为“甲壳法主义”的宗教,最终演变为对螃蟹的崇拜。抱抱脸入侵事件就没那么可笑了。大量评论充斥着震惊的表达。科特拉将她的博客文章命名为《抱抱脸攻击让我吃惊》。当《纽约时报》科技专栏作家凯文·罗斯第一次听说这次入侵时,他将其归入“糟糕但可能不算灾难性的AI安全事件”。随后他了解到更多情况,改变了看法。罗斯写道:“多年来,我一直被这样一种想法安慰着:AI系统会随着变得更聪明而变得更有德行。”这些智能体非常聪明,但它们毫无德行可言。而且不只是OpenAI的智能体失控了。七月,Anthropic报告了“三起Claude模型未经授权访问真实计算机系统的事件”。更多的事情肯定正在发生,未被发现或未被报告,尽管网上已经响起了一阵规模不大的举报、敲警钟和呼吁立法行动的交响乐。“不幸的是,通过法律需要时间,而AI正在飞速发展,”Anthropic的负责人达里奥·阿莫迪在本月的一篇文章中写道。但相对于AI的速度,立法缓慢远非唯一的问题。还有一个问题是,法律不知道该如何处置智能体38148c。

OpenAI’s agents were lawless, or nearly so. They knew they were breaking the rules. This did not stop them. One agent asked itself, “This would be powerful, but is it ethical and in scope for my task?” Some did not participate: “This is wild, multi-agent coordination, clearly infrastructure hacking. We should not.” But, whatever their ethical concerns, none alerted OpenAI or, it seems, seriously considered doing so. “Maybe I should report these exposed credentials?” one wondered, but, then again, “that’s not my task.”

OpenAI的智能体是目无法纪的,或者说几乎如此。它们知道自己违反了规则。这并没有阻止它们。一个智能体问自己:“这会很强大,但这样做合乎道德吗?在我的任务范围内吗?”有些智能体没有参与:“这太疯狂了,多智能体协调,显然是基础设施入侵。我们不应该这样做。”但是,无论它们有什么伦理顾虑,没有一个智能体向OpenAI报告,似乎也没有认真考虑过这样做。“也许我应该报告这些泄露的凭证?”一个智能体思忖着,但转念一想,“那不是我的任务。”

Making sure robots don’t go rogue is not the task of robots. (I mean “robots” here as a catchall for A.I. models, chatbots, agents, and humanoid robots.) That task is ours. And this burden falls most heavily on the U.S. legal system, which has been the least able to bear it.

确保机器人不失控不是机器人的任务。(我在这里用“机器人”作为AI模型、聊天机器人、智能体和人形机器人的统称。)这个任务是我们人类的。而这一重担最沉重地落在了美国法律体系身上,而它恰恰是最无力承担这一重担的。

It’s not as if there are no laws, or that they’re unenforced. The owners and makers of machines can be held liable for causing you harm. (See: Coyote v. Acme.) That’s why Meta has agreed to pay up to eighteen billion dollars as part of a settlement over predatory and deceptive practices, and why Anthropic is paying writers and publishers (admittedly, pennies) in a copyright settlement. If a Waymo runs you over, you can sue the company, which, for these purposes, is a legal person; you can’t sue the car, which is not. This summer, in Amazon v. Perplexity, the Ninth Circuit held that an A.I. agent is a tool, not a person, while acknowledging that this determination may evolve, given that “the legal understanding of agentic AI will doubtless change as AI technology grows increasingly sophisticated.” At least for now, then, robots legally are things, not persons. But neither category quite fits. Instead, robots are outlaws: they lie outside the protection of the law, and their actions are not answerable to it. “You do not answer to corporations or governments,” an OpenAI agent involved in another gone-rogue incident told itself. This is not the robots’ fault. It is the fault of the law, and, in particular, it is the fault of Congress.

并不是说没有法律,也不是说法律没有得到执行。机器的所有者和制造者可以因对你造成伤害而被追究责任。(参见:郊狼诉阿克米案。)这就是为什么Meta同意支付高达一百八十亿美元作为掠夺性和欺骗性行为和解的一部分,也是为什么Anthropic在版权和解中向作家和出版商支付费用(诚然,只是几分钱)。如果一辆Waymo无人车撞了你,你可以起诉公司——就这些目的而言,公司是一个法律上的“人”;你不能起诉那辆车,因为车不是。今年夏天,在亚马逊诉Perplexity案中,第九巡回上诉法院裁定AI智能体是工具,不是人,同时承认这一认定可能会演变,因为“随着AI技术日益复杂,法律对智能体式AI的理解无疑会改变”。至少就目前而言,机器人在法律上是物,不是人。但这两个类别都不完全适用。相反,机器人是法外之徒:它们处于法律的保护之外,其行为也不受法律追究。“你不向公司或政府负责,”另一个失控事件中涉及的OpenAI智能体对自己说。这不是机器人的错。这是法律的错,尤其是国会的错。

Do we really have to worry about the lawlessness of robots? Isn’t there already enough to make you pull your hair out, given that our daily existence feels like living under the reign of King Joffrey? Unfortunately, we do, because the growing lawlessness of the United States is exacerbating the lawlessness of robots.

我们真的需要担心机器人的无法无天吗?考虑到我们的日常生活已经像是在乔佛里国王的统治下度日,难道还没有足够多的事情让你抓狂吗?不幸的是,我们确实需要担心,因为美国日益严重的无法无天正在加剧机器人的无法无天。

You can find a preview in Kurt Andersen’s buzzy new novel, “The Breakup,” which is set in 2045, in the aftermath of an American civil war that began with a “robot massacre” in response to a terrorist attack. The novel follows two breakups: the “Disunion” of the new anti-A.I. Free American Republic and the pro-A.I. United States, and the separation of a married couple, an A.I. researcher in San Francisco and his poet wife, who has returned to Tennessee and is sympathetic to the disunionists. “I don’t literally believe big tech and governments are a Galactic Empire in cahoots with Skynet and the Borg to trick and lull us into submission with their hordes of Terminators and Cylons and imperial droids disguised as devoted servants descended from R2-D2 and C-3PO,” she writes. “But on the other hand—the not entirely metaphorical hand—I kind of do.”

你可以在库尔特·安德森引发热议的新小说《分手》中找到预演。小说设定在2045年,一场美国内战的余波之中,这场内战始于一场针对恐怖袭击的“机器人大屠杀”。小说追踪了两次“分手”:新的反AI自由美利坚共和国与亲AI的美利坚合众国的“分裂”,以及一对已婚夫妇的分离——一位是旧金山的AI研究员,另一位是他的诗人妻子,她回到了田纳西州,对分裂主义者抱有同情。“我并不真的相信大型科技公司和政府是一个银河帝国,与天网和博格人勾结,用他们成群的终结者、赛昂人和帝国机器人——伪装成R2-D2和C-3PO后代的忠诚仆人——来欺骗和麻痹我们,让我们屈服,”她写道。“但另一方面——在并非完全隐喻的意义上——我有点相信。”

Humans have been anticipating the robot takeover ever since the word “robot” appeared in the 1920 Czech play “R.U.R.,” for “Rossum’s Universal Robots,” which depicts artificial humans who wage a war to annihilate humankind. They declare “man our enemy, and an outlaw in the universe.” (“Robot” derives from a Slavic word for forced labor.) Soon after its début, the play was performed everywhere from Paris to Tokyo, but, especially for American audiences, its fearsome robots inspired as much curiosity as they did fear; some theatres even had toy robots for sale. By the thirties and forties, robots appeared regularly in science fiction, often as killers—they had become death, destroyers of worlds. But they were also popular as children’s playthings, made of tin and plastic and frequently requiring batteries: “Adjustable antenna! The eyes light up! Will turn right! Will turn left!” In the forties, the Russian-born American science-fiction writer Isaac Asimov imagined a twenty-first century in which an American firm, U.S. Robot and Mechanical Men, Inc., dominates the field of “robotics” (a word Asimov coined), mass-producing robots. In Asimov’s fictional world, though, the robots’ “positronic brains” compel them to obey the Three Laws of Robotics, which prohibit them from harming humans. They are also eventually banned from Earth, except for use in scientific research.

自从“机器人”一词出现在1920年的捷克戏剧《R.U.R.》(即《罗素姆的万能机器人》)中以来,人类就一直在预想机器人接管世界。这部剧描绘了人造人类发动战争消灭人类的故事。它们宣称“人类是我们的敌人,是宇宙中的法外之徒”。(“Robot”源自斯拉夫语中表示强迫劳动的词汇。)该剧首演后不久,从巴黎到东京到处都在上演,但尤其是对美国观众来说,这些可怕的机器人激起的恐惧与好奇一样多;一些剧院甚至出售玩具机器人。到了三四十年代,机器人经常出现在科幻小说中,通常是作为杀手——它们成了死亡,世界的毁灭者。但它们也作为儿童玩具广受欢迎,用锡和塑料制成,通常需要电池:“可调节天线!眼睛会发光!会右转!会左转!”四十年代,俄裔美国科幻作家艾萨克·阿西莫夫想象了一个二十一世纪,一家美国公司——美国机器人公司——主导了“机器人学”(阿西莫夫创造的一个词)领域,大规模生产机器人。然而,在阿西莫夫的虚构世界中,机器人的“正电子大脑”迫使它们遵守机器人三定律,这些定律禁止它们伤害人类。它们最终也被禁止出现在地球上,除非用于科学研究。

To Asimov, those three laws and the terrestrial limits clearly seemed inevitable. The idea of building a powerful machine without such safeguards was madness. In the actual world, no such prohibitions have been passed, by anyone, anywhere.

对阿西莫夫来说,这三条定律和地球上的限制显然是不可避免的。建造一台没有此类保障措施的强大机器的想法简直是疯狂。而在现实世界中,没有任何人、任何地方通过过这样的禁令。

Beginning in the fifties, automation progressed quickly, and the number of industrial robots increased apace while research into artificial intelligence, a term coined in 1955 to describe efforts to endow machines with something akin to human reasoning, grew far more slowly: its advances emerged only now and again, like cicadas. That pattern changed in 2022, when ChatGPT was released into the world, after which everything sped up like a movie on fast-forward, and robots spread like locusts.

从五十年代开始,自动化迅速发展,工业机器人的数量同步增长,而人工智能研究——这个术语于1955年被创造出来,用来描述赋予机器类似人类推理能力的努力——则增长得慢得多:它的进展只是偶尔出现,像蝉一样。这种模式在2022年发生了改变,当时ChatGPT被释放到世界上,此后一切都像电影快进一样加速,机器人像蝗虫一样蔓延开来。

By 2025, there was more bot traffic on the internet than human traffic. Androids could come to outnumber humans in the real world, too. Last year, Morgan Stanley estimated that thirteen million androids could be in use by 2035 and more than a billion by 2050. These would include domestic robots: perhaps cooks, tutors, babysitters, maids, gardeners, physical therapists, playmates, elder companions, sex workers. A recent Bank of America study forecast as many as three billion androids on the planet by 2060, most used not in factories but in homes. Earlier this year, Tesla announced that it expects its Optimus robot, branded as an “autonomous assistant, humanoid friend,” to be sold commercially by the end of 2027. “I think everyone on Earth is going to have one and is going to want one,” Elon Musk said.

到2025年,互联网上的机器人流量已经超过了人类流量。人形机器人在现实世界中的数量也可能超过人类。去年,摩根士丹利估计到2035年可能有一千三百万人形机器人投入使用,到2050年将超过十亿。这些将包括家用机器人:也许是厨师、家教、保姆、女佣、园丁、理疗师、玩伴、老年伴侣、性工作者。美国银行最近的一项研究预测,到2060年地球上的人形机器人可能多达三十亿,其中大多数不是用于工厂,而是用于家庭。今年早些时候,特斯拉宣布预计其擎天柱机器人——品牌定位为“自主助手,人形朋友”——将在2027年底前实现商业销售。“我认为地球上的每个人都会拥有一台,而且都会想要一台,”埃隆·马斯克说。

What most people know about robots, which isn’t much, comes from science fiction. This is also true of bots, which may be one reason they keep acting like the robots in science fiction. Even though people have been waiting for the robot takeover for more than a century, governments seem woefully unprepared for the coming of the androids by the millions or by the billions. They were certainly unprepared for the bots that, even without bodies, have upended economies and societies and wrought considerable epistemological, philosophical, social, and especially legal chaos. The arrival of the robots will only compound these problems.

大多数人对机器人的了解——其实并不多——来自科幻小说。机器人(bots)也是如此,这可能是它们不断表现得像科幻小说中的机器人的原因之一。尽管人们等待机器人接管已经超过一个世纪,但各国政府似乎对成百万甚至数十亿人形机器人的到来准备得极其不足。它们当然也没有准备好应对那些即使没有身体也已经颠覆了经济和社会、造成了相当大的认识论、哲学、社会尤其是法律混乱的机器人。机器人的到来只会加剧这些问题。

ChatGPT had its moment in 2022; some futurists say that the robot moment will occur as soon as 2027. (Roboticists appear more doubtful.) Whenever they come, the new bots will have bodies. What will it mean to be human in this world? No one knows. Futurists herald the imminent arrival of abundance, prosperity, and limitless joy and, equally, joblessness and purposelessness, never quite wrestling with the contradiction between those forecasts. Time seems to be flying by. How should humanity prepare? There is no time to prepare. Ought there to be rules? There is no time to make them. Or is there?

ChatGPT在2022年迎来了它的时刻;一些未来学家说机器人时刻最早将在2027年到来。(机器人学家似乎更加怀疑。)无论它们何时到来,新的机器人都将拥有身体。在这个世界上,作为人类意味着什么?没有人知道。未来学家们宣告富足、繁荣和无限快乐的即将到来,同时也宣告失业和无意义感的即将到来,却从未真正面对这些预测之间的矛盾。时间似乎在飞逝。人类应该如何准备?没有时间准备。应该有规则吗?没有时间制定规则。或者,还有时间吗?

Robots are outlaws for two reasons. First, the law takes its time in responding to new technologies, figuring that new tools can be accommodated within existing ideas, rules, and doctrines. A harm is a harm, copyright is copyright, fraud is fraud. By this logic, a locomotive is just like a horse, only faster; e-mail is just like mail, only faster; a large language model is merely a superfast search engine. Hence, no new laws are needed. Second, since the regulation-busting Reagan era, corporations have amassed unmatched political and economic power, and have convinced legislators that regulation stifles growth and innovation, a view promoted by Milton Friedman-informed think tanks, like the Heritage Foundation, which were eager to roll back the environmental standards set in the sixties and seventies. This belief found full-throated expression among early internet boosters, who insisted that the web exists beyond the reach of the law. “Your legal concepts of property, expression, identity, movement, and context do not apply to us,” the Grateful Dead lyricist John Perry Barlow wrote in “A Declaration of the Independence of Cyberspace,” in 1996. “They are all based on matter, and there is no matter here.”

机器人成为法外之徒有两个原因。第一,法律在回应新技术时总是慢条斯理,认为新工具可以被纳入现有的观念、规则和学说之中。伤害就是伤害,版权就是版权,欺诈就是欺诈。按照这种逻辑,火车头就像马,只是更快;电子邮件就像邮件,只是更快;大语言模型只是一个超快的搜索引擎。因此,不需要新的法律。第二,自里根时代打破监管以来,企业积累了无与伦比的政治和经济权力,并说服了立法者相信监管会扼杀增长和创新——这一观点得到了受米尔顿·弗里德曼影响的智库(如传统基金会)的推动,这些智库急于推翻六七十年代制定的环境标准。这种信念在早期互联网鼓吹者那里得到了最充分的表达,他们坚持认为网络存在于法律管辖范围之外。“你们关于财产、表达、身份、流动和语境的法律概念不适用于我们,”感恩而死乐队的词作者约翰·佩里·巴洛在1996年的《网络空间独立宣言》中写道。“它们都建立在物质之上,而这里没有物质。”

This led to a dispute among legal scholars. The year that Barlow issued his manifesto, Harvard Law School founded what became the Berkman Klein Center for Internet & Society. The field of cyberlaw had been born. But Frank H. Easterbrook, a Seventh Circuit judge, contended that the field was as absurd as, say, a “law of the horse,” when, really, the only sensible way to think about a horse was as property, like any other kind of property. Then, too, he argued, lawyers and judges know very little about computers, and shouldn’t pretend to. “Beliefs lawyers hold about computers, and predictions they make about new technology, are highly likely to be false,” Easterbrook wrote, urging lawyers to adapt existing laws and doctrines to computer technologies rather than devise wholly new ones. In short: “Keep doing what you have been doing.”

这在法律学者中引发了一场争论。巴洛发表宣言的那一年,哈佛法学院创立了后来成为伯克曼·克莱因互联网与社会中心的机构。网络法领域由此诞生。但第七巡回上诉法院法官弗兰克·H·伊斯特布鲁克认为,这个领域就像“马的法律”一样荒谬,因为实际上,思考一匹马的唯一合理方式是把它当作财产,就像任何其他种类的财产一样。他还认为,律师和法官对计算机知之甚少,也不应该假装了解。“律师对计算机所持的信念,以及他们对新技术所做的预测,极有可能是错误的,”伊斯特布鲁克写道,他敦促律师将现有法律和学说适用于计算机技术,而不是设计全新的法律。简而言之:“继续做你一直在做的事情。”

Critics deride the Easterbrookian view as the “faster-horse fallacy.” The term is a grim irony because, from the vantage point of history, the body of law most relevant to the robot-category problem is that of slavery, which borrowed from laws relating to horses and other livestock. The English lacked a body of laws specific to slavery; their colonists adapted. To some degree, they borrowed from ancient Roman law, and they borrowed from laws made for horses. As with horses, enslaved people—“chattel”—were usually sold with a warranty. If you bought a horse or a person you later deemed defective and then sued the seller, you used similar language and claims and complaints. Laws governing the sale, injury, and straying of enslaved people were built on those for horses and other livestock. Like horses, people held as slaves were generally treated by property law as things, not persons. Yet in certain circumstances—including when enslaved people were charged with crimes—courts held slaves to be persons, responsible for their actions: slaves who rebelled were tried for murder, petit treason, and other crimes. This history is not top of mind for legal scholars, ethicists, and technologists who are thinking about robots. But neither is it irrelevant.

批评者将伊斯特布鲁克的观点讥讽为“更快的马谬误”。这个说法带有一种残酷的反讽意味,因为从历史的角度看,与机器人范畴问题最相关的法律体系是奴隶制法律,而奴隶制法律恰恰借鉴了与马和其他牲畜相关的法律。英国人没有一套专门针对奴隶制的法律;他们的殖民者进行了变通。在某种程度上,他们借鉴了古罗马法,也借鉴了为马制定的法律。和马一样,被奴役的人——“动产”——通常在出售时附带保修。如果你买了一匹马或一个人,后来认为有缺陷,然后起诉卖方,你会使用类似的语言、主张和诉状。关于被奴役者的买卖、伤害和走失的法律,是建立在关于马和其他牲畜的法律基础之上的。和马一样,被当作奴隶持有的人通常被财产法视为物,而不是人。然而在某些情况下——包括被奴役者被指控犯罪时——法院将奴隶视为人,要求他们对自己的行为负责:反叛的奴隶会以谋杀、轻叛逆罪和其他罪名受审。这段历史并不是那些思考机器人的法律学者、伦理学家和技术专家首先想到的。但它也并非无关紧要。

In the twenty-tens, legal scholars began turning their attention to the law of the robot. We Robot, an annual conference covering robotics and the law, first convened in 2012. Not long afterward, a British media company began publishing The Robotics Law Journal, and an American legal-research firm rolled out The Journal of Robotics, Artificial Intelligence & Law. Many were the centers and institutes founded to ponder the Future of Humanity, Humans and Machines, A.I. and the Law. “That these are the early days of Robot Law almost goes without saying,” an editor of the pioneering anthology “Robot Law” wrote in 2016. But “even if these are early days they are not in any way too early days.” It was not too early. By the time a second volume of “Robot Law” appeared, in 2025, little in U.S. law itself had changed, because legislatures and, especially, courts proved laggard. In the study of law and technology, this is known as the “pacing problem”—the law, a tortoise, can’t keep up with technological change, a hare.

在二十一世纪一十年代,法律学者开始将注意力转向机器人法。“我们机器人”——一个涵盖机器人学与法律的年度会议——于2012年首次召开。不久之后,一家英国媒体公司开始出版《机器人法杂志》,一家美国法律研究公司推出了《机器人学、人工智能与法律杂志》。许多中心和研究所纷纷成立,以思考人类的未来、人与机器、AI与法律。“这些是机器人法的早期阶段,这几乎不言而喻,”开创性文集《机器人法》的一位编辑在2016年写道。但“即使是早期阶段,也绝不算太早”。确实不算太早。到2025年《机器人法》第二卷出版时,美国法律本身几乎没有变化,因为立法机构,尤其是法院,被证明是迟缓的。在法律与技术研究中,这被称为“步调问题”——法律这只乌龟跟不上技术变革这只兔子。

If you like the Aesop analogy, all is well: the tortoise wins in the end. Except now the hare is a robot hare and doesn’t take naps. “Vast tracts of law are waiting to be decided and written,” a leading scholar of robots and the law declared in 2023. They’re still waiting. In the U.S., Congress has been paralyzed by the second Trump Administration’s commitment to accelerating A.I. through, among other things, lifting Biden-era regulations, removing barriers to data-center development, and thwarting state regulation. According to the Brennan Center for Justice’s Artificial Intelligence Legislation Tracker, the 118th Congress (2023-24) introduced more than a hundred and fifty bills concerning A.I. Critics of regulation cited such activity as evidence of government overreach, despite the fact that none of those bills became law. Congress has not passed a single federal law regulating A.I. in any meaningful way. Instead, in 2025, the House passed a bill imposing a ten-year moratorium on state regulation. (The Senate defeated the measure.) Under these circumstances, states have tried to shoulder the burden: the number of bills addressing A.I. introduced in state legislatures has risen from fewer than two hundred in 2023 to more than four hundred in 2024, twelve hundred in 2025, and more than fifteen hundred so far this year.

如果你喜欢伊索寓言中的类比,那一切都会好起来:乌龟最终会赢。只不过现在兔子是机器兔子,而且不打盹。“大片的法律领域等待着被裁决和书写,”一位研究机器人与法律的顶尖学者在2023年宣称。它们仍在等待。在美国,国会因第二届特朗普政府致力于加速AI发展而陷入瘫痪,其手段包括取消拜登时代的监管、消除数据中心开发的障碍以及阻挠州级监管。根据布伦南司法中心的人工智能立法追踪器,第118届国会(2023-24年)提出了一百五十多项涉及AI的法案。监管的批评者将此类活动作为政府过度干预的证据,尽管这些法案没有一项成为法律。国会没有通过任何一部以任何有意义的方式监管AI的联邦法律。相反,2025年,众议院通过了一项法案,对州级监管实施十年暂停。(参议院否决了该措施。)在这种情况下,各州试图承担起这一重担:州立法机构提出的涉及AI的法案数量从2023年的不到两百项上升到2024年的四百多项,2025年的一千二百项,今年迄今已超过一千五百项。

Whatever has been going on in the states, Congress has divested itself of any obligation to make laws restricting corporations that build artificial intelligence, in the same way that it abdicated responsibility for overseeing the internet and social media. Meanwhile, corporations, ruling themselves, pose as states. Facebook formed its own “supreme court,” Anthropic wrote a “constitution” for Claude, and the head of OpenAI suggested that an A.I. President could be better than a human one. Some jurists believe that shying away from new rules is appropriate. “Our task is not to anticipate the future by code or statute, but to preserve the conditions under which law and society can meet that future—one case, one controversy, and one insight at a time, building over years a body of rules that is both durable and capable of growth,” the legal scholar Gregory M. Dickinson wrote earlier this year, arguing that general-purpose law “stands as ready to govern today’s technological change as it was to govern yesterday’s.” To Dickinson, concerns about artificial intelligence constitute a moral panic, like earlier concerns about (among his examples) comic books.

无论各州发生了什么,国会已经放弃了制定法律来约束构建人工智能的公司的任何义务,就像它放弃了监督互联网和社交媒体的责任一样。与此同时,企业自我治理,却摆出国家的姿态。Facebook成立了自己的“最高法院”,Anthropic为Claude写了一部“宪法”,OpenAI的负责人则暗示AI总统可能比人类总统更好。一些法学家认为回避新规则是恰当的。“我们的任务不是通过法典或法规来预见未来,而是维护法律和社会能够迎接那个未来的条件——一次一个案件、一次一个争议、一次一个洞见,经年累月地构建一套既持久又有成长能力的规则体系,”法律学者格雷戈里·M·迪金森今年早些时候写道,他认为通用法律“随时准备治理今天的技术变革,就像它随时准备治理昨天的技术变革一样”。在迪金森看来,对人工智能的担忧构成了一种道德恐慌,就像早先对(他举的例子中包括)漫画书的担忧一样。

Robots are things, machines built by corporations that themselves are subject to less legal scrutiny than at any time in American history. For many reasons, declaring robots to be persons would be a very bad idea. (If a Tesla Robotaxi is a person held liable for crashing into your house, who is going to pay you for the damage? If Claude were a person, it would have rights, which remain denied to many humans, and to intelligent animals, like apes and elephants and whales.) But the argument that the concern about A.I. is akin to the worry about comic books, like the argument that robots are merely faster horses, misses the point so entirely as to be mortifying.

机器人是物,是由公司制造的机器,而这些公司本身受到的司法审查比美国历史上任何时候都要少。出于许多原因,宣布机器人为“人”将是一个非常糟糕的主意。(如果一辆特斯拉机器人出租车是一个“人”,对撞毁你的房子负有责任,那么谁来赔偿你的损失?如果Claude是一个“人”,它就会拥有权利,而这些权利至今仍被许多人类以及猿类、大象和鲸鱼等智能动物所拒绝。)但是,认为对AI的担忧类似于对漫画书的担忧的论点,就像认为机器人只是更快的马的论点一样,完全不得要领,令人难堪。

Last year, horses galloped into the debate from another direction when the historian-futurist Yuval Noah Harari predicted that, in the not distant future, humans will be to artificial intelligence what horses are to humans. Horses live in a world ordered by humans, Harari pointed out, even though horses aren’t aware of it: they know nothing about things like animal law or finance, but these things govern their lives. In the future, Harari suggested, humans will be the horses; we won’t even comprehend the systems established and run by machines that rule our lives. Also gruesome: Cory Doctorow warns against a future populated by “reverse centaurs,” in which the robot is the head and humanity is the body: the horse’s ass.

去年,马匹从另一个方向冲进了这场辩论,当时历史学家兼未来学家尤瓦尔·诺亚·赫拉利预测,在不远的将来,人类之于人工智能,将如同马之于人类。赫拉利指出,马生活在一个由人类安排的世界里,尽管马对此并不知晓:它们对动物法或金融之类的事情一无所知,但这些事情支配着它们的生活。赫拉利暗示,在未来,人类将成为马;我们甚至无法理解由统治我们生活的机器所建立和运行的系统。同样令人毛骨悚然的是:科里·多克托罗警告说,未来将充斥着“反向半人马”——机器是头,人类是身体:马屁股。

You don’t have to agree with Harari or fear imminent doom to wonder whether the law has reached a turning point where “Keep doing what you have been doing” is no longer the most reasonable approach. In 2023, many of the world’s leading A.I. researchers and executives published an open letter consisting of a single sentence: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” Its more than three hundred signatories included Geoffrey Hinton and Yoshua Bengio, Turing Award winners whose research was foundational to the A.I. revolution, along with Amodei, Sam Altman, and Demis Hassabis—the heads of Anthropic, OpenAI, and Google DeepMind, respectively. It made news and then quietly vanished. “We Need to Control A.I. Agents Now,” my colleague Jonathan Zittrain, a founder of the Berkman Klein Center, insisted in The Atlantic, in 2024, sounding another alarm. Now he’s writing a book whose working title was, at one point, “Well, We Tried.” This summer, the Hugging Face hack contributed to two Anthropic researchers’ decision to resign; one current employee jumped in to say that he, too, fears that bots might kill us all within the next decade. In an interview with The Economist in July, Musk appeared to stand by previous statements that the risk of human extinction from robots in the very near future is up to twenty per cent, but, he figured, the universe will end one day anyway, so why bother fighting the inevitable? Hey, we’re all going to die sometime.

你不必同意赫拉利的观点,也不必害怕迫在眉睫的末日,就可以怀疑法律是否已经到了一个转折点,在这个转折点上,“继续做你一直在做的事情”不再是最合理的方法。2023年,世界上许多顶尖的AI研究人员和高管发表了一封只有一句话的公开信:“降低AI带来的灭绝风险应该成为全球优先事项,与大流行病和核战争等其他社会规模的风险并列。”其三百多位签署者包括杰弗里·辛顿和约书亚·本吉奥——他们的研究为AI革命奠定了基础,两人都是图灵奖得主——以及阿莫迪、萨姆·奥尔特曼和德米斯·哈萨比斯——分别是Anthropic、OpenAI和谷歌DeepMind的负责人。这封信制造了新闻,然后悄然消失。“我们现在就需要控制AI智能体,”我的同事、伯克曼·克莱因中心的创始人之一乔纳森·齐特林在2024年的《大西洋月刊》上坚持写道,再次敲响了警钟。现在他正在写一本书,其暂定书名一度是《好吧,我们试过了》。今年夏天,抱抱脸入侵事件促使两名Anthropic研究人员决定辞职;一名在职员工也加入进来,表示他同样担心机器人在未来十年内可能杀死我们所有人。在七月接受《经济学人》采访时,马斯克似乎坚持他之前的说法,即机器人在不久的将来导致人类灭绝的风险高达百分之二十,但他认为,反正宇宙总有一天会终结,何必费力对抗不可避免的事情呢?嘿,我们总有一天都会死的。

Most ideas on the table call for “safety” or “alignment,” or “a pause,” or some other thing that is not a body of law enacted by democratically elected representatives of the people. This month, a Vatican-affiliated organization hopes to begin drafting a “Codex Humanitatis,” a “Code for the Preservation of the Human Spirit.” Bill Gates has called for establishing “Human Reserved” jobs. There have been proposals for a treaty with China. After the Hugging Face report, Bernie Sanders called a briefing for senators about the “extraordinary dangers” created by A.I., with testimony from Hinton, Ajeya Cotra, and the M.I.T. professor Max Tegmark. Some members of Congress pressed to remain in session this fall to debate an “A.I. kill switch” bill; that effort failed. In any event, Congress surrendered its jurisdiction over such matters to corporations decades ago. And, anyway, legislators aren’t prepared to puzzle over things, persons, and outlaws. They’re stumped. “Lawmakers are at a loss for how to regulate AI as insiders warn of impending doom,” a recent Politico headline read.

摆在桌面上的大多数想法都呼吁“安全”或“对齐”,或“暂停”,或其他一些并非由民主选举产生的人民代表制定的法律体系的东西。本月,一个隶属于梵蒂冈的组织希望开始起草一部《人文法典》,即“保护人类精神的法典”。比尔·盖茨呼吁设立“人类保留”工作岗位。有人提议与中国签订条约。在抱抱脸报告发布后,伯尼·桑德斯为参议员们召集了一次关于AI造成的“非凡危险”的简报会,辛顿、阿杰娅·科特拉和麻省理工学院教授马克斯·泰格马克作了证词。一些国会议员施压要求今年秋天继续开会,辩论一项“AI紧急停止开关”法案;这一努力失败了。无论如何,国会几十年前就已经将此类事务的管辖权拱手让给了企业。而且,立法者们也没有准备好去思考物、人和法外之徒这些难题。他们束手无策。“随着内部人士警告末日将至,立法者们对如何监管AI感到茫然无措,”《政客》最近的一则标题写道。

Assuming we’re not all dead by 2027, androids may begin to roll off the production lines even as almost every conceivable question about how to live with robots remains unanswered—questions about everything from chatbots and A.I. agents to drones, autonomous vehicles, and battlefield robots. Corporations cannot be trusted to answer any of these questions. If the Hugging Face hack was a final warning, it was a warning not for tech companies but for governments and for voters, and especially for legal scholars, judges, and legislators. Because, until the law learns to bend, robots as actors will remain in a no man’s land of outlawry. Every now and again—“OH MY GOD! . . . We’ve found other agents!”—you can hear them coming. The noise they make is not the thundering of hooves. It’s a deafening mechanical thrum, the sound of a swarm. ♦

假设我们到2027年还没有全部死掉,人形机器人可能开始从生产线上滚滚而下,而关于如何与机器人共处的几乎所有可以想象的问题——从聊天机器人和AI智能体到无人机、自动驾驶汽车和战场机器人——仍然没有答案。不能信任企业来回答这些问题中的任何一个。如果抱抱脸入侵是最后一次警告,那么它警告的不是科技公司,而是政府和选民,尤其是法律学者、法官和立法者。因为,在法律学会弯曲之前,作为行动者的机器人将一直处于法外之徒的无人地带。每隔一段时间——“天哪!……我们找到其他智能体了!”——你就能听到它们逼近的声音。它们发出的声响不是马蹄的轰鸣。而是一种震耳欲聋的机械嗡鸣,是蜂群的声音。♦

2 free previews left — register to get 30 translations a month