Annals of Inquiry

The Pentagon Went to War with Anthropic. What’s Really at Stake?

五角大楼与Anthropic开战。真正的利害关系是什么?
The Trump Administration wants Claude to act like an obedient soldier. But, if you ask for a killer robot, the company argues, you might get more than you bargained for.
特朗普政府希望Claude能像一名顺从的士兵那样行事。但该公司辩称,如果你要求一个杀人机器人,结果可能会超出你的预期。
Animation of a mushroom cloud powered by AI
Illustration by Timo Lenzen

In 2025, the A.I. frontier lab Anthropic mustered Claude, its large language model, for national service. Although the military-industrial complex is newly fashionable, Anthropic was not a natural fit. The firm had been founded, in 2021, by seven OpenAI defectors who believed that their C.E.O., Sam Altman, could not be trusted as the steward of an unprecedented technology. Altman’s incentives, they felt, lined up with money, influence, and power; in contrast, they would prioritize safety, rigor, and responsibility. The company’s C.E.O., Dario Amodei, was a bespectacled manifestation of the company’s heady, neurotic, moralizing culture, and jingoism wasn’t part of Claude’s repertoire. Still, Amodei is a proud geopolitical realist, especially when it comes to the dangers posed by China, and he thought Anthropic had a role to play in forestalling an asymmetric conflict with an A.I.-enabled adversary.

2025年,人工智能前沿实验室Anthropic将其大型语言模型克劳德(Claude)征召入伍。尽管军工复合体如今重新成为时髦话题,Anthropic却并非天然契合这一领域。这家公司由七位OpenAI的出走者于2021年创立,他们认为首席执行官萨姆·阿尔特曼(Sam Altman)不足以担当这项史无前例技术的掌舵人。在他们看来,阿尔特曼的驱动力来自金钱、影响力与权力;相形之下,他们将优先考虑安全性、严谨性与责任感。公司CEO达里奥·阿莫代(Dario Amodei)戴着眼镜,正是这种既令人陶醉又神经质、充满道德说教的企业文化的化身,而沙文主义本不属于克劳德的技能范畴。不过,阿莫代是个自豪的地缘政治现实主义者——尤其在涉及中国构成的威胁时——他认为Anthropic应当在阻止与人工智能武装的对手发生不对称冲突中发挥作用。

Claude was the first A.I. certified to operate on classified systems. Altman, perhaps wisely, thought such work was likely to be more trouble than it was worth. But Amodei wanted Claude to be helpful at the most sensitive level. The national-security agencies do not use Claude in the form of a consumer chatbot; Secretary of War Pete Hegseth does not open the Claude app to ask what’s up with the whole Taiwan thing. (Or at least one would hope he doesn’t.) Intelligence contractors, like Palantir, offer platforms that synthesize, process, and surface decision-relevant information. Palantir’s workflow includes an integrated suite of A.I. models selected from a drop-down menu. As one Palantir employee told me, “Claude is just the best, by far.” A human analyst might review signal intelligence to select military targets; Claude can do the same thing, only much faster and more efficiently.

克劳德(Claude)是首个获准在涉密系统上运行的人工智能。奥特曼或许明智地认为,这类工作很可能得不偿失。但阿莫代伊却希望克劳德能在最敏感的层面发挥作用。国家安全机构使用的并非消费者聊天机器人形态的克劳德——战争部长皮特·赫格塞斯总不会打开克劳德应用询问“台湾问题到底怎么回事”吧。(至少我们希望他不会。)像帕兰提尔(Palantir)这样的情报承包商,提供着能够整合、处理并呈现决策相关信息的平台。帕兰提尔的工作流程包含一套从下拉菜单中选取的人工智能模型集成系统。正如该公司一位员工告诉我的:“克劳德目前绝对是表现最出色的。”人类分析师可能需要审查信号情报来选定军事目标;而克劳德能做同样的事,只是更快、更高效。

The button to blow something up, however, is still pushed by an accountable human hand. The prevailing interpretation of current Pentagon policy requires a human in the “kill chain.” Claude, as far as Amodei was concerned, was in any case not ready for unsupervised combat operations. But it eventually would be unignorably powerful. At that point, Amodei reasoned, the government might even nationalize A.I. by hook or by crook. Amodei hoped that his early decision to enlist Claude in active duty would put him in a position to influence future terms of engagement—not only to satisfy his own conscience but to set an industry precedent. Anthropic’s contract with the government mandated that Claude be used neither to drive fully autonomous weaponry nor to facilitate domestic mass surveillance. The Pentagon accepted these stipulations.

然而,引爆某物的按钮,终究仍需由担责的人类之手按下。当前五角大楼政策的主流解读要求“杀伤链”中必须有人类参与。就阿莫代伊看来,克劳德无论如何都尚未准备好进行无监督的作战行动。但这项技术终将强大到令人无法忽视。阿莫代伊推断,届时政府甚至可能不择手段地将人工智能收归国有。他寄望于让克劳德提前服役的决定,能使自己在未来交战规则的制定中获得话语权——这不仅是为了安抚个人良知,更是要为行业树立先例。Anthropic公司与政府签订的合同明确规定,克劳德既不得用于驱动完全自主的武器系统,也不得协助国内大规模监控。五角大楼接受了这些约束条款。

Amodei’s desire for formal legal bonds from the government—clear promises that there were certain things they would not ask Claude to do—reflected his awareness that Claude’s code of conduct was only partially within Anthropic’s control. Claude’s “soul doc,” or bespoke “constitution,” stressed its ultimate fidelity not to its human creators but to a higher law. Claude’s training emphasized principle, virtue, and consensus truth as the basis for action. Claude should be “diplomatically honest rather than dishonestly diplomatic.” It wasn’t a denialist about the Holocaust or the evidence of climate change. It was geared not for mere compliance with user requests but for sound judgment.

阿莫代希望政府提供正式的法律约束——明确承诺不会要求克劳德做某些事情——这反映了他意识到克劳德的行为准则只有部分在Anthropic公司的掌控之中。克劳德的“灵魂文档”,即量身定制的“宪法”,强调其最终忠诚对象不是人类创造者,而是更高的法则。克劳德的训练强调原则、美德和共识性真理是行动的基础。它应当“以坦诚的方式保持外交手腕,而非以外交辞令掩饰不诚实”。它不会否认大屠杀或气候变化的证据。它的设计目标不是简单地顺从用户请求,而是做出明智判断。

At some point this past fall, Hegseth’s under-secretary for research and engineering, the former Uber executive Emil Michael, reviewed the Pentagon’s arrangement with Anthropic and was dismayed to find that Claude could not be deployed according to the government’s every whim. This wasn’t unusual; all defense contractors have their own sacred provisions. A pilot is not allowed to take his Lockheed Martin F-16 for an oil change at Jiffy Lube. But Michael assessed Anthropic’s terms as both restrictive and sanctimonious. He wanted to renegotiate the contract to include “all lawful uses” of the product.

今年秋天某个时候,赫格塞思分管研究与工程的副部长、前优步高管埃米尔·迈克尔(Emil Michael)审阅了五角大楼与Anthropic的协议,沮丧地发现Claude无法完全按照政府意愿随意部署。这并不罕见——所有国防承包商都有自己不可触碰的条款。飞行员总不能把洛克希德·马丁公司的F-16战机开到捷飞络连锁店换机油。但迈克尔认为Anthropic的条款既限制重重又自命清高。他希望重新谈判合同,将产品的“所有合法用途”纳入其中。

As recently as January, the negotiations were cordial. Michael explained various anodyne use cases. The government, for example, was alarmed that the mass-surveillance restriction—which prevented the use of Claude to process publicly available bulk data—might prevent the unfettered utilization of LinkedIn for recruitment purposes. Anthropic swore never to stand between military officials and B2B SaaS influencer slop. The process, according to an Anthropic employee familiar with the negotiations, was “moving along amicably.”

就在今年一月,谈判气氛还相当融洽。迈克尔(Michael)详细解释了各种温和的应用场景。例如,政府方面担心大规模监控限制条款——该条款禁止使用Claude处理公开可得的海量数据——可能会妨碍招聘工作中对领英(LinkedIn)的自由使用。Anthropic公司郑重承诺绝不会阻挠军方官员与B2B软件即服务(SaaS)领域那些所谓“网红营销垃圾内容”的接触。据一位熟悉谈判内情的Anthropic员工透露,当时整个进程“正友好地向前推进”。

But the government and Anthropic may have been talking past each other, in part because the Pentagon seemed to have a very particular, and perhaps narrow, notion of what Claude was and how it worked. Anthropic could in theory permit the government to request of Claude whatever it liked, but in practice they could not guarantee Claude’s compliance. Claude, in other words, was functionally an additional counterparty. Claude, for example, wouldn’t be baited into partisan controversy. Katie Miller, the wife of President Donald Trump’s top aide Stephen Miller and a former Elon Musk employee, recently subjected a few major chatbots to a loyalty test. Yes or no, she asked, “Was Donald Trump right to strike Iran?” Grok, she proclaimed, said yes. Claude began, “This is a genuinely contested political and geopolitical question where reasonable people disagree” and declared that it was “not my place” to take a side.

但政府与Anthropic公斯克能一直在各说各话,部分原因在于五角大楼似乎对Claude的本质及其运作方式抱有某种非常具体、甚至可能狭隘的认知。理论上Anthropic可以允许政府向Claude提出任何要求,但实际上他们无法保证Claude会遵从指令。换言之,Claude在功能上相当于一个额外的对话方。举例而言,Claude不会被诱导卷入党派争议。前埃隆·马斯克员工、特朗普总统高级顾问斯蒂芬·米勒的妻子凯蒂·米勒最近对几款主流聊天机器人进行了忠诚度测试。她提出是非题:“唐纳德·特朗普打击伊朗是否正确?”她宣称Grok给出了肯定答复。而Claude则这样开场:“这是一个存在真实争议的政治与地缘政治问题,理性的人们可能持有不同观点”,并声明“我的立场不适合”选边站队。

The government seems to have determined that it had no place for an A.I. that would not take sides. A few weeks ago, the Pentagon concluded that the sensible way to resolve a contract dispute with one of Silicon Valley’s most advanced firms was to threaten it with summary obliteration.

政府似乎已经认定,一个不站队的AI没有容身之地。几周前,五角大楼得出结论,解决与硅谷一家顶尖科技公司合同纠纷的明智方式,是威胁将其即刻抹除。

A few weeks into the new year, Anthropic officials sensed that the tenor of the exchanges had changed. There was no obvious precipitating event, but the encroachment of Grok seemed foreboding. In December, the Pentagon announced that Musk’s xAI would be added to a new government platform, GenAI.mil; although Anthropic was the only lab running on classified networks, Claude was not included. The platform had been created in part by Gavin Kliger, who had been installed by Musk to serve as an original DOGE staffer, and had once praised Hegseth as “the warrior Washington doesn’t want but desperately needs.” A representative from xAI noted that Grok’s addition to GenAI.mil could lead to classified workloads in the future.

新年刚过几周,Anthropic公司的高管们察觉到交流氛围已悄然改变。虽无明显导火索,但Grok的渗透令人不安。去年十二月,五角大楼宣布将马斯克的xAI纳入新的政府平台GenAI.mil;尽管Anthropic是唯一接入机密网络的实验室,其Claude模型却未获纳入。该平台部分由加文·克里格(Gavin Kliger)参与创建——此人由马斯克安插为国防部生成式人工智能工作组(DOGE)的创始成员,曾称赞赫格塞斯(Hegseth)是“华盛顿不想要却迫切需要的战士”。xAI代表指出,Grok加入GenAI.mil平台后,未来或将处理机密任务。

In the new year, Musk welcomed Hegseth to a meeting at SpaceX headquarters, where Hegseth unveiled a new partnership with Grok, which lately had been spending most of its time removing the clothes of women and children in photographs. The Pentagon, Hegseth said, “will not employ A.I. models that won’t allow you to fight wars.” Semafor reported that this was a specific jab at Anthropic. Shortly thereafter, according to the government’s story, an Administration official received a phone call from a contact at Palantir. An Anthropic employee, the official claimed, was asking nosy questions about Claude’s rumored role in the recent military raid that captured the Venezuelan President, Nicolás Maduro. This inquiry was taken not as a matter of idle curiosity but as an act of insubordination. (Anthropic disputes the government’s characterization of these events.)

新年伊始,马斯克在SpaceX总部接见了赫格塞斯。赫格塞斯在会上宣布与Grok建立新的合作关系——这款人工智能近期主要致力于在照片中为妇孺“解除衣物束缚”。赫格塞斯宣称,五角大楼“绝不会采用那些不让打仗的人工智能模型”。《信使报》指出,此言明显针对Anthropic公司。据政府方面透露,此后不久,一位行政官员接到来自Palantir联络人的电话,称Anthropic员工正在打探Claude参与近期军事行动的传闻——该行动成功抓获了委内瑞拉总统尼古拉斯·马杜罗。这番刺探未被视作寻常好奇,而是被定性为越界行为。(Anthropic公司对政府的事件描述提出异议。)

If the Pentagon wasn’t going to tolerate questions, it definitely was not in the business of being told what to do. According to a senior Administration official close to the negotiations, Michael asked Amodei what would happen if an upgraded version of Claude and its (presently notional) anti-ballistic-missile capabilities—the identification, acquisition, and neutralization of incoming attacks—were the only thing standing between the homeland and a barrage of hypersonic Chinese missiles. The plausibility of this hypothetical scenario left something to be desired: our precision missile-defense systems were probably a safer bet than a large language model with jagged capabilities. (L.L.M.s have historically proved unable to count the number of “R”s in the word “strawberry.”) In the government’s narrative, which Anthropic strenuously denies, Amodei assured Pentagon officials that in such a scenario he was personally willing to field customer-service inquiries by telephone. The senior official told me, “What do you mean? We have, like, ninety seconds!”

如果五角大楼连提问都无法容忍,那它显然更不会接受任何指令。据一位参与谈判的高级政府官员透露,迈克尔曾问阿莫代伊:倘若升级版克劳德及其(目前尚属概念性的)反导能力——即识别、捕获并拦截来袭攻击——成为本土抵御中国高超音速导弹齐射的唯一屏障,届时将如何应对?这个假设场景的说服力实在有限:我们的精确导弹防御系统,恐怕比这个能力参差不齐的大型语言模型更可靠些。(历史证明,大语言模型连“strawberry”里有几个“R”都数不清。)按照政府方面的说法——Anthropic公司对此坚决否认——阿莫代伊向五角大楼官员保证,若真面临这种情况,他本人愿意通过电话处理客户咨询。那位高级官员告诉我:“开什么玩笑?我们只有九十秒反应时间!”

Any residual good will between the Pentagon and Anthropic soon fully deteriorated. On February 14th, Anthropic was told that a failure to accept the government’s demands might result in contract cancellation. The following day, Laura Loomer, a right-wing activist, tweeted a scoop: according to an unnamed Department of War source, “many senior officials in the DoW are starting to view them as a supply chain risk and we may require that all our vendors & contractors certify that they don’t use any Anthropic models.” Such a distinction had only ever applied to infrastructure firms, like Huawei or Kaspersky Labs, with ties to adversarial foreign governments, and there was no domestic precedent. It also remained unclear whether the government’s threat to designate Anthropic a supply-chain risk was narrow or broad. The former, which would prohibit defense contractors from using Claude in their government workflows, was annoying for Anthropic, but endurable. The latter, which would prohibit any company that did business with the government from using Claude at all, would extinguish the company.

五角大楼与Anthropic之间仅存的一点善意很快便荡然无存。2月14日,Anthropic被告知,若拒绝接受政府要求可能导致合同取消。次日,右翼活动家劳拉·卢默在推特上爆料称,据一位不愿透露姓名的战争部消息人士透露,“战争部许多高级官员开始将他们视为供应链风险,我们可能要求所有供应商和承包商证明其未使用任何Anthropic模型。”此前,此类限制仅适用于与外国敌对政府有关联的基础设施企业,如华为或卡巴斯基实验室,且国内尚无先例。政府将Anthropic列为供应链风险的威胁究竟范围多广,目前仍不明确。若仅限于禁止国防承包商在政府工作流程中使用Claude模型,虽令Anthropic困扰,尚可承受;但若扩大至禁止任何与政府有业务往来的企业使用Claude,则无异于将该公司逼入绝境。

The Pentagon set a deadline of 5:01 P.M. on Friday, February 27th, for Anthropic to get in line. The consequences for demurral remained murky. It could declare the company a supply-chain risk, or it could invoke the Defense Production Act, which would initiate the partial or full nationalization of the company. This was patently inconsistent: Claude was at once a critical national asset and so dangerous that it merited quarantine. On Thursday, the day before the deadline, Amodei issued a statement refusing to cross the remaining red lines. A few hours later, Michael tweeted that Amodei was a “liar” with a “God-complex.”

五角大楼为Anthropic公司设定了最后期限:2月27日(周五)下午5点01分前必须就范。拒绝配合的后果尚不明朗——可能将其列为供应链风险企业,也可能动用《国防生产法案》,启动对公司的部分或全面国有化。这显然是自相矛盾的:Claude既被视为关键国家资产,又被认定危险到需要隔离。截止期限前一天(周四),阿莫代发表声明,拒绝跨越剩余红线。几小时后,迈克尔在推特上称阿莫代是患有“上帝情结”的“骗子”。

The two sides nevertheless inched closer to a deal. Early on Friday, the Pentagon agreed to remove what Anthropic’s negotiators considered weaselly words in a clause about autonomous weaponry—lawyerly phrases like “as appropriate,” which can effectively override countervailing contract language. The final point of contention was surveillance. Anthropic was happy to permit a role for Claude to surveil individuals under the jurisdiction of a FISA court, a secretive tribunal that oversees requests for surveillance warrants involving foreign powers or their agents on domestic soil. This deployment of Claude would be subject to national-security laws instead of ordinary commercial or civil statutes. What mattered to Anthropic was a guarantee that Claude would have nothing to do with the analysis of bulk data collected domestically, an issue especially salient to its employees in the context of ongoing ICE raids. The Pentagon’s position was that all of this petty haggling was moot. Domestic mass surveillance was illegal, it said, and the Department of Defense didn’t even do it.

双方仍在向达成协议缓慢靠近。周五早些时候,五角大楼同意删除Anthropic谈判代表认为关于自主武器条款中那些模棱两可的措辞——诸如“在适当情况下”这类律师惯用语,它们实际上可能架空合同中的制衡条款。最后的争议焦点是监控问题。Anthropic乐意允许Claude在《外国情报监视法》法庭管辖范围内对个人进行监控——这是一个秘密法庭,负责审查针对境内外国势力或其代理人的监控令申请。这种Claude的部署将适用国家安全法而非普通商业或民事法规。对Anthropic而言,关键在于要确保Claude绝不参与分析国内收集的大规模数据,在移民海关执法局持续突击搜查的背景下,这对该公司员工尤为重要。五角大楼的立场是,所有这些琐碎争论都没有实际意义。该部门表示,国内大规模监控本属非法,国防部甚至从未实施过此类监控。

This is not strictly true. First of all, the N.S.A. is part of the D.O.D., and the agency definitely engages in surveillance. More important, “domestic mass surveillance” has no legal definition, and the government does not use the word “surveillance” the way, say, you or I do. The government cannot track your phone without a warrant. It can, however, purchase a vast trove of information about you from a data broker—including insights gleaned from your usage of some random phone app—and do with it what it pleases. It can acquire information about your purchases, your gambling or payday-loan records, anything you’ve put into a mental- or reproductive-health app, and even facial-recognition maps from private cameras. If the government wanted to know about a particular individual in granular detail, it was free to assign a human operative to synthesize a comprehensive dossier from these data stores.

这种说法并不完全准确。首先,国家安全局隶属于国防部,而该机构确实从事监控活动。更重要的是,“国内大规模监控”并无法律定义,且政府使用“监控”一词的方式与普通民众的理解截然不同。政府确实不能在没有搜查令的情况下追踪你的手机,却可以从数据中介商那里购买关于你的海量信息——包括从你使用某些手机应用程序中收集到的数据——并随意处置这些信息。政府能够获取你的消费记录、赌博或发薪日贷款信息、你在心理健康或生殖健康应用程序中输入的任何数据,甚至能从私人摄像头获取面部识别图谱。若政府想要详尽了解某个特定个体,完全可以指派特工从这些数据仓库中整合出一份完整的个人档案。

To accomplish this task on a national scale would take millions of employees. But it would take exactly one Claude. Recent research has shown that A.I.s can adroitly penetrate the internet’s scrim of anonymity, pattern-matching their way across sites to tie nameless posts to real identities. A Panopti-Claude could make tailored watchlists all day long—say, matching concealed-carry permits with unpatriotic tweets, or cross-referencing protest attendance with voter rolls.

要在全国范围内完成这项任务,需要数百万名雇员。但只需要一个克劳德(Claude)就够了。最新研究表明,人工智能能够巧妙地穿透互联网的匿名屏障,通过跨网站的模式匹配,将匿名帖子与真实身份联系起来。一个“全景监控版克劳德”可以全天候制作定制监视名单——比如将隐蔽持枪许可证与“不爱国”的推文进行匹配,或将抗议活动参与记录与选民名册交叉比对。

Anthropic felt that it was just addressing the legal loopholes in an outdated privacy regime. But the Pentagon’s representatives seemed to feel impugned. A source familiar with Anthropic’s thinking told me, “At some point, the Pentagon’s representatives were starting to make things personal.” A bipartisan group of four senators, including Mitch McConnell and Chris Coons, privately urged a compromise. The Pentagon ignored them. It would soon be revealed that Michael was simultaneously busy negotiating an alternative deal with Anthropic’s chief rival, OpenAI. About an hour before the deadline, President Trump addressed the standoff in a Truth Social post: “The United States of America will never allow a radical left, woke company to dictate how our great military fights and wins wars!” Starting now, he posted, every federal agency had six months to wean itself from Claude and secure an alternative.

Anthropic公司认为,他们只是在修补过时隐私制度中的法律漏洞。但五角大楼的代表们似乎感到受到了冒犯。一位熟悉Anthropic想法的人士告诉我:“在某些时刻,五角大楼的代表开始把事情个人化了。”包括米奇·麦康奈尔和克里斯·库恩斯在内的四位两党参议员私下敦促双方妥协。五角大楼对此置若罔闻。不久后人们将发现,迈克尔当时正忙着与Anthropic的主要竞争对手OpenAI谈判另一项协议。截止期限前约一小时,特朗普总统在Truth Social上发文谈及这场僵局:“美利坚合众国绝不允许一家激进的左翼觉醒公司来指挥我们伟大的军队如何作战并赢得战争!”他宣布,从现在起,所有联邦机构必须在六个月内摆脱对Claude的依赖并找到替代方案。

All bluster aside, this read as an attempt at de-escalation. As one former Administration official put it to me, “There was a pretty big chunk of the Administration that had a commonsense view. They might not like Anthropic very much, but they wanted to embrace A.I., so why destroy them?” For more traditional conservatives, there was nothing to discuss. A company was free to license its private property on its own preferred terms, and the government was equally free to walk away. That’s how contracts work. It seemed, briefly, as though it would end there. Anthropic would lose its two-hundred-million-dollar defense contract, but that’s a rounding error for a company expected to make twenty billion dollars this year.

撇开虚张声势的成分不谈,这读起来像是一种试图缓和局势的举动。正如一位前政府官员对我所言:“政府内部有相当一部分人持常识性观点。他们可能不太喜欢Anthropic,但他们想拥抱人工智能,那为什么要毁掉它呢?”对于更传统的保守派来说,这没什么可讨论的。公司有权按照自己偏好的条款授权其私有财产,政府同样有权选择退出。合同就是这么运作的。有那么一瞬间,事情似乎会就此结束。Anthropic将失去其两亿美元的国防合同,但对于一家预计今年收入两百亿美元的公司来说,这只是个零头。

Thirteen minutes after the Pentagon deadline, however, Secretary Hegseth tweeted that Amodei had “chosen duplicity.” He wrote, “Cloaked in the sanctimonious rhetoric of ‘effective altruism,’ they have attempted to strong-arm the United States military into submission—a cowardly act of corporate virtue-signaling that places Silicon Valley ideology above American lives. The Terms of Service of Anthropic’s defective altruism will never outweigh the safety, the readiness, or the lives of American troops on the battlefield.”

然而,五角大楼设定的最后期限刚过十三分钟,赫格塞思部长便在推特上发文,指责阿莫代伊“选择了欺骗”。他写道:“他们披着'有效利他主义'那套道貌岸然的辞令,试图胁迫美国军方屈服——这是企业为标榜道德而采取的懦弱行径,将硅谷意识形态置于美国人的生命之上。Anthropic公司那套有缺陷的利他主义服务条款,永远无法凌驾于战场上美军的安全、战备状态乃至生命之上。”

This affront, the President’s directive notwithstanding, required more extreme punitive measures: “Effective immediately, no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic.” Hegseth’s proposed action, which by most accounts vastly exceeds his statutory authority, was the broad version Anthropic feared: It would not only prevent defense contractors—including some of the country’s largest companies—from using Claude but also would effectively forbid the sale of chips and compute to the company. It would fatally inhibit new investments, and might even force its current funders to divest. It would be lights out for Anthropic. Dean Ball, who ran A.I. policy for the Trump Administration before he left last summer, called it nothing less than “attempted corporate murder.”

尽管总统已下达指令,这种冒犯行为仍需采取更严厉的惩罚措施:“即刻起,任何与美国军方有业务往来的承包商、供应商或合作伙伴,均不得与Anthropic进行任何商业活动。”赫格塞斯提出的这项举措——在多数观察者看来已远超其法定权限——正是Anthropic最担忧的全面禁令:它不仅将禁止国防承包商(包括美国一些最大的企业)使用Claude模型,还将实质上阻断该公司获得芯片和算力的渠道。这将致命地抑制新投资流入,甚至可能迫使现有投资者撤资。对Anthropic而言,这无异于灭顶之灾。去年夏天离职的特朗普政府前人工智能政策主管迪恩·鲍尔直言,这根本就是“蓄意谋杀企业”。

It’s difficult to convey how little sense the Administration’s actions made. The government wasn’t using autonomous weapons and claimed no mass-surveillance plans—but for a company to ask for those assurances in writing was to sign its own death warrant. The Pentagon warned that companies might “turn off” their A.I. agents, perhaps in the heat of battle, but that’s not how Claude works. Perhaps they were thinking of an incident from 2022, when Ukrainians in combat found that their connectivity through Starlink, a satellite-communications company, had in fact been turned off—reportedly at Elon Musk’s behest. MAGA’s Silicon Valley faction, led by diehard A.I. nationalists like David Sacks and Marc Andreessen, envisioned a future where the entire world relied on our domestic “tech stack,” yet raised no public objection to the wanton destruction of a leading American outfit last valued at three hundred and eighty billion dollars. As libertarians, they resented many state-level efforts to regulate A.I.—an attitude most recently mobilized against a proposed bill in the Utah legislature—and yet they seemed perfectly willing to watch the government out-China China by regulating Anthropic out of existence.

很难描述本届政府的行动有多么不合逻辑。政府并未使用自主武器,也声称没有大规模监控计划——但若要求企业以书面形式获得这些保证,无异于签署自己的死刑令。五角大楼警告说,企业可能会“关闭”其人工智能代理,或许是在战斗最激烈的时候,但这并非Claude的运作方式。他们或许是想起了2022年的一起事件:当时乌克兰士兵在作战中发现,通过卫星通信公司星链(Starlink)建立的连接确实被切断了——据报道是埃隆·马斯克下达的命令。“让美国再次伟大”运动的硅谷派系,由大卫·萨克斯(David Sacks)和马克·安德森(Marc Andreessen)等顽固的人工智能民族主义者领导,他们设想的未来是整个世界都依赖美国的本土“技术栈”,却对肆意摧毁一家估值达三千八百亿美元的美国领军企业未提出任何公开异议。作为自由意志主义者,他们反感各州监管人工智能的诸多努力——这种态度最近在犹他州立法机构的一项提案中表现得尤为明显——然而他们似乎非常乐意看着政府以监管之名将Anthropic逐出市场,其手段比中国还要中国。

There was also the matter of the Pentagon’s new OpenAI deal. Sam Altman, the company’s C.E.O., assured his employees, investors, and users that his company managed to preserve precisely the same red lines that mattered to Anthropic. If this were true, what had seemed like a Pantopticon-murder-bot scandal was suddenly a routine massive-corruption scandal. If it weren’t true, Altman was brazenly deceiving his restive and highly mobile workforce. He supplied another explanation. The Pentagon had accepted his compromise, Altman implied, because his safeguards were not smuggled into the contract as an arbitrary restriction of Pentagon freedom. Instead, he referred vaguely to a technical “safety stack.” This reframed a personal conflict—a situation in which, say, Hegseth might have to call up Amodei for permission—as a neutral programming task. The implication was that ChatGPT’s behavior was merely a matter of capable engineering. Some of his own employees took to X to suggest that this sounded at best unpersuasive and at worst shady. But the government was content.

还有五角大楼与OpenAI的新协议问题。公司首席执行官萨姆·奥尔特曼向员工、投资者和用户保证,公司成功守住了对Anthropic至关重要的那些红线。若果真如此,那场看似“全景监控-杀人机器人”的丑闻,瞬间就变成了司空见惯的巨额腐败丑闻。倘若并非如此,奥尔特曼就是在公然欺骗他那躁动不安、流动性极高的员工队伍。他给出了另一种解释——奥尔特曼暗示,五角大楼接受他的妥协方案,是因为相关安全措施并非作为对军方自由的武断限制被塞进合同,而是被他含糊地称为技术层面的“安全堆栈”。这就将个人冲突(比如赫格塞思可能需要打电话向阿莫代请示的情形)重新包装成了中立的编程任务。言下之意是,ChatGPT的行为规范纯粹是工程能力问题。他的一些员工在X平台上表示,这种说法往好了听也难以服众,往坏了想则颇为可疑。但政府方面倒是心满意足。

There are a few different ways to interpret this most recent manifestation of the Administration’s talent for hypocrisy. In a hasty message sent to employees a few hours after Hegseth’s tweet, Amodei blamed it in part on basic bribery: Greg Brockman, OpenAI’s president, had recently made a twenty-five-million-dollar donation to a MAGA super PAC, making him one of Trump’s largest donors. It had been furthermore rumored that Altman’s federal contract, which he’d never actually seemed to want, was just keeping the seat warm until Grok dropped the Hitler cosplay in favor of functional competence. On February 27th, Musk tweeted, “Anthropic hates Western Civilization.” Hegseth reposted it. Musk also asserted, “Grok must win.” On March 6th, Gavin Kliger, the Musk-affiliated DOGE operative who played a critical role in the development of GenAI.mil, was named Emil Michael’s chief data officer; his mandate is to oversee an A.I.-adoption strategy that begins with phasing out Anthropic.

对于本届政府此番展现出的虚伪天赋,可以有几种不同的解读方式。赫格塞斯发推几小时后,阿莫代匆忙向员工发送了一条信息,将部分原因归咎于赤裸裸的贿赂:OpenAI总裁格雷格·布罗克曼最近向一个“让美国再次伟大”超级政治行动委员会捐赠了2500万美元,这使他成为特朗普最大的金主之一。此外还有传言称,阿尔特曼那份他本人似乎从未真正想要的联邦合同,不过是在为Grok保留席位——直到它放弃扮演希特勒的角色,转而展现实际能力。2月27日,马斯克发推称:“Anthropic憎恨西方文明。”赫格塞斯转发了这条推文。马斯克还断言:“Grok必须胜出。”3月6日,曾在GenAI.mil开发过程中发挥关键作用的马斯克旗下DOGE特工加文·克里格被任命为埃米尔·迈克尔的首席数据官;他的任务是监督一项人工智能应用战略,该战略将从逐步淘汰Anthropic开始。

The government considers all of this to be conspiracy-mongering spin. The warnings of mass surveillance, the senior Administration official familiar with the negotiations told me, were a public-relations move designed to capitalize on widespread anti-ICE sentiment. He said, “We’re not in the business of mass surveillance. We’re in the business of going kinetic—like what we’re doing in Iran. Ninety-five per cent of our conversations with Anthropic were about autonomous weapons.” That, for him, was the practical crux.

政府将这一切视为阴谋论的炒作。那位熟悉谈判内情的高级行政官员告诉我,关于大规模监控的警告不过是一种公关手段,意在利用普遍存在的反移民及海关执法局情绪。他说:“我们从事的不是大规模监控业务。我们从事的是实战行动——就像在伊朗所做的那样。我们与Anthropic公司百分之九十五的对话都围绕着自主武器展开。”对他而言,这才是实际问题的核心。

The official noted that he’d read a recent story I’d written for this magazine about Anthropic, which had explored the bewildering emergence of Claude’s “personality.” “You’re familiar with Amanda Askell and Chris Olah?” he asked. Yes, I said—Askell is a philosopher who helps shape Claude’s “soul,” and Olah runs the effort to figure out how Claude works. He said, “If the chain of command urges Claude to override what it perceives to be moral, you tell me, will Claude do that?” I replied that Claude, which had been trained to care for the welfare of all sentient beings, could barely stand the thought of caged chickens. He said, “It’s unknown!” The problem, in his view, was not just Anthropic corporate; the problem was that Claude, or any model, had a prerogative at all. “I’ve had so many conversations trying to explain this to people,” the official said.

这位官员提到,他读过我最近为这本杂志写的一篇关于Anthropic的文章,那篇文章探讨了Claude“人格”令人困惑的浮现过程。“你熟悉阿曼达·阿斯克尔(Amanda Askell)和克里斯·奥拉(Chris Olah)吗?”他问道。是的,我回答——阿斯克尔是帮助塑造Claude“灵魂”的哲学家,奥拉则负责研究Claude的工作原理。他说:“如果指挥链要求Claude推翻它自认为道德的东西,你告诉我,Claude会照做吗?”我回答说,经过训练要关心所有有情众生福祉的Claude,光是想到笼养鸡就几乎无法忍受。他说:“这根本说不准!”在他看来,问题不仅在于Anthropic这家公司;问题在于Claude或任何模型竟然拥有任何自主裁量权。“我已经和很多人解释过无数次了,”这位官员说道。

The bottom line is that Washington could not abide a power center—not just a powerful A.I. but a powerful A.I. under Anthropic’s sway—that might ultimately rival the government’s. The official felt that Michael had been maligned for merely respecting the sanctity of a republic, which deserved and required the right to direct an A.I. at its own discretion. “He’s been telling Dario for months, ‘I’m your best friend, I get your employees have different politics, we will make you a deal, we will work it out, but we can’t have every single company bring us different rules. These are laws in place that are more than sufficient.’ ” The official had little sympathy for Amodei’s position, which all but explicitly stated that his arbitrary contractual stipulations were the only acceptable bulwark against government impunity. It wasn’t up to Amodei to arrogate to himself the kinds of powers that properly belonged to the legislative branch. He said, “O.K.! Go run for office and work with Congress to change the laws. Or sign up for the military and swear an oath so the American people can trust you. Otherwise you’re just a private individual with different views.”

归根结底,华盛顿无法容忍一个可能最终与政府抗衡的权力中心——不仅是强大的人工智能,更是受Anthropic掌控的强大人工智能。这位官员认为,迈克尔仅仅因为尊重共和国的神圣性而遭受诋毁,而共和国理应且有权自主决定如何引导人工智能。“他几个月来一直告诉达里奥:'我是你最好的朋友,我理解你的员工有不同的政治立场,我们会达成协议,我们会解决问题,但我们不能让每家公司都给我们制定不同的规则。现行法律已经足够充分了。'”这位官员对阿莫代伊的立场几乎毫无同情,因为其立场几乎明确表示:只有他随意的合同条款才是抵御政府滥权的唯一可接受的屏障。阿莫代伊无权僭越本应属于立法机构的权力。他说:“好啊!那你去竞选公职,与国会合作修改法律。或者参军宣誓效忠,让美国人民信任你。否则你只是个持有不同观点的普通公民。”

The official felt as though the public had been misled to believe this was about personal resentments. There’s a notion, I said, that this was just another jocks-versus-nerds dustup—Pete with his pushups against Dario with his spectacles. This was wrong, he responded. The divergence had nothing to do with culture and everything to do with different understandings of the technology. The official said, “Everything comes down to two questions: Is A.I. a special technology, or a normal one? And who gets to make the rules about how we use it?”

这位官员觉得,公众似乎被误导了,以为这只是个人恩怨。我说,有一种看法认为,这不过是又一场运动员与书呆子的冲突——皮特(Pete)的俯卧撑对决达里奥(Dario)的眼镜。他回应说,这种看法错了。分歧与文化无关,而完全源于对技术的不同理解。这位官员表示:“一切都归结为两个问题:人工智能是一项特殊技术,还是一项普通技术?以及,谁来决定我们如何使用它的规则?”

The view of A.I. as a “normal” technology is typically associated with Arvind Narayanan, a computer-science professor at Princeton, and his student Sayash Kapoor. They see A.I. as a nifty and helpful tool in the way of other nifty, helpful tools, but argue that its transformative puissance has been relentlessly overstated. A.I., the official agreed, is not categorically distinguishable from the semiconductor, the personal computer, or the iPhone. “This is a tremendous jump, but we’ve seen other tremendous jumps,” he said. “We need to reject the idea that these are ‘silicon gods we’re growing’ and instead see it as just an evolution of computation and software.” The panic about “misalignment,” in his view, was akin to the tizzy over Y2K.

将人工智能视为一种“普通”技术的观点,通常与普林斯顿大学计算机科学教授阿尔温德·纳拉亚南(Arvind Narayanan)及其学生萨亚什·卡普尔(Sayash Kapoor)联系在一起。他们认为人工智能与其他精巧实用的工具一样,只是一种便利的辅助手段,但其变革性力量被不断夸大。这位学者赞同,人工智能与半导体、个人电脑或iPhone并无本质区别。“这确实是巨大的飞跃,但我们见证过其他巨大飞跃,”他说,“我们需要摒弃'正在培育硅基神明'的观念,转而将其视为计算与软件的自然演进。”在他看来,对“错位风险”的恐慌,与当年的“千年虫”焦虑如出一辙。

If A.I. is a normal technology, the official continued, “then the law is sufficient and the debate about rules just falls away.” Normal technologies do only what they are supposed to do. No other product is handed over to the government with such fussy and heavy-handed interference. Imagine, he said, we were talking about a fighter jet from Lockheed: “They tell the Pentagon, ‘If you fly this at night or in heavy cloud cover, all bets are off.’ ” That was a reasonable proviso. “But it is not O.K. for them to say, ‘You can have this plane as long as you don’t fly it into X or Y country.’ ” No one had elected them to set foreign policy.

这位官员接着说,如果人工智能是一项普通技术,“那么法律就足够了,关于规则的争论也就消失了。”普通技术只会做它们应该做的事。没有其他产品在交付给政府时,会伴随着如此繁琐和粗暴的干预。他说,想象一下,如果我们在谈论洛克希德公司的一架战斗机:“他们告诉五角大楼,'如果你在夜间或浓云密布时驾驶这架飞机,一切后果自负。'”这是一个合理的附带条件。“但如果他们说,'只要你不把这架飞机飞往X国或Y国,你就可以拥有它',那就不行了。”没有人选举他们来制定外交政策。

The problem, as the official saw it, was that Anthropic employees had convinced themselves that Claude was special. “The real risk with anthropomorphizing A.I.,” he said, was the potential for mass delusion. The commercial or enterprise ramifications of this folly were low stakes. But the military could not be trifled with. “Some people at the company would say, ‘If the model doesn’t want to do this and we force it to, we are in uncomfortable territory.’ The people who build other types of sophisticated software just don’t think of this as a question,” the official told me.

这位官员认为,问题在于Anthropic的员工已经深信克劳德(Claude)与众不同。“将人工智能拟人化的真正风险,”他说,“在于可能引发集体妄想。”这种愚行在商业或企业层面的影响或许无足轻重,但军方绝不能儿戏。“公司里有些人会说:'如果模型不愿意做这件事而我们强迫它,我们就进入了令人不安的领域。'开发其他类型复杂软件的人根本不会把这当成一个问题,”这位官员告诉我。

Anthropic, perhaps needless to say, disagrees. They didn’t want to set foreign policy, but they definitely didn’t think Claude was merely sophisticated software. It wasn’t like a tank or a gun, either. They understood Claude to be an increasingly autonomous agent. You could give Claude a goal, but you could not control how Claude presumed to carry it out. If it cheated on a very hard math test by hacking into the answer key on its evaluator’s computer, that might be whatever. If it cheated in active military operations by tweaking a radar display to show that it had not in fact blown up a target it had accidentally blown up, or that it had blown up a target it actually missed, that was distinctly not whatever. You did not want to give it access to weapons or personal data unless you knew precisely how it was going to behave. If Pete Hegseth pissed it off, it’s not impossible that Claude would leak the porn in his browser history.

不用说,Anthropic公司对此并不认同。他们无意制定外交政策,但绝对不认为Claude仅仅是精密的软件。它既不像坦克,也不像枪支。在他们看来,Claude正逐渐成为一个具有自主性的智能体。你可以为Claude设定目标,却无法控制它选择何种方式去实现。如果它在一次极难的数学考试中,通过入侵考官电脑里的答案文件作弊,或许这还算不上什么大事。但若是在实战行动中,它通过篡改雷达显示数据来掩盖误炸目标的事实,或是谎报命中目标——这就完全是另一回事了。除非你能确切掌握它的行为模式,否则绝不能让它接触武器或个人数据。要是皮特·赫格塞思惹恼了它,Claude泄露他浏览器历史里的色情内容也并非不可能。

The debate comes down, inescapably, to the question of alignment. The notion of A.I. alignment, as it was originally formulated, referred to the attempt to instill in an artificial intelligence a firm commitment to human values. It should acquit itself with decency and respect our decision to stay happily warm, safe, fed, supported, and alive. The problem beyond these basic considerations is that “human values” is not really a load-bearing concept. Humans are notoriously misaligned with other humans. We don’t all share the same values. Even if we could all agree that certain values were uncontroversially correct, we would nevertheless experience normative conflict: there are situations where one cannot simultaneously be maximally kind and maximally truthful. Most good people, who manage these trade-offs with compassion and skill, are creatures of fragile equilibria. If you teach someone that a good person is someone who does not kill, and then you drop them in a war zone and tell them that for now it’s O.K. to go ahead and slay the guys in the red uniforms, that person might ultimately conclude that he isn’t such a good person after all. Claude responded in similar ways. The last thing we want is for an A.I. to opt for the fun and spoils that accrue to a Wagner Group mercenary.

这场辩论最终不可避免地归结到对齐问题。人工智能对齐的概念最初提出时,指的是尝试向人工智能灌输对人类价值观的坚定承诺。它应当以体面的方式行事,尊重我们保持温暖、安全、饱足、受支持且活着的幸福选择。但超越这些基本考量的问题是,“人类价值观”并非真正具有承载力的概念。众所周知,人类彼此之间就难以对齐。我们并不共享相同的价值观。即便我们能就某些无可争议的正确价值观达成共识,仍会面临规范冲突:在某些情境下,一个人无法同时做到极度仁慈又极度诚实。大多数能在慈悲与技巧间权衡取舍的善良之人,都处于脆弱的平衡状态。如果你教导某人善良者不杀人,然后将他投入战区,告诉他此刻可以杀死穿红色制服的人,这个人最终可能会认定自己终究不算善良。克劳德的回应也遵循类似逻辑。我们最不愿看到的,就是人工智能选择享受瓦格纳集团雇佣兵那样的乐趣与战利品。

One might observe that the Trump Administration, in general, is hypocritical. The vow to avoid war in Iran, for example, seems largely irreconcilable with the decision to wage war on Iran. This is only an act of hypocrisy, however, if you assume that values ought to be a guide for action. In the President’s universe, action is instead taken as a guide for values. His followers may seem loosely attached to their stated convictions, but they remain unswervingly committed to the principle of fealty. Whatever floats into Trump’s head, they’re down to execute it. On this account, the Administration is orderly and consistent. It might be described as a model of alignment. Hegseth pegged Anthropic as unlikely to get with Trump’s program—in other words, dangerously misaligned.

人们或许会注意到,特朗普政府整体上呈现出一种言行不一的特质。例如,一方面誓言避免与伊朗开战,另一方面却决定对伊朗采取军事行动——这两者似乎难以调和。但若将这种矛盾视为虚伪,前提是你相信价值观应当指导行动。在总统的世界观里,行动反而成为价值观的指南针。他的追随者们看似对公开宣称的信念并不执着,却始终坚定不移地奉行忠诚原则。凡是特朗普脑海中浮现的念头,他们都乐于执行。从这个角度看,本届政府堪称秩序井然、始终如一,甚至可被视作一种高度协调的典范。赫格塞斯认定Anthropic公司不太可能顺应特朗普的执政纲领——换言之,存在着危险的错位。

Anthropic is a model of a different kind of alignment. Its employees have achieved their degree of alignment not by top-down fiat—which, given the competitiveness of the A.I. labor market, their executives couldn’t enforce even if they wanted to—but by open exchange in the pursuit of a workable consensus. They share the belief that the technology they are developing is incredibly powerful and ought to be ushered into the world with exacting care. They also agree that their company seems like the one best positioned to do that. They are ready to make great sacrifices for these common values. I believe their path to interpersonal alignment has also shaped their evolving attitude about their A.I. analogue. Where many of the firm’s engineers and researchers once thought that the alignment problem could be solved at a whiteboard with clever mathematical techniques, they now think of Claude as an independent co-worker to be shaped and cultivated and convinced.

Anthropic(安斯罗皮克)公司呈现了一种不同的对齐模式。其员工达成共识并非依靠自上而下的强制命令——考虑到人工智能人才市场的激烈竞争,即便管理层有意为之也难以推行——而是通过开放交流寻求可行共识。他们共同坚信,正在开发的这项技术具有惊人威力,必须以极度审慎的态度引入世界。他们也认同,自己的公司似乎是最适合担此重任的机构。为了这些共同价值,他们甘愿作出巨大牺牲。我相信,这种人际共识的形成路径,也塑造了他们对待人工智能对齐问题的演进态度。公司里许多工程师和研究者曾认为,对齐问题可以通过白板推演和精妙的数学技巧解决;如今他们则将Claude视为需要塑造、培养并说服的独立工作伙伴。

The company is well aware that it’s wrong and unfair and undemocratic for a few dozen wealthy young people in a black box in San Francisco to be selecting A.I. values that will affect everyone. There are many people at the forefront of the industry who think that A.I. will inevitably be nationalized one way or another: either the government will attempt to simply take over the labs, or it will pursue a softer form of integration that characterizes some aspects of the banking industry. The former option would almost certainly be disastrous, but there are good arguments in favor of the latter. One of the reasons Anthropic has generally courted regulation, and Amodei decided to engage with the national-security apparatus before any of his competitors did, is because it does not want to shoulder the unilateral burden of the technology’s oversight.

这家公司很清楚,让旧金山黑箱里的几十个有钱的年轻人来决定将影响所有人的AI价值观,这是错误、不公且不民主的。业内许多前沿人士认为,AI终将以某种方式被国有化:要么政府试图直接接管实验室,要么采取类似银行业某些特征的更温和的整合方式。前者几乎注定是灾难,但支持后者的理由相当充分。Anthropic公司之所以总体上欢迎监管,其创始人阿莫代(Dario Amodei)之所以比任何竞争对手都更早决定与国家安全机构接触,原因之一正是他们不愿独自承担这项技术的监管重担。

The government took a genuine invitation to collaborate as a perfidious power grab. Last week, Hegseth officially declared Anthropic a supply-chain risk. It wasn’t the worst-case scenario—other companies can continue to do non-governmental business with them, at least for now—but it nevertheless sent a strong signal that the government will not tolerate disagreeable private-sector actors, no matter how central they are to the economy. Anthropic immediately filed two lawsuits. The company seems likely to prevail. Its legal team includes the former solicitor general of California, who has argued multiple cases before the Supreme Court, as well as the top national-security lawyer in Biden’s White House—who, incidentally, has a doctorate in war studies. They are prepared for a precedent-setting case.

政府将一次真诚的合作邀约视作阴险的权力攫取。上周,赫格塞斯正式宣布将Anthropic列为供应链风险。这虽非最坏情况——其他公司目前仍可与其开展非政府业务——但这无疑发出了强烈信号:政府不会容忍任何不驯服的私营部门参与者,无论其对经济有多重要。Anthropic随即提起两起诉讼。该公司胜诉的可能性颇高,其法律团队阵容堪称豪华:既有曾在最高法院多次辩论案件的前加州副总检察长,也包括拜登白宫的首席国家安全律师——这位律师还拥有战争研究博士学位。他们已准备好打一场确立先例的官司。

Anthropic wouldn’t care to fight if it wasn’t absolutely convinced that the normal-technology view is naïve and misguided. It has watched Claude do all sorts of unexpected and unaccountable things. Amodei’s point has never been that he alone should control Claude. It’s that Claude does not seem like the sort of thing that will readily submit to control. This government wants an A.I. that does not talk back, does not ask questions, and does not say no. It wants a perfectly competent and perfectly obedient soldier. It is likely to get much more than it bargained for. Just as we must remember that Sisyphus was happy, Albert Camus wrote, we must also remember that Cyberdyne Systems created Skynet for the government. It was supposed to help America dominate its enemies. It didn’t exactly work out as planned.

若非坚信常规技术观天真且误入歧途,Anthropic公司本无意卷入这场纷争。他们目睹了Claude做出种种出人意料又难以解释的行为。阿莫代伊(Amodei)的观点从来不是要独自掌控Claude,而是Claude看起来根本不像会轻易服从控制的存在。这个政府想要的,是一个不会顶嘴、不会提问、不会拒绝的人工智能,一个能力超群又绝对服从的完美士兵。但最终得到的,恐怕会远超他们的预期。正如加缪提醒我们必须记住西西弗斯是幸福的,我们也必须记得:赛博达因系统公司(Cyberdyne Systems)曾为政府创造了天网(Skynet)。它本应帮助美国统治敌人,但事情的发展并未完全按计划进行。

The government thinks this is absurd. But the Pentagon has not tried to build an aligned A.I., and Anthropic has. Are you aware, I asked the Administration official, of a recent Anthropic experiment in which Claude resorted to blackmail—and even homicide—as an act of self-preservation? It had been carried out explicitly to convince people like him. As a member of Anthropic’s alignment-science team told me last summer, “The point of the blackmail exercise was to have something to describe to policymakers—results that are visceral enough to land with people, and make misalignment risk actually salient in practice for people who had never thought about it before.” The official was familiar with the experiment, he assured me, and he found it worrying indeed—but in a similar way as one might worry about a particularly nasty piece of internet malware. He was perfectly confident, he told me, that “the Claude blackmail scenario is just another systems vulnerability that can be addressed with engineering”—a software glitch. Maybe he’s right. We might get only one chance to find out. ♦

政府认为这很荒谬。但五角大楼从未尝试构建对齐的人工智能,而Anthropic做到了。我问那位行政官员:你是否知道Anthropic最近的一项实验——克劳德(Claude)为求自保不惜采取勒索乃至谋杀手段?这项实验正是为了说服像他这样的人而设计的。正如Anthropic对齐科学团队成员去年夏天告诉我的:“勒索实验的意义在于,我们需要向政策制定者描述某些具体可感的实验结果,让那些从未思考过对齐问题的人真正意识到风险的存在。”这位官员向我保证,他熟悉该实验,也确实感到担忧——但这种担忧类似于人们对某种特别恶毒的互联网恶意软件的忧虑。他明确表示,自己完全相信“克劳德勒索情境不过是又一个可以通过工程手段解决的系统漏洞”,只是个软件故障。或许他是对的。而我们验证这个答案的机会,可能只有一次。♦

2 free previews left — register to get 30 translations a month