The Mark of the Machine
Last week, Anthropic announced an unexpected policy change: text created with Claude, the company’s large language model, would now contain an invisible watermark. If a person were to copy and paste a chatbot-crafted passage into a separate document, an A.I. detector would be able to determine the likelihood that it had been processed by Claude. A.I.-detection systems aren’t new, of course; programs such as Pangram and Turnitin have become entrenched in academic and publishing practices. But, while those tools train on huge data sets to recognize statistical patterns that constitute a discernible A.I. writing “style,” Claude’s watermark is embedded within the text itself, at the individual word level, making it potentially harder for L.L.M. users to edit away evidence that they used A.I.
Across the internet, criticism of the allegedly broken contract between machines and their human prompters ballooned. Weren’t L.L.M.s specifically advertised as being able to present a machine’s work as one’s own? Wasn’t that possibility indeed why many people used L.L.M.s in the first place? Now, for the students who’d grown accustomed to writing essays with chatbots, the journalists and authors who secretly used L.L.M.s to draft their articles and newsletters and books, and even the lawyers who relied on Claude to write briefs and motions, the jig was seemingly up. “Those guys come out of the process with a digital tattoo on their forehead,” one Reddit user wrote in a viral post that itself appeared to be A.I.-generated. (I ran it through an A.I.-detection system and, yeah, one hundred per cent L.L.M.-“assisted.”) “The stigma,” the poster continued. “Jesus, the stigma.”
In the absence of comprehensive federal regulations for A.I., and with inconsistent and often contradictory rules governing its use in professional, academic, and artistic contexts, few forces have proved as powerful at tempering A.I.’s threat to human ingenuity as public shaming has. “Put plainly, you should feel bad for using AI,” the editors of n+1 wrote, last year, in a polemic against L.L.M.s. “Stigmatization is a powerful force, and disgust and shame are among our greatest tools.” As generative A.I. is increasingly deployed across our cultural and political life—from winners of prestigious short-story prizes to major-label album releases to Presidential propaganda—the average person is now tasked with policing machine-generated infractions in an effort to preserve whatever ethical dignity and critical acumen our species has left. Accusers can occasionally be overzealous, making allegations that aren’t always ironclad, especially when the phrase “100% AI Generated” has emerged as a kind of cultural scarlet letter. But with Claude’s new watermark, maybe this process of discerning the “real” from the “fake” would become more streamlined, and more foolproof—a stepping stone to a more transparent and cautious relationship with L.L.M.s and gen A.I. writ large.
Surely Anthropic did not institute its watermark to discourage people from using Claude, or to make its product less appealing—so why, then, would it choose to make its automated text more obviously traceable in the first place? The company’s official announcement explained that the change was made to comply with the E.U. A.I. Act, which requires L.L.M. providers to make A.I.-generated text identifiable as such. This identification could take the form of a watermark, metadata, or a cryptographic signal—statistical signatures that allow detection systems to determine whether text, images, or audio was created or manipulated by A.I. Google Gemini, for instance, uses a watermark called SynthID in its A.I.-generated material. OpenAI currently applies detection methods only to images and audio, but, after the E.U. A.I. Act’s “transparency obligations” went into effect, the company stated that it was “working to expand provenance measures” for ChatGPT text. These A.I. companies emphasize that tracking properties like the watermark won’t be discernible to the average reader and can’t be traced to whoever prompted the L.L.M. to generate the text—unless, of course, said person attempts to publish or present the text as his own and is outed by a detection system.
Anthropic’s motives, however, may not be limited to complying with the E.U. and upholding its stated values of safety and transparency. The company may, quite simply, want to make it easier for its models to verify human-written text and exclude A.I.-generated material when training its L.L.M.s, out of fear that training an A.I. model on A.I.-generated content might lead to a decline in reliability and quality. (The irony!) It also may be angling to assert “provenance”—a cherished word among A.I. providers—over their L.L.M.s’ output, establishing a sort of intellectual property over what their product creates, unique from its competitors. It’s hard not to perceive the whole identification operation as, in some ways, an advancement in A.I. surveillance—a way of tracking where a chatbot’s words go after they leave the machine, creating a Rolodex of use cases for future referral. “You know what this reminds me of?” the same addled Reddit poster wrote in his anti-watermark manifesto. “Those police operations that arrest the drug user and leave the dealer alone. Watermarking is the same thing.” A company like Anthropic may reasonably view watermarking as a harmless and sensible evolution: a boon to their training data, a legally compliant expansion of A.I. transparency, and a way to determine if something originated with its model. But, to its users, the stain of a watermark mainly forecasts a senseless form of social punishment, a tactic to tarnish anyone who dares use A.I.-generated material instead of plumbing the muck of their own mind.
Some critics have argued that the watermark will prove largely ineffective for flagging anything short of blatant chatbot use. When an L.L.M. like Claude generates text, it typically makes its word choices from a range of statistically probable options. With a watermark, however, the model hews toward a more predetermined set of outcomes. These outcomes can accumulate into a statistical pattern that allows a detection system to calculate the likelihood that Claude generated a piece of text. If one were to rigorously edit, or rewrite, text that the L.L.M. spits out, the detection system may struggle to pick up the pattern, but arguably that’s a feature and not a bug—isn’t catching the most obvious abuses of using A.I. for writing the point of something like a watermark, or at least a slightly more ethical internal system than the current system of nothing?
Aside from anxieties that the watermark is merely a red herring, an easy-to-evade compliance strategy, some watermark detractors fear that these statistical signatures will make chatbots worse at writing. John Gruber, a tech blogger, posited that Claude would now more readily default to watermarked words, rather than unleash the full scope of its linguistic range, which might result in lower-quality text, or in some cases unusable prose. This outcome is certainly possible if L.L.M.s begin to gravitate toward specific synonyms or phrases that hinder the explanatory prowess and precision of, say, an instruction manual or a doctor’s note—documents that are increasingly being outsourced to L.L.M.s. But many experts agree that Gruber’s concerns are overblown; Steven Murdoch, a professor of security engineering at University College London, told the Guardian that the change “probably wouldn’t have any noticeable impact.”
As L.L.M. devotees debated whether the watermark would worsen A.I.’s writing capabilities, Ben Thompson, who writes the popular tech and media newsletter “Stratechery,” bristled at the notion that A.I. could be considered an author at all. (Unfortunately, his argument wasn’t aesthetic but, rather, more technical.) He contended that L.L.M.s were “wielded by humans” and thus could not be attributed as creators of anything: “To insist on watermarking is no different than insisting that a ballpoint pen advertise itself as the author, a concept that is clearly absurd,” he wrote. It’s a breathtakingly myopic view of A.I. authorship to assume that people deploy L.L.M.s as they would a pen—an empty technological vessel adopted to transcribe conscious thought, absent of any mediation that may alter ideas, let alone language itself. Look past all the hand-wringing and false equivalences, though, and the critics of A.I.-detection systems appear to be arguing for something very simple; there are ethical and uncontroversial ways to engage with chatbots, they claim, such as copy editing or proofreading, and people who use chatbots appropriately should not have to fear public shaming or stigmatization every time they paste a passage from Claude into a word processor.
Such philosophical delineations between authenticity and fraudulence, merit and duplicity, are collapsing and distorting amid our current A.I. upheaval, morphing into what tech conglomerates want us to understand as “collaboration” with their products. Although Thompson argues that L.L.M.s do not create ideas but assist humans in actualizing their preëxisting ones, thereby allowing humans to evolve in harmony alongside A.I., it is clear that most of us still care about maintaining the distinction between human work and a machine’s statistical output, or between taking pride in something that is wholly our own and doing so for something that was “assisted” or aided by an L.L.M.
In his essay “Poetry and the Primitive,” the writer Gary Snyder defined poetry as “the skilled and inspired use of the voice and language to embody rare and powerful states of mind that are in immediate origin personal to the singer, but at deep levels common to all who listen.” This is why hysteria strikes every time an artist is caught using A.I.—often without disclosing their usage—and also why professors remain flabbergasted when students automate even the simplest of assignments. Writing essays, songs, stories, poems, screenplays, articles, blogs, e-mails—whatever—should not merely be a means to an end, an efficiency project evaluated on the metrics of whether it “works” or accomplishes its supposed objective; it should emerge from the parts of ourselves machines can never access. “A hand pushing a button may wield great power, but that hand will never learn what a hand can do,” Snyder wrote. “Poetry must sing or speak from authentic experience.” Anthropic’s watermark may appear to presage a future where humans and machines can honestly and ethically coexist, where people who use L.L.M.s do so in a fully pellucid manner. And maybe it will—but here’s to hoping that our stigmatization of A.I. continues to protect the dignity of our hands, which can push the button, but can also do so much more. ♦