The Watermarked Shadow of AI: Quality in Peril?

The latest development in large language models (LLMs) has left many in the industry puzzled. Anthropic, a prominent player, is set to start watermarking its AI-generated text to comply with an upcoming EU regulation. While this might seem like a straightforward solution to combat disinformation and ensure accountability, it raises concerns about quality.

At first glance, watermarks may seem innocuous. However, they could affect the output of LLMs, which rely heavily on randomness to create human-like prose. This randomness allows them to avoid getting stuck in loops and producing repetitive text.

John Gruber, a veteran tech blogger, has expressed concerns that watermarks will constrain LLMs like Claude, forcing it to make worse word choices overall. According to him, this might not affect accuracy but could lead to less precise and engaging writing.

Current LLMs already struggle with making optimal sentence construction choices. Steven Murdoch, a professor of computer science at University College London, pointed out that these models rely on randomness to generate text. Without it, they’d quickly become stuck in loops and repeat themselves ad infinitum.

The real question is whether the watermarked version of Claude will be significantly different from its current iteration. Murdoch suggests it might not be noticeable, but Gruber’s concerns can’t be dismissed so easily. The watermark will alter the small choices an LLM makes when generating text – precisely the decisions that make or break writing quality.

The regulation aims to address the proliferation of AI-generated content across industries. From academic papers to social media posts, there has been a marked increase in instances where chatbot-written material has been passed off as human work. While watermarks might help mitigate this issue, they raise questions about LLMs’ ability to create original content.

The impact of watermarks on quality will be a litmus test for the entire industry. Will these models adapt and find new ways to compensate for the constraint, or will they suffer from a loss in creativity? As we navigate this uncharted territory, one thing is clear: the lines between human and machine writing are becoming increasingly blurred.

The stakes are high not just for LLMs but also for those who rely on them. Students, lawyers, and university professors will have to contend with more stringent regulations regarding AI-generated content. The most pressing concern lies in the potential damage that AI-written material can do to models themselves. Training AI on AI-written content creates “model collapse,” leading models to confuse concepts.

In the end, watermarking AI-generated text is not just about accountability or preventing disinformation – it’s also a way to ensure these chatbots don’t become less effective. Whether this comes at the cost of quality remains to be seen.