定义人工智能精神错乱。第二部分:“多产型人工智能精神错乱”
Defining AI Psychosis. Part 2: "Prolific AI Psychosis"

原始链接: https://jeffs.blog/p/defining-ai-psychosis-part-2-prolific

本文探讨了“多产人工智能精神错乱”(prolific AI psychosis)这一状态,即过度依赖人工智能工具导致批判性思维丧失以及与现实世界效用脱节的现象。 尽管现代人工智能代理能够加速编码等任务,但它们往往产出高频率、低质量的内容。用户可能会陷入“间歇性强化”的陷阱——将人工智能视为老虎机,在获得频繁且看似奇迹般的成功时,也伴随着掩盖了潜在混乱或技术债务的“伪胜利”。作者指出,该病症的标志是评估自身工作质量和目的的能力下降,这种情况往往因过度专注、睡眠不足和孤立而加剧。 作者认为,虽然这些工具提供了强大的杠杆作用,但它们需要严格的人工监督和“匠心”。对于那些受过高野心或对被淘汰的恐惧所驱动的人来说,风险最高,这会导致他们优先考虑纯粹的产出而非实际价值。归根结底,要避免这种状态,需要设定明确的界限,在人工智能驱动的工作之外保持生活,并依靠人类的判断来定义什么是真正有用、理想且制作精良的。

这场 Hacker News 的讨论围绕着“多产 AI 精神病”(Prolific AI Psychosis)这一概念展开。精神科医生 Jeff S. 将其定义为:个人在不为工作增加实质价值的情况下,大量产出 AI 生成内容的现象。 评论者对该术语的有效性和实用性持不同意见: * **怀疑态度:** 许多人认为这只是“糟糕同事”行为的重新包装,这种行为在 AI 出现之前就已存在。批评者认为,该术语可能会被那些反感 AI 的人武器化,以此给低产员工贴上标签,这可能掩盖了准确评估工作表现的难度。 * **支持与细微差别:** 另一些人认为,该术语在临床或管理背景下是有用的,可以识别出那些原本高效、但因过度依赖 AI 而导致工作能力下降的员工。一些贡献者指出,尽管低质量工作并非新鲜事,但 AI 显著加剧了这一问题,使其变得更明显、更容易识别。 * **技术背景:** 专业人士强调,尽管初级员工可能会产出“垃圾内容”,但经验丰富的导师对于指导如何将 AI 作为有效工具仍然至关重要。他们区分了那些利用 AI 提升能力的人,以及那些利用 AI 逃避工作的人。
相关文章

原文

Last week, I suggested that the term AI psychosis is applied to at least three different situations:

1) Genuine psychotic experiences related to LLM use, a phenomenon I described as “true AI psychosis.”

2) Hyperengagement with AI tools that is accompanied by a softer disconnection from reality. I’m calling this “prolific AI psychosis.”

3) Maladaptive relationships with AI chatbots that mimic human connection. I’m calling this “parasocial AI psychosis.”

I then discussed true AI psychosis. I clarified what psychosis means to a psychiatrist. And I concluded that true AI psychosis is generally a variant of pre-existing psychosis rather than a new syndrome.

This week, I’ll describe my current understanding of prolific AI psychosis.

Prolific AI Psychosis

Prolific AI psychosis occurs when a person generates a large quantity of AI output without significantly increasing the real value of their work. In some cases, the new AI workflow may even destroy value.

For example, a software engineer with prolific AI psychosis can produce thousands of lines of code every day, but the code itself has little real-world utility. In contrast, a productive developer may create less code, but the created code is valuable to users and to their organization. Some days will include large additions to the codebase, other days will be spent deleting unnecessary code, and occasionally, a software-breaking bug will be fixed by changing a single character. Lines of code are only loosely correlated with productivity.

Let me translate that into more familiar terms. Consider your favorite author. If you know that they currently write a respectable 1,000 words per day, you would rightfully worry if they begin writing 100,000 words per day. The care that goes into choosing 1,000 great words can’t be maintained at 100x volume [1].

The problem in prolific AI psychosis is not increased output. (More code and more text may actually be more productive.)

The problem lies in the subject’s perception of their output: they can’t assess the quality of their own work. The phenomenon mimics psychosis because the subject experiences a mild disconnection from reality: a defect in critical thinking.

Current AI Tools

Before I go further, it’s important to describe current AI tools. For most people, AI still means a ChatGPT conversation or a Gemini summary attached to a Google search. While a few people may experience prolific AI psychosis with these tools, state-of-the-art tools pose a much greater threat.

There are similarities between current AI tools and chatbots: you enter a prompt; you get a response. But state of the art tools are now wrapped in a program called an agent harness. Instead of answering a simple question through conversation, a harness is designed to work on complex tasks until a solution is found. It acts as an agent by making decisions, opening programs, accessing the internet, and writing software to help you achieve your goals.

A great harness is especially impressive in software development. If you wanted to create an iPhone app ten years ago, you’d need to spend dozens of hours reading and writing code to get a basic app working. Today, you can type a few sentences into Claude Code, respond to a few questions, and a new app will pop out a few minutes later. It won’t be great, but it might solve your problem. (Getting it listed on the app store is an entirely different issue that can’t be solved so easily.)

A developer can do much more. Harnesses can create loops which repeatedly call AI models, working through a software development todo list until a full-fledged app is built. With discipline, the outcomes are astounding. It feels miraculous when an AI agent completes a difficult task for you.

But every superpower has its drawbacks, and AI sometimes makes decisions that are impossibly bad. Multiple new files are created when a one-line fix would have done the job. You’re confidently told that a new solution works, while your actual software is more broken than ever. The code becomes a tangled mess that is difficult to read and almost impossible to extend. Simple, decades-old problems that have been solved by open-source solutions are rebuilt from scratch, making your product painfully slow. We’ve found ways to mitigate some of these issues, but it’s a skill that takes time to develop.

Some claim that current AI models work at the level of a graduate student. That doesn’t match my experience. They’re one part senior engineer and one part toddler-running-across-white-carpet-with-a-jug-of-red-Kool-Aid. Your job is to determine which is which. Unfortunately, both sides speak with complete confidence in their abilities, and they can’t always tell when they’ve spilled the Kool-Aid. And so the software engineer’s job now includes a large amount of difficult supervision [2].

Here’s a different way to think about it: AI software development feels like playing the world’s most favorable slot machine. Most pulls on the lever are big wins! Most of the losses are obvious and small. Occasionally, a loss will look just like a win. And unless you have the skill, focus, and patience to reject counterfeit wins, your mistakes will eventually create chaos.

The Development of Prolific AI Psychosis

To illustrate the progression of LLM-tool use into prolific AI psychosis, consider a hypothetical example.

Imagine that you are a software engineer who now has a tool which can easily complete some tasks 100x faster than ever [3].

If you like building software, this is beyond exciting. You can accomplish more at work. You can build the hobby project you’ve always wanted. You can finally create the video game of your dreams. Every possibility is on the table.

And so you pull the slot machine lever and get started. While the wheels spin, you set up more machines. You rush back to review results as they come. They’re good this time. But you keep catching occasional bugs. And so you set up new machines to help you check your work. Sometimes, they catch problems, and sometimes they don’t. So you refine their instructions. You give them more capabilities. You create networks of bots which work together to solve problems. The code keeps coming.

Your ambitions grow. You’re not just creating new software; you’re going to be the first developer to run a one-billion-dollar company. It’s been predicted. Why not you? The scope of your project increases. You can’t sleep at night because you’re so excited to work. You spend more and more time with computers, and less and less time with people. Those who question your efforts don’t understand. Their criticisms are short-sighted. And even if the models aren’t perfect now, the next set of tools will be better.

Over time, your lack of sleep and hyperfocus draw you away from reality. Your software is impressive in many ways. But is it useful? Is it good? Is it something that people actually want? You don’t have time for these questions. You need to “escape the permanent underclass.”

I feel stressed just thinking about it.

While I don’t think I’ve ever experienced full-on AI psychosis, I have been through brief periods where I was headed down this path. It feels incredible to make rapid progress. And there are times when the model’s output is nothing short of miraculous.

The illusion broke when I realized that I couldn’t understand my own project. That’s not an unfamiliar feeling in tech: every advance is built upon layers of progress that I’ll never fully appreciate [4]. But I had never before written dozens of custom files without the faintest idea of how they worked. And it became impossible for me to add new features without starting over from scratch.

I’ve spoken with many developers who’ve been through a similar experience. Life is much better on the other side. There are dozens of productive AI workflows—each with advantages and disadvantages. But the most effective developers all seem to prioritize human judgment, sleep, and at least some semblance of a life outside of work.

The Cause of Prolific AI Psychosis

I can only speculate on the cause of prolific AI psychosis.

The slot-machine-style rewards of AI tools are partially responsible. Intermittent reinforcement—the receipt of positive or negative outcomes on an unpredictable schedule—is one of the most powerful drivers of human behavior. It’s also an important feature of LLM technology that can’t be wholly removed.

Individual factors are also important. I suspect that people with ADHD and impulse-control problems are at increased risk of prolific AI psychosis. Likewise, personality traits like high openness and low conscientiousness may correlate with risk.

Experience may play a role. It’s not yet clear whether new developers or senior engineers are more at risk. I suspect that new programmers are more susceptible, but accounts of prominent developers experiencing apparent prolific AI psychosis are relatively common [5].

Cultural excitement also influences developer behavior. These tools are powerful when used well. But every technological revolution is accompanied by unwarranted hype and get-rich-quick schemes. It’s easy to ignore downsides when the upside is so appealing.

The narrative that AI will replace all jobs doesn’t help [6]. This message has softened in recent months, but the fear remains significant. This is especially true for software developers, who often find it difficult to find a new role after layoffs. More output—even when it is not truly productive—still drives rewards in metrics-driven environments.

Finally, “taste” is increasingly relevant. Unfortunately, it’s also difficult to define. I prefer the term “craft,” which to me encompasses both quality and aesthetics. Regardless of language, I believe that product assessment is a conversation between humans. Is it useful? Is it good? Is it desirable? Is it delightful? These are subjective qualities that are difficult to find within an LLM [7].

I’ll close by adding that prolific AI psychosis isn’t just a problem for software developers. Anything that can be automated with AI can turn to slop with insufficient human guidance. Placing intentional boundaries is one of the most important tasks of the AI era.

That’s enough for today. Next week, I’ll write about parasocial relationships with LLM chatbots.

Footnotes:

[1] This comparison is imperfect. Software and novels are fundamentally different. Every software project has boilerplate code and a limited vocabulary. Likewise, the problems that arise in software development are different in kind from the problems seen in other creative endeavors.

[2] Much of the recent work that has advanced agentic engineering is an attempt to address this challenge. But the best work still requires a significant amount of human supervision.

[3] I’m skeptical of claims that AI makes people 100x more effective at work. I have found instances where a few minutes with a model can save a day’s worth of work, which matches the 100-fold order of magnitude. But the average increase in productivity is not this significant. Every moment saved also includes an increase in AI supervision; this is a non-trivial task. In addition, there are other bottlenecks to productivity. These include coordination, experimentation, understanding of the customer, access to resources, and internal policies.

[4] This comic is legendary for a reason: https://xkcd.com/2347/.

[5] Anecdotes are not data, and so further research will be necessary. Likewise, accusations of “AI psychosis” are not always merited. “This doesn’t meet my personal standard of quality” is a criticism that predates generative AI.

[6] Cultural beliefs that are shared within a community are not delusional. Tech communities have their share of quirks. It is not a delusion to believe (or disbelieve) in “a permanent underclass,” artificial superintelligence, or the existence of an AI bubble.

[7] I’m aware that people are working on agentic economies which sell directly to AI agents. It’s not an area I’m interested in, but it does raise questions about non-human taste.

联系我们 contact @ memedata.com