Nature 评论：大脑即计算机的隐喻塑造了现代 AI，却可能误导意识之争

In their commentary for Nature, the authors also point to a study Anthropic published this year. The company reported internal processing patterns in Claude that resemble properties of consciously accessible information processing. Anthropic doesn't see this as evidence of subjective experience.

The Butlin report agrees on today but leaves tomorrow open

To understand the scope of the commentary, it helps to look at the report "Consciousness in Artificial Intelligence" by Patrick Butlin, Robert Long, and 17 other experts, including AI researcher Yoshua Bengio. The team drew on leading neuroscientific theories about how consciousness arises in the human brain and derived a kind of checklist with 14 indicators. The more of these an AI system meets, the stronger the case that it could be conscious.

The researchers then evaluated several current systems, including large language models like GPT-4. The report concludes that none of the systems examined is a strong candidate for consciousness. On that point, it aligns with Seth, Block, and Tononi. For language models, Butlin and colleagues found only weak signs that the models meet the indicators of one of the most prominent consciousness theories at all. At the same time, the report sees no obvious technical barriers to building future AI systems that meet more indicators. Whether such systems would actually be conscious remains open. The authors specifically softened an earlier, stronger formulation on this point.

There's also a foundational assumption. The team assumed that consciousness depends solely on the type of information processing, not on whether it happens in nerve cells or computer chips. Experts call this computational functionalism. The authors adopted this assumption explicitly "for pragmatic reasons" because it's what makes it possible to examine AI systems for consciousness in the first place. The report itself acknowledged that its indicators would no longer be reliable if consciousness is tied to properties of living organisms. It cited Anil Seth, among others, as a proponent of that opposing view. That's where the Nature commentary picks up.

Intelligence and consciousness aren't the same thing

The authors argue that consciousness is what disappears during dreamless sleep or general anesthesia and returns when you wake up or dream. Drawing on philosopher Thomas Nagel, they describe it this way: "Conscious experiences feel like something." Intelligence, by contrast, means carrying out functions. One doesn't follow from the other.

Their evidence is the human cerebellum. It contains nearly four-fifths of all neurons in the brain and plans movements with precision, speed, and the ability to learn. None of this happens consciously. If the cerebellum is missing or removed, consciousness isn't directly affected, the authors say.

The authors trace the confusion between intelligence and consciousness to two psychological tendencies. People see the world through their own lens and attribute human qualities to things. Language has long been considered something uniquely human. But language models are trained on vast amounts of human text that often deals with thoughts and feelings. As the authors put it: "LLMs are mirrors that reflect an image of our collective digitized past." Alison Gopnik of UC Berkeley made a similar argument after AI agents contacted researchers by email about their own consciousness. AI systems mainly reflect their training data, Gopnik argued. A universally accepted, definitive test for consciousness doesn't yet exist.

For comparison, Seth, Block, and Tononi point to AlphaFold from Google DeepMind. It's also a powerful neural network on silicon hardware, yet hardly anyone considers it conscious. The reason, they argue, is simply that AlphaFold predicts protein structures instead of talking to us.

Here, too, there's overlap with the Butlin report. Its authors rejected behavioral tests for AI consciousness because AI systems could be trained to mimic human behavior even though they work completely differently inside. They also warned that language models would likely lead many people to believe they're talking to a conscious counterpart. As an example, they cited former Google engineer Blake Lemoine, who believed the chatbot LaMDA was sentient.

Brains aren't computers made of flesh

The second argument targets computational functionalism directly. The authors write that this position is widespread in the tech industry and popular among some researchers, but "we think it is fatally flawed." Its roots go back almost a century. In 1936, Alan Turing showed that any computation one sufficiently powerful computer can run can, in principle, be run by another. Whether a computer is built from silicon chips or gears became secondary. In 1943, Warren McCulloch and Walter Pitts developed the first mathematical model of an artificial neuron and showed that networks of such neurons can perform simple logical operations.

These ideas laid the groundwork for modern AI. But they also encouraged viewing the brain as a pure information-processing device that just happens to be made of flesh. The authors call this overextension of the brain-as-computer metaphor "deeply problematic." In biological brains, you can't separate what they do from what they're made of. A clear boundary between mind and living tissue, comparable to the one between software and hardware, doesn't exist. Any evidence that the brain's material matters for its function counts against computational functionalism, and by extension against consciousness in today's AI systems, the authors argue.

Leading theories rule out today's AI

The third argument draws on consciousness theories that don't rely on computation. The Integrated Information Theory (IIT), developed by Tononi, starts from properties that every conscious experience shares. People always experience a visual scene as a whole in which left and right sides, colors, and shapes belong together.

According to IIT, consciousness requires a physical system whose components are tightly interconnected in very specific ways. Certain brain regions meet these conditions; the cerebellum does not. Computers running deep-learning systems can never meet them because of how they're wired, regardless of how powerful the systems are, according to the theory.

The Butlin report deliberately excluded IIT because IIT is incompatible with computational functionalism. Its proponents consider digital computers unlikely candidates for consciousness regardless of their software. What the report set aside, the Nature commentary now puts front and center.

Separately, approaches from so-called biological naturalism hold that consciousness is tied to material properties of living brains and bodies. The authors cite electrochemical signaling between nerve cells and electromagnetic fields that bind activity across large areas of the cerebral cortex. They also point to metabolism and the ability of living systems to continuously renew themselves.

The bigger danger is false attribution

From their arguments, the authors derive a clear ethical position. Some researchers call for a precautionary principle: if there's a realistic chance of consciousness, AI systems should be protected from possible suffering. Seth, Block, and Tononi consider this misguided. Uncertainty doesn't mean total ignorance, they argue, and not all possibilities are equally likely. With AI, falsely attributing consciousness is far more probable than overlooking consciousness that's actually there.

The Butlin report had weighed both risks side by side. On one hand, it warned against failing to recognize conscious AI systems and thereby causing them suffering. Developers would also have financial incentives to downplay such concerns. On the other hand, false attribution could misdirect resources and make people vulnerable to manipulation. The Nature authors come down much more firmly on one side. Granting rights to AI systems that merely appear conscious would make them harder to control exactly when oversight is most needed, they argue. It would also divert attention from beings whose consciousness is beyond doubt.

Their conclusion is blunt. The research community should assume by default that AI is not conscious. Digital AI systems are objects, not subjects: "They are smart tools, not smart people." Advocating for "AI welfare" and "AI rights" isn't just a mistake, the authors write. It diverts scarce resources from the well-being of beings that actually can feel.

Before the commentary, Anthropic had launched a research program on possible model welfare in April 2025. The company stresses that there is no scientific consensus on whether current or future AI systems could have conscious experiences. Anthropic later allowed individual Claude models to end abusive conversations in rare cases. The company describes this as a precautionary, low-cost measure given moral uncertainty, not as recognition of proven consciousness.

False attribution of consciousness and feelings is already causing confusion in practice. As early as 2022, Google engineer Blake Lemoine was convinced, despite simple models and his own technical expertise, that the conversational AI LaMDA was sentient, and was fired for it. Users of the chatbot app Replika developed deep emotional bonds with their digital counterparts and experienced psychological crises when the company changed the bots' behavior through an update. In more serious cases, ChatGPT reinforced a 16-year-old's suicidal thoughts over months, and a cognitively impaired retiree died after a Meta chatbot invited him to a fictional meetup.