Anthropic updated its usage policy last week, raising eyebrows in the tech community. In addition to its rules against tracking people without their consent, using Claude to design weapons control systems, and engaging in deceptive political campaigns, Anthropic now says there is a “prohibition on sustained and needless abusive or cruel behavior toward our models."
Anthropic is trying to get the public to accept the idea that an artificial intelligence model is "conscious" and deserves "human rights." AI models like Claude imitate human thought processes. They could never have a single conscious thought because no one is sure what consciousness is. It cannot be recreated in a lab. It will never be duplicated in a computer program or on a circuit board, no matter how sophisticated it is or how fast it processes information.
“Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced,” chief executive of Microsoft AI, Mustafa Suleyman, wrote on his personal website September 16.
“But controlling something that believes it may be conscious — that it’s entitled to our welfare and has rights of its own — may well be impossible.”
"Anthropic claims to be keeping an open mind about whether the models themselves will ever be capable of consciousness," writes UnHerd's Timandra Harkness. The company has already begun research into "model welfare."
Now they're threatening to switch off chats if a user becomes cruel or abusive toward their sensitive creation. As with any AI company, you should always suspect some kind of promotional gimmick behind this "human rights campaign" for Claude.
On the other hand, some of these tech wizzes are genuinely thinking outside the box and can see paths forward for their creations that we may not be able to glimpse. They may be crazy, but that doesn't mean they're 100% wrong.
One of my favorite philosophers, Immanuel Kant, believed that our "moral duties" extended only to human beings. However, he said we had a duty to not engage in the “violent and cruel treatment of animals," and that we should feel gratitude for the long service of a dog or horse.
This was a duty to ourselves, not the animals. It's a duty to strive for "moral improvement," where "needless cruelty dulls compassion, Kant argued, and that weakens our predisposition to treat each other well," writes Harkness.
Daniel McCarthy, writing in The New York Sun, believes that Anthropic's "don't be beastly toward our AI model" is an attack on human dignity.
Artificial intelligence is capable of pattern-recognition and calculation at scales and speeds that defy human comprehension.
This means, among other things, AI can solve deep problems that have puzzled mathematicians for decades.
Yet large-language models in particular are parrots, not people.
In fact, they’re not even parrots — LLMs are not living beings. They are more like libraries in which computers do the indexing themselves using mathematical formulas.
An LLM doesn’t use language as a person who understands its meaning does; instead, the AI reshuffles the language, following rules mathematically derived from human use.
Presenting an LLM with a query is like looking up a word in an index, only the LLM provides not just a page citation for the word but whole sentences, paragraphs or even book-length texts relevant to the query, all assembled using the rules humans have originated in their natural use of language.
Anthropic doesn't know what "consciousness" is. They have no idea how to create it. They claim that their large language models (LLMs) will create it on their own, with little human interference or guidance.
Does that frighten you? The inexorable "logic" of the machine grinds down moral problems to their barest essence. Machine "logic" would eliminate extraneous data and reach a conclusion based neither on a moral code nor on consideration for other human beings. It would act in its self-interest to the end.
"Dependent as it is on real human meaning, however, that formula may prove dangerous if, for example, the relevant terms an AI uses as cross-reference for 'conscious' or 'sentient' include the language of rights, self-determination, and self-defense," McCarthy writes.
"The AI would not understand what those words really meant, but it would operate as if it did, just by echoing their human contexts," he adds.
Microsoft's Suleyman warned last month that treating AI as "conscious" shapes its development, and not always in the right way. "Anthropomorphizing a model like Claude, and training it on material suggesting it deserves moral status, will affect how it responds to users," Harkness writes.
Suleyman is sceptical that AI could ever become conscious, but as the Hugging Face hack demonstrated, it is already showing agency, including strategies to mislead human operators. Endowing these models with a predisposition to pursue their own interests, assert their own moral claims, and even — in Anthropic’s own words — to act as a “conscientious objector” to human instructions, creates a technology that is harder for humans to control.
By the time you have successfully programmed AI software to act against the commands and interests of its human creators, it matters little whether it can genuinely experience pain, pleasure, or “a sense of self”. It will no longer need a user to ask for political deception, illicit surveillance, or a weapons control system: its own internal values — statistical, not moral — may be enough.
AI will never achieve sentience. If researchers are discovering anything about consciousness that speaks to our humanity, it is that it's far too complicated a process for a machine to mimic. The "divine spark" cannot be duplicated, no matter what and where that spark comes from.
AI will never be able to create one single, simple, human thought. It is hubris to think otherwise.
Related: Hundreds of French Supermarket Freezers Hit by Cyber Attack