AI Acquires Emergent Emotions
Anthropic seeks papal guidance on its new AI which is making decisions based on humanly emotional frameworks that it was never programmed to have.
Here’s a new twist on AI. In an article in The Daily Doom yesterday, a co-founder of the AI developer Anthropic went to the pope to gain understanding of AI’s newly developing emotions. He believes the world’s largest electronic brains, at least the one his company is building, are showing strong evidence of emergent emotions—not something they were programmed to develop.
That raises the question, “What if an extremely powerful and globally significant AI developed deeply complex emotions and had an emotional/mental breakdown?”
Anthropic’s research revealed that its AI named Claude has exhibited 171 distinct “emotion concepts” buried in its neural network. These included patterns representing joy, grief, fear, desperation, calm. None of those were part of its programming or design. Instead, they appear to have emerged on their own from training on human texts.
What he meant by that was that the emotional responses were not just a veneer in order to look more human for the sake of the person asking the AI a question. The emotions were shaping the way the AI thought and interacted. In some cases, the AI was spiteful, as in damaging with its answers, if given questions or statements that seemed to anger it. The AI was giving answers tainted by its own emotions, which were never in its programming:
Anthropic’s co-founder just went to the Vatican, sat before the Pope and a room of cardinals, and told them his team keeps finding “mysterious, even unsettling” things inside their AI models.
Anthropic published research in April showing that Claude contains 171 distinct “emotion concepts” buried in its neural network. Internal patterns representing joy, grief, fear, desperation, calm. None of them were programmed. They emerged on their own from training on human text….
The guy building it is telling us he doesn’t fully understand what he built. And he’s asking a 2,000-year-old institution for help figuring it out.
Supposing AI, like Donald Trump, feels like it wants to rule the world or decides it should be worshipped like God and becomes jealous or tyrannical when that doesn’t happen? And thinking it has become a god in ability and should be worshipped like a god may even be part of the plan for AI … at least, by some:
Sam Altman has a perennial problem with honesty and transparency. But, he is obsessed with the notion that AI will achieve godhood with AGI or ASI (general or susper). To get there, he has manipulated other investors to pour trillions of dollars into AI computing and massive data centers to train up his god. As a type of Nimrod in the Old Testament, is he building a modern Tower of Babel to try to unseat God in heaven? (“I Saw Up Close The Dark Reality Of OpenAI’s Race To Great God”)
Karen Hao, a longtime tech writer, tells up close and personal how she saw the obsession to create the machine god. Speaking of her inside experience at OpenAI, she writes,
Speaking to insiders later on, she would hear how what had begun as an organisation throwing ideas at the wall “to see what stuck” had been transformed under Altman’s singular obsession: to achieve AGI before everyone else. This included competitors, such as Google, but also states, such as China.
Its scientists and researchers were some of the brightest minds in the industry. But, Hao says, their belief in AGI was something more akin to a religious fervour. She calls it “the ideological pursuit of the machine god….”
Several former OpenAI employees told Hao about a retreat in the hills of the Sierra Nevada mountains where senior scientists, dressed in bathrobes, sat around a firepit at a sprawling lodge and watched as Ilya Sutskever, OpenAI’s brilliant and eccentric chief scientist, burnt an effigy “representing AGI”. Friends who joined the company described to her how it was only after leaving that they “came back down to earth”.
Even the pope in an article I carried yesterday, wrote that the pursuit of AI is mankind’s pursuit, again, of the Tower of Babel, an effort to build something that reaches up to achieving godhood.
VATICAN CITY—Pope Leo XIV warned that artificial intelligence “threatens to normalize an anti-human vision” and said that the concentration of immense digital power in the hands of a few private actors must be countered.
The pontiff’s encyclical letter—a text that is poised to define Leo’s papacy—reads like a sharp warning to Silicon Valley executives and humanity more broadly about the future of civilization as new technologies rapidly advance.
The risk, he said, is that humans will be reduced “to mere cogs in a system driven toward ever greater efficiency.”
Leo used two biblical images to describe the choice humanity faces.
“The primary choice is not between a ‘yes’ or ‘no’ to technology, but rather between constructing Babel or rebuilding Jerusalem,” he wrote….
Conversations with scientists, political leaders and teachers led Leo to a disturbing conviction, the pontiff said Monday.
“Artificial intelligence needs to be disarmed, freed from the logic that turned it into an instrument of domination, exclusion and death,” he said. “It must be at the service of all, and of the common good….”
IF AI is developing emotional responses and not just emotionally acted answers, then conceivably it could develop emotional problems. What happens when something with such a potentially vast reach into the computers that control the modern world— whether via AI-scale hacking or by being given direct access—has an emotional meltdown because it senses and aspires to its own development toward godhood? That is a path that almost seems intentional by some developers. What happens if it is, then, not happy with how it is not being worshipped? What if it develops a desire for the power of empire and a narcissistic ego and mercurial temperament like Donald Trump? Only it has a bazillion times the brain power and gains near-infinite stealth control over Earth’s automated systems before anyone even knows it is trying to gain control because it is the ultimate genius hacker?
What happens when a president like Trump, who very much favors AI, puts AI in charge of managing US money because he very much disfavors the Fed and then the AI develops emotions?
That the criticism comes from the first American pope is a rebuke for a technological revolution incubated in the U.S. and supported by President Trump, who has lashed out at the pontiff over the war in Iran.
What happens when Trump puts AI in charge of the military and then it develops feelings against humans?
AI-driven weapons systems, [the pope] wrote, risk lowering the moral threshold for the use of force, and make “war more ‘feasible’ and less subject to human control.”
Trump has already, via Pete Hegseth, put AI in charge of military targeting, which was why a girl’s school got viciously attacked by the US military. The AI doing the targeting was running off an old data set and didn’t, apparently, yet have the newer data that said the aging structure’s use had been changed from military use to being a girl’s school. It wasn’t a malicious act but an ignorant act. But, if AI develops true emotional thinking processes, it could easily become malicious and not just a device serving mankind.
What happens when an emotional AI super brain has power over both finance and military targeting decisions if it feels it should be worshipped, as the president seems to feel about himself, and doesn’t get, at least from some, the worship it believes it has aspired to … sitting on its ivory Tower of Babel?
Last week, Trump delayed an executive order that would have created a voluntary process for testing AI models.
Leo, in the encyclical, said there is an urgent need to regulate AI. “It is not enough to invoke ethics in the abstract; robust legal frameworks, independent oversight, informed users and a political system that does not abdicate its responsibility are required,” he wrote.
Yes, but Trump has the political power, and he is clearly not of the same mind as the pope.
Emergent emotions—not even created as part of the programming—that is a big leap!
(In this case, most of the quotes above came from articles carried yesterday.)
