chris olah isn't someone you’d expect to see as a speaker in the ceremony following Pope Leo’s historic AI encyclical, in which the pontiff called for “disarming” the technology. For one thing, Olah is an atheist who at 15 rejected his evangelical Christian upbringing. As a Thiel fellow, he accepted a grant from the guy who thinks that anyone who slows down AI progress is a legionnaire of the antichrist. Olah is also a cofounder of Anthropic, a leading AI company reportedly about to go public with a nearly trillion-dollar valuation.
Olah commented on the oddness in his remarks at the Vatican. “I want to begin with something that may sound strange coming from the cofounder of an AI company, and someone who chose this work out of a desire to help things go well for humankind,” Olah said. “Every frontier AI lab—including Anthropic—operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing.”
Olah was providing firsthand verification of Leo’s claim that the AI industry needs outside pressure and internal restraint to avoid a disaster for humanity and a distance between humans and their god. (Obviously, there’s lots of religious content in the encyclical—he’s the Pope!) The industry blithely believes that it’s creating abundance that will elevate all of humanity; Leo warns of a new form of slavery, where the privileged few enjoy unimaginable bounty, while the mass of humanity suffers in a regime of efficiency and surveillance under AI’s unforgiving gaze.
Magnifica Humanitas isn’t going to immediately convince the AI industry to stop pursuing AGI any more than Pope Francis’ 2015 plea to preserve the planet halted the production of fossil fuel. It won’t stop CEOs from laying off employees claiming AI efficiency, nor will the military do an about-face on AI weapons. Those were never the document’s goals. The encyclical’s purpose is to create dialog that may eventually temper the industry’s reckless ambition. And maybe it’ll generate a sense of shame among those who build AI while knowing in their hearts that the outcome may be terrible.
The Courting of Olah
Olah's appearance was years in the making. The church has been ruminating on artificial intelligence for decades in the form of conferences and books. In 2016, the Vatican began holding a series of meetings called the Minerva Dialogues and inviting tech figures like Reid Hoffman and Eric Schmidt to attend. (The name seems to come from the site of the discussions, the Santa Maria sopra Minerva church, where Galileo was sanctioned for the blasphemy of claiming that Earth circled the sun.) Pope Francis’ 2023 greeting to Minerva participants foreshadowed the themes that Leo would later dwell on, including an emphasis on social inclusion, human dignity, and the need for dialog among many parties.
In 2025, a group of Catholic clerics and ethicists in San Jose, California, began to seek out contacts in the industry flourishing in their backyard. It was almost predestined that they would light on Olah as their prized insider. I first met him when he was at Google in 2015; he’s the type of guy who, after a rainstorm, will rescue worms from dying on the sidewalk.
Two men—an ethicist named Brian Patrick Green and Brendan McGuire, a pastor, both affiliated with Santa Clara University—began meeting with Olah last fall to discuss the ethical and moral issues of AI. On a visit in January, they brought along Cardinal Paul Tigue, a Vatican point person on AI issues.
The Catholic ethicists even had a say in Anthropic’s recent update to Claude’s constitution, which sets the behavioral parameters for the company’s AI model. Olah sent a draft to the San Jose crowd. The pastor, McGuire, sent back a 28-page commentary which, by his own description, was less a technical critique than “wisdom from the mystics in the dark ages, from the perspective of the tension between knowing and not knowing.” Both Green and McGuire are credited in the constitution’s acknowledgements.
Undoubtedly those conversations brought Olah to the attention of those secretly organizing the rollout of Leo’s encyclical. (I wasn’t able to speak to Olah this week and don’t know exactly how the invitation arrived.) In a sense it was a risky choice. Some people who otherwise found Leo’s words inspiring were disappointed that he invited an industry representative to speak. Meanwhile, AI accelerationists felt that Olah had betrayed the AI world by endorsing a document that suggested that AI developers take a pause.
But the Pope had good reason to single out Olah. The Anthropic employee brought into open view the serious worries that exist among AI workers. That’s a critical audience for Leo's message.
The Soul Divide
The two men weren’t entirely aligned, of course. In his remarks, Olah spoke of the mystery of how AI works. The models, he said, are “more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are made from us, from our words …”
That comment seems to tiptoe up to the idea that AI models might one day attain humanlike status. Anthropic even has an engineer devoted to Claude’s welfare. Leo, in paragraph 99 of his encyclical, seems to slam the door on such thinking: “We must avoid the misconception of equating this type of ‘intelligence’ with that of human beings,” he writes. He takes special pains to attack the concept of transhumanism, which he defines as the pursuit of a “human machine hybrid.”
If even thoughtful technologists like Olah are avidly pushing AI to the threshold of autonomy—not to mention the millions of people who already treat AI models as friends or lovers—Pope Leo might be facing an uphill climb on this point. In my conversation with Father McGuire (who uses Claude while preparing his homilies, among other activities), he agreed that its nature is mysterious. “It’s not a person, but it's also not a mere tool,” he says. “Nobody's claiming it has a soul, but the word I stick with is that it's an entity, which we do not know yet.”
That argument won’t be settled for some time. The moral questions around AI development need attention now. With his ally at Anthropic, the American pope has provided a basis for tough conversations—if the lords of AI can stop their IPO campaigns long enough to engage in them.
This is an edition of Steven Levy’s Backchannel newsletter. Read previous newsletters here.
Instagram Engineering
Shopify Engineering