Taking warnings about runaway AI seriously is one thing. No sane person involved in developing and scaling AI believes there are no risks.
But what if the warnings of an AI apocalypse are coming from people who see AI only through a doomsday lens, who believe that the adoption of AI would lead only to the total eradication or enslavement of the human race, while ignoring other, more mundane dangers like hacking and theft?
It's called "Effective Altruism," or EA, and many Silicon Valley adherents work for AI companies like Anthropic and OpenAI. EA is a social and philosophical movement that uses evidence and reason to figure out how to help others as much as possible. One of those EA adherents, Jacob Coxon, caused a ruckus a couple of weeks ago by stating that AI will kill us all in the next decade.
"EA’s philosophy of AI doom could have a self-fulfilling quality," writes Washington Post columnist Jason Willick. "It imagines humanity as essentially a machine for calculating and maximizing the well-being of the greatest number of people, now and in the future, ignoring other theories of morality."
Coxon's warning points to the primary danger of EA thinking: It oversimplifies ethics into measurable metrics and ignores systemic political change and other moral choices.
Not all AI doomers are EA adherents. And let's face it: Some EA doomers are a little strange, like some in the movement believing that shrimp welfare is an important animal rights issue. Also, taking the emotional element out of charitable donations by focusing on the charity's "metrics" (helping more people more efficiently) rather than the human reason for donating seems cold.
The EA element in the AI industry is driving progress. What if they get it wrong? What if their efforts to align AI with human moral codes result in even more bizarre behavior?
The AI industry talks about “aligning” artificial intelligence with human value systems. Most human value systems look nothing like EA. But if the EA moral vision is triumphant in the technology world, the industry might end up making bots that are more prone to bizarre and dangerous behavior.
Fear of an apocalyptic AI takeover is rooted in the perception that machines can have motivations like people do. It isn’t true, as Microsoft recently emphasized in a helpfully “humanist” AI manifesto. But the illusion that it is true, rampant among doomers, can itself be dangerous.
If the machines are seen as having their own motivations, their creators might be increasingly absolved of responsibility for shipping unsafe products. Legal liability for one’s own creations — one of those systems developed over the centuries for managing technological risk — will ultimately collapse if AI is anthropomorphized. That will reduce the industry’s incentive to police itself.
"The fact that EAs are mostly well-meaning rather than cynical — that their alarm over artificial intelligence is ideologically sincere — doesn’t exempt their ideas from criticism," writes Willick. One need only look at the hundreds of millions of people who have been killed as a result of "sincere" communists to know how that might turn out.
“Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive," as C.S. Lewis put it in a 1949 essay, "The Humanitarian Theory of Punishment." Too many 20th- and now 21st-century catastrophes have been the result of people simply believing they're only trying to do good for all humanity.
The great Christian scholar and theologian added, “Those who torment us for our own good will torment us without end, for they do so with the approval of their own conscience. They may be more likely to go to Heaven yet at the same time likelier to make a Hell of earth.”
EA is a performative philosophy; it only works if others see the halo around your head. I would prefer any AI policy we create be based on a mix of self-interest and the good of the community. That's what's worked for 250 years, and I see no reason to change it.
Also for our VIPs: After the AI Bubble Bursts
