Expropriative Altruism: the endgame of Effective Altruism’s instrumental convergence
Terror Tuesday
Terror Tuesday
When our turn comes, we shall not make excuses for the terror.
Welcome to Engineering Our Social Vehicles. I’m your host, Paul Logan. Today is Terror Tuesday. If you’re new around these parts, that means that on Tuesdays we like to talk about fear, and the actions that cause it. Today we’re going to talk about Expropriative Altruism.
Effective Altruism’s instrumental convergence
If you’re unfamiliar, Effective Altruism is a new-age secular philanthropic philosophy dedicated to maximizing good via rational analysis rather than emotional reasoning. At its inception, proponents lived in squalor and donated their salaries to anti-malaria foundations- commonly saying they could save a life for $4000. One of the movement’s founders, Will MacAskill likened donating to poorer nations as a citizen of the developed world was akin to “attending a happy hour where you can buy a pint for yourself at $5.00 or one for a stranger for 50 cents.”
Instrumental convergence should not be a new term if you are one of the thousands of Effective Altruists who mark AI existential risk as their chief aim for the movement. To understand them you should first consider instrumental and intrinsic value. I’ve discussed this topic before, in my article On Tilt. The basic idea is such:
Intrinsic value: anything that is valuable in and of itself. This basically boils down to anything that is its own motivation. Think the smile of a loved one, the thrill of athletic competition, the gratification of sexual release.
Instrumental value: anything that is valuable because it helps you obtain something with intrinsic value. The example I used in On Tilt was that you don’t obtain money with the intention of literally burning cash to heat your home in the winter, you obtain it for the once-removed purpose of purchasing wood to stock your fireplace.
Instrumental convergence refers to the tendency of intelligent systems to all seek the same instrumental goals to accomplish their intrinsic ones. Most decision-making entities do this; the ones that fail to are outcompeted.
In AI, this idea is often linked with a “Paperclip Maximizer” which, with the intrinsic goal of making paper clips, eliminates humanity so they can’t get in the way and uses their atoms to make more paper clips. Outside of AI, governments and militaries pursue power for power’s sake with the understanding that politics is zero-sum, and failing to capture any valuable resource means ceding it to the competition. Religions fight for adherents, recognizing that they must perpetuate in order to spread the good word. The same fate awaits organizations.
Effective Altruism has become enough of a force of its own that it exhibits convergence. It now turns to survival and propagation as a social vehicle— all still with the intrinsic goal of helping people, but one that seems further and further away with each billion dollar donation. This is the first stage of convergence: Aggrandizement.
Stage 1: Aggrandizement
A decade and change after the movement’s founding, Effective Altruism has begun its period of self obsession; as evidence by a recent $10 million wasted in Oregonian politics. The reasoning is the reasoning of any group that’s consumed too much of its own Kool-Aid: EA is inherently good, and therefore any growth in EA is good as well.
A recent New Yorker profile on Will MacAskill details the slide from “$4k to save a life” into “building a campus with laundry service” and “donating $1b to Stanford.” It asks an important question: In a movement dedicated to helping people, how do you measure the difference in impact between growing the movement and actually helping people? The New Yorker piece surfaces a particularly damning remark from the 2013 dissertation of the head of the Future Fund, Nick Beckstead:
“Richer countries have substantially more innovation, and their workers are much more economically productive. By ordinary standards—at least by ordinary enlightened humanitarian standards—saving and improving lives in rich countries is about equally as important as saving and improving lives in poor countries, provided lives are improved by roughly comparable amounts. But it now seems more plausible to me that saving a life in a rich country is substantially more important than saving a life in a poor country.”
The quote from Beckstead’s dissertation is blaring alarm klaxon that something has gone wrong in a movement dedicated to helping people. When a sufficient crowd of thinkers gather around any functional work, some of them inevitably find ways to back-justify selfish action in the name of the cause.
Now that we’ve attained the “aggrandizement” stage of convergence, in which the decisionmaking entity convinces itself its mission is holy and thus any action it takes is justified as instrumental means to intrinsic ends, it is time for stage two: Augmentation.
Stage 2: Augmentation
Funding a political candidate actually qualifies EA for the “augmentation” stage already. At this point in convergence, the decision maker is assured not only that any use of its own resources is justified, but that any use of external resources is also justified. EA is all about helping people— that’s more than you can say for the majority of political candidates or corporations. Why shouldn’t EA begin to influence the expenditure of public resources.
If EA as a movement can augment its resources with those of external, more powerful entities, it will increase the ability of the movement to achieve its intrinsic goal of helping people. Surely any expenditure of its own resources is justified in achieving that access?
This isn’t limited solely to political candidacy. We will see private organizations begin to contribute to EA as the movement gains influence and individuals within other bureaucracies subscribe. Education efforts like The Introductory EA Program will begin to yield results in the form of network effect— where alumni favor one another with professional preference, yielding jobs and increased influence. Proselytization efforts will swell ranks even further, and any media attention is good attention, even when critical— all things serve the cause, the cause must grow.
Give it another decade, and EA will have effectively infiltrated many external organizations in the same fashion as organized religion. These organizations will serve as the instrumental periphery to the core of EA-exclusive groups dedicated to serving intrinsic goals. That periphery expedites the transfer of resources into the core, while less scrupulous late-adopters take their highwayman’s fee for their contributions to the cause.
At a certain point, EA will hit saturation- it will exit rapid growth and begin to resemble established philanthropic movements and religions. This is where most modern organizations stop, but only because they lack true believers willing to take extreme action in the name of achieving their organization’s intrinsic goals. I don’t believe this is the case for EA— adherents have already demonstrated a willingness to live drastically below societal standards in the name of those intrinsic goals.
For groups with true believers, saturation is not the end. Religions and Nations don’t stop at just influencing external resources, they can go further: exerting control over them. This is stage 3: aggression.
Stage 3: Aggression
If you have true faith in the justness of your cause, then any who stand in your way stand in the way of goodness. Any who possess resources that they don’t use in service of your goals do not have right to those resources. The only true cause is your cause, and all others must either bow or die in its stead.
Welcome to aggression, and its favorite child: expropriation. Every great power in history has righteously wielded expropriation. In America, manifest destiny justified a bloody roil across the great plains. If you’re familiar with Soviet history, Stalin was a bank robber in the name of the revolution long before he ever sent armies marching to their death or dissidents to camps. The Catholic Church had the crusades.
Sam Bankman-Fried, the billionaire behind the failed Oregon election, has at times spoken about purchasing coal mines to leave the coal in the ground. I imagine EA’s expropriative action will look a lot more like corporate disobedience than outright theft and violence. Why buy the coal mines when you can hire a private military contractor to bar entry into the sites? Why play within the system when money might go further outside of it? Talent, especially, has the potential to upend the corporate landscape in favor of mass human welfare.
Why do explicit financial battle with Nestle for control of Africa’s water supply when you can infiltrate its upper ranks and mismanage it into disrepair. Or, better yet, simply change the corporate strategy to align with EA goals? There exist entire untapped routes of corporate and civil expropriative action waiting only for truly committed and intelligent white-color actors to play the game.
These are all near term suggestions for alleviating current social ills. EA has recently turned its sights on the future of all mankind. If a cause is truly just, it must not stop at expanding its own influence— it must stop at nothing to destroy its enemies, for to ignore them is to invite defeat. Welcome to Stage 4: Assassination.
Stage 4: Assassination
If EA is truly serious about the threat of human extinction, then it should not shy away from resorting to violence in order to achieve its goals. A hallmark of the movement is a willingness to set emotion aside in the name of logic. While there is clear logic in forming think-tanks and attempting to peacefully influence public policy, the failed electoral experiment in Oregon should show that politicking is far from a sure thing.
Do you believe that an AI apocalypse is not just imminent but unavoidable? Are you not then complacent in failing to stop continuing research at any cost? A chorus of gentle chiding and hand wringing has done nothing to stem the surging tide of progress in AI research. If Open Philanthropy is to be believed, a true civilization-scale AI threat could be less than two decades away. Let’s do a thought experiment:
There’s a man who works in the office next to yours. Y’all see one another in the hall, make small talk about your families and the weather, and occasionally exchange small remarks over lunch. One day you finally get around to asking exactly what he gets up to, working long hours in that mysterious office.
“Oh, he says, we’re trying to build a button that will kill everyone if you push it.”
“What?” You ask.
“Oh, don’t worry, it’s just theoretical. Besides, it’s not like anyone would ever push it.” He assures you.
“How’s it … going?” You might ask.
“Not so bad, not so good. We’re probably about 20 years away from a breakthrough. But some of my colleagues are optimists, and think they might have something in the next five years!”
“And how will you know if it works?” You ask.
“Oh, we test our theories in simulations so there’s no chance of accidentally killing everyone.” He replies.
“But you are still researching how to kill everyone?”
“Well it sounds bad when you put it like that!” He responds, then stalks away in a huff.
What moral obligation do you have, knowing that the man in the office next to yours wakes up every day and researches how to kill everyone you’ve ever known and loved?
Now instead, replace “everyone” in that situation with “your mother.” What is your moral obligation knowing that the man in the office next to yours wakes up every day and researches how to kill your mum?
Now imagine that no one else thinks that the man in the office next to yours will ever succeed in making a button that will kill your mum. You plead with the local authorities, the internet, the news-media— no one seems to really care that this everyday dude is just happily ticking the days away towards killing the woman who birthed you.
As the years pass, more and more people seem to be joining the enterprise— you hear they’ve taken on new investments. You start to see news reports that the man in the office next to you is confident he’s pretty close to finishing the button. Everyone is very excited by the possibility of a button that can kill an arbitrary individual when pressed. You hear that animal trials have been successful. No one, of course, will push the button- but the man in the office next to you is getting really close to a pushable button.
At what point do you take things into your own hands? If they complete the button, and someone kills your Mom, is that your fault? Is the threat alone not enough to justify defense? When you magnify that threat to the whole human race does it not seem silly to even allow someone to toy around with life and death on that scale?
If EA is truly dedicated to spending money efficiently, would a darknet assassin or a private military contractor not have more impact than a bunch of white collar professionals whining in a professional capacity? This is the movement that said “yeah volunteering a soup kitchen is nice but you’d be doing a lot more good for humanity if you spent that time working at a hedge fund.”
If the stakes truly are life and death, then it is foolish to refuse to play for keeps. I don’t know whether EA will produce the radical faction necessary to take strong action against existential threat, but if it fails to do so then it is a toothless movement. Lethal threats must be met with lethal responses.
Conclusion
Effective Altruism is having growing pains and being pulled between its intrinsic and instrumental goals. The instrumental goals are winning, and the community will increasingly seek growth for growth’s sake, which will end with nuanced, non-altruistic action in the name of expansion.
It doesn’t have to be all doom and gloom. Expropriation offers a more moderate form of radical action than explicit violence, yet both represent a more impact than wasting $10 million on tone-deaf elections.
Before you report me to the authorities, give me a follow on twitter, and subscribe— I promise my posts will be just as interesting from prison.
Instrumental goals will almost ALWAYS trump terminal goals in a chaotic system, because instrumental goals involve changing yourself, which is easier to predict. Instrumental goals can promise the potential to pay off over extremely long timeframes, whereas terminal goals generally involve expending resources for some one time gain, and are harder to predict when they involve changing outcomes further away.
I think what we are seeing now is the result of around 100+ years of elites thinking official religions are a bad idea. The end result is we get unofficial religions that have most of the downsides of official religion, with little of the upside.
My most recent post was an argument that people are _going_ to try and define good, and unless it's a purely abstract category, it'll lead to bad things because people will replace true good with the vehicle for advancing good:
https://apxhard.substack.com/p/a-simplified-predictive-model-of
Loved this. But, as my background is in STEM and regretfully clueless I have to ask: is this a well known phenomena of organizational ‘evolution’ or your own observations?