Hi there, welcome to Engineering Our Social Vehicles! I’m your host, Paul Logan. Today is Slow Collapse Saturday. If you’re new around these parts, that means that on Saturdays we like to talk about the subtle ways in which our society has already collapsed around us and what it will look like on its loooooooong slide into the dustbin of history. Today, we’re going to talk about Algorithmic Genocide.
What is Algorithmic Genocide?
Previous incidents of mass murder have allowed some lucky few to slip through cracks because they don’t look a certain way, or were hidden by friends. Future genocide will not look like freedom fighters sizing civilians up on the side of the road. It will look like querying a database to return the complete data footprint of all individuals who have posted, liked, followed, or joined groups in favor of a certain ideology or political party. It will look like machine-learning recommendation algorithms for high-risk individuals in an officer’s precinct. It will look like facial recognition systems flagging those citizens. If we’re really unlucky, it may even look like a drone automatically dispatching and disposing of flagged dissidents.
Perhaps the most poignant moment in the 8-minute video is when an insurgent group analyses all publicly available social media profiles for tendencies and political preferences opposite their own. They program swarms of autonomous, face recognizing, hunter-killer quadcopters to execute their political enemies en masse with this data.
The narrative is very similar to “Hated in the Nation,” a Black Mirror episode with in which a swarm of autonomous bees are directed to kill everyone who used a specific hashtag on twitter. 2
The following season Black Mirror followed up autonomous killer bees with autonomous killer dogs in Metalhead. The dogs are indiscriminate, and don’t really fall under “algorithmic genocide” unless that algorithm was “kill everyone.”
The most shocking part of Slaughterbots for me was the realization that we already have the technology to separate out the “to kill” population in the same way you’d split them into advertising demographics. Facial recognition technology to enforce that list is already deployed in some parts of the world. With those two things, all that’s left is a finger to pull the trigger.
It was equally shocking to me when I realized I got the wrong message from the Slaughterbots. People are much more worried about Metalhead than they are Hated in the Nation. This makes sense: the stakes of human extinction are much higher than the stakes of selective oppression, which has been the norm for all of organized human history. The Future of Life Institute is focused on the “autonomous” part of mass murder, not the “algorithmic” part, and released a sequel that really drove home the “autonomous bad” message in late 2021:
This brings up an important question: why should we care about algorithmic genocide when AGI (Artificial General Intelligence) extinction is around the corner? Don’t get me wrong, I think autonomous weapons are an objectively terrible threat to humanity, I’m just equally worried about the data science needed to create targeting profiles. I don’t think these two issues are opposed. In fact, I think preventing algorithmic genocide can only help build the solidarity structures that will be necessary in enforcing AGI alignment.
The AGI alignment community is pretty vague on the mechanism of human extinction. The pundits that do state a potential mechanism of extinction are a lot more concerned about super viruses with 2-month incubation periods than terminators. It doesn’t build credibility to immediately launch into science fiction when asked how the human race will go extinct. People are dumb, and won’t feel threatened if they can’t see specific images of a weapon to channel their fears into.
My bet is that we see algorithmic genocide before we see AGI extinction, and that it will follow the general process of automation by being manual before it is autonomous. Finding those situations and stopping them is a great way for the AGI alignment community to prove impact and simultaneously throw up AGI roadblocks. In addition, it would supply specific examples to point to and say “this is what the beginning of AGI-driven extinction looks like.”
If it comes down to fearing autonomous bees that kill specific people based on online data or autonomous dogs that kill everyone; I agree with the guards in Arrested Development. The bees are the more immediate danger:
Daemon and Civil War II
In Daniel Suarez’s 2006 speculative science fiction novel Daemon, a distributed computing entity wrests control of society from the powers that be. Suarez really flexes on all the ways decentralized control would outcompete central power, but perhaps the most frightening is the chapter in which he murders all spammers.
In a single afternoon, thousands of agents are independently coordinated to burst into the offices and homes of spammers all over the globe and execute them without remorse. It’s an… interesting inclusion on Suarez’s part, especially because there’s a small news article celebrating it in the novel. Who doesn’t want an end to spam, after all?
The sad thing is I’m not sure the Daemon’s depiction of a positive public response to algorithmic genocide is that far off. If news emerged that all ardent pro-lifers had been massacred in the span of an afternoon, I think there would be celebration from many of the pro-choice crowd. I have trouble imagining the same wouldn’t occur with the roles switched.
We’re hitting the point at which things are so divisive people are worried we’re going to start killing one another over opposed politics. When I read that half of americans anticipate a civil war soon, I’m flummoxed not by the belief, but the logistics. The first civil war occured along a sensical boundary between two contiguous nations:
What contiguous geography do americans have to fight over? This?
People aren’t anticipating a geopolitical power struggle, they’re anticipating a sociopolitical one. If we really do get down to fighting, the lines of battle will be drawn in consultation with personal politics. Uniforms will be few and far between. It will be war amongst the people. Any organized effort to fight the “the libs” would involve either indiscriminate slaughter of everyone who lives in an urban area, or algorithmic tagging of enemy ideological combatants.
Unsurprisingly the sequel to Daemon — Freedom™️ — runs into this problem while trying to set up a war between “new” distributed society and “old” centralized society. Daemon depicted all agents of the malicious AI’s shadow society as blending in with normal society when they aren’t donning knockoff SWAT gear and killing people with 3D printed pistols. Suarez pulls a cop out in Freedom™️ by handwaving the first book’s situation away and replacing it with “oh I guess the dark web goons have left their old lives to pursue cloistered living in self sufficient isolated communities that are clearly delineated.”
There’s a battle scene with PMC’s (Private Military Contractors) airdropping into a midwestern town to scare the now clearly marked dark web goons back into old, non-distributed ways of life. I’m still mad about how this one section of an otherwise fantastic book manages to fail in not just its goal of speculating near-future warfare, but how it manages fail at even current understanding of asymmetric warfare.
Future kill squads will be outfitted with the tech needed to tell “target” from “civilian” based on digital thumbprint. If you’ve got a face and have ever expressed a view online, the future will likely be rough for you. This is tech organized militaries have been salivating over for decades:
Some believe that mastering this [data] problem actually holds the solution to ending insurgency as a phenomenon once and for all. In a world of mass surveillance, follows this thinking, if we can just sift through the information rapidly enough, insurgents will not be able to operate effectively. AI algorithms won’t just instantly identify insurgents via face recognition or gait detection analysis, but even move to predict their activities before they plan them. Indeed, projects like PreData already mine open source intelligence like social media posts to predict the very outbreak of insurgencies.
An illustration of Elastic Bunch Mapping for facial recognition.Source: Federal Bureau of Investigation
Included in the code alongside tags like “rec_gender” and “rec_sunglasses” was “rec_uygur,” which returned a 1 if the software believed it had found a Uighur. Within the half million identifications the cameras attempted to record, the software guessed it saw Uighurs 2,834 times. Images stored alongside the entry would allow the police to double check.
Facial recognition is paired with DNA surveillance already ongoing in Xinjiang, the main location of the re-education camps for the Uighur genocide. This is algorithmic genocide. They’re even incorporating GPS data from cars. A slowly drawn noose of data looks like this; a swiss cheese approach to victim selection that will catch targets in one filter even if they evade the rest.
I mention Facebook’s involvement in the Myanmar genocide in relation to China’s ongoing Xinjiang campaign to underscore how deadly these two components will be when paired together. You can’t see someone’s political disposition by looking at them in the same way you might be able to observe their race3. Your political disposition is exactly what Facebook sees. As technological connective tissue spreads and ties all data and insights into central platforms, it’s only a matter of time until the wrong group ends up with the right access to single out and murder an ideological population.
Conclusion
We have already entered into the age of algorithmic genocide. China is pioneering best practices whilst domestic security agencies sharpen the long knives with impunity on whichever groups have sparked public ire. Online platforms already have all of the information needed to derive the politics of you and your loved ones. Before we see true AGI, we’ll see a genocide, not of race, but of ideology; powered by these already-available tools. I’d stake my name on it.
If you’re as jazzed as me about the societal shithole we’ve dug ourselves into, grab a shovel, drop trou, and hit subscribe. You probably won’t be executed for it in the near future.
I basically always want to comment here and have little time to do so, but feel compelled here.
The question about how the civil war would be fought, and the absence of clear geographical boundaries, is I think the portal to understanding a plausible future.
I think giant nation states are like the equivalent of massive single called organisms. What makes multicellular organisms work is that literally every cell enforces it’s own borders.
I suspect you’ll see an era of chaos leading to lots of smaller communities erecting and enforcing local borders around themselves. “Nobody allowed in unless you can prove you live or work here”, followed by walls and even things like glass or (eventually, as a carbon capture mechanism + sign of wealth, synthesized crystal).
Old walls and barriers used to be for making it physically impossible to attack. I think the new walls will be more about making it impossible to hide an attack. I think even the most ardent that pro-choice progrssives would oppose people going in and murdering pro-lifers with guns or knives. But custom engineered viruses with lethal payloads + genetically encoded kill lists? That, I think people might be a little more ok with.
Net net here I think there’s a reason China bans footage of tienanmen square instead of narrativing around it. Nazi concentration weren’t advertised as killing camps among the third reich. Everyone really does want to see themselves as the good guys. Social media won’t go away so I suspect we won’t see much of anything besides performative violence at scale.
Thanks for making me think today. My guess is that such technology (killer drone and algorithmic targeting) already exist. Government versions most likely have a Human in the Loop to provide ethical decisions about who to target. I bet the human can be pulled out of the loop when needed. It certainly wouldn't be hard to design such systems.
I speculate that the US might go down the path of balkanization rather than a full blown Civil War 2. The boundaries would be more complex: The Republic of Texas, Greater Idaho, Ecotopia, The State of Jefferson etc. Energy, food and water might be as much a driver as politics.
I basically always want to comment here and have little time to do so, but feel compelled here.
The question about how the civil war would be fought, and the absence of clear geographical boundaries, is I think the portal to understanding a plausible future.
I think giant nation states are like the equivalent of massive single called organisms. What makes multicellular organisms work is that literally every cell enforces it’s own borders.
I suspect you’ll see an era of chaos leading to lots of smaller communities erecting and enforcing local borders around themselves. “Nobody allowed in unless you can prove you live or work here”, followed by walls and even things like glass or (eventually, as a carbon capture mechanism + sign of wealth, synthesized crystal).
Old walls and barriers used to be for making it physically impossible to attack. I think the new walls will be more about making it impossible to hide an attack. I think even the most ardent that pro-choice progrssives would oppose people going in and murdering pro-lifers with guns or knives. But custom engineered viruses with lethal payloads + genetically encoded kill lists? That, I think people might be a little more ok with.
Net net here I think there’s a reason China bans footage of tienanmen square instead of narrativing around it. Nazi concentration weren’t advertised as killing camps among the third reich. Everyone really does want to see themselves as the good guys. Social media won’t go away so I suspect we won’t see much of anything besides performative violence at scale.
Thanks for making me think today. My guess is that such technology (killer drone and algorithmic targeting) already exist. Government versions most likely have a Human in the Loop to provide ethical decisions about who to target. I bet the human can be pulled out of the loop when needed. It certainly wouldn't be hard to design such systems.
I speculate that the US might go down the path of balkanization rather than a full blown Civil War 2. The boundaries would be more complex: The Republic of Texas, Greater Idaho, Ecotopia, The State of Jefferson etc. Energy, food and water might be as much a driver as politics.