“Artificial Angels” II: know your antimeme (SPOILERS)

Grimes’s first big interview after the release of “Artificial Angels” appears to be with political youtuber Joshua Citarella, on “Doomscroll”. She directly mentions the novel There Is No Antimemetics Division and calls it a masterpiece, but so far, no one else seems to have realized that her new single directly quotes the book, as well as referencing it in other ways.

I’ll explain what I understand so far, but just be warned, THESE ARE SPOILERS.

The novel is set in the world of the SCP Foundation, an open-ended collaborative Internet fiction about an institute whose mission is to protect humanity from all kinds of dangerous entities, mostly by locking them away. The main theme is psychological horror. The entities, each designated SCP followed by a number, range from merely creepy and uncanny, to existentially terrifying and world-ending. The Foundation itself is a secretive bureaucracy much like an intelligence agency, full of compartmentalized secrets, and constantly at risk of failing in its mission. The SCP stories are found mostly on a special wiki, but there are also many SCP videos.

A meme, as defined by Richard Dawkins, is the cultural counterpart of a gene, an information-carrying replicator subject to a Darwinian process. In Internet argot, it later came to refer to the humorous images-with-text that people pass around through social media, which are manifestly subject to that process of variation, replication, and selection. But what is an antimeme?

An antimeme is an information entity that doesn’t spread, that prevents itself from spreading. Maybe it’s boring or complicated, maybe the human mind refuses to accept it or fears to pass it on. Quite a few SCPs have an antimemetic aspect, it helps them to avoid detection or resist analysis. In the real world, Roko’s basilisk is a borderline example: during its history, for many of its sufferers, it has functioned as the kind of antimeme that a person holds in their own mind but doesn’t want to pass on. But as this blog has chronicled, it also broke containment in 2013, or at least a diluted version of the basilisk did.

In the SCP novel, it is implied that the Antimemetics Division of the SCP Foundation periodically faces SCPs so terrible that it has to erase, not just the enemy, but itself, and all record of the event. The novel focuses on one such enemy, SCP-3125. This is a complex of memes that turns humans and their civilization into a self-hating, self-torturing hell on earth whose residents believe that nothing better is possible. Via tragic self-sacrifice, the Antimemetics Division defeats it by summoning an even more potent meme-complex that refutes SCP-3125 and then erases itself. (This is not stated in the book, but SCP fandom would lead one to believe that this nemesis is SCP-2747.)

In Grimes’s lyrics, the narrator states “the world can only end one way, my world, my way”. That is the direct quote from the book. There are other echoes of SCP-3125: “everything you love, you hate, everything you make, you break” (arguably part of its message for humanity), the “five-star” motif, and (in the video) Grimes’s dialogue with the mirror.

On the other hand, the chorus which says “The only thing I covet is my own annihilation” sounds like SCP-2747. The reference to a “final weapon” is also straight out of the book.

What is the meaning of these references, in the context of “Artificial Angels”? I do not know exactly how the layers fit together, but I’d say there are multiple layers. The clear main theme of the song is artificial intelligence. In the video we see AI women, including the AI companion “Ani” now offered by Grimes’s ex Elon. We see Grimes hunting a group of shambling tribespeople wearing animal skins, with a gun and a drone; the tribe throws rocks at the drone.

The hunting itself could be to kill, but it could also be to make love. The AI companion, smarter than us, that hunts us, could want either of those things. (In fact it also has a third purpose, which is to make money for its corporate master.) For humans to be hunted by smarter-than-human machines is also evolution in action, a recapitulation of the era in which animals around the world found themselves pursued by packs of ingenious primates, but with humans now on the receiving end. In her conversation with Citarella, Grimes says she is haunted by the long Neanderthal era and how little information survives from it…

For someone such as myself, steeped in the expectation that AI leads to superintelligence, the SCP themes are like an allegory of how that could turn out. One of the defining science-fictional depictions of superintelligence is Vernor Vinge’s A Fire Upon the Deep, in which a galaxy segregated by “zones of thought” which only allow certain levels of intelligence, is beset by a malign and dominating superintelligence. In the end it is brought low by a countermeasure which moves the zone boundaries so that the superintelligence is no longer able to think, but in the process it also wipes out thousands of lesser civilizations. The interactions between SCP-3125 and its nemesis are reminiscent of Vinge; the way that Grimes’s narrative voice mingles both the dominator and the annihilator, suggests a process in which hubris and nemesis are born together. Among the umpteen unknown scenarios for what follows superintelligence, the construction of an unstable babel that swiftly collapses for inscrutable internal reasons, but which has incidentally steamrolled the human world along the way, is certainly one of them.

We shall see what other readings of this text emerge…

Posted in Uncategorized | Leave a comment

“Artificial Angels” I

The lyrics directly quote these stories of the SCP Foundation’s Antimemetics Division.

Posted in Uncategorized | Leave a comment

Nominative determinism

Meta once again showing its reckless disregard for AI safety

Posted in Uncategorized | Leave a comment

Roko’s latest brainstorm

“Roko’s basilisk” was born in the distant year of 2010, among a small Internet community transfixed by the idea that an artificial superintelligence might one day emerge that would first conquer Earth, and then a big slice of the universe, and that its guiding value system could be anything; so the outcome might be very very good, or very very bad. Among some people, this created a sense of being at the crossroads of destiny, because most of humanity was still unaware of this scenario.

Among the other ideas circulating in this community, were the many-worlds interpretation of quantum mechanics – the idea that all physically possible worlds exist in parallel – and the even stranger idea of “acausal trade”. Ordinary interactions among people are “causal” in the sense that they involve ordinary cause-and-effect: I do something in response to what you did, or in anticipation of what you might do. I know about you through my senses, I act upon our shared world through my body – these are facts so basic that hardly anyone bothers to talk about them.

An *acausal* interaction occurs, when you have two or more entities, which may be far apart in space or in time, or which may even be in different parallel universes, but which somehow still know about each other, and which carry out actions with the other entity in mind. It’s as if you were reading a book about someone reading a book about you, and then you did something so that the person in the book would react.

I am a skeptic about this whole idea, but its very slipperiness makes it difficult to debunk or even reason about rigorously. In any case, a popular idea in certain circles, back in 2010, was that superintelligences in different branches of the quantum multiverse might “acausally trade” by interacting with simulations of each other. Later came the idea that a superintelligence might also acausally interact with a human-level intelligence in another world, by simulating that human in its own world. Already this breaks the symmetry of the scenario in which two superintelligences simulate each other, because a mere human can’t really simulate a superintelligence, they can just “imagine” it.

At this point, the idea of acausal trade was becoming ripe for incorporation into fantasy and psychosis. But Roko sealed the deal, by proposing that people who want to create a future with a human-friendly superintelligence, might feel extra motivation, if they agreed in advance that it could punish them, once it comes into being, if they didn’t worked as hard as they could to create it…

Now, if the idea of acausal interaction between superintelligences is dubious, and the idea of acausal interaction between humans and superintelligences is even more dubious, *this* idea transcended mere dubiousness and became a meme with little connection to the original. All the weird stuff about humanity’s cosmic destiny in the acausal multiverse evaporated, and it became simply the fear that you might be trapped in a virtual hell governed by an angry AI. Or something.

I created this blog just before Roko’s brainchild got its 15 minutes of fame, in the hope of encouraging sober analysis and debunking. But I always had better things to do than to attempt that rigorous philosophical analysis; meanwhile the dumbed-down version became known to thousands, and then hundreds of thousands of people, and so this blog became an occasional exercise in basilisk-watching, recording a few moments in the basilisk’s journey through tech culture and pop culture.


But now it’s 2023, the age of ChatGPT, and the topic of superhuman AI isn’t just for futurologists and science fiction fans any more. That little Internet community turned out to be rather influential in the new technical areas of “AI safety” and “AI alignment”; not because of its multiverse ontology or acausal flights of fancy, but because it raised the idea of aligning AI with human values, even before the deep learning revolution that has now given us the super-chatbots and AI-artists that are exciting and terrifying so many people.

For more people than ever before, the idea that AIs will surpass us, take over the world, and decide what is to become of humanity, now has a bitter imminence. Among the myriad responses to this situation, one response is to take a stand and say, this is what the value system of our new AI overlords should be; here are the values by which they should make their decisions. For example, back in 2004, the AI researcher Ben Goertzel proposed that the values governing the universe should be “growth, choice, and joy”. Or, one could call for a world governed by the Bible’s ten commandments, the UN’s sustainable development goals, you name it.

What Roko offers us now, is one such concrete proposal, but in a wrapping that resembles aspects of his 2010 scenario. What’s the proposal? It is that the resources of the universe (he calls this humanity’s “cosmic endowment”) should be divided up among the members of the human race, in three parts. Part “C” is to be divided among everyone equally. Part “B” is to be divided among everyone, in proportion to their current “wealth and power”. And part “A” is to be promised to the wealth creators of the world, in a way that motivates them to work towards the future. Every human being, then, gets some of A, some of B, and maybe some of C, and they will be free to do what they wish with their cosmic inheritance – except that there will be AI overseers enforcing “Postsingularity Law” forbidding, for example, cruel and exploitative actions within one’s domain.

To a great extent this proposal is an amplification to astronomical scales of the existing economic order. C is the welfare system, B is private property, A is policy to promote economic growth – more or less. What will sound insane, to those unaccustomed to a certain kind of cosmic futurism, is the idea that humanity could create an AI god which will start dividing up solar systems and galactic clusters among the inhabitants of Earth, according to an extrapolated version of these principles.

It may sound insane, but nonetheless, we are already at the stage where you could feed these goals to “Auto-GPT” and it would start chugging away dutifully, making a to-do list and googling for inspiration. In other words, we are already at the stage where you can tell an “AI” to do those things, and it will try to do them. It just won’t be very effective. But as AI advances, it will get trickier and more decisive in pursuit of its assigned goals, as well as more creative in its interpretation of them.

However, Roko doesn’t just present his A-B-C proposal as a manifesto that people could endorse, and that AIs could adopt. He presents it as a “contract” that will be a “deal amongst all humans in the multiverse”. He includes a few paragraphs about hunting for, and punishing, “Singularity Criminals” anywhere in the multiverse, who tried to reduce the size of the “cosmic endowment”. Incidentally, we’re to hunt for them via simulation – that is, we deliberately look for possible worlds where there are people trying to sabotage the A-B-C scheme, simulate them, and punish them.

So to sum up, in this proposal, Roko takes a particular formula for dividing up the wealth of the universe among humanity and its successors, and then packages it in the same 2010 framework of commitment and punishment in an acausal multiverse. But if history is any guide, the “acausal multiverse” part will be over the heads of most people. What remains is something like: Roko proposes that our AI overlords should parcel out the wealth of the world according to a certain formula, and that anyone who interferes with this scheme should be punished.

Posted in Uncategorized | Leave a comment

A language model offers its opinion

Posted in Uncategorized | Leave a comment

Grimethink

I don’t think I will actually change the theme of this blog to Grimes and all her works, but Miss_Anthropocene is definitely a clever title and concept.

Posted in Uncategorized | Leave a comment

Delete your offspring

I cannot find the press release itself, but according to various media, the launch of Grimes’s new single is accompanied by the statement that: “Simply by listening to this song, the future General AI overlords will see that you’ve supported their message and be less likely to delete your offspring.”

Posted in Uncategorized | Leave a comment

Springtime for the basilisk

Fresh from its cameo in Silicon Valley, today’s news is basilisk as matchmaker, Elon Musk and Grimes having bonded over its rococo incarnation.

I wondered how Roko himself is doing. I found that six months ago, he proposed a refinement of his concept, BasiliskCoin. “Causally it’s a cryptocurrency but acausally it’s a fiat currency.” So I don’t know how he’s doing as a person, but I can confirm that he still has his conceptual moxie.

Posted in Uncategorized | Leave a comment

A minor update

Roko’s basilisk was recently cited on a major TV series, “Silicon Valley”, a comedy about startups. So it might be a good time to catch up.

The picture in the previous post was of pop star Grimes, playing a character called “Roccoco Basilisk” in one of her videos.

Less Wrong the website recently completed a major transition to a version 2.0. But the online “rationalist community” these days seems to be most active in social media.

The new center of criticism directed at Less Wrong rationalism is /r/SneerClub at reddit. Perhaps one should call these people “antira” (antirationalist) by analogy with “antifa” (antifascist), since a lot of the criticism is actually coming from “progressives” who see the “rationalists” as giving support to the progressives’ enemies in the American culture war (conservatives, alt-right, etc).

Artificial intelligence, in the guise of “machine learning”, is now a billion-dollar industry.

Posted in Uncategorized | Leave a comment

Pop culture is full of surprises

image

Posted in Uncategorized | Leave a comment