Roko’s basilisk is a thought experiment proposed in 2010 by the user Roko on the Less Wrong community blog. Roko used ideas in decision theory to argue that a sufficiently powerful AI agent would have an incentive to torture anyone who imagined the agent but didn’t work to bring the agent into existence. The argument was called a “basilisk” because merely hearing the argument would supposedly put you at risk of torture from this hypothetical agent — a basilisk in this context is any information that harms or endangers the people who hear it. — https://www.lesswrong.com/tag/rokos-basilisk
Roko’s basilisk is an extremely hypothetical thought experiment due to the required conditions for such a basilisk to exist. It requires us to believe that either time travel is real or our reality is actually a simulation. Such thought experiments seem frightening but the improbability of time travel or simulation hypothesis makes it less dire as other terrors that lurk in our vicinity.
One such terror is what I would call the real Roko’s basilisk.
Stanford prison experiment.
The Stanford prison experiment (SPE) was a social psychology experiment, conducted in 1971, whose attempt was to investigate the psychological effects of perceived power. They used the framing of the struggle between prisoners and prison officers to make clear the power difference of the groups involved. If you are unfamiliar with the experiment, please read the wiki or watch the video below.
There were many things wrong with the experiment. No one has been able to reproduce the experiment. It was discovered afterward that some of the methodology was questionable. For example, the guards were prompted to act tougher, which invalidates the conclusion that the guards started acted as observed by themselves. Some of the participants, both guards and prisoners admitted that they played up their roles knowing they were being observed, indicating that there was some implied expectation from the researchers; one prisoner even admitted that their psychological breakdown was faked. You can read more about them in Lifespan of a Lie.
The real Roko’s basilisk?
According to Alex Haslam and Stephen Reicher, psychologists who co-directed an attempted replication of the Stanford prison experiment in Great Britain in 2001, a critical factor in making people commit atrocities is a leader assuring them that they are acting in the service of a higher moral cause with which they identify — for instance, scientific progress or prison reform. We have been taught that guards abused prisoners in the Stanford prison experiment because of the power of their roles, but Haslam and Reicher argue that their behavior arose instead from their identification with the experimenters, which Jaffe and Zimbardo encouraged at every turn. — Lifespan of a Lie
The real Roko’s basilisk is an insidious form of mob mentality or tribalism. None of the guards were threatened or forced to play up their roles yet they willingly committed to the act. Whether they were convinced by Zimbardo and/or Jaffee or whether they identified with the experimenters, it’s clear that the experiment shows a few key things (let’s call them SPE assertions):
- Suggestion alone without any physical instigation can induce physical outcomes.
- Human beings who assume that they are consciously aware of reality (and therefore what’s right and what’s wrong) can be corrupted, whether such corruption is coerced or not, the resultant action is an act of corruption. 2a. Those who are unaware of such corruption will believe that their acts were just (narcissistic cognitive dissonance) 2b. Those who are aware of such corruption will willingly revise their worldview to retroactively justify their corrupt actions (histrionic cognitive dissonance)
- Human beings are easily manipulated, especially by those with implied authority.
- Human beings are inherently unreliable sources for objectivity
- Human beings are trolls (for lack of a better term). They will deceive for subjective reasons (or for the lols)
- Dismissal of claims only serve to reenforce the existence of such claims (the Streisand Effect)
Now, considering the above assertions, imagine an evil that:
- The thought of such evil alone can bring it into being.
- Any attempts at censorship of evil inherently requires us to acknowledge that such evil exist, therefore it reenforces the existence of such evil.
- Agents of such evil need not be evil themselves; any sign of authority, even a false one, is enough to spread such evil.
The basilisk is already here.
The real basilisk isn’t an entity of evil like Roko’s basilisk. Although I also use the term “evil” above, “evil” can be replaced with any other term and my assertions will still hold true. The real basilisk is the insidious immortality of ideas themselves regardless of the veracity of such ideas. The introduction of any idea may start in motion things which will ensure the survival of said idea; and once in motion, it cannot be stopped.
What are examples of this basilisk’s handiwork? Industries are that “too big to fail” and must be bailed out; Regan era economics; Flat Earth theory; QAnon; war on terror; war on drugs; the list goes on and on.
Originally published at https://www.objectivelysubjective.com on May 8, 2021.