Recently, I was investigating a potential information hazard, of which I will not mention due to the potential risks it could cause to others, given the unlikely scenario in which the information indeed turns out to be true. During this time, I stumbled across a variant to the famous thought experiment Kavka’s Toxin Puzzle. I am attributed this variation of the experiment to Mark Green on Quora, as I could not find any other reference to the variation in my short time researching. It goes essentially as follows.

- A scientist has acquired or created a supercomputer which can perfectly predict the actions of people

- This computer does so by creating simulations

- The scientist gives you a vile of poison that will make you ill for a day, before achieving a complete recovery.

- Next Tuesday, the scientist will ask the computer if you drink the poison on Wednesday, if the computer says yes, they will deposit $1,000,000 into your account.

- The scientist then states that you can keep the money, regardless of whether you drink the poison.

- Receiving the money on Tuesday, you realise it would be completely irrational to drink the poison on Wednesday, except-

- For the computer to accurately predict what you would do, the simulated you would need to be unaware that you are in a simulation-

- It’s impossible to prove that you are not the simulation, thus the only rational choice would be to drink the poison.

Given that the computer is considered a perfect predictor and can create simulations in which its inhabitants believe not to be in a simulation, anyone who doesn’t do as the computer predicts, is certain to be simulated. For example, not drinking the poison after receiving the money would prove that you are in a simulated reality. These simulated realities would end when either Wednesday ends without you drinking the poison, or you are drinking the poison. Yet, we fail to see the horrifying truth that this creates. The abhorrent situation that could be created and abused under such a system. The universe in which we exist is not inherently a simulation, in fact it is more likely than not, not a simulation. Yet, a universe in which the prediction of the computer is not followed, is certain to be simulated.

We should simplify this experiment. Ask the computer what your next word will be. Wait for its response, and do not say a word. Consider that the computer suggests that your next word will be ‘run’, and you intentionally say a different word, ‘expiration…’ immediately, the simulation is ended, the universe is over, and everyone who existed in it, 7 billion people, effectively die.

This isn’t too big a deal, of course, unless someone wanted to destroy our universe, in which case doing so becomes terrifyingly easy.

1. Create an AI or Super intelligence capable of running such simulations, (Potentially possible and likely.)

2. Ask it a simple prediction, and then do the opposite of its answer.

3. YAY! Dead universe, we are all simulated.

Ironically, this is a form of Timeless Decision Theory. The universe could have existed outside of a simulated reality, right up until the point that the answer to the question was acted on, in which case the universe becomes certainly simulated. In essence, the future caused something that happened in the past.

So, logically, someone could destroy the universe as we know it, by simply asking a question to a supercomputer.

But it gets worse. Consider that the person who does this and abuses the computer, hates society so much that it chooses to do it over, and over, and over. Such a person would effectively kill over 8,400 billion people in an hour. Wiping them from the face of existence over, and over, and over, in an endless cycle of vengeance and torment by eradicated the reality the exist in from existence.

So, I would be interested to hear if anyone has any thoughts or ideas on this that could help us understand this. Also, if you replace any mention of computer or AI with a post-singularity ai it becomes an even more concerning situation, given how quickly we appear to be approaching said singularity!