The Vulnerable World Hypothesis

SpaceDweller

The question is, does scientific progress ultimately lead to self-destruction or major destabilization of human civilization?

"The vulnerable world hypothesis" is a coined term that deals with this modified question according to the following paper:
https://nickbostrom.com/papers/vulnerable.pdf
Note that the paper does not say we're doomed, it only analyzes this question.

Scientific and technological progress might change people’s capabilities or incentives in ways that would destabilize civilization.

For instance discovery of nuclear weapons is one such scientific progress which is capable of destroying the civilization.
According to the paper, reason why this didn't happen is because we're very lucky it didn't happen rather than civilization preventing it from happening upfront.
Primary reason we're lucky is because developing nuclear weapons is not easily accessible to the public and the process of weaponizing uranium is complex and expensive (etc). and we know that there was no power in place to prevent use of the tech when it was discovered.

In the future we're likely to discover similar dangerous technologies, not necessarily in the field of physics), ex. a deadly virus (biotech), artificial intelligence (computer science) etc. meaning we might not be as lucky next time and thus some solution(s) should exist to prevent accidental catastrophe.

One of which is unpopular:

A general ability to stabilize a vulnerable world would require greatly amplified capacities for preventive policing and global governance.

Other 2 proposed solutions are:
1. Restrict technological development
2. Ensure that there does not exist a large population of actors representing a wide and recognizably human distribution of motives

Both of which are "unrealistic" and "costly" therefore favouring "preventive policing and global governance"
Further, preventive policing and global governance IMO can be simplified to just global governance because without global government preventive policing can be influenced by local governments in the name of "national security" and similar reasons.

Therefore global government and policing supported by global government is it appears the most effective solution to prevent self-destruction caused by scientific progress.

Problem with global government is that it's unpopular for obvious reasons so my primary question here is what do you think, which other methods do you think might yield same or better options to prevent self-destruction or major destabilization?
Or otherwise do you support establishment of global government for the sake of this question?

Secondary question for debate is fundamental because if you don't agree with it then it doesn't make sense to search for solutions, so, do you think scientific progress ultimately poses a risk of self-destruction or destabilization?
Or do you think is it legitimate question worth paying significant attention to? (perhaps there are other priorities greater or more dangerous than this? ex. global warming etc)

Note that I did not mention "fermi paradox" which deals with why we don't find intelligent life in the universe - where one possible hypothesis is because technologically advanced civilization tends to self-destruct due to lack of control of discovery of dangerous technologies capable of destabilization or destruction.
I don't know if this hypothesis is derived from the vulnerable world hypothesis or vice versa but it's interesting to take it into consideration for the secondary question above.

Leontiskos

The traditional Judeo-Christian Biblical view says that the demons give humans precocious knowledge so that the human race will destroy itself. The weaponry aspect is especially seen in the line of Cain, culminating in Tubal-Cain.

I think this traditional narrative is instructive with respect to your OP, for powerful knowledge is dangerous precisely insofar as it is precocious. Someone who is mature is able to wield powerful knowledge in a way that is safe and wise.

my primary question here is what do you think, which other methods do you think might yield same or better options to prevent self-destruction or major destabilization? — SpaceDweller

I think maturation is needed, including moral maturation. As far as quick fixes go, I'm not sure, for they are all problematic. Government is not a terrible quick fix. Perhaps it would be possible to discover technologies that can only be used for good and never for evil, but I doubt it.

Prescinding from liberal individualism for a moment, I do not think our cultural intelligence is sufficiently attuned to these dangers in a way that allows us to avoid them. We are apt to rush ahead, blindly grasping after the newest and most powerful technology, with no regard for side effects or deleterious consequences. We lack a robust recognition of our own destructive capacities, both as individuals and as groups. Rectifying this situation would go a long way towards inculcating the sort of moral maturation I have in mind.

SpaceDweller

I think maturation is needed, including moral maturation. — Leontiskos

I think no matter how morally perfect or morally enlightened world population is there will always be individuals willing to do immoral things with the help of destructive knowledge or technology.

What you seem to have problem with is to prevent censorship of knowledge or to ensure public access to knowledge.
From one side this also means bad actors having free access to dangerous knowledge increasing the risk of destabilization.
The opposite is that such knowledge should be censored to prevent destabilization which you compare to Biblical view in that people should have the knowledge regardless of what some "evil" world government (or God) says.

Since you like neither of these 2 options you propose that moral maturity of the population is the answer.

But the chain is only as strong as its weakest link, and so is the population only as mature as it's most immoral (or insane) individual.
For this reason I don't find maturity any better than world government or censorship of knowledge.

Leontiskos

But the chain is only as strong as its weakest link, and so is the population only as mature as it's most immoral (or insane) individual. — SpaceDweller

Yes, I realize you are thinking along the lines of liberal individualism, which is why my last paragraph began, "Prescinding from liberal individualism for a moment..."

If you want to think in terms of individualism then consider the fact that three year-old children are "weak links." If the three year-old gets hold of a gun or a button that launches a nuclear missile, then we will be in a great deal of trouble. So do we address this weak link by destroying it or strengthening it? No, we just keep guns out of the hands of three year-olds. This is easy as long as the adult population is able to control the toddler population.

So you could think of the individualism-framed problem as a problem of the ratio between adults and toddlers, and the ensuing probability that any given random individual will be a "toddler." The government solution is based on the idea that a minority of adults will maintain control over time, thus preventing toddlers from accessing dangerous weapons. But as a society degrades and the proportion of toddlers-to-adults grows, the chance that a toddler is able to grab power will increase. The only long-term solution to this problem is to raise the average moral IQ, because that is the only way to address the proportion of toddlers-to-adults directly.

Or in other words, we need to think about this issue in abductive terms, as searching for the best possible solution; rather than in deductive terms of finding the perfect solution. There is no perfect solution.

ssu

Why use scientific progress and not simply technological progress?

Our weapons have become more deadly because of technological advances. Science is a bit different. Quantum physics tells us something about how the world works. Yet that Science itself didn't give us nuclear weapons, they are applications of science. Hence use of technology gave us nuclear weapons. So why be against science and think scientific progress makes us vulnerable?

There's a lot of examples that show that we aren't as vulnerable as earlier: we don't die as early as before. If there are bad harvests, we don't in the industrialized World die of famine. Actually famines have become more rare. We don't just have to raise our hands up and hope that the God's wouldn't be angry at us, we have an idea just how we changing (destroying?) our environment.

Therefore global government and policing supported by global government is it appears the most effective solution to prevent self-destruction caused by scientific progress. — SpaceDweller

I don't think so.

Besides, global government would lead to excessive centralization and yes, a lot lof people all around the World might come together... to oppose this singular government.

Sovereign states working not as an union, but a confederacy, would in my view work far more better. The concept of nation state has been proven quite effective. Also when there are multiple different countries trying different things, we aren't all then destined to make the same mistakes.

Leontiskos

Why use scientific progress and not simply technological progress? — ssu

Because almost everyone today is a Baconian, thus, "scientia potentia est" ("knowledge is power"). "Science" in our modern lexicon often just means, "The thing that provides us with technology."

ssu

↪Leontiskos

Sorry, if I relate 'science' to using the scientific method, not a tool for gaining power.

Leontiskos

↪ssu

- The scientific method derives in large part from Francis Bacon and the desire to manipulate nature. Modern science was a shift away from speculative knowledge, or knowledge for the sake of knowledge. It moved in the direction of practical knowledge, or knowledge for the sake of power, namely the power to manipulate nature.

SpaceDweller

There's a lot of examples that show that we aren't as vulnerable as earlier: we don't die as early as before. If there are bad harvests, we don't in the industrialized World die of famine. Actually famines have become more rare. We don't just have to raise our hands up and hope that the God's wouldn't be angry at us, we have an idea just how we changing (destroying?) our environment. — ssu

Yes, there are many benefits of scientific progress but the thing is that only one wrong technology can devastate all benefits.

I don't think science is inherently evil because of this, only that it has the potential of self-destruction if not controlled.

So you could think of the individualism-framed problem as a problem of the ratio between adults and toddlers, and the ensuing probability that any given random individual will be a "toddler." The government solution is based on the idea that a minority of adults will maintain control over time, thus preventing toddlers from accessing dangerous weapons. — Leontiskos

So the world government idea becomes even more undesirable in the eyes of those who oppose it, because it implies censorship of destructive knowledge further raising suspicion and conspiracies about world government.
Overall world government, censorship of knowledge and moral development play together, inclusively instead of either one exclusively.

ssu

↪Leontiskos

Yet we don't call engineers scientists, but engineers.

There really is a difference between science and technology. Your simply not using the definition of technology and just putting it together with everything being 'science'. However there's a reason why the standard definitions are different. Let's define first what they mean:

Science is the study of the natural world by collecting data through a systematic process called the scientific method. (Human sciences then study the world that we have created ourselves)

Technology is when we apply that science to create devices that can solves problems and do tasks.

There's a fundamental difference between the two. The power that you are talking about comes really from technology, all those devices starting from steam engines to microchips to nuclear weapons.

And I wouldn't say that modern science has shifted away from speculative knowledge. Science has become a huge field with academia and research institutes where a lot of that new study has gone to applied sciences and hence basically to technological research. How you describe it would mean that scientific study has somehow diminished when it hasn't done so. There's still going on that original science stuff, no matter how much studies are directed by grants.

Hence what you refer to is more the situation when people first have a problem, want to solve the problem with new technology and then the engineers ask scientist if the technology would be even possible. Well, the scientists cannot just invent new scientific realities to give the engineers new ways to make technology work.

ssu

Yes, there are many benefits of scientific progress but the thing is that only one wrong technology can devastate all benefits. — SpaceDweller

Must be really some awesome technology of the future, because the fact is that even an all-out nuclear war between US/Russia and China wouldn't devastate everything and kill everybody. It might be a well respected mantra to say to voice opposition to nuclear weapons, but destroying everything is a lot harder than we think.

The Library of Congress and it's 173 million items might burn up in a nuclear fireball, yet the libraries in Argentina and Chile would likely stay intact. And those libraries have a lot more information stored than the great library of Alexandria had. Also many Argentinians and Chileans would survive, even if there would be a nuclear winter of some sort.

I don't think science is inherently evil because of this, only that it has the potential of self-destruction if not controlled. — SpaceDweller

My father was a professor of virology, and while he has now passed away, he did live to see the Covid pandemic. What he was really afraid back then was the possibility that the Covid-virus had indeed been created by research and then had spread out of the laboratory. He personally believed it was a real, worrisome possibility and feared what damage such a thing would do to medical research and in the trust in medicine in general. With seven million deaths, one million dead in the US, you can bet that it to be a laboratory "Oops!" isn't something people actually want to hear.

So yes, you do have a point.

And just like the scientists that cloned the first animal, Dolly the sheep, in 1996 who insisted that governments should look at the ethics and have regulations on the subject, there is a large acceptance from the scientific community that regulation and limitations should be enacted to the possible dangers. Yet those who would prosper from even dangerous research can have other incentives.

Yet I think this isn't a problem of science itself. It's a problem of when technological research isn't given any limits. It's more of a problem of lax regulation of industry, I would say.

There can be limits to scientific knowledge and you can indeed keep technology at bay. The perfect example of this is just how ancient is now the technology to make nuclear weapons. Yet there's a great incentive for scientists not to make the technology public: they'll face long jail sentences or simply will be killed. That's a great incentive (to stay alive) not to openly market nuclear technology to someone just with the money wishing to buy it.

Hence technology can indeed be regulated and you don't need a global government. You need just common sense.

Leontiskos

There's still going on that original science stuff, no matter how much studies are directed by grants. — ssu

If science is just a means to technology, and science is funded almost entirely by a desire for technology (or other forms of power), then science is not about speculative knowledge in any real sense. We have seen science moving in this direction for hundreds of years now.

You are right that in theory science should be this separate, autonomous thing. But in practice it turns out not to be.

SpaceDweller

Must be really some awesome technology of the future, because the fact is that even an all-out nuclear war between US/Russia and China wouldn't devastate everything and kill everybody. It might be a well respected mantra to say to voice opposition to nuclear weapons, but destroying everything is a lot harder than we think. — ssu

Yeah we don't know what the future holds, so we can't dismiss the possibility.
General preventive measures are desired, just like there are preventive measures being discussed about the use of AI today even though the AI does not pose any risks for now but has the capacity to be weaponized in the future.

There really is a difference between science and technology. Your simply not using the definition of technology and just putting it together with everything being 'science'. However there's a reason why the standard definitions are different. Let's define first what they mean: — ssu

Agree but I think it's not wrong to say "scientific progress" when addressing this question because without scientific progress there is no technological progress.

ex. discovery of nuclear fission is science without which there would be no nuclear weapons (concrete technology)
inventing nukes was really easy thanks to scientific progress.

The point is that scientific progress leads to potentially devastating technologies.

180 Proof

Why use scientific progress and not simply technological progress? — ssu

:up: :up:

↪SpaceDweller

Every 'civilization' is always most vulnerable to (thermodynamic and/or information) entropy.

ssu

If science is just a means to technology, and science is funded almost entirely by a desire for technology (or other forms of power), then science is not about speculative knowledge in any real sense. We have seen science moving in this direction for hundreds of years now.

You are right that in theory science should be this separate, autonomous thing. But in practice it turns out not to be. — Leontiskos

A lot of people do think that science is just one part of the process of how our technology will improve and that tech is just there to improve our lives. But talk to a scientist and you will notice that they are actually interested in science itself. That isn't something irrelevant.

And then think about what science was hundreds of years ago: A tiny cabal of men that had the ability to think about science and not simply to work to feed themselves and their families. If they were aware of each other, likely they were writing to each other. Even at the start of the 20th Century the scientific circles were small.

Just look at this picture from 1926 of the "Fifth Solvay International Conference on Electrons and Photons". That's what a "golden age" of science looks like: a tiny cabal of people that we now know from the names of the various theories and models that make Quantum Physics.

https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F039e8c19-4b94-46c8-9204-5c31902ef50e_1746x956.png

There are 28 men and one woman in that picture. Now you couldn't take a picture of all scientists that are working in the field, even of those that really are just doing the science part. At the time of Newton, if such a conference on Physics/Mathematics in general would have been held and there would have been 29 participants, I think there would be nearly all of the people at that time studying these issues.

ssu

The point is that scientific progress leads to potentially devastating technologies. — SpaceDweller

So we agree that it's the potentially devastating technology, or the use of this tech, which is the real threat.

Because otherwise it's a bit difficult to say "Stop Science! It may discover things that can potentially lead to devastating use of technology." Once when you stop science, then there's not even the harmless technologies to help us. Everything will be just engineers improving current machines and concepts. But once you have developed the pencil, the written book, the spoon etc. there's not much to improve there. Spoons and books have stayed the same for quite a while. No incentive or reason to improve a technology that works so well.

Spoon from the Roman period, 4th or 5th Century. Looks quite the same as I have in my kitchen and likely used the same way as now:

main-image

When we don't have any new technologies, the problems that our current technology cannot be solved will mean that there likely won't be any solution for the problem. And that's why earlier societies collapsed as they didn't have the capability to deal with the changes that happened.

Michael

The question is, does scientific progress ultimately lead to self-destruction or major destabilization of human civilization? — SpaceDweller

I suspect so. Global travel increases the likelihood of a global pandemic, excessive industrialisation increases the use of non-renewable resources and the likelihood of harmful climate change, and automated systems controlled by an artificial intelligence is vulnerable to coding errors and sabotage.

I suspect we die off before we're capable of interstellar colonisation.

We'd need excessive regulation to avoid that, but capitalism won't allow it.

SpaceDweller

Every 'civilization' is always most vulnerable to (thermodynamic and/or information) entropy. — 180 Proof

Maybe you want to explain?, I don't get it.

Everything will be just engineers improving current machines and concepts. But once you have developed the pencil, the written book, the spoon etc. there's not much to improve there. Spoons and books have stayed the same for quite a while. No incentive or reason to improve a technology that works so well. — ssu

Spoons will likely never go out of scope, but for written books there already are alternatives, PDF's and similar which we read on PC and mobile etc.

Fire was first invention to prepare meals followed by stoves and now wait until food replicator is discovered like the one in star trek series.

The point is that it takes centuries until one seemingly irreplaceable tech goes out of scope and is replaced with newer one.
Only because spoons and similar are used for very long time doesn't mean they'll be used for eternity.

I suspect so. Global travel increases the likelihood of a global pandemic, excessive industrialisation increases the use of non-renewable resources and the likelihood of harmful climate change, and automated systems controlled by an artificial intelligence is vulnerable to coding errors and sabotage. — Michael

Good point, so that's an alternative danger other than concrete technology, maybe it should be called self-destruction caused by cumulative scientific discoveries.
Climate change is good real-world example.

SpaceDweller

↪Michael

Thinking about what you said, this should probably be classified as passive danger and active danger.
concrete tech ex. nukes correspond to active danger while what you said corresponds to passive danger, one which develops over time and does general destabilization.

ssu

Fire was first invention to prepare meals followed by stoves and now wait until food replicator is discovered like the one in star trek series. — SpaceDweller

Yet we won't get "food replicators", at least in the way in Star Trek, without new scientific insights.

And there are ample amount of global problems where science and technology would help. And just implementing current technologies globally would help. For example, the second largest exporter of food after the US is actually tiny Netherlands. Hence if other countries had as advanced agricultural sector as Netherlands, then there would be ample space for wildlife and nature and still everybody would be fed. Still, for example famines have eased even with the current development.

SpaceDweller

↪ssu

I think this is wrong approach, counting good things about science to encourage discovery of potentially devastating technologies is not an argument IMO.

Main problem about this hypothesis is how to contain potential devastating effects caused by scientific progress.
I did agree that stopping research is not an option and so does the linked paper say it's unrealistic and costly, so this is not a solution, global governance and policing is a better solution but not popular, so we seek something better than that.

@Leontiskos so far is the only one to provide one possible solution, not that I find it better but it's not that bad IMO.

ssu

Main problem about this hypothesis is how to contain potential devastating effects caused by scientific progress. — SpaceDweller

Again, just what are the devastating effects caused by scientific progress?

Without scientific progress there sure would be devastating effects. Not just potential. Have you thought about this question from this viewpoint?

So let's assume there wouldn't have been any Renaissance and further age of Enlightenment in the West, but the Church would have held power as in the Muslim World. Where would be now?

I did agree that stopping research is not an option and so does the linked paper say it's unrealistic and costly, so this is not a solution, global governance and policing is a better solution but not popular, so we seek something better than that. — SpaceDweller

Ok, but why isn't then this more of a problem of basically the abuse of technology?

Many times these problems actually need very nuanced and specific solutions, not radical and dramatic solutions like "World government".

wonderer1

Many times these problems actually need very nuanced and specific solutions, not radical and dramatic solutions like "World government". — ssu

:up:

NOS4A2

↪SpaceDweller

Tech has evolved at an astronomical pace while the species itself hasn't. Given this disparity it is quite possible we could destroy ourselves with it. But it is for this same reason that a world government is out of the question.

Leontiskos

So the world government idea becomes even more undesirable in the eyes of those who oppose it, because it implies censorship of destructive knowledge further raising suspicion and conspiracies about world government. — SpaceDweller

Rather, the objection is that any solution which requires that a small minority maintain power indefinitely will eventually fail. Governmental solutions, including a world government, require that a small minority maintain power indefinitely. The democratic case is slightly more complicated, but presumably the government that is doing the policing will need to be filled with superior humans, and insofar as such policing is necessary there must be large numbers of inferior humans who need to be policed. Thus even on the democratic case a similar reality ensues.

Governmental approaches can often be useful alongside other tactics, but they cannot be a standalone silver bullet, and in general I don't think "minority approaches" are sustainable.

Leontiskos

↪ssu

We actually see a symptom of this pragmatization of scientia in recent philosophy of language, for example:

The meaning of "water boils at 100℃" is what we are able to do with it. — Banno

...Speculative claims are thus reframed to mean practical claims. Many such users on the forum seem to actually labor under the idea that speculative beliefs do not exist. This is reflected in places like Banno's thread on belief.

SpaceDweller

Again, just what are the devastating effects caused by scientific progress? — ssu

We don't know yet, it's a matter of the future, for now the only evidence is nuclear weapons and AI and perhaps some other developments like genetics etc.
According to this trend we can be sure to discover more such destabilizing technologies (unless you suspect science is capable of this?)

Without scientific progress there sure would be devastating effects. Not just potential. Have you thought about this question from this viewpoint? — ssu

We already established that stopping research is unrealistic and costly option.

So let's assume there wouldn't have been any Renaissance and further age of Enlightenment in the West, but the Church would have held power as in the Muslim World. Where would be now? — ssu

I see this there is a hidden argument among commentators here who oppose this hypothesis which is that there are 2 camps now, traditionalist current and liberalist current, where liberalists label this hypothesis with traditionalism and compare it with "church style" opposition against science, but this is so wrong.

If your answer to my secondary question (see OP) is negative then I can understand your stance.

Ok, but why isn't then this more of a problem of basically the abuse of technology? — ssu

I guess because the purpose of devastating tech (nukes) is to destroy, there is no abuse since there is only one purpose.

Tech has evolved at an astronomical pace while the species itself hasn't. Given this disparity it is quite possible we could destroy ourselves with it. But it is for this same reason that a world government is out of the question. — NOS4A2

Yes agree, we're not evolved enough and are behind tech, the paper however says that it's politics that's behind tech and suggests that improvements in politics should be improved, suggesting world government and policing which is a political matter.

Rather, the objection is that any solution which requires that a small minority maintain power indefinitely will eventually fail. — Leontiskos

Yes agree.

NOS4A2

↪SpaceDweller

Yes agree, we're not evolved enough and are behind tech, the paper however says that it's politics that's behind tech and suggests that improvements in politics should be improved, suggesting world government and policing which is a political matter.

Government is a kind of technology, except it’s an immoral one. It’s premised on monopoly, plunder, and coercion. Not only that but it’s entirely inefficient. Besides, Government has been the greatest progenitor of the threat of mass-extinction since the meteor.

A world government, which would become the largest monopoly ever seen, would be grossly inefficient. The bureaucratic stupidity would be immense, leading to a failure of communication like found at Chernobyl. The people employed in it are just job-holders, including the politicians, each of them possessing the inherent tendency to satisfy their wants through the easiest means available. I think a political solution is ridiculous one, quite frankly.

SpaceDweller

Government is a kind of technology, except it’s an immoral one. It’s premised on monopoly, plunder, and coercion. Not only that but it’s entirely inefficient. Besides, Government has been the greatest progenitor of the threat of mass-extinction since the meteor. — NOS4A2

Therefore we need to improve on political philosophy to find better ways on how society should be governed which in turn should address scientific progress.

I find this to be plausible solution in addition to methods to improve development of sociology and moral education of population as suggested by @Leontiskos.

The Vulnerable World Hypothesis

Welcome to The Philosophy Forum!

Categories

More Discussions