I'll assay a quick answer.

or is it all simply a product of intelligence? — Benj96

It seems to me a product of intelligences - a many. Give thought to the phenomenology of the estimating. Everything is dumb, except the intelligences. Each assesses separately and it's unlikely that all will be increasingly wrong. And the mean probably wont be right, but close.

The answer, then, seems to lie in a close understanding of exactly what is happening. In few words, the several applications of relevant experience.

Why does a quantity of invalid information indicate the whereabouts of valid information? Is there some intelligence to this behaviour of maths and statistics or is it all simply a product of intelligence? — Benj96

The phenomenon of many guesses centered on the right answer may be called "The Bell Curve" or The "Wisdom of Crowds". Chaotic randomness has been found to have an intrinsic hidden core of Order. Chaotic systems are unpredictable, but deterministic. Opinions vary on an explanation for the ultimate cause of that emergent order within chaos, but without it our world would have decayed into dust long ago. I call that anti-entropy organizing force "Enformy". And it seems to be a sign of intelligence & intention underlying the laws of Nature. :smile:

Wisdom of Crowds : https://en.wikipedia.org/wiki/The_Wisdom_of_Crowds

Chaos Theory : https://en.wikipedia.org/wiki/Chaos_theory

Enformy : http://blog-glossary.enformationism.info/page8.html

Bear in mind that this phenomenon only works when people are at least slightly more likes to be right than wrong.

If the opposite is true, then bigger crowds will give wronger answers.

Due to probability a player is just as likely to overestimate as to underestimate the quantity in the jar. And due to this the more players that player, even if they are all wrong, the more accurate the calculation of the real answer gets. I find this remarkable. It's as if by understanding a simple rule of maths you can transform a whole lot of wrong into a single piece of right. — Benj96

I wonder if this is so...or if it is just something you are supposing is so.

Has it been tested?

There is something gnawing at me saying, "The general thesis here may be all wrong in many cases."

Even in variations on the example you gave, I doubt there is a bell curve resultant that aims at the correct answer.

Have you indication that it is so...or is it just something that randomly occurred to you?

Nope, the "wisdom of crowds" is false. There is no rational reason to believe that averaging naive guesses increases accuracy, unless by fluke the average of these naive guesses is the right answer.

Enformy — Gnomon

This is so interesting

Has it been tested? — Frank Apisa

It has. Numerous times with replicable results

Enformy — Gnomon
This is so interesting — Benj96

In case you missed the sidebar, here's a little more detail on the concept of positive Enformy, which is called "Negentropy" by physicists. They probably missed the progressive implications of evolution because they believed it to be totally random & directionless. :smile:

Entropy :
A quality of the universe modeled as a thermodynamic system. Energy always flows from Hot (high energy density) to Cold (low density) -- except when it doesn't. On rare occasions, energy lingers in a moderate state that we know as Matter, and sometimes even reveals new qualities and states of material stuff .
The Second Law of Thermo-dynamics states that, in a closed system, Entropy always increases until it reaches equilibrium at a temperature of absolute zero. But some glitch in that system allows stable forms to emerge that can recycle energy in the form of qualities we call Life & Mind. That glitch is what I call Enformy.
http://blog-glossary.enformationism.info/page8.html

Nope, the "wisdom of crowds" is false. There is no rational reason to believe that averaging naive guesses increases accuracy, unless by fluke the average of these naive guesses is the right answer. — sime

Does empirical statistical evidence count as rational? This intelligent "fluke" seems to be built-in to the mathematical foundation of Nature. :smile:

Wisdom of Crowds : "There is growing evidence that the wisdom of crowds can be really powerful," Kao says. "A lot of studies show that you can calculate the average of estimates and that average can be surprisingly good."
https://phys.org/news/2018-04-crowd-wisdom.html
"Do you believe that a crowd can be more intelligent than any individual in the crowd?"
https://nrich.maths.org/9601

Law of Large Numbers : https://en.wikipedia.org/wiki/Law_of_large_numbers

Does empirical statistical evidence count as rational? This intelligent "fluke" seems to be built-in to the mathematical foundation of Nature. :smile: — Gnomon

Your very article points out the biased nature of people's predictions.

"However," adds Berdahl, "there is a great deal of evidence that people have strong biases in estimation and decision tasks."

Hence the crowd might be wise, but it has nothing inherently to do with probability or the laws of nature, per se. All that the law of averages can say is that ignorant opinions cannot completely drown out expertise.

Benj96
171
Has it been tested?
— Frank Apisa

It has. Numerous times with replicable results — Benj96

Sounds like fantasy to me.

IF what follows is the question proposed, are you saying that the guesses would lead us to "valid information" on the truth of the situation?

Are there any sentient beings living on any of the planets circling the nearest 25 stars to Sol?

And are you also suggesting that because 95% of Americans guess that a GOD exists...that represents "valid information" on the truth of whether or not a GOD actually does exist?

I once witnessed a similar mistaken belief expressed in a technical report written at MIT discussing algorithms for Bayesian statistics.

They argued that if you only know that your data is Gaussian distributed, but that you don't know it's location and scale parameters, then you should assign a particular 'non-informative prior' over these parameters, say a uniform distribution, and then integrate out these parameters analytically in order to obtain a 'predictive distribution' - which is essentially to accept the wisdom of crowds.

But ignorance about a distribution's parameters cannot imply any particular distribution of outcomes, rather it implies every distribution of outcomes that is consistent with one's state of ignorance.

For example, if we don't know anything about the bias of a coin, then we shouldn't assign a particular distribution to the probability of getting heads, say P(heads)= 0.5 - as is done by naive Bayesian practitioners who appeal to maximum entropy or the principle of indifference, rather we should assign the interval (0,1) i.e.

P(heads) = (0,1) i.e. 0 < P(heads) < 1

And are you also suggesting that because 95% of Americans guess that a GOD exists...that represents "valid information" on the truth of whether or not a GOD actually does exist? — Frank Apisa

No because "means" refer to numerical values which are subject to probability in the statistical sense no in the philosophical/metaphysical sense. Plus these experiments encounter the phenomenon of "educated guesses" where direct information is being perceived but relatively poorly or inaccurately. It's as if the sum of all semi accurate perceptions balance eachother in such a way as to give a realistic perception of the quantity if something.

I would imagine it also works for some qualities. For example the ratio of different basic tastes in a foodstuff. If you had hundreds of people taste the flavour of a food and asked them to rate it in each category and you took the average of all subjective interpretations it's likely to result in a reasonably objective actual ratio of sugar (sweet) , salt (salty), glutamate (umami) etc flavours.

Basically there is an intelligence to collectives that is not possessed by any specific individual but is derived from the interaction of their personal guesses/ perceptions.

Basically there is an intelligence to collectives that is not possessed by any specific individual but is derived from the interaction of their personal guesses/ perceptions. — Benj96

Okay. You do seem convinced of that. I am not.

But, it has no practical purpose for me in my life at the moment, so I'll leave it be.