1. The wave function describes a stochastic process.


Some view the wave function as representing an actual physical state that exists at a given time and changes during the course of an experiment. Viewing it this way: when unobserved, individual particles do not have definite properties and can be seen as occupying multiple different states simultaneously in superposition. Particles take on definite properties only when we measure them; however, which particle state we eventually observe when measured occurs non-deterministically. We can only know the probability that a given state will occur, which can be derived directly from the wave function.

The statistical interpretation removes the assumption that the wave function represents some unobserved physical state. Instead, it focuses on the fact that measurement outcomes for a given point in time during an experiment can be predicted using the probabilities derived directly from the wave function. Hypothetically, if we were to perform repeated experimental runs for that same particular experimental context, the long run relative frequencies of the outcomes would be what is described by these derived probabilities. By not representing a physical state per se, this makes the wave function solely a predictive model for possible measurement outcomes. Its evolution is not describing changes in some actual physical state but changes in the probabilities of finding a particle in various possible states as the experimental context changes with time. Specifically then, what the wavefunction is describing is the behavior of a stochastic (random) process: collections or sequences of random variables that represent the state of a particle over different points in time. While not representing the enigmatic quantum state as more commonly envisioned, it should be seen as no different from how stochastic processes are widely used to describe the behavior of many other real physical systems in the natural sciences.

A motivation for deflating the wave function's status as a representation of some specific physical state is that we can restore some intuitive aspects of "realism" with regard to particles: during an experiment, particles do have definite properties at a given point in time, we just do not know which states they are in unless we measure them because of the random nature of their behavior. The famous wave-particle duality then becomes completely deflated given that quantum mechanics just becomes about particles with definite states which behave according to probability distributions given by the wave function.

In orthodox interpretations, treating the wave function as a physical state is accompanied by the introduction of the collapse postulate in order to reconcile, and provide a mechanism that transitions between, the indefinite properties of the unobserved quantum state and the definite properties we observe with measurement. The statistical approach obviously has no requirement for this: on the one hand, particles have definite properties both when unobserved and measured, intuitively conforming to our sense of "realism"; on the other hand, the wave function does not represent a single given particle anyway, just probabilities regarding particle behavior under particular conditions. Without physical collapse, we don't need a special role played by observation or measurement that needs further explanation concerning when and why collapse occurs - i.e. the measurement problem. Nor do we have to deal with various ambiguities and difficulties regarding relativistic causation, which are especially salient when applying collapse to entangled pairs of particles. The emergence of the classical world is also in principle less complicated given that particles always have definite properties; it can be noted that limits where quantum mechanics approximates classical behavior can be derived without any reference to collapse.



2. Bell violations are direct consequences of non-commutativity.


An obvious question for any interpretation is: how does it approach the infamous Bell violations? Works by a number of people have indicated that the violation of Bell inequalities by a set of observables is equivalent to the absence of a joint probability distribution that encompasses all of those observables. One notable example of this work is Fine's theorem; from this, it can be inferred that Bell violations are consequences of non-commutativity, which will be defined shortly.


--- --- ---
(Some references for Fine's theorem)

Hidden variables, joint probability, and the Bell inequalities

https://scholar.google.co.uk/scholar?cluster=2543155278787880428&hl=en&as_sdt=0,5&as_vis=1

Joint distributions, quantum correlations, and commuting observables

https://scholar.google.co.uk/scholar?cluster=8813695518940155915&hl=en&as_sdt=0,5&as_vis=1
--- --- ---


Also in a similar vein:


--- --- ---
George Boole's 'conditions of possible experience' and the quantum puzzle

https://scholar.google.co.uk/scholar?cluster=16366977415123888164&hl=en&as_sdt=0,5&as_vis=1

Possible Experience: From Boole to Bell

https://scholar.google.co.uk/scholar?cluster=301320604491795906&hl=en&as_sdt=0,5&as_vis=1
--- --- ---



Fine's theorem specifically demands the following as equivalent:


"(3) There is one [global] joint distribution for all observables of the experiment, returning the experimental probabilities.

(4) There are well-defined, compatible joint distributions for all pairs and triples of commuting and non-commuting observables.

(5) The Bell inequalities hold."


The Kochen-Specker theorem stipulates that (4) from above is impossible in quantum mechanics due to its Hilbert space structure. More specifically, (4) is violated simply because in quantum mechanics there is always a presence of non-commuting pairs of observables and these cannot have valid pairwise joint probability distributions under usual assumptions.

Pairs of observables without joint probability distributions can be said to be incompatible. When we try to define joint probability distributions for these incompatible pairs of observables, their distributions violate the rules of probability, notably the law of total probability which equates marginal probabilities to sums of joint probabilities:


https://en.m.wikipedia.org/wiki/Law_of_total_probability


This prevents the resultant probabilities from describing a conventionally valid probability distribution (though I suppose this doesn't necessarily stop someone using unconventional rules of probability). In contrast, compatible pairs of observables do have valid joint probability distributions.

Non-commutativity just means that the order of applying a pair of quantum measurement operators affects the measurement results for that pair: essentially, measurements on non-commuting observables disturb each other. On the contrary, commuting pairs of observables will not disturb each other: the outcome of one observable in the pair will not be affected by the measurement of the other, and so the measurement order has no effect.

Since (4) doesn't hold, we see that (5) doesn't either. The fact that a quantum system generates profound correlations to the extent of Bell violation is equivalent to the fact that it does not have a global joint probability distribution (also by violating the law of total probability), which is caused by the presence of incompatible pairs of variables that do not commute. Without a global joint probability description, the statistics of Bell experiments can only be described using the multiple separate joint probability spaces that describe each of the compatible pairs.

The non-commuting pairs are therefore the root of Bell violations, something that has been emphasized by some recent physicists including proofs in the case of the CHSH inequality for the necessity and sufficiency of non-commutativity for Bell violation.


--- --- ---
Get rid of nonlocality from quantum physics

https://scholar.google.co.uk/scholar?cluster=11575548674791370584&hl=en&as_sdt=0,5&as_vis=1

Making sense of Bell's theorem and quantum nonlocality:

https://scholar.google.co.uk/scholar?cluster=6010274925746687086&hl=en&as_sdt=0,5&as_vis=1

Nonlocality claims are inconsistent with Hilbert-space quantum mechanics:

https://scholar.google.co.uk/scholar?cluster=18053424246858448382&hl=en&as_sdt=0,5&as_vis=1

In praise of quantum uncertainty:

https://scholar.google.co.uk/scholar?cluster=4615841789462490335&hl=en&as_sdt=0,5&as_vis=1

Experimental Counterexample to Bell's Locality Criterion:

https://scholar.google.co.uk/scholar?cluster=4769324801739580243&hl=en&as_sdt=0,5&as_vis=1
--- --- ---



It is worth noting briefly that in any Bell experiment, the non-commuting pairs of observables that cause the Bell correlations are always locally related to each other in the sense of being properties of the same particle. Therefore, we cannot explain Bell correlations as being a direct result of disturbances between non-commuting observables acting across spatially separated locations. On the contrary, the spatially separated pairs of observables in these experiments are always pairwise compatible and this is typically interpreted as suggesting that non-locally separated observables cannot disturb each other in ways that violate speed-of-light limits (non-signalling).

It might also be worth noting that the link between non-commutativity and Bell-type violations also seems to occur in classical systems, most notably in classical polarization optics. Bell-type violations have also been derived in the context of Brownian motion as a consequence of Heisenberg-like uncertainty relations. An important distinction from quantum violations is that none of the classical examples involve nonlocal correlations between spatially separated variables: i.e. they are local intrasystem violations. While clearly not quantum, this perhaps adds more evidence that Bell violations are clearly a formal necessity due to non-commutativity, regardless of the system.


--- --- ---
Quantum Mechanics and Classical Optics: New Ways to Combine Classical and Quantum Methods:

https://scholar.google.co.uk/scholar?cluster=9708108079894379453&hl=en&as_sdt=0,5&as_vis=1


Entanglement in Classical Optics:

https://scholar.google.co.uk/scholar?cluster=9687012103438601910&hl=en&as_sdt=0,5&as_vis=1


Brownian Entanglement:

https://scholar.google.co.uk/scholar?cluster=11916823875626356065&hl=en&as_sdt=0,5&as_vis=1
--- --- ---



3. Heisenberg's uncertainty principle and non-commutativity are generic properties of stochastic processes.


First, it can be noted that generally, canonical non-commuting variables in quantum mechanics are those that correspond to cases of generalized coordinates (e.g. position) and generalized momentum in Hamiltonian mechanics. The major exception is the mutually non-commuting angular momentum operators along x, y and z axes (which can then be generalized to spin observables as seen in Bell experiments). Though deriving their commutation relations requires the use of the regular canonical commutation relations for position and momentum, this non-commutativity is essentially inherited from a generic non-commutativity that affects all descriptions of 3D rotation (The SU(2) or SO(3) group). It is also the root of Bell-type violations in classical polarization optics. Even with everyday objects, you can see that successive rotations along different planes from the same starting positions will result in different ending positions if you perform those same rotations in different orders: the different planes of rotation just don't commute.

From non-commutativity, Heisenberg's uncertainty relations can be derived. These relations dictate that the variance or uncertainty of measurement for one observable of a non-commuting pair is inversely related to the uncertainty for the other observable in that pair. In the statistical interpretation, we can interpret this purely in terms of probability distributions that are realized by the long run relative frequencies of outcomes. For instance: if, over many repeated iterations of some experimental context, the measured position of a particle tends to be bunched up in one location, then the repeated measurements of momentum will give values be dispersed in all directions.

There is a very simple example of this which can be described in classical optics. If we send a beam of light through a slit, the width of the slit (denoting position) is inversely related to the size of the angle or spread of directions (denoting momentum) by which the light exits the other side of the slit. In the (statistical) quantum description, this pattern is predicted when repeatedly sending single photons one by one through the slit. Importantly, it can be gleaned from the example that the Heisenberg uncertainty relation is not about measurement itself, but the experimental context which constrains the behavior of the particles. If the experimental context necessitates a particular spread of position measurements (e.g. because of the width of the slit), then this constrains (i.e. disturbs a la non-commutativity) the spread of momentum measurements, and vice versa.

We can also see (Old Edit: ignore this paragraph; do not think this intuition is correct) in an intuitive sense how this might lead to pairwise incompatibility, precluding a valid joint probability distribution. Just as the statistics of the global joint probability distribution referred to earlier can only be represented in multiple separate probability spaces or contexts, it seems that joint probability distributions for position and momentum might in principle only be represented validly within distinct experimental contexts (e.g. different widths of slit) which each ascribe mutually exclusive sets of variances / spreads to the different observables. (New edit: having thought about it, I am pretty sure this paragraph is actually correct. Some experimental set up might be constituted of two variables A and B, each with marginal probabilities that will be realized in the experimental outcomes. There will be incompatibility when the law of total probability (LTP) is violated so that p(A) will not be the same as [sum p(A, B)] when A is considered jointly with B, perhaps under some specific measurement setting. This means B is disturbing p(A). [sum p(A, B)] is still a marginal probability but the question is: for what? Skipping some explanation for brevity, It will just be a marginal probability for A in some specific context involving B that must be somehow different to our original p(A) where no contexts have been explicitly differentiated in the experimental set up. For position-momentum, we might see the disturbance as linked to the Heisenberg Uncertainty - since the variance of one observable is inversely related to the other, their marginal probability distributions are necessarily constrained / altered by each other. The marginal distribution of one of the non-commuting pairs will depend on the specific distribution of the other, disturbing the notion of any context independent marginal probabilities. The contextual joint probabilities are then always dependent on how particular experimental contexts constrain position/momentum and so there would be no possible joint probability that we can construct just using p(A) and p(B) taken at face value from the experimental set up. Any experimental set up which subsumes or fails to differentiate multiple different contexts for non-commuting observables will have marginal probabilities which fail to produce a single valid joint probability distribution for those observables that attempts to generalize across all of those different contexts simultaneously... Only joint probabilities in individual contexts induced by disturbances.

Andrei Khrennikov has a whole series of papers informative on this, talking about the link between the law of total probability, interference and experimental contexts. The following is just one example:


https://scholar.google.co.uk/scholar?cluster=4642651957428255714&hl=en&as_sdt=0,5&as_vis=1 )


We can now look at specifically why position and momentum do not commute and have particular uncertainty relations. When looked at in terms of the path integral formulation, the non-commutativity in quantum mechanics can be derived directly from the fact that the particle trajectories are non-differentiable. This can be seen as a direct result of the erratic, zig-zagging nature of the paths, embodying the inherently random, probabilistic nature of measurement outcomes in quantum mechanics.


https://en.m.wikipedia.org/wiki/Path_integral_formulation (Section: path integral in quantum mechanics; canonical commutation relations)


This property seems to be directly inherited from the Wiener process / integral that the path integral formulation is related to by Wick rotation.


https://en.m.wikipedia.org/wiki/Wick_rotation

Note: Where is the Commutation Relation Hiding in the Path Integral Formulation?
https://scholar.google.co.uk/scholar?cluster=11872738296616463941&hl=en&as_sdt=0,5&as_vis=1


The Wiener process is a very broadly applicable stochastic process, perhaps most well known in physics as a model for Brownian motion which describes the random behavior of a particle suspended in a medium (i.e. liquid or gas). The evolving probability density function for the Brownian motion of a particle can then be described by a diffusion equation. Interestingly, the Schrodinger equation is also related by Wick rotation to a diffusion equation so that the Schrodinger equation can effectively be seen as a diffusion equation with a complex constant or describing a diffusion process in imaginary (related to complex numbers) time.

Paths realized by Wiener processes are also characterized by the same kinds of random jumps, rendering them non-differentiable. This non-differentiability is well known, leading to the construction of tools such as stochastic calculus designed specifically to deal with this non-differentiable nature. Consequently, we can actually derive non-commutativity properties for classical Wiener processes in a similar way as in the quantum case:


Ito’s stochastic calculus and Heisenberg
commutation relations:

https://www.sciencedirect.com/science/article/pii/S0304414910000256


Given that the Heisenberg uncertainty relations can be derived from non-commutativity, it is then no surprise that analogous uncertainty relations are actually well documented in diffusion processes. Not only can these be derived from the same kind of non-differentiability but it can be shown that this seems to apply generically for a broad range of stochastic systems. Examples, subsumed into this also include hydrodynamics and well known uncertainty relations in statistical thermodynamics:


--- --- ---
Generalization of uncertainty relation for quantum and stochastic systems:

https://www.sciencedirect.com/science/article/abs/pii/S0375960118303633

Classical uncertainty relations and entropy production in non-equilibrium statistical mechanics:

https://scholar.google.co.uk/scholar?cluster=6026419417934600498&hl=en&as_sdt=0,5&as_vis=1

Non-quantum uncertainty relations of stochastic dynamics:

https://scholar.google.co.uk/scholar?cluster=12722751213412558053&hl=en&as_sdt=0,5&as_vis=1
--- --- ---



Given that quantum mechanics is clearly a special case of stochastic systems where non-commutativity and uncertainty relations occur, this hints that they are just a direct result of the random, probabilistic nature of quantum mechanics.

There have also been some attempts to derive uncertainty relations directly from the probability density functions of stochastic processes in a comparatively generic manner. Here, probability densities (analogous to generalized coordinates/position) are contrasted with the probability gradient or flow/dynamics (analogous to generalized momentum) in the context of systems characterized by random behavior. These can also be described in terms of the Fourier transform in the same way as can be done for quantum position-momentum uncertainty relations:


--- --- ---
Indeterminacy relations in random dynamics:

https://scholar.google.co.uk/scholar?cluster=6176854777283805481&hl=en&as_sdt=0,5&as_vis=1

Information dynamics: temporal behavior of uncertainty measures:

https://scholar.google.co.uk/scholar?cluster=6481107725040031189&hl=en&as_sdt=0,5&as_vis=1

Parcels and particles: Markov blankets in the brain:

https://scholar.google.co.uk/scholar?cluster=13034249073504028456&hl=en&as_sdt=0,5&as_vis=1

A free energy principle for a particular physics:

https://scholar.google.co.uk/scholar?cluster=10954599080507512058&hl=en&as_sdt=0,5&as_vis=1
--- --- ---



The author of the bottom paper has given an interesting possible intuition for thinking about why these types of uncertainty relations might exist generically:


"Intuitively, if the probability mass of a particular state is concentrated around one point in phase-space, then the flow must [be vigorously rebuilding gradients – in all directions – to counter the dispersive effects of random fluctuations]. This means that the flow is as dispersed as the fluctuations. Conversely, if flow is limited to a small range, random fluctuations would disperse particular states over state-space. In short, the dispersion of states and their flow must complement each other at a nonequilibrium steady-state."


As an example: if you want to stop a droplet of some liquid dissolving in a glass of water, you will want forces to act on the liquid particles from all directions to keep them within a small vicinity. If force is applied solely from one direction without counteracting forces from all of the others, the liquid droplet will still be able to disperse everywhere else except from where the force came from. It would be impossible to keep the liquid particles all in one place if the forces acting on them are all acting in / from a single precise direction. Conversely, the liquid particles could never disperse across a broad range of positions if there were forces acting from every possible direction gathering them up.


The author also expresses this similarly in the more specific context of thermodynamics which also has well known thermodynamic uncertainty relations:


"Intuitively speaking, random fluctuations always increase the entropy through dispersing the ensemble density, while flow decreases entropy by ‘rebuilding’ probability gradients (i.e., where probability currents flow towards regimes of greater density). In other words, random fluctuations disperse states into regimes of high surprisal (and implicitly thermodynamic potential), while gradient flows – due to forces – counter the implicit dispersion."



4. Do we need to give up realism, locality or free choice?


As has been said, the wave function does no more than describe long run relative frequencies which manifest as compatible, non-signalling joint probability distributions between the spatially separated measurements of a Bell experiment. The wave function does not describe individual particles whose superpositions physically collapse either. When combined with the fact that the Bell violations are formally rooted in locally non-commuting variables which are necessitated by stochastic systems, it is tempting to think that Bell nonlocality is more or less just a strange statistical artifact that signifies the absence of global joint probability distributions (or conditions of possible experience according to Boole). This may explain why nonlocality seems to co-exist happily with non-signalling even though they prima-facie contradict each other. Therefore, even though there are definitely nonlocal correlations in quantum mechanics, we seem to be able to explain them away locally without a need to refer to spooky nonlocal forces that act between individual particles and violate speed-of-light limits. Given that the non-commutativity of locally related observables can be naturally explained as a consequence of stochasticity, there also does not seem to be a need to appeal to distant causes in the past that are influencing measurement settings a la superdeterminism, i.e. giving up free choice.

It seems that giving up realism may be what is preferable: giving up the notion that particle states have pre-existing values. Under a statistical interpretation, we can retain definite properties of particles at every point in time during any run of an experiment without contradicting ideas of contextuality, incompatibility or the notion of irrealism. This is because the wave function and superposition is about probability distributions describing long run frequencies, not individual particles. The idea that particles do not have definite pre-existing states prior to measurement is then simply replaced by, or rather, shifted toward the notion that non-commuting variables do not have meaningful joint probability distributions for their long run frequencies that are independent of the particular experimental context. Therefore, even though realism is given up in a way compatible with the requirements of Bell's theorem, particles can still retain their definite properties as individuals in a realist way.



5. Underlying causes of indeterminacy?


The main benefits of a statistical or stochastic interpretation is that there is no measurement problem and some realism is restored in the sense that particles have definite properties. It is also arguably the most straightforward way of interpreting the math of quantum mechanics without injecting any additional ontology into the formulas; after all, the Schrodinger equation is just a complex (number) diffusion equation, diffusion equations describing the behavior of stochastic processes. At the same time, this leaves a lot to be desired in terms of explanation: it might be asked why exactly particles behave non-deterministically, perhaps hinting at some underlying cause yet to be discovered. A deterministic explanation would not necessarily be ruled out, in the same way that stochastic models of Brownian motion are in principle explainable in a classical, deterministic way through collisions between a Brownian macro-particle and the micro-particles that constitute the medium it is suspended in. Given how Brownian motion is related to the behavior of Brownian particles suspended in some background medium, we might ask if there is something analogous in quantum mechanics which can provide the type of underlying explanation we might want. Potential hypotheses could relate to quantum foam or random fluctuations in the underlying spacetime and quantum vacuum fields.