Cfxtrjtrk

It is well known that the observed energy density of the vacuum is many orders of magnitude less than the value calculated by quantum field theory. Published values range between 60 and 120 orders of magnitude, depending on which assumptions are made in the calculations. Why is this not universally acknowledged as a strong empirical falsification of quantum mechanics?

Experimentally, based on cosmological observations, there seems to be a vacuum energy (the "dark energy" component of the cosmological energy budget), with a certain value. A the present epoch, there seems to be about three times as much vacuum/dark energy as there is "dark matter" and about fifteen or twenty times as much dark energy as there is visible matter. This concentration of dark energy poses two very serious puzzles, but neither of them is at all suggestive of a breakdown of quantum mechanics.

The first problem, mentioned in the question, is the "hierarchy" problem. There is no quantum mechanical prediction for the absolute energy density of vacuum. However, it is possible to make some very crude "guesstimates" about this quantity. We know that some new fundamental physics must take over at the Planck energy scale $E_{P}$, where gravitational interactions are in the deeply quantum regime. We may therefore guess that the vacuum energy density is proportional to $E_{P}^{4}$. (This is certainly not a prediction of quantum mechanics though. Strictly, according to quantum field theory, without including gravity, the energy of the vacuum is unobservable and therefore not even well defined.) The problem with the $propto E_{P}^{4}$ guess for the vacuum energy is that it is off by 275 nepers or so. But that does not falsify quantum mechanics, since our guess was not based on rigorous quantum theory anyway.

The other puzzle with the vacuum energy density (the "coincidence" problem) is that its value at the present cosmological epoch is pretty close to the energy density of matter in the universe, even though there is no a priori reason why the two should be related. The fact that the (light plus dark) matter and dark energy densities are relatively close suggests that what we are observing as apparent vacuum energy might very well be something else entirely anyway. But what that "something else" might be, no one knows.

"Quantum mechanics" is actually a very general, broad theory that "really", at least working from many more modern understandings of the topic, is about information, and more specifically, it is a language for writing theories that describe (in some way) physics in which information content is limited, just as relativity is actually a theory of space and time in which information propagation speed is limited. And moreover, that the information is limited in such a way that there are trade-offs between information determining various physical parameters of a system, and not a simplistic "pixelization". That is, in effect, what the "true" meaning of Planck's constant $hbar$, and the fact that $hbar > 0$, means. Check out Scott Aaronson's page here for the idea of quantum mechanics as a language for writing theories, instead of per se a theory in its own right:

https://www.scottaaronson.com/democritus/lec9.html

though it doesn't specifically touch on the "information limit" notion, for that, try:

https://iopscience.iop.org/article/10.1088/0143-0807/36/1/015010

e.g. section 3.8, mentions the idea of QM as an information-limited theory, at least in touching, though doesn't quite go about it in the same way as I had worked it out.

The way then to "falsify" quantum mechanics would be to show an instance where its informational limits are violated, e.g. if someone finds a way to create a particle that has position and momentum (or another pair of incompatible physical parameters) more precisely defined than Heisenberg's limit allows. Merely finding a failure of certain theories built on it (e.g. "quantum field theories" - QFTs) to account for a cosmological parameter's value which is already going to be well in the range of those limits is not going to necessarily falsify QM, as another theory written in its language might still work and be able to account for that result, even spectacularly. It will simply falsify that particular theory built using it, namely Standard Model QFTs. (Whether QFTs entirely are out, at at least a fundamental level, is disputable, but the SM is at least guaranteed to have something wrong with it.)

Our back of the envelope prediction for the order of magnitude of the vacuum energy is indeed very wrong! However, keep in mind that

1. It is possible to precisely fine-tune free-parameters of the theory to match the measurement. This is achieved through a delicate cancellation between so-called tree-level parameters and corrections. When we make the back of the envelope calculation, we implicitly assume that such cancellations don't occur.




2. This isn't a test of quantum mechanics per se; but a test of a particular theory that obeys a combination of quantum mechanics and special relatively. Such theories are called quantum field theories. There are many such theories as we may introduce lots of types of fields and let them interact in lots of different ways.

So, quantum mechanics isn't falsified as measurements of the vacuum energy don't directly test it. And even the theories that the measurements do test aren't falsified because we can find extremely fine-tuned combinations of parameters that match observations.

The fact that fine-tuning is required is considered problematic and arguably means that our theories might be somewhat implausible; read about naturalness/fine-tuning in physics for more information.

Since we need quantum mechanics to describe atoms, metals, anything to do with particle physics, etc., it is not something that can be put aside so easily. Any alternative to quantum mechanics, sought because of the vacuum energy problem, would have to nonetheless behave like quantum mechanics in all those cases where it does work.

In any case, an impossibly large vacuum energy is not a generic prediction of quantum mechanics. There are plenty of quantum theories in which the vacuum energy equals zero, notably all supersymmetric quantum theories in which supersymmetry remains unbroken.

Since the large vacuum energies come from quantum theories which combine the standard model, or supersymmetric extensions of the standard model, with gravity in a certain way, the usual supposition has been that a small vacuum energy will be obtained through identifying the right combination of fields, and/or the right approach to quantum gravity, rather than by abandoning quantum mechanics per se.

Also, even if your theory predicts a large vacuum energy, if it is a field theory you can simply postulate a separate cosmological constant term that cancels out most of it. (You may see this in action in "The New Minimal Standard Model", in comments around equation 3.) This is considered unsatisfactory for a fundamental theory, because it requires a vastly unlikely near-coincidence between two independent parameters, but it does provide a way for a quantum theory to produce a net vacuum energy of the correct size.

So the problem of the vacuum energy is a real problem but hardly anyone seeks an answer through the abandonment of quantum mechanics, except people who are already developing an alternative to quantum mechanics for other reasons. (I think "stochastic electrodynamics" may partly be motivated by vacuum energy issues?) Incidentally, yet another reason that people don't proceed that way, is that the known alternatives to quantum mechanics don't equal all the other things that quantum mechanics can do. One basic example: fermion fields, in which there are antiparticles as well as particles, and they can be created and destroyed in pairs. Bohmian mechanics is one of the leading alternatives to quantum mechanics, and yet to my knowledge, there is no Bohmian model of fermion fields. My point here is that physicists who care about describing physics at the level where vacuum energy is an issue, actually have no existing alternative to the quantum framework.

This is a very good question. As noted above it is not a prediction of "ordinary" quantum mechanics but of QED/QFT. As it is probably the wrongest prediction ever, something fundamental is wrong. However since predictions of QED tend to be very accurately confirmed by experiment, it cannot be a complete falsification. Something in the theory is false, however. Unresolved so far.

On a philosophical level, quantum field theory makes many predictions that have been empirically tested and verified correct, and just a few predictions that have been falsified. (Off the top of my head, I can't remember any others besides the vacuum energy density, although I vaguely recall reading that there are more.) That does mean QFT is missing something, but it doesn't mean it's nonsense. The "correct" theory might use totally different equations and "laws" than QFT does, but it will agree with the predictions of QFT for all the cases that have been verified, including all of the counterintuitive ones. It will still have matter waves, delocalization, entanglement, and the uncertainty principle, for instance.

A good historical analogy is to the replacement of Newtonian gravitation with general relativity. Newtonian gravitation is mostly correct. It gives good predictions for the behavior of small objects moving slowly near Earth's surface, and for all of the planets' orbits except Mercury. The mathematics of general relativity is totally different and much more complicated, but it gives very nearly the same predictions for small objects moving slowly near Earth's surface, and for most of the planets' orbits. And it gives a correct prediction for Mercury's orbit.

We can't measure the energy density of the vacuum as such. What we measure is the large-scale curvature of spacetime. Actually, we don't measure that either: what we measure is light in the vicinity of Earth, and we relate that by a complicated (and theory-laden) calculation back to spacetime curvature, and from there (via more theory) to an alleged vacuum energy density, which disagrees spectacularly with an alleged vacuum energy density obtained by a different theoretical calculation.

We know that there is something wrong with this process, because there is an inconsistency, but we don't know where the problem is. Knowing where the problem is is tantamount to knowing how to fix it.

It's a he-said-she-said kind of situation. If Alice says "P implies Q", and Bob says "P", and Carol says "not Q", why is this not universally acknowledged as a falsification of Alice's claim? Because we don't know that Alice is the one who is wrong; it could be any of the three.