The Most Embarrassing Graph in Modern Physics

Scientists don’t always agree with each other. Yes, I know; shocking but true. In cases of collegial disagreement, it’s often fun to quantify the extent of opinion by gathering a collection of experts and taking a poll. Inevitably some killjoy will loudly grumble that “scientific questions aren’t decided by voting!”, but that misses the point. A poll of scientists isn’t meant to decide questions, it’s meant to collect data — mapping out the territory of opinion among people who have spent time and effort thinking carefully about the relevant questions.

There’s been a bit of attention given recently to one such poll, carried out by Maximilian Schlosshauer, Johannes Kofler, and Anton Zeilinger at a quantum foundations meeting (see John Preskill at Quantum Frontiers, Swans on Tea). The pollsters asked a variety of questions, many frustratingly vague, which were patiently answered by the 33 participants.

This plot gives the money shot, as they say in Hollywood:

Quantum Poll

It’s a histogram of the audience’s “favorite” interpretation of quantum mechanics. As we see, among this expert collection of physicists, philosophers, and mathematicians, there is not much of a consensus. A 42% percent plurality votes for the “Copenhagen” interpretation, while the others are scattered over a handful of alternatives.

I’ll go out on a limb to suggest that the results of this poll should be very embarrassing to physicists. Not, I hasten to add, because Copenhagen came in first, although that’s also a perspective I might want to defend (I think Copenhagen is completely ill-defined, and shouldn’t be the favorite anything of any thoughtful person). The embarrassing thing is that we don’t have agreement.

Think about it — quantum mechanics has been around since the 1920’s at least, in a fairly settled form. John von Neumann laid out the mathematical structure in 1932. Subsequently, quantum mechanics has become the most important and best-tested part of modern physics. Without it, nothing makes sense. Every student who gets a degree in physics is supposed to learn QM above all else. There are a variety of experimental probes, all of which confirm the theory to spectacular precision.

And yet — we don’t understand it. Embarrassing. To all of us, as a field (not excepting myself).

I’m sitting in a bistro at the University of Nottingham, where I gave a talk yesterday about quantum mechanics. I put it this way: here in 2013, we don’t really know whether objective “wave function collapse” is part of reality (as the poll above demonstrates). We also don’t know whether, for example, supersymmetry is part of reality. Wave function collapse has been a looming problem for much longer, and is of much wider applicability, than the existence of supersymmetry. Yet the effort that is put into investigating the two questions is extremely disproportionate.

Not that we should be spending as much money trying to pinpoint a correct understanding of quantum mechanics as we do looking for supersymmetry, of course. The appropriate tools are very different. We won’t know whether supersymmetry is real without performing very costly experiments. For quantum mechanics, by contrast, all we really have to do (most people believe) is think about it in the right way. No elaborate experiments necessarily required (although they could help nudge us in the right direction, no doubt about that). But if anything, that makes the embarrassment more acute. All we have to do is wrap our brains around the issue, and yet we’ve failed to do so.

I’m optimistic that we will, however. And I suspect it will take a lot fewer than another eighty years. The advance of experimental techniques that push the quantum/classical boundary is forcing people to take these issues more seriously. I’d like to believe that in the 21st century we’ll finally develop a convincing and believable understanding of the greatest triumph of 20th-century physics.

  1. Sean,
    Don’t worry, the laws underlying the physics of everyday life are completely understood.

  2. I think the answer to molyneux’s problem applies here. Children born with congenital cataracts whose sight is restored can distinguish shapes by touch but not by sight until they have had time to touch and see the shapes together.

    Our abstract imagination is limited by our cognitive sensory experience. This survey result springs from those kind of limits. We will always be limited in our capacity to understand nature.

    http://www.nature.com/neuro/journal/v14/n5/abs/nn.2795.html

  3. In practice, I suspect that “Copenhagen” is mostly shorthand for “stop bothering me” or “shut up and calculate”. It’s the traditional default answer you give when you don’t want to talk about the interpretation of quantum mechanics, though, or perhaps because, nobody seems precisely sure what it is. And that’s not necessarily bad, though I suppose more interest would be expected at a foundations conference.

    I think I’m with the people who don’t find the disagreement all that embarrassing. Like Chad, I do not believe that people are going to be able to pick a correct interpretation from this list through pure reason. There are some, like the objective-collapse models, that actually make different experimental predictions from pure QM, and those might actually be tested (some crude forms of objective collapse one might propose have already been experimentally falsified). But any interpretation that’s consistent with all the results of quantum mechanics is going to stand unless and until quantum mechanics itself is wrong.

  4. I interrupted calculating the theoretical UV spectrum of a series of isomeric organic molecules by QM in order to read this thread, well I was waiting for the calculation to finish. I am a chemist not a physicist, therefore by definition I don’t understand QM but then we all don’t.

    I’m doing this calculation for the mundane and practical purpose of helping to identify an unknown compound in a pharmaceutical dosage form. QM is OK for FAPP, however I am not sure I want to shut up and calculate. The purely philosophical desire for physical realism leads me towards the Everett interpretation.

    What drives me to distraction is that in biology then from the many worlds point of view evolution must occur in Hilbert space. As Schrödinger pointed out in “What is Life” since X-rays cause mutations the source of mutation must be quantum mechanical enabling him to correctly predict that the genetic material was at the molecular level. Monod pointed out evolution was driven by chance and necessity http://ichemicalscum.blogspot.ca/2012/09/what-drives-evolution-mechanics-on.html .

    As Gould has suggested if you could rerun evolution again you would not get the same outcome. So all possible outcomes should on the basis of many worlds be real. I’ve wondered if there was some way of testing this using Bayesian statistics but I cant see one so I am driven to distraction.

  5. This is simply a proof (if anyone needed it) that a century later QM is still the frontier of our understanding of reality.

  6. I suspect that the poll allowed participants to give multiple answers, thus explaining why the sum > 100%.

  7. I disagree with the claim that the Copenhagen interpretation is ill-defined.

    There is only one assumption in QM, that the physical observables form a noncommutative algebra. While the elements of commutative algebras can be represented as numbers, noncommutative algebras should be represented as matrices, or more generally, as linear operators defined on Hilbert-spaces.

    The expectation value should preserve the linearity of this structure.
    So we can use (psi, A psi) for pure states or, more generally Tr(rho*A) for mixed states.

    The whole mathematical construction is completely natural and well defined.

    Of course, it is hard to understand QM in an intuitive way, because our intutition is based on classical physics = commutative algebras. But this simply doesn’t mean that QM is ill-defined. Noncommutative algebras are completely well defined.

  8. I’ve been quite sympathetic to the idea that Scott Aaronson articulated, that Quantum Mechanics is not a jealous god (such as in the model of JHVH.) If thinking about MWI helps you perform a calculation in the morning and Copenhagen gets you through an afternoon problem, you have not angered your deity by thinking in two different ways. So long as you don’t upset any experimentalists, we can say every interpretation is just fine. Even if you have realist sensibilities, what is it that’s supposed to be real, other than the results of experiments? If it can’t be experimentally determined, there isn’t really some obligation to have a definite answer, that we should feel inadequate for failing to give.

    I was surprised though that consistent histories was at 0, I tend to like it.

  9. A particle is a point in space that has no dimension, so, how can it be something that can be measured or observed, it’s just a small bit of a wave that continuously moves about through space and time, so it can’t be measured exactly any more than an ocean wave, you can only guess measure where it was but never, never a location. Just a thought. Am I WAY OFF the mark.

  10. @Matt McIrvin “In practice, I suspect that “Copenhagen” is mostly shorthand for “stop bothering me” or “shut up and calculate”.”

    Unlikely as this poll was taken at a conference dedicated to the foundations of quantum theoy, so people who do not want to think about interpretations of quantum theory would not have been there.

    I think that one of the main things I learned from this poll is that if you conduct a poll at a conference organized by Zeilinger then Copenhagen will come out top, whereas if you conduct a poll at a conference organized by Tegmark then many worlds will come out top. Is this a surprise to anyone?

  11. Many Worlds Interpretation seems like the most plausible in my opinion. I’ve never really been a big believer in fundamental randomness, it just doesn’t make sense (even to my post quantum strangeness incorporating mind).

  12. Personally I find that data from only 33 participants was used to publish a paper to be fairly embarrassing. No where near 5 sigma confidence of anything in the community, just those who happened to be at one conference.

  13. How does the many worlds interpretation reproduce probabilities when they are generally irrational numbers?

  14. Matt Leifer,

    I totally agree with you. The work
    of John Bell, Cock Inspector, and
    PBR, etc is definitely a step in the
    right direction.

  15. It doesn’t really matter. Physics is all about explaining the results of experiments, and predicting the results of experiments by using a made-up “model” and mathematical rules. That’s it. The model may not have any physical significance to it or correspond to anything “actually existing”. If the model and the mathematical rules allow you to explain stuff and predict stuff, then it’s useful and we keep it around and say things like “that’s how the world works”. At the heart of quantum mechanics is “the wavefunction” and the mathematical rules that govern it. It allows us to do pretty well, and that’s all that matters. Who cares what the wavefunction is or what it does, as long as it gives us results.

    One thing I’ve noticed is how, as we go to smaller and smaller distances, the world seems to get more and more abstract and mathematical. We go from planets, which behave like actual objects with independent existence, to things like atoms and elementary particles, which are much more mathematical, less like anything “physical” we have experience with. Each electron is exactly the same as every other electron because electrons are just names we attach to a mathematical concept that seems to act in accordance with certain mathematical rules attached to that mathematical concept. It works, so who cares about interpretation.

  16. Here’s the answer to everything.

    Evidently, an electron does not possess the property of position before it’s measured. But it cannot have *no* properties at all before measurement, since then it would never get measured. Thus, quantum particles have some properties but not others before measurement.

    The electron is ‘real’ at all times.

    Is there anything wrong with this ‘realist’ interpretation?

  17. @ … and physics:

    In addition to “real” electrons, there are also “virtual” electrons, that don’t have well-defined properties of real electrons (like mass, charge, spin, etc.). They are never observed directly (for precisely the reasons you state), but their “secondary effects” can be (and indeed are) observed.

    You don’t want to think about an electron as a particle, but rather as an excitation of the electron field, just like a photon is an excitation of the electromagnetic field. Particles don’t exist in nature, fields do.

    HTH, 🙂
    Marko

  18. The raison d’etre for all these interpretations of QM is physicists struggle to resolve the so-called “measurement problem”, which is at the core of the theory itself, and it is a real problem.

    Stated simply, QM doesn’t describe the quantum world, but rather the interaction between the quantum world with the classical world. There is always an (implicit or explicit) assumption that the “measurement device” does not and must not obey the laws of QM. This assumption introduces a fundamental dualism into our understanding of nature, saying that the laws of physics do not apply universally for all the “pieces” of the Universe.

    So one is forced to choose which “pieces” do obey QM and which don’t, and how to determine the boundary between the two sets. Consequently, people can either have an opinion (i.e. pick one of the interpretations), or choose to ignore the problem (the shut-up-and-calculate strategy).

    But do keep in mind that the interpretations are not a matter of just personal taste, or our psychological means of coming to terms with the reality of quantum physics. They are actually frameworks that try to address a very real (and so far a very unsolved) problem of quantum physics. Furthermore, they all fall short of resolving this problem in a satisfactory way.

    For example, as soon as you consider the Universe to be your quantum system, and a hydrogen atom to be a measuring device (which “observes” the Universe and collapses its wavefunction), the “measurement problem” blows right in your face, showing the incompleteness of any and every formulation of QM:

    * there is no concept of probability, since you don’t have an ensamble of Universes;
    * there is no decoherence, since the number of degrees of freedom of your measuring device is very limited;
    * there is no arrow of time, since the measuring device doesn’t have a large enough phase space (and as a consequence, “evolution in time” can not be formulated).

    And then what? The shut-up-and-calculate philosophy is unmaintainable, since QM doesn’t tell you a damn thing about the predictions of the measurements. That is the main issue that various interpretations are trying to resolve.

    The QM formalism, as well as it works on table-top experiments, is fundamentally incomplete. Completing it requires one to have a “philosophical interpretation” of QM, one that can be used to extrapolate our understanding of the QM formalism beyond the “small quantum and large classical” physical systems.

    Give all that, it is only natural that there is no consensus about any of these interpretations, and the graph seems quite ok from my point of view.

    HTH, 🙂
    Marko

  19. I have to agree with Matt Leifer. We are not ascertained that we have enough information to test specific QM theories as is, as they then should be called. We need to continue to try to derive tests.

    @ Nicholas Malaya: How do you define and test a boundary between “physical sense” & “direct observable” and any other set of constraints?

    Doing observations by way of experiments are using constraints, which can be more or less put there by theory. In that sense there is no difference between ad hoc observations (say, temperature before it was defined by statistical physics) and observational tests of measured values by way of theory (say, entropy).

    Testing for existence is at the heart of every mechanics, classical, relativistic or quantum, by “constrained reaction on constrained action”, as action-reaction or observation-observable (re Deutsch’s testable definition). We can’t bootstrap up any kind of physics without having done that test as first order of business!

    @ Rezso: I think the problem is that it is ill-defined physically. How fast does the wavefunction “collapse”? I have never heard of any such prediction from the Copenhagen theory. (Admittedly I am no theoretical physicist.)

    There doesn’t seem to be a dynamics for a crucial part of the theory.

  20. @ vmarko: I don’t think this moves your point (I bow out on decoherence), but I don’t think QM is necessarily incomplete just because it doesn’t explicitly handles characteristics of macrosystem such as universes or arrows of time.

    Susskind derives the arrow of time (and yes, probabilities over ensembles of universes) from the dynamics of an inflationary multiverse. (More precisely terminal vacuums sets up an arrow.) That trickles down to microscale, where we don’t want any irreversibility correlated to an arrow, we want microscopic reversibility (“detailed balance” of chemistry). There would be no incompleteness between these two theories, QM within the observable universe/causal patch, and inflation from the causal patch up to the overall system.

  21. @ doc c: Your religious comment is a veritable deepity. The inflationary standard cosmology is observable with only 6 parameters, which is about as many as you use to make images from a telescope in the first place.

    And that cosmology predicts and test that universes are zero energy, so can only happen spontaneously. This is perhaps what Hawking started in the 70’s, when he discovered that universes could start so. But now we also know that all universes must start so. I believe that is what Krauss has written a book about (but I can be mistaken).

    Humanity killed all gods 2004. Or at least the theistic creationist ones, with the implication of how inflation smooths the universe and so decides at least some physical laws, that the deist ones could soon be dead too. Or, reminding of our host’s predilections, since physicalism is a testable and now tested theory, all magic can now be rejected beyond reasonable doubt.

    The mistake was to believe that we will not be able to understand the universe, or that invisible bearded men had anything to do with it.

  22. @ Torbjörn Larsson:

    Yes, Susskind uses the Multiverse ideas to circumvent some of the problems. This idea initially began with the Everett’s many-worlds interpretation of QM, which was devised precisely for the purpose of introducing “an ensamble of Universes”, so that one can keep talking about probability.

    But Multiverse ideas have their own bag of problems (for example, the inability to perform the preparation of the ensamble into the same initial quantum state), so it’s not as easy as it might look.

    Of course, there are quite some valiant and bold attempts, with even more to come over time, but so far there is no solution which is convincing enough. The result of the poll that Sean displayed says precisely that. 😉

    Best, 🙂
    Marko