Science and Unobservable Things

Today’s Bloggingheads dialogue features me and writer John Horgan — I will spare you a screen capture of our faces, but here is a good old-fashioned link.

John is the author of The End of Science, in which he argues that much of modern physics has entered an era of “ironic science,” where speculation about unobservable things (inflation, other universes, extra dimensions) has replaced the hard-nosed empiricism of an earlier era. Most of our discussion went over that same territory, focusing primarily on inflation but touching on other examples as well.

You can judge for yourself whether I was persuasive or not, but the case I tried to make was that attitudes along the lines of “that stuff you’re talking about can never be observed, so you’re not doing science, it’s just theology” are woefully simplistic, and simply don’t reflect the way that science works in the real world. Other branches of the wavefunction, or the state of the universe before the Big Bang, may by themselves be unobservable, but they are part of a larger picture that remains tied to what we see around us. (Inflation is a particularly inappropriate example to pick on; while it has by no means been established, and it is undeniably difficult to distinguish definitively between models, it keeps making predictions that are tested and come out correct — spatial flatness of the universe, density fluctuations larger than the Hubble radius, correlations between perturbations in matter and radiation, fluctuation amplitudes on different scales that are almost equal but not quite…)

If you are firmly convinced that talking about the multiverse and other unobservable things is deeply unscientific and a leading indicator of the Decline of the West, nothing I say will change your mind. In particular, you may judge that the question which inflation tries to answer — “Why was the early universe like that?” — is a priori unscientific, and we should just accept the universe as it is. That’s an intellectually consistent position that you are welcome to take. The good news is that the overwhelming majority of interesting science being done today remains closely connected to tangible phenomena just as it (usually!) has been through the history of modern science. But if you instead ask in good faith why sensible people would be led to hypothesize all of this unobservable superstructure, there are perfectly good answers to be had.

The most important point is that the underlying goal of science is not simply making predictions — it’s developing an understanding of the mechanisms underlying the operation of the natural world. This point is made very eloquently by David Deutsch in his book The Fabric of Reality. As I mention in the dialogue, Deutsch chooses this quote by Steven Weinberg as an exemplar of hard-boiled instrumentalism:

The important thing is to be able to make predictions about images on the astronomers’ photographic plates, frequencies of spectral lines, and so on, and it simply doesn’t matter whether we ascribe these predictions to the physical effects of gravitational fields on the motion of planets and photons or to a curvature of space and time.

That’s crazy, of course — the dynamics through which we derive those predictions matters enormously. (I suspect that Weinberg was trying to emphasize that there may be formulations of the same underlying theory that look different but are actually equivalent; then the distinction truly wouldn’t matter, but saying “the important thing is to make predictions” is going a bit too far.) Deutsch asks us to imagine an “oracle,” a black box which will correctly answer any well-posed empirical question we ask of it. So in principle the oracle can help us make any prediction we like — would that count as the ultimate end-all scientific theory? Of course not, as it would provide no understanding whatsoever. As Deutsch notes, it would be able to predict that a certain rocket-ship design would blow up on take-off, but offer no clue as to how we could fix it. The oracle would serve as a replacement for experiments, but not for theories. No scientist, armed with an infinite array of answers to specific questions but zero understanding of how they were obtained, would declare their work completed.

If making predictions were all that mattered, we would have stopped doing particle physics some time around the early 1980’s. The problem with the Standard Model of particle physics, remember, is that (until we learned more about neutrino physics and dark matter) it kept making predictions that fit all of our experiments! We’ve been working very hard, and spending a lot of money, just to do experiments for which the Standard Model would be unable to make an accurate prediction. And we do so because we’re not satisfied with predicting the outcome of experiments; we want to understand the underlying mechanism, and the Standard Model (especially the breaking of electroweak symmetry) falls short on that score.

The next thing to understand is that all of these crazy speculations about multiverses and extra dimensions originate in the attempt to understand phenomena that we observe right here in the nearby world. Gravity and quantum mechanics both exist — very few people doubt that. And therefore, we want a theory that can encompass both of them. By a very explicit chain of reasoning — trying to understand perturbation theory, getting anomalies to cancel, etc. — we are led to superstrings in ten dimensions. And then we try to bring that theory back into contact with the observed world around us, compactifying those extra dimensions and trying to match onto particle physics and cosmology. The program may or may not work — it’s certainly hard, and we may ultimately decide that it’s just too hard, or find an idea that works just as well without all the extra-dimensional superstructure. Theories of what happened before the Big Bang are the same way; we’re not tossing out scenarios because we think it’s amusing, but because we are trying to understand features of the world we actually do observe, and that attempt drives us to these hypotheses.

Ultimately, of course, we do need to make contact with observation and experiment. But the final point to emphasize is that not every prediction of every theory needs to be testable; what needs to be testable is the framework as a whole. If we do manage to construct a theory that makes a set of specific and unambiguous testable predictions, and those predictions are tested and the theory comes through with flying colors, and that theory also predicts unambiguously that inflation happened or there are multiple universes or extra dimensions, I will be very happy to believe in the reality of those ideas. That happy situation does not seem to be around the corner — right now the data are offering us a few clues, on the basis of which invent new hypotheses, and we have a long way to go before some of those hypotheses grow into frameworks which can be tested against data. If anyone is skeptical that this is likely to happen, that is certainly their prerogative, and they should feel fortunate that the overwhelming majority of contemporary science is not forced to work that way. Others, meanwhile, will remain interested in questions that do seem to call for this kind of bold speculation, and are willing to push the program forward for a while to see what happens. Keeping in mind, of course, that when Boltzmann was grounding the laws of thermodynamics using kinetic theory, most physicists scoffed at the notion of these “atoms” and rolled their eyes at the invocation of unobservable entities to explain everyday phenomena.

There is also a less rosy possibility, which may very well come to pass: that we develop more than one theory that fits all of the experimental data we know how to collect, such that they differ in specific predictions that are beyond our technological reach. That would, indeed, be too bad. But at the moment, we seem to be in little danger of this embarrassment of theoretical riches. We don’t even have one theory that reconciles gravity and quantum mechanics while matching cleanly onto our low-energy world, or a comprehensive model of the early universe that explains our initial conditions. If we actually do develop more than one, science will be faced with an interesting kind of existential dilemma that doesn’t have a lot of precedent in history. (Can anyone think of an example?) But I’m not losing sleep over this possibility; and in the meantime, I’ll keep trying to develop at least one such idea.

This entry was posted in Science, Time. Bookmark the permalink.

94 Responses to Science and Unobservable Things

  1. Peter Shor says:

    Sean said

    There is also a less rosy possibility, which may very well come to pass: that we develop more than one theory that fits all of the experimental data we know how to collect, such that they differ in specific predictions that are beyond our technological reach. … Can anyone think of an example?

    Do you mean something like the solar neutrino problem? For over 30 years, nobody knew whether our models of the sun’s interior were incorrect, or whether neutrinos had mass and the mass eigenstates didn’t match up with the flavor eigenstates. But remarkably, the experimentalists came through and observed neutrino oscillations.

    Or are you looking for a different kind of example.

  2. Sean says:

    Should have been more clear — there are plenty of examples where multiple models fit the data but made different predictions that we just weren’t yet able to test, but it was clear how such a test could be done. I was thinking more of examples where it was entirely unclear how the theories could possibly be distinguished, within the realm of technological feasibility.

  3. observer says:

    Sean, do you agree that, in principle, that a new theory that truly unifies general relativity and the standard model, might not need any new experiments to verify it, and that instead, a mathematical proof that the new theory appropriately contains the two old ones would suffice?

  4. frdn says:

    Why is John Horgan qualified to criticize theories in physics? It was painfully obvious in the diavlog that he doesn’t understand 1 thing about physics.

  5. Luis says:

    Sean said

    There is also a less rosy possibility, which may very well come to pass: that we develop more than one theory that fits all of the experimental data we know how to collect, such that they differ in specific predictions that are beyond our technological reach. … Can anyone think of an example?

    I’m not a physicist –only a modest theoretical linguist, but nonetheless, here is something from our field (excerpt from the preface to “Compositionality in formal semantics: selected papers of Barbara Partee”). No relation to physics, but I like the anecdote.

    When Mike Flynn was finishing up, we went for a farewell lunch. During lunch he said: “I want to ask you what’s probably a really stupid question: You and Edwin Williams –it has always seemed to me that your theoretical frameworks are at odds with each other […] Are they actually compatible in a way that I just don’t see, or are they incompatible and you don’t bring up the subject for some reason?” […] We [Williams and Partee] immediately decided to co-teach Introduction to Semantics next spring […] followed by two or three weeks for debate where we would jointly try to identify some empirical prediction on which the two approaches disagreed. […] By the end, we had identified one sentence type, illustrated below, about which our theories made opposite predictions

    On whom can you depend to do the dishes?

    We asked the students about whether the sentence was well-formed with “whom” controlling the implicit subject of “to do the dishes” (i.e., on the reading “which person can you depend on such that he will do the dishes?”). When we asked the class for judgements, the class split 50-50

  6. Sean says:

    observer — that’s exactly the pickle I was speculating about above. I don’t think it would “suffice,” absent some reason to believe that no as-yet-undiscovered model were equally viable, but it would certainly be quite an achievement, one well worth working toward.

  7. Ellipsis says:

    All true — however methinks thou doth protest a bit too much. 😉

    A clear example of “two theories” that are undifferentiable in that they give the same predictions up to essentially all imaginable experimental tests is Copenhagen interpretation vs. many worlds.

  8. Anne says:

    I’m far from an expert, but if you’re looking for two competing interpretations for which we can’t see a way experiment will ever distinguish them, how about quantum mechanics? The Copenhagen interpretation and the multiple-worlds interpretation are wildly different, but I don’t see how they could ever be experimentally distinguished.

  9. Peter Woit says:

    frdn,

    And what are your qualifications to criticize John Horgan? Other than being one of the all too numerous juvenile cretins who think writing in to physics blogs anonymously to attack people is a good way to spend your time?

  10. Sean says:

    Peter, thanks for setting an example of high-minded discourse. Your deep concern for the tone of discussion has been duly noted, many times. If you feel a need to reiterate it further, please do so somewhere else.

    Everyone else, please try to keep the discussion on-topic and free of insults.

  11. Sean says:

    Anne, most often different interpretations of quantum mechanics have precisely the same set of predictions for every conceivable experiment, so they are not really different theories. I was wondering about theories that may differ in principle — e.g., for dynamics at the Planck scale — but give identical predictions for every conceivable experiment within our technological reach. There’s certainly no reason why two such models couldn’t exist, especially in the case of gravity where the force is so weak.

  12. John Ramsden says:

    It’s perhaps not quite what you had in mind when asking for examples of multiple theories that fit all the physical facts, but as these theories would both presumably be expressed mathematically maybe it’s not too far off the mark…

    There are many examples in maths of more than one “rubric” being applicable to the same range of problems, with some of the latter better suited to one approach and different problems the other.

    One example would be modern(ish) algebraic geometry, where you had the analytic approach as expounded in the tome by Griffith and Harris for example and by contrast the algebraic approach of Weil and Zariski. These developed in parallel over roughly the first half of the 20th C.

    But they were pretty much united by Grothendieck’s theory of schemes, and I imagine the same would occur with the multiple physical theories – In due course some overarching formalism and abstract understanding would be developed so that both ended up under the same umbrella.

    Another example, this time within algebra, would be the theory of ideals: In the 19th century one had Kummer’s system, and then Kronecker came along and developed his theory of primary decompositions. It was only some years later that Dedekind devised the simple formalism which united both and is used today.

  13. Peter Woit says:

    Sean,

    The question is whether you should provide a forum for anonymous insults. I gather from your comment that you intend to continue to do so, that it’s only my objections to this that you see as being worth criticizing. It’s your blog….

  14. George Musser says:

    A sociological question: why does John Horgan continue to get any traction with his argument, considering that positivism was thoroughly discredited within science and within the philosophy of science more than half a century ago?
    George

  15. Mike says:

    Even if a branch of physics were not empirical, that would not mean it’s theology. It would mean it’s ‘just’ mathematics. There’s a big difference between mathematics and theology. In the context of curved geometries or closed universes, there’s a question as to whether pi is an element of reality. But this is very different than the question of whether reality contains a god or not.

  16. Elliot says:

    I, Elliot Tarabour, also believe that Horgan’s arguments are without merit. With respect to Copenhagen vs. MWI, it is not at all obvious to me that there is no way in principle to experimentally distinguish them.

    e.

  17. chemicalscum says:

    Sean
    Anne, most often different interpretations of quantum mechanics have precisely the same set of predictions for every conceivable experiment, so they are not really different theories. I was wondering about theories that may differ in principle

    David Deutsch has proposed an experiment that may be able, when future advances in the technology of quantum computing become available, to distinguish the many worlds interpretation from any interpretation involving state vector reduction. Indeed as I remember he discusses it in “The Fabric of Reality”.

  18. chemicalscum says:

    Sean

    corrected block quote (damn no preview)

    Anne, most often different interpretations of quantum mechanics have precisely the same set of predictions for every conceivable experiment, so they are not really different theories. I was wondering about theories that may differ in principle

    David Deutsch has proposed an experiment that may be able, when future advances in the technology of quantum computing become available, to distinguish the many worlds interpretation from any interpretation involving state vector reduction. Indeed as I remember he discusses it in “The Fabric of Reality”.

  19. miller says:

    I find this post extremely interesting, because I actually disagree with Sean for once. At least, I sort of disagree.

    You draw a distinction between making predictions and understanding. But if a particular understanding of a mechanism doesn’t in any way help to make predictions, what have you understood? It seems to me that you’ve understood something that has no effect whatsoever on reality, and is thus not real. I would argue that all science must in some way make predictions. These need not be the immediate goal of science, nor do they need to be practically testable. It’s also allowable to develop simpler ways to reach old predictions. “Ironic science”, as John Horgan calls it, is ok because it does make predictions, as obscure as those predictions are.

    As for that “less rosy possibility,” it’s already happened with miracles, hasn’t it? Theory: The laws of physics are time-invariant, with the single exception of two thousand years ago, when someone’s body spontaneously decayed into neutrinos and anti-neutrinos. This makes the prediction that if we went back in time, we could observe it directly, but this is untestable in practice. Quite the existential dilemma we’ve got there, huh? How do we stay sane in light of this mystery?

  20. anonymous says:

    “There is also a less rosy possibility, which may very well come to pass: that we develop more than one theory that fits all of the experimental data we know how to collect, such that they differ in specific predictions that are beyond our technological reach. … Can anyone think of an example?”

    The Continuum Hypothesis in Math.

  21. anonymous says:

    woops… the quote in the above post should be this one:

    “If we actually do develop more than one, science will be faced with an interesting kind of existential dilemma that doesn’t have a lot of precedent in history. (Can anyone think of an example?)”

  22. observer says:

    Re: Sean#6, Me#3, Not to nitpick, Sean, but actually my pickle was intended as a subpickle of your pickle.

    Re: John Ramsden#12, the difference is that in mathematics it is usually clear that a unification or generalization has been found, or at least it can be proved, while in physics you also have to contend with the connection between model and reality for each of the old and new theories.

    On a practical level, it seems most theories don’t have such lofty ambitions as to unify GR and SM, but at the same time they still manage to be theoretically robust enough to stick their neck out and make a falsifiable prediction. There are exceptions, of course.

  23. HB says:

    Sean,

    At around 10:30 you said “… the amount of curvature will grow …” I thought K(t)=k/a(t)^2 meant that amount of curvature wouldn’t grow. What am I missing?

    HB

  24. John Merryman says:

    Here is one I’ve raised a number of times; If time is a fundamental dimension, than physical reality moves along it from past events to future ones, but if time is a consequence of motion, than events are caused by this motion and proceed from future potential to past circumstance. So does time go from past to future, or from being in the future to being in the past?
    (Does the rotation of the earth proceed along the passage from one day to the next, or are days the consequence of the rotation of the earth and go from being in the future to being in the past?)

    a black box which will correctly answer any well-posed empirical question we ask of it.

    Reminds me of the old “What would a real guy do?” joke;

    If an alien came down to earth and gave you a black box and said it would save humanity from future calamity, would you; 1) Give it to the President? 2) Give it to the Pope? 3) Take it apart and figure out how it works?

  25. Peter Woit says:

    George,

    Painting John Horgan (or Steven Weinberg, or just about anyone else) as a pure positivist is just a way of trying to not deal with the real issue here. Some parts of particle physics (string theory) and some parts of cosmology (the multiverse) just inherently don’t predict or explain anything (using “explain” in the scientific sense, which carries with it a requirement that your explanation of how the world works be testable by experiment). The amount of attention and effort that has gone into such theories over the last quarter century is remarkable and very unusual in the history of science. Horgan more than ten years ago was one of the first science journalists to notice that something funny was going on.

    As more and more time goes on, and some serious physicists start engaging in more and more speculative behavior, getting farther and farther from any hope of testability (see, the Landscape and the multiverse), the argument that there’s a problem here is getting more and more traction. Among serious physicists these days, I don’t think this is even any more a very controversial claim, with many of them prepared to admit that parts of the subject are in a bit of a crisis.

    Horgan’s complaints about inflation are somewhat more unusual, and Sean is right that for the general idea there is serious circumstantial evidence. But even there, Horgan has a point and some versions of inflationary cosmology have serious problems with the question of testability.

    Horgan’s truly controversial views are not in this area, but in his arguments about the “end of science”. For those, he definitely does have trouble getting traction, especially among scientists, who pretty uniformly believe that what they are doing is not at an “end”, or they wouldn’t be doing it….