Beyond Falsifiability

I have a backlog of fun papers that I haven’t yet talked about on the blog, so I’m going to try to work through them in reverse chronological order. I just came out with a philosophically-oriented paper on the thorny issue of the scientific status of multiverse cosmological models:

Beyond Falsifiability: Normal Science in a Multiverse
Sean M. Carroll

Cosmological models that invoke a multiverse – a collection of unobservable regions of space where conditions are very different from the region around us – are controversial, on the grounds that unobservable phenomena shouldn’t play a crucial role in legitimate scientific theories. I argue that the way we evaluate multiverse models is precisely the same as the way we evaluate any other models, on the basis of abduction, Bayesian inference, and empirical success. There is no scientifically respectable way to do cosmology without taking into account different possibilities for what the universe might be like outside our horizon. Multiverse theories are utterly conventionally scientific, even if evaluating them can be difficult in practice.

This is well-trodden ground, of course. We’re talking about the cosmological multiverse, not its very different relative the Many-Worlds interpretation of quantum mechanics. It’s not the best name, as the idea is that there is only one “universe,” in the sense of a connected region of space, but of course in an expanding universe there will be a horizon past which it is impossible to see. If conditions in far-away unobservable regions are very different from conditions nearby, we call the collection of all such regions “the multiverse.”

There are legitimate scientific puzzles raised by the multiverse idea, but there are also fake problems. Among the fakes is the idea that “the multiverse isn’t science because it’s unobservable and therefore unfalsifiable.” I’ve written about this before, but shockingly not everyone immediately agreed with everything I have said.

Back in 2014 the Edge Annual Question was “What Scientific Theory Is Ready for Retirement?”, and I answered Falsifiability. The idea of falsifiability, pioneered by philosopher Karl Popper and adopted as a bumper-sticker slogan by some working scientists, is that a theory only counts as “science” if we can envision an experiment that could potentially return an answer that was utterly incompatible with the theory, thereby consigning it to the scientific dustbin. Popper’s idea was to rule out so-called theories that were so fuzzy and ill-defined that they were compatible with literally anything.

As I explained in my short write-up, it’s not so much that falsifiability is completely wrong-headed, it’s just not quite up to the difficult task of precisely demarcating the line between science and non-science. This is well-recognized by philosophers; in my paper I quote Alex Broadbent as saying

It is remarkable and interesting that Popper remains extremely popular among natural scientists, despite almost universal agreement among philosophers that – notwithstanding his ingenuity and philosophical prowess – his central claims are false.

If we care about accurately characterizing the practice and principles of science, we need to do a little better — which philosophers work hard to do, while some physicists can’t be bothered. (I’m not blaming Popper himself here, nor even trying to carefully figure out what precisely he had in mind — the point is that a certain cartoonish version of his views has been elevated to the status of a sacred principle, and that’s a mistake.)

After my short piece came out, George Ellis and Joe Silk wrote an editorial in Nature, arguing that theories like the multiverse served to undermine the integrity of physics, which needs to be defended from attack. They suggested that people like me think that “elegance [as opposed to data] should suffice,” that sufficiently elegant theories “need not be tested experimentally,” and that I wanted to “to weaken the testability requirement for fundamental physics.” All of which is, of course, thoroughly false.

Nobody argues that elegance should suffice — indeed, I explicitly emphasized the importance of empirical testing in my very short piece. And I’m not suggesting that we “weaken” anything at all — I’m suggesting that we physicists treat the philosophy of science with the intellectual care that it deserves. The point is not that falsifiability used to be the right criterion for demarcating science from non-science, and now we want to change it; the point is that it never was, and we should be more honest about how science is practiced.

Another target of Ellis and Silk’s ire was Richard Dawid, a string theorist turned philosopher, who wrote a provocative book called String Theory and the Scientific Method. While I don’t necessarily agree with Dawid about everything, he does make some very sensible points. Unfortunately he coins the term “non-empirical theory confirmation,” which was an extremely bad marketing strategy. It sounds like Dawid is saying that we can confirm theories (in the sense of demonstrating that they are true) without using any empirical data, but he’s not saying that at all. Philosophers use “confirmation” in a much weaker sense than that of ordinary language, to refer to any considerations that could increase our credence in a theory. Of course there are some non-empirical ways that our credence in a theory could change; we could suddenly realize that it explains more than we expected, for example. But we can’t simply declare a theory to be “correct” on such grounds, nor was Dawid suggesting that we could.

In 2015 Dawid organized a conference on “Why Trust a Theory?” to discuss some of these issues, which I was unfortunately not able to attend. Now he is putting together a volume of essays, both from people who were at the conference and some additional contributors; it’s for that volume that this current essay was written. You can find other interesting contributions on the arxiv, for example from Joe Polchinski, Eva Silverstein, and Carlo Rovelli.

Hopefully with this longer format, the message I am trying to convey will be less amenable to misconstrual. Nobody is trying to change the rules of science; we are just trying to state them accurately. The multiverse is scientific in an utterly boring, conventional way: it makes definite statements about how things are, it has explanatory power for phenomena we do observe empirically, and our credence in it can go up or down on the basis of both observations and improvements in our theoretical understanding. Most importantly, it might be true, even if it might be difficult to ever decide with high confidence whether it is or not. Understanding how science progresses is an interesting and difficult question, and should not be reduced to brandishing bumper-sticker mottos to attack theoretical approaches to which we are not personally sympathetic.

This entry was posted in arxiv, Philosophy, Science. Bookmark the permalink.

40 Responses to Beyond Falsifiability

  1. Paul Hayes says:


    I still don’t like it. What if I can find – or at least can’t exclude the possibility of finding – some D’ which “falsifies” H1 more strongly than D “falsifies” H0?

  2. Raza Husain says:

    There is a great difference between mathematical models models and physical reality. It doesn’t cost me an extra dime or energy to think of the number one or infinity but to go from one real apple to infinite real apples you need infinite real energy, infinite space and infinite time. So much so you will see nothing but apples, which you don’t, which is proof that anything that is real is finite.

  3. JimV says:

    The idea of a multiverse as one connected region of space, comprised of regions with different properties, is not how I previously, naively, thought the cosmological multiverse was defined. (I thought there was no connection between the regions.)

    If they connect, then one or more connections could exist within our light-cone, and we could make some observations which might support the hypothesis. The fact that, as far as I know, no such evidence has been found, for me weighs against the hypothesis. (Similar to, though not as strongly as, how it weighs against the God hypothesis.)

  4. The Black Swan says:

    “…in an expanding universe there will be a horizon past which it is impossible to see. If conditions in far-away unobservable regions are very different from conditions nearby, we call the collection of all such regions the multiverse.”

    Assume the the Big Bang and metric expansion are true and as a consequence create a horizon at the edge of our observable universe.

    If we did astronomy in a different galaxy, galaxy A, at our horizon then we must see a similar universe with a horizon in every direction relative to galaxy A. The argument for this is because galaxy A evolved in a universe with the same laws of physics as the Milky Way. If we did astronomy in a different galaxy on our horizon, galaxy B, which is located at the antipode with respect to galaxy A then we should still see the same horizon phenomenon due to the Big Bang and expansion of the space-time metric

    Galaxies A and B should have galaxies A_1 and B_1 located on their respective horizon antipodes that are beyond our horizon but where galaxy A_1 and B_1 astronomers observe a universe with the same laws. We can do this for A_n and A_n+1 galaxies to prove the universe is infinite and has the same laws everywhere.

    Inflationary cosmology is one proposed solution to explain the horizon / homogeneity problem. The regions of the universe are disconnected now because light has not had enough time to traverse the universe if the universe is only 13 billion years old. According to inflation the entire universe was connected in the Big Bang and then the universe underwent inflation to lock in the homogeneity of the early connected universe at just the right size and at just the right time. The size of the universe when inflation starts and stops as well as the rate the universe inflates can be arbitrarily set to any value. A prediction of a multiverse outside our universe that cannot be observed does nothing to demonstrate why inflation is better in the space of scientific ideas. We should expect falsifiable theories to disappear while theories lacking precision and by their very nature unfalsifiable to survive. This is not evidence the theory is true. If these are the only theories in town now then maybe it is due a failure of imagination.

  5. marten says:

    As I understand it beyond falsifiability does not mean beyond doubt.

  6. KC Lee says:

    Raza Husain,

    From your conclusion “anything that is real is finite”:

    1. Since counterfactual definiteness implies items are physically real whether measured or not, classical reality, embodying counterfactual definiteness, deals with physically real items. As such, it loses definition when encountering infinities (e.g. black hole singularities).

    2. If there is a reality that is not described by such physical classical reality, it will not suffer from the above limitation. Rephrased, a “nonphysical” reality remains valid even when cencountering infinities.

    3. Whereby, should one have to chose which reality is more fundamental, “nonphysical” reality is favored because:
    a) it is more generally applicable (unaffected by infinities) and,
    b) physical (classical) reality could be viewed as a subset of a “nonphysical” reality, but not vice versa.

    Is there such a thing as “nonphysical” reality?

    A thought experiment described in an October, 2017 comment in Mind-Blowing Quantum Mechanics (QM),, says QM reality could be such a “nonphysical” reality.

    If true, this suggests that QM is more fundamental than classical physics.


  7. Gaehazzi says:

    Closely behind every man’s back there follows a little green imp.
    Only the man being followed can see his little green imp, which is invisible to everybody else but him, including to mirrors.
    And he’s very shy, that little green imp: as soon as the man looks back, the little green imp hides away.
    And he’s very quick, that little green imp: there’s no way the man can look back fast enough to catch a glimpse of him.
    And he’s very smart, that little green imp: he can hide even where there’s nowhere to hide.

    And, if not for Popper, that little green imp would scientifically explain why it starts raining whenever I forget my umbrella.

  8. Gaehazzi says:

    In 1979 Zev Bechler, a professor of philosophy at Haifa University (currently emeritus at Tel Aviv University) published a little Hebrew book titled “Philosophy of Science.” Each chapter of the book was a weekly lesson in a course, part of the IDF Radio Broadcast University series. In essence, the book is a contribution by Prof. Bechler to the the anti-science side in the Science Wars of that period.

    The book ridicules scientists’ claim that scientific truth is superior to that of other intellectual endeavors, a claim based on an alleged iron-clad “scientific method” and a clear demarcation between science and non-science. Moreover, Prof. Bechler goes on to ridicule past generations of science philosophers—from Aristotle, through Bacon, Einstein, Duhem, Popper to Kuhn. In his view, the philosophy of science trade is to explain, justify and defend whatever scientists practically do, and to apologize and come up with new demarcations when newer science transgresses the previously set methodological limits—as if scientists care. In particular, he claims that Popper’s philosophy had been actually demolished by Duhem’s, since the latter precludes not only empirical confirmation of theories but also their falsification. And he sees in the keen embrace of Popper by scientists a false pretense of rationality not actually there.

    The point I want to make is this. For ardent physics theoreticians, breaking free from the oppression of any demarcation is an understandable passion—“to boldly go where no [theoretician] has gone before.” But it plays into the hands of the Antiscience, for that’s exactly what they were arguing all along. And bold ventures admirable in science fiction do not necessarily honor science fact.

  9. Shecky R says:

    Why has no one yet suggested a multi-multiverse… or, a multi-multi-multiverse, or…. 😉

  10. Ben Goren says:

    So…falsifiability is not so bad as a rough approximation, but it’s not how reality actually works.

    Aristotelian Metaphysics is a good analogy. It actually is true, in most domains of human experience, that stuff only moves so long as you keep pushing on it. Don’t believe me? Slide your coffee mug across the desktop, stop pushing, and see how long it keeps going. Of course, we now know that the table (via friction) is pushing back; Newton’s is unquestionably the fuller picture. But, insofar as the human environment is concerned, our personally-observable daily universe is much more Aristotelian than Newtonian.

    So, too, with falsification. In the typical human environment, you can trivially make up an infinite variety of conspiracy theories, none of which can be disproven, to explain anything you like. You’re hungry because the CIA is using alien mind ray technology to control your appetite via your dental implants; you’re actually a computer simulation that’s part of the Matrix; the Fates have woven your destiny into their fabric — whatever. And, in that context, the ideas that you can test and disprove tend to be an awful lot more useful than the ones that you can’t falsify.

    But…it’s also trivial to construct actual real-world scenarios which are immune to falsification. Toss a coin; catch it; don’t look at it; toss it again. It is true that the coin was either heads-up or tails-up in your hand, but neither statement is falsifiable.

    The astute will note that, in this context, falsifiability looks an awful lot like Sean’s favorite equation, that S = k * log W on Boltzmann’s tombstone pictured at the top of the page. In our coin example, the final macro states are indistinguishable: a coin in your hand. But the intermediate micro states are, literally, heads and tails apart.

    This suggests to me that the multiverse question should be approached in a similar manner. Ultimately, the Universe is what it is, regardless of what we think it is or should be. The multiverse either is or isn’t a reasonable description of reality. After all, we really could be characters on the Holodeck. But we can also attempt to enumerate the various micro states that would result in the macro state that we observe…and there’re many more ways to create the world we live in honestly than through conspiracy theories. As Sean has academically noted, Boltzmann Brains and similar conspiracies really aren’t a problem for physics. And, if you take honest non-conspiratorial physics seriously, as Sean likes to suggest we do, you wind up with many more ways to construct the observable Universe with lots more beyond our Big Bang horizon than you do with the Big Bang as all there is.

    The cosmological multiverse theory even passes the Aristotelian-inspired sniff test. In Aristotle’s world, humans were the center, the focal point, of all reality. Ever since, our horizons have been rapidly receding with us rendered even more insignificant. The Earth isn’t the center of the Solar system, the Sun isn’t the center of the Galaxy, the Milky Way isn’t the center of the Universe — and even matter itself isn’t the dominant element of physics. So why should we continue to be so hubristic as to insist that our own Big Bang is the only such?

    Especially since the only objection to multiverses always boils down to the same anthropocentric arguments as were originally lobbed against Galileo….



  11. Anders says:

    If falsifiability is the gold standard, shouldn’t we throw out the Standard Model? From what I’ve heard, it predicts that neutrinos should be massless yet they have mass…

  12. Neil says:

    The SM does NOT predict massless neutrinos. The zero mass was put in by hand because they were assumed massless. And though we now know neutrinos have mass, it has not been measured. (But we do know they are very light.) Thus, although massive neutrinos indicate the need for a BSM theory, they do not falsify the SM. The SM is not wrong, just incomplete, and works fine even though neutrinos have a small mass.

  13. John B says:

    Maybe everything we know about the multiverse is wrong. For all we know, just our galaxy is a universe, and every other galaxy is another universe that is further out in space than our Milky Way universe. Then the universe isn’t expanding, and it is actually the multiverse expanding. There would be as much evidence to disprove this idea as there are stars in the sky. Prove me wrong… No one would ever do that, since it is so far far outside of our conventional way of thinking about the multiverse which isn’t based on science.

  14. Ben Andrews says:

    Although we need both philosophy and science to make progress in our understanding of the universe we must keep in mind that philosophy in itself is not science

  15. Mark says:

    In a previous blog Sean Caroll mentioned that many models that arise from general relativity (and presumably cosmic inflation) lead somewhat naturally to predictions of a multiverse. In that case, one could say that mutliverse does not need to be falsifiable in itself to be considered a well motivated hypothesis and as such a part of science, because it is a prediction of theories that are themselves falsifiable. More generally the individual theories of science, based on observation (and so falsifiable), form a partially completed jigsaw and it seems perfectly rational to propose theories for those areas that are not complete by using predictions from the surrounding complete areas. In the same way you can use the known pieces around an incomplete area to predict the pieces you need for completion in a real jigsaw.