On Determinism

Back in 1814, Pierre-Simon Laplace was mulling over the implications of Newtonian mechanics, and realized something profound. If there were a vast intelligence — since dubbed Laplace’s Demon — that knew the exact state of the universe at any one moment, and knew all the laws of physics, and had arbitrarily large computational capacity, it could both predict the future and reconstruct the past with perfect accuracy. While this is a straightforward consequence of Newton’s theory, it seems to conflict with our intuitive notion of free will. Even if there is no such demon, presumably there is some particular state of the universe, which implies that the future is fixed by the present. What room, then, for free choice? What’s surprising is that we still don’t have a consensus answer to this question. Subsequent developments, most relevantly in the probabilistic nature of predictions in quantum mechanics, have muddied the waters more than clarifying them.

Massimo Pigliucci has written a primer for skeptics of determinism, in part spurred by reading (and taking issue with) Alex Rosenberg’s new book The Atheist’s Guide to Reality, which I mentioned here. And Jerry Coyne responds, mostly to say that none of this amounts to “free will” over and above the laws of physics. (Which is true, even if, as I’ll mention below, quantum indeterminacy can propagate upward to classical behavior.) I wanted to give my own two cents, partly as a physicist and partly as a guy who just can’t resist giving his two cents.

Echoing Massimo’s structure, here are some talking points:

* There are probably many notions of what determinism means, but let’s distinguish two. The crucial thing is that the universe can be divided up into different moments of time. (The division will generally be highly non-unique, but that’s okay.) Then we can call “global determinism” the claim that, if we know the exact state of the whole universe at one time, the future and past are completely determined. But we can also define “local determinism” to be the claim that, if we know the exact state of some part of the universe at one time, the future and past of a certain region of the universe (the “domain of dependence”) is completely determined. Both are reasonable and relevant.

* It makes sense to be interested, as Massimo seems to be, in whether or not the one true correct ultimate set of laws of physics are deterministic or not. He argues that we don’t know, and that’s obviously right, since we don’t know what the final theory is. But that’s a rather defeatist attitude all by itself; we can look at the theories we do understand and try to draw lessons from them.

* Classical mechanics, which you might have thought was deterministic if anything was, actually has some loopholes. We can think of certain situations where more than one future obeys the equations of motion starting from the same past. This is discussed a bit in the Stanford Encyclopedia of Philosophy article on causal determinism. But I personally don’t find the examples that impressive. For one thing, they are highly non-generic; you have to work really hard to find these kinds of solutions, and they certainly aren’t stable under small perturbations. More importantly, classical mechanics isn’t right; it’s just an approximation to quantum mechanics, and these finely-tuned classical solutions would be dramatically altered by quantum effects.

* General relativity is a classical theory, so it’s also not correct, but we don’t have the final theory of quantum gravity so it’s worth a look. As Massimo points out, there are good examples in GR where traditional global determinism breaks down; naked singularities would be an example. (Basically, determinism breaks down when information can in principle “flow in” from a singularity or boundary that isn’t included in “the whole universe at one moment of time.”) We might sidestep this problem by arguing that naked singularities aren’t physical, which is quite reasonable. But there are much more benign examples, such as anti-de Sitter space — a maximally symmetric spacetime with a negative cosmological constant. This universe has no singularities, but does have a boundary at infinity, so a single moment of time only determines part of the universe, not the whole thing. On the other hand, like the classical-mechanics examples alluded to above, this seems like a technicality that can be cleared up with a slight change of definition, e.g. by imposing some simple boundary condition at infinity.

Much more importantly, these kinds of GR phenomena are very far away from our everyday lives; there’s really no relevance to discussions of free will. GR violates global determinism in the strict sense, but certainly obeys local determinism; that’s all that should be required for this kind of discussion.

* Quantum mechanics is where things get interesting. When a quantum state is happily evolving along according to the Schrödinger equation, everything is perfectly deterministic; indeed, more so than classical mechanics, because the space of states (Hilbert space) doesn’t allow for the kind of non-generic funny business that let non-deterministic classical solutions sneak in. But when we make an observation, we are unable to deterministically predict what its outcome will be. (And Bell’s theorem at least suggests that this inability is not just because we’re not smart enough; we never will be able to make such predictions.) At this point, opinions become split about whether the loss of determinism is real, or merely apparent. This is a crucial question for both physicists and philosophers, but not directly relevant for the question of free will.

The traditional (“Copenhagen”) view is that QM is truly non-deterministic, and that probability plays a central role in the measurement process when wave functions collapse. Unfortunately, this process is extremely unsatisfying, not just because it runs contrary to our philosophical prejudices but because what counts as a “measurement” and the quantum/classical split are extremely ill-defined. Almost everyone agrees we should do better, despite the fact that we still teach this approach in textbooks. Someone like Tom Banks would try to eliminate the magical process of wave function collapse, but keep probability (and thus a loss of determinism) as a central feature. There is a whole school of thought along these lines, which treats the quantum state as a device for tracking probabilities; see this excellent post by Matt Leifer for more details.

The other way to go is many-worlds, which says that the ordinary deterministic evolution of the Schrödinger equation is all that ever happens. The problem there is comporting such a claim with the reality of our experience — we see Schrödinger’s cat to be alive or dead, not ever in a live/dead superposition as QM would seem to imply. The resolution is that “we” are not described by the entire quantum state; rather, we live in one branch of the wave function, which also includes numerous other branches where different outcomes were observed. This approach (which I favor) restores determinism at the level of the fundamental equations, but sacrifices it for the observational predictions made by real observers. If I were keeping a tally, I would certainly put this one in the non-determinism camp, for anyone interested in questions of free will.

* Then there is the question of whether or not the lack of determinism in QM plays any role at all in our everyday lives. When we flip a coin or play the lottery, one might think that the relevant probabilities are “purely classical” — i.e. they stem from our lack of knowledge about the state of the muscles and nerves in my hand and the wind and the coin that is about to be flipped, but if I knew all of those things I could make a perfectly deterministic prediction about what would happen to the coin. (Indeed, a well-trained magician can flip a coin and get whatever result they want.)

This is actually a tricky problem, to which the answers aren’t clear. Yes, there may be a level of classical description in terms of a probability distribution; but where does that probability distribution come from? Physicists disagree about whether or not quantum mechanics plays a crucial role here. Since I have friends in high places, this weekend I emailed Andy Albrecht, who answered and brought David Deutsch into the conversation. They both argue — plausibly, although I’m not really qualified to pass judgment — that essentially all classical probabilities can ultimately traced down to the quantum wave function. And indeed, that this reasoning provides the only sensible basis for talking about probabilities at all! (David mentions that Lev Vaidman seems to disagree, so it’s not uncontroversial by any means.) They are both, in other words, firmly anti-Bayesian in their view on probability. A good Bayesian thinks that probabilities are always statements about our fundamental ignorance concerning what is “really” going on. Albrecht and Deutsch would argue that’s not true, probabilities are ultimately always statements about the wave function of the universe. If they’re right — and again, it looks plausible, but I need to think about it more — then QM effects are indeed of crucial importance in accounting for our inability to predict the future in the everyday world.

* I should say something about chaos, which always comes up in these discussions. In classical mechanics, even when the underlying model is perfectly deterministic, it can often be the case that a small uncertainty in our knowledge of the initial state can lead to large uncertainty in the future/past evolution. (E.g. for the tumbling of Hyperion.) This is sometimes brought up as if it causes problems for determinism: “since tiny mistakes propagate, you couldn’t realistically predict the future anyway.” This is about as irrelevant as it is possible to be irrelevant. The Laplacian viewpoint was always that if you had perfect information, you could predict the past and future. But that was always a statement of principle, not of practice. Of course, in practice, you have nowhere near enough information to make the kinds of calculation that Laplace’s vast intellect likes to do. That was perfectly obvious long before the advent of chaos theory. The correct statement is “in a classical deterministic system, with perfect information and arbitrary computing power you can predict the future in principle, but not in practice,” and that statement is completely unaltered by an understanding of chaos.

So where does that leave us? My personal suspicion is that the ultimate laws of physics will embody something like the many-worlds philosophy: the underlying laws are perfectly deterministic, but what happens along any specific history is irreducibly probabilistic. (In a better understanding of quantum gravity, our notion of “time” might be altered, and therefore our notion of “determinism” might be affected; but I suspect that there will still be some underlying equations that are rigidly obeyed.) But that’s just a suspicion, not anything worth taking to the bank. For everyday-life purposes, we can’t get around the fact that quantum mechanics makes it impossible to predict the future robustly.

Of course, this is all utterly irrelevant for questions of free will. (I’m sure Massimo knows this, but he didn’t discuss it in his blog post.) We can imagine four different possibilities: determinism + free will, indeterminism + free will, determinism + no free will, and indeterminism + no free will. All of these are logically possible, and in fact beliefs that some people actually hold! Bringing determinism into discussions of free will is a red herring.

It matters, of course, how one defines “free will.” The usual strategy in these discussions is to pick your own definition, and then argue on that basis, no matter what definition is being used by the person you’re arguing with. It’s not a strategy that advances human knowledge, but it makes for an endless string of debates.

A better question is, if we choose to think of human beings as collections of atoms and particles evolving according to the laws of physics, is such a description accurate and complete? Or is there something about human consciousness — some strong sense of “free will” — that allows us to deviate from the predictions that such a purely mechanistic model would make?

If that’s your definition of free will, then it doesn’t matter whether the laws of physics are deterministic or not — all that matters is that there are laws. If the atoms and particles that make up human beings obey those laws, there is no free will in this strong sense; if there is such a notion of free will, the laws are violated. In particular, if you want to use the lack of determinism in quantum mechanics to make room for supra-physical human volition (or, for that matter, occasional interventions by God in the course of biological evolution, as Francis Collins believes), then let’s be clear: you are not making use of the rules of quantum mechanics, you are simply violating them. Quantum mechanics doesn’t say “we don’t know what’s going to happen, but maybe our ineffable spirit energies are secretly making the choices”; it says “the probability of an outcome is the modulus squared of the quantum amplitude,” full stop. Just because there are probabilities doesn’t mean there is room for free will in that sense.

On the other hand, if you use a weak sense of free will, along the lines of “a useful theory of macroscopic human behavior models people as rational agents capable of making choices,” then free will is completely compatible with the underlying laws of physics, whether they are deterministic or not. That is the (fairly standard) compatibilist position, as defended by me in Free Will is as Real as Baseball. I would argue that this is the most useful notion of free will, the one people have in mind as they contemplate whether to go right to law school or spend a year hiking through Europe. It is not so weak as to be tautological: we could imagine a universe in which there were simple robust future boundary conditions, such that a model of rational agents would not be sufficient to describe the world. E.g. a world in which there were accurate prophesies of the future: “You will grow up to marry a handsome prince.” (Like it or not.) For better or for worse, that’s not the world we live in. What happens to you in the future is a combination of choices you make and forces well beyond your control — make the best of it!

  1. Collins’ position, or a logically equivalent statement about free will, seems superficially plausible as long as you’re only looking at values of a single operator, like the position of a particle. The distribution has to have a certain shape over the long term–that leaves a lot of wiggle room for different patterns of results, right? You could do a lot with that! God could do a lot with that!

    Not so. QM makes predictions about the distribution of values of any observable operator, not just x. God, or free-will power, would have to work in ways so mysterious that they cannot be detected in the distribution of values of any observable quantity. That’s actually a pretty powerful constraint.

  2. Yes, the real issue isn’t whether freedom is compatible with determinism, it’s whether freedom is compatible with the completeness of physics (or, if one rejects physicalism, with the completeness of natural laws).

    I would certainly put [many worlds] in the non-determinism camp, for anyone interested in questions of free will.

    Really? It strikes me as even more deterministic than classical mechanics in a way — it seems like on this account it would be nearly impossible to avoid any action (since in some world that action will occur).

    At least given classical determinism I can choose to perform some actions and avoid others. With many worlds, it will (often? always?) be the case that the action both will be performed and won’t be performed.

    I suppose choices will make a difference to the extent that quantum uncertainties wash out and are irrelevant to large-scale bodily behavior, but given the multiplicity of real worlds, it’s not really clear how much of a difference this makes. It’s pretty hard to make sense of rationality and morality in a many-worlds scenario.

  3. Oh please.

    You get one three-body collision in there — in the past or the future — and all your predictive power is gone.

    And surely if Newton knew this, LaPlace knew it too.

  4. Greg Egan wrote some science-fiction stories set in a world in which post-human beings made sure their brains were specially engineered such that their choices would never be indeterminate on the quantum level, because they felt that only then would their decisions really matter!

  5. Ethan Siegel says:

    You get one three-body collision in there . . . and all your predictive power is gone.

    That’s why Sean listed “arbitrary computing power” in there. We’re interested in the metaphysics, not in our practical abilities to predict. (Which is also why Sean rightly dismisses chaos as irrelevant.)

    The relevant point is that the behavior does follow the physical laws.

    And note that even though we can’t solve the three-body problem to predict future behavior, once we have the behavior in hand, we can confirm that it didn’t violate the laws. If I give you the solution, it’s generally pretty straightforward to confirm that it is indeed a solution to the equations of motion. That’s all we need to establish the completeness of physics.

  6. No physicalist @6, not the “three body gravitational problem,” but a simultaneous collision between three particles.

    You know, where particles “1” “2” and “3” collide, each with some initial momentum, simultaneously.

    That, classically, is a totally unpredictable system. As you can imagine, if “1 hits 2 which then hits 3” you get a different answer than if “3 hits 2 which then hits 1.” But if “1, 2, and 3 collide simultaneously,” your system is under-constrained, and there are multiple possible solutions to what the final momenta of the three particles will be. Classically.

    It’s non-deterministic, and — like I said — LaPlace surely knew this.

  7. The free will discussion has never really seemed that interesting for reasons that Sean outlines above. Everyone has a different definition, and everyone just seems to talk over each other. We would really need a robust theory of consciousness before we’d be able to tackle free will in a more meaningful way, and we’re nowhere near that.

    For what it’s worth, the compatibilist view seems like the sensible way to think about it. If you want to do something and you choose to do it and you do do it, then you’ve acted freely. It doesn’t matter whether your base desires are “freely” chosen or not (whatever that might mean).

  8. Ethan, as I said in the post, there are non-deterministic problems in classical mechanics (albeit they appear to be a set of measure zero). Whether Laplace knew this is an empirical question, about which I don’t have any data. But he certainly did write the famous passage about a vast intelligence, as quoted in Wikipedia.

  9. “a simultaneous collision between three particles.”

    Ah. Then I take it that this will be a case like the one that Norton discusses, and that Sean refers to in the S.E.P. entry.

    Then the relevant question (as Sean points out) is whether such a scenario is realistic enough to worry about. Two issues:

    (1) Does the real world ever admit of actually instantaneous interactions of this sort? As you say, you get different answers depending on which collision happens first — but if as a matter of fact one does occur first, then the outcome is determined (though we might be in a position to predict that outcome). So, are there real collisions in which three bodies really collide at precisely the same instant?

    Of course, when you try to get down to such fine-grained detail, you notice that classical mechanics’ description of colliding rigid bodies isn’t exactly right — so the answer is that there are not collisions of this sort in the real world. Which leads us to the second issue:

    (2) Given that the real world is quantum mechanical, what should we say about determinism? Sean has given his answer above. The main thing that I would add is that we shouldn’t rule out non-local hidden variable interpretations like Bohm’s. It seems to me that such a theory is robustly deterministic. (And I’m inclined to say that many worlds and spontaneous collapse are just as non-local as Bohm’s theory, but that’s an argument for another day.)

  10. Sean,

    I don’t think you give the simple example of a three-body collision its due; if you were willing to consider it, you’d arrive at the same non-determinism that arises in the final momenta in neutron decay. While the examples you link to in the Stanford Encyclopedia may be of measure zero, this one isn’t; it’s far more general than the very specific example given there. You can take a simultaneous collision between any three classical objects with a combination of any initial momenta, and your final state is not determined.

    LaPlace didn’t know about the densities and energies of the early stages of the Big Bang (where the three-body collision, at those high densities, are simultaneous if one considers timescales smaller than the Planck time to be simultaneous), nor of simple matter-antimatter processes (electron-positron annihilation, treating it as classically as possible), but if one is willing to accept these basic things that happen, you cannot keep determinism even if you don’t step into the quantum realm.

    Not saying LaPlace didn’t “forget” (or ignore) a 3-body collision when he wrote his passage, just that — given some very basic things that we know — that argument holds no water, even classically (that is, without any reference to or consideration of a quantum mechanical wavefunction).

  11. Pingback: On Determinism | Quantum Pie with Krister Shalm

  12. That, classically, is a totally unpredictable system. As you can imagine, if “1 hits 2 which then hits 3″ you get a different answer than if “3 hits 2 which then hits 1.” But if “1, 2, and 3 collide simultaneously,” your system is under-constrained, and there are multiple possible solutions to what the final momenta of the three particles will be. Classically.

    Hmm, a really interesting point that gets to the core of the failures of classical mechanics that led to both relativity and QM. Einstein was pretty explicit that a lot of his reasoning about special relativity came from questions about what “simultaneity” meant. And, although I don’t have the acumen to fairly judge, it seems to me like demanding that A, B, and C collide simultaneously violates the uncertainty principle (you are demanding absolutely zero error in your time measurements of the collisions of A, B, and C pairwise).

    Would you mind providing some kind of citation or link where I can read more, though? Wikipedia doesn’t seem to treat this particular version of the “three body problem” on the relevant page.

    I’d be really curious to hear more about the conversation with Albrecht and Deutch, Sean. I’ve been trending Bayesian (I guess the pun is intended…sorry folks) but I’d be love to hear some of the arguments against.

  13. Ethan, three particles in three dimensions describe an eighteen-dimensional phase space. The constraint that all three particles are at the same location at some time is a six-dimensional constraint (choose one position, the other two positions are determined). We’re left with a twelve-dimensional submanifold of an eighteen-dimensional space. That is bigger than a point, but is certainly a set of measure zero in the larger space. Said another way, more directly relevant to this discussion: a generic perturbation in phase space moves you off the constraint surface.

    Everyone agrees that classical mechanics allows for non-deterministic evolution, but it’s not the generic case, so I’m not sure what the argument is here.

  14. Whatever biologists make out of consciousness and free will my feeling is that it will have very little to do with physics and specially QM. Just like nature abhors a vacuum biology abhors indeterminacy. You can see this on biological process that could involve quantum phenomenona but don’t, like the electron transport chain and photoreceptors. Both of these processes have quite deterministic outputs.

    Consciousness will be a matter of competition and selection, with an aggregator that measures the output of different neuronal constellations that represent each competing thought. This aggregator will turn out with something as mundane as the level of neurotransmitter or number of active synapses. It will not involve physics or QM in an interesting way.

  15. Pingback: Determinism and free-will « Only Yes

  16. @Somite Says #16,

    On the contrary, indeterminacy is sometimes useful in biological systems. Molecular signaling pathways can be “engineered” to be more or less deterministic versus stochastic, and there are probably good reasons (i.e., fitness benefits) for both kinds of circuit function. Just Google Scholar “lambda phage stochastic” for examples. My own bias is that this sort of stochastic understanding will play an important role in any good model of behavior/decision making/consciousness/free will/whatever-you-want-to-call-it (though this is still an open question). I agree that there is no need for biologists to call on QM directly, as yet, but the mathematics that underlie it may be useful (perhaps have already been used? I’m no expert here) for developing stochastic kinetic models.

  17. I agree. Stochastic yes. QM no. When I wrote indeterminacy I meant that of QM and now I realize I meant something like uncertainty.

  18. Hi, I’ve been following your blog for a while and I love it. I am not a physicist, just an avid follower of physics. This is my first time submitting a comment. Sorry it’s a bit off-topic, but maybe it’s something you could spend a future post on? I imagine many of your readers would have the same question.

    There’s something about many-worlds that I’ve always wondered: does it explain probabilities, and if so, how?

    If I imagine a 50:50 choice, like Schrodinger’s Cat, then it makes sense that (after opening the box) the wavefunction contains 2 pieces, and , and that I have a 50:50 chance of being part of either piece. One can certainly build up many rational-number probabilities with lots of 50:50 choices. However, I can change the setup so the probability is an irrational ratio, let’s say P():P() = 1:sqrt(2). How does many-worlds handle that case? The wavefunction still has two pieces, and . They have different amplitudes, but the “me” that is part of each feels just as real. Yet there must be some way in which “I” am more likely to be the “me” that is part of than . Does that arise out of the many-worlds program, or is it a further postulate, as in “My probability of existing in any branch of the wavefunction depends on the amplitudes at each branch.”

  19. Tom– This is an excellent question, the subject of much current research. It is, at least, not perfectly obvious how to get probabilities out of the many-worlds interpretation. See e.g.


    There are promising approaches (e.g. from Deutsch, Wallace, Zurek, and others), but I don’t think there’s a consensus. Of course many other approaches “get” probability, essentially by putting it in by hand; MWI doesn’t have that freedom, since it’s just supposed to be the Schrodinger equation and nothing else.

  20. I used to argue about free will, but now I don’t. Whether the universe is deterministic or not is, as pointed out, pretty much irrelevant, because the possession of free will is a subjective state. You either feel free, or you don’t.
    It’s interesting to speculate on this though: If the universe is deterministic, then the conclusion from human behavior is that doing nothing is, evolutionarily speaking, advantageous. If choice is a non-concept, it follows that the time my ancestors spent making decisions, i.e., not doing anything when confronted with a stimulus, increased their reproductive success. This thought pleases me more than freedom.

  21. Bizarrely I was just reading David Wallace, a philosopher with Oxford Flu (symptoms include feeling you keep splitting into multiple copies). His entry in this years Oxford Handbook on Quantum Mechanics is a good look overview of some of these issues.


    He covers Many Worlds (Everett) generally, and the preferred basis problem, which he has a strong view on (spoiler alert: it’s emergent). For probability – which has been, and remains a hotly debated issue – he sketches out the main arguments on various sides.

    Probability is far from ‘solved’, but Deutsch, Wallace et al. have shown pretty convincingly that you can derive probability amplitudes (mod squared) from MWI + decision theory or symmetry arguments.

    To answer Tom’s question, branches are a continuum in Quantum Theory, so irrational weightings are just as easy as 50:50. The fact that probability amplitudes (the Born Rule) can be derived from Everett mean that no further postulates are required (if the derivation is valid of course!)

    As Sean pointed out (and Wallace agrees) there are a lot of questions with probability at determinacy, in Many Worlds or not. Arguably Deutsch’s derivation has put probability on a much firmer footing under Everett, than we had before. Giving a principled argument for uncertainty in a deterministic physics.

  22. Regarding lawfulness and free will, does anyone have any thoughts on the following quote from Chapter One of Stephen Hawking’s “A Brief History of Time”:

    “Now, if you believe that the universe is not arbitrary, but governed by definite laws, you ultimately have to combine the partial theories into a complete unified theory that will describe everything in the universe. But there is a fundamental paradox in the search for such a complete unified theory. The ideas about scientific theories outlined above assume we are rational beings who are free to observe the universe as we want and to draw logical deductions from what we see. In such a scheme it is reasonable to suppose that we might progress ever closer toward the laws that govern the universe. Yet if there really is a complete unified theory, it would also presumably determine our actions. And so the theory itself would determine the outcome of our search for it! And why should it determine that we come to the right conclusions from the evidence? Might it not equally well determine that we draw the wrong conclusion? Or no conclusion at all?

    So that is Hawking’s fundamental paradox…how is science possible if we aren’t “free rational beings”?

    Conway and Kochen raise a similar point in their paper, The Free Will Theorem:

    “It is hard to take science seriously in a universe that in fact controls all the choices experimenters think they make. Nature could be in an insidious conspiracy to ‘confirm’ laws by denying us the freedom to make the tests that would refute them. Physical induction, the primary tool of science, disappears if we are denied access to random samples.”


  23. Thanks to Matt for the clear explanation of why free will can’t be “hiding” in quantum probability. The necessity for such free will to leave a detectable signature in the distribution of results is indeed a strong constraint.

    I wonder however if we can still hide free will there if we assume that it exists only in the waveform and disappears when the wave is collapsed upon measurement. Uncollapsed waves still have a real effect on the world. If free will also disappeared upon measurement then it could not be detected in the distribution pattern.