The Trouble With Physics

I was asked to review Lee Smolin’s The Trouble With Physics by New Scientist. The review has now appeared, although with a couple of drawbacks. Most obviously, only subscribers can read it. But more importantly, they have some antiquated print-journal notion of a “word limit,” which in my case was about 1000 words. When I started writing the review, I kind of went over the limit. By a factor of about three. This is why the Intelligent Designer invented blogs; here’s the review I would have written, if the Man hadn’t tried to stifle my creativity. (Other reviews at Backreaction and Not Even Wrong; see also Bee’s interview with Lee, or his appearance with Brian Greene on Science Friday.)

——————————————————————

It was only after re-reading and considerable head-scratching that I figured out why Lee Smolin’s The Trouble With Physics is such a frustrating book: it’s really two books, with intertwined but ultimately independent arguments. One argument is big and abstract and likely to be ignored by most of the book’s audience; the other is narrow and specific and part of a wide-ranging and heated discussion carried out between scientists, in the popular press, and on the internet. The abstract argument — about academic culture and the need to nurture speculative ideas — is, in my opinion, important and largely correct, while the specific one — about the best way to set about quantizing gravity — is overstated and undersupported. It’s too bad that vociferous debate over the latter seems likely to suck all the oxygen away from the former.

Fundamental physics (for want of a better term) is concerned with the ultimate microscopic laws of nature. In our current understanding, these laws describe gravity according to Einstein’s general theory of relativity, and everything else according to the Standard Model of particle physics. The good news is that, with just a few exceptions (dark matter and dark energy, neutrino masses), these two theories are consistent with all the experimental data we have. The bad news is that they are mutually inconsistent. The Standard Model is a quantum field theory, a direct outgrowth of the quantum-mechanical revolution of the 1920’s. General relativity (GR), meanwhile, remains a classical theory, very much in the tradition of Newtonian mechanics. The program of “quantum gravity” is to invent a quantum-mechanical theory that reduces to GR in the classical limit.

This is obviously a crucially important problem, but one that has traditionally been a sidelight in the world of theoretical physics. For one thing, coming up with good models of quantum gravity has turned out to be extremely difficult; for another, the weakness of gravity implies that quantum effects don’t become important in any realistic experiment. There is a severe conceptual divide between GR and the Standard Model, but as a practical matter there is no pressing empirical question that one or the other of them cannot answer.

Quantum gravity moved to the forefront of research in the 1980’s, for two very different reasons. One was the success of the Standard Model itself; its triumph was so complete that there weren’t any nagging experimental puzzles left to resolve (a frustrating situation that persisted for twenty years). The other was the appearance of a promising new approach: string theory, the simple idea of replacing elementary point particles by one-dimensional loops and segments of “string.” (You’re not supposed to ask what the strings are made of; they’re made of string stuff, and there are no deeper layers.) In fact the theory had been around since the late 1960’s, originally investigated as an approach to the strong interactions. But problems arose, including the unavoidable appearance of string states that had all the characteristics one would expect of gravitons, particles of gravity. Whereas most attempts to quantize gravity ran quickly aground, here was a theory that insisted on the existence of gravity even when we didn’t ask for it! In 1984, Michael Green and John Schwarz demonstrated that certain potentially worrisome anomalies in the theory could be successfully canceled, and string mania swept the particle-theory community.

In the heady days of the “first superstring revolution,” triumphalism was everywhere. String theory wasn’t just a way to quantize gravity, it was a Theory of Everything, from which we could potentially derive all of particle physics. Sadly, that hasn’t worked out, or at least not yet. (String theorists remain quite confident that the theory is compatible with everything we know about particle physics, but optimism that it will uniquely predict the low-energy world is at a low ebb.) But on the theoretical front, there have been impressive advances, including a “second revolution” in the mid-nineties. Among the most astonishing results was the discovery by Juan Maldacena of gauge/gravity duality, according to which quantum gravity in a particular background is precisely equivalent to a completely distinct field theory, without gravity, in a different number of dimensions! String theory and quantum field theory, it turns out, aren’t really separate disciplines; there is a web of dualities that reveal various different-looking string theories as simply different manifestations of the same underlying theory, and some of those manifestations are ordinary field theories. Results such as this convince string theorists that they are on the right track, even in the absence of experimental tests. (Although all but the most fervent will readily agree that experimental tests are always the ultimate arbiter.)

But it’s been a long time since the last revolution, and contact with data seems no closer. Indeed, the hope that string theory would uniquely predict a model of particle physics appears increasingly utopian; these days, it seems more likely that there is a huge number (10500 or more) phases in which string theory can find itself, each featuring different particles and forces. This embarrassment of riches has opened a possible explanation for apparent fine-tunings in nature — perhaps every phase of string theory exists somewhere, and we only find ourselves in those that are hospitable to life. But this particular prediction is not experimentally testable; if there is to be contact with data, it seems that it won’t be through predicting the details of particle physics.

It is perhaps not surprising that there has been a backlash against string theory. Lee Smolin’s The Trouble With Physics is a paradigmatic example, along with Peter Woit’s new book Not Even Wrong. Both books were foreshadowed by Roger Penrose’s massive work, The Road to Reality. But string theorists have not been silent; several years ago, Brian Greene’s The Elegant Universe was a surprise bestseller, and more recently Leonard Susskind’s The Cosmic Landscape has focused on the opportunities presented by a theory with 10500 different phases. Alex Vilenkin’s Many Worlds in One also discusses the multiverse, and Lisa Randall’s Warped Passages enthuses over the possibility of extra dimensions of spacetime — while Lawrence Krauss’s Hiding in the Mirror strikes a skeptical note. Perhaps surprisingly, these books have not been published by vanity presses — there is apparently a huge market for popular discussions of the problems and prospects of string theory and related subjects.

Smolin is an excellent writer and a wide-ranging thinker, and his book is extremely readable. He adopts a more-in-sorrow-than-in-anger attitude toward string theory, claiming to appreciate its virtues while being very aware of its shortcomings. The Trouble with Physics offers a lucid introduction to general relativity, quantum mechanics, and string theory itself, before becoming more judgmental about the current state of the theory and its future prospects.

There is plenty to worry or complain about when it comes to string theory, but Smolin’s concerns are not always particularly compelling. For example, there are crucially important results in string theory (such as the fundamental fact that quantum-gravitational scattering is finite, or the gauge/gravity duality mentioned above) for which rigorous proofs have not been found. But there are proofs, and there are proofs. In fact, there are almost no results in realistic quantum field theories that have been rigorously proven; physicists often take the attitude that reasonably strong arguments are enough to allow us to accept a claim, even in the absence of the kind of proof that would make a mathematician happy. Both the finiteness of stringy scattering and the equivalence of gauge theory and gravity under Maldacena’s duality are supported by extremely compelling evidence, to the point where it has become extremely hard to see how they could fail to be true.

Smolin’s favorite alternative to string theory is Loop Quantum Gravity (LQG), which has grown out of attempts to quantize general relativity directly (without exotica such as supersymmetry or extra dimensions). To most field theorists, this seems like a quixotic quest; general relativity is not well-behaved at short distances and high energies, where such new degrees of freedom are likely to play a crucial role. But Smolin makes much of one purported advantage of LQG, that the theory is background-independent. In other words, rather than picking some background spacetime and studying the propagation of strings (or whatever), LQG is formulated without reference to any specific background.

It’s unclear whether this is really such a big deal. Most approaches to string theory are indeed background-dependent (although in some cases one can quibble about definitions), but that’s presumably because we don’t understand the theory very well. This is an argument about style; in particular, how we should set about inventing new theories. Smolin wants to think big, and start with a background-independent formulation from the start. String theorists would argue that it’s okay to start with a background, since we are led to exciting new results like finite scattering and gauge/gravity duality, and a background-independent formulation will perhaps be invented some day. It’s not an argument that anyone can hope to definitively win, until the right theory is settled and we can look back on how it was invented.

There are other aspects of Smolin’s book that, as a working physicist, rub me the wrong way. He puts a great deal of emphasis on connection to experimental results, which is entirely appropriate. However, he tends to give the impression that LQG and other non-stringy approaches are in close contact with experiment in a way that string theory is not, and I don’t think there’s any reasonable reading on which that is true. There may very well be certain experimental findings — which haven’t yet happened — that would be easier to explain in LQG than in string theory. But the converse is certainly equally true; the discovery of extra dimensions is the most obvious example. As far as I can tell, both string theory and LQG (and every other approach to quantizing gravity) are in the position of not making a single verifiable prediction that, if contradicted by experiment, would falsify the theory. (I’d be happy to hear otherwise.)

Smolin does mention a number of experimental results that have already been obtained, but none of them is naturally explained by LQG any more than by string theory, and most of them are, to be blunt, likely to go away. He mentions the claimed observation that the fine-structure constant is varying with time (against which more precise data has already been obtained), certain large-angle anomalies in the cosmic microwave background anisotropy, and the possibility of large-scale modifications of general relativity replacing dark matter. (Bad timing on that one.) I don’t know of any approach to quantum gravity that firmly predicts (or even better, predicted ahead of time) that any of these should be true. That’s the least surprising thing in the world; gravity is a weak force, and most of the universe is in the regime where it is completely classical.

Smolin also complains about the tendency of string theorists to hype their field. It is hard to argue with that; as a cosmologist, of course, it is hard to feel morally superior, either. But Smolin does tend to project such a feeling of superiority, often contrasting the careful and nuanced claims of LQG to the bombast of string theory. Yet he feels comfortable making statements such as (p. 232)

Loop quantum gravity already has elementary particles in it, and recent results suggest that this is exactly the right particle physics: the standard model.

There are only two ways to interpret this kind of statement: either (1) we have good evidence that quantum spacetime alone, without additional fields, supports excitations that have the right kinds of interactions and quantum numbers to be the particles of the Standard Model, which would be the most important discovery in physics since the invention of quantum mechanics, or (2) it’s hype. Time will tell, I suppose. The point being, it’s perfectly natural to get excited or even overenthusiastic when one is working on ideas of fantastic scope and ambition; at the end of the day, those ideas should be judged on whether they are right or wrong, not whether their proponents were insufficiently cautious and humble.

To date, the string theorists are unambiguously winning the battle for support within the physics community. Success is measured primarily by faculty positions and grant money, and these flow to string theorists much more than to anyone pursuing other approaches to quantum gravity. From an historical perspective, the unusual feature of this situation is that there are any resources being spent on research in quantum gravity; if string theory were suddenly to fall out of favor, it seems much more likely that jobs and money would flow to particle phenomenology, astrophysics, or other areas of theory than to alternative approaches to quantum gravity.

It seems worth emphasizing that the dominance of string theory is absolutely not self-perpetuating. When string theorists apply for grants, they are ultimately judged by program officers at the National Science Foundation or the Department of Energy, the large majority of whom are not string theorists. (I don’t know of any who are, off the top of my head.) And when string theorists apply for faculty jobs, it might very well be other string theorists who decide which are the best candidates, but the job itself must be approved by the rest of the department and by the university administration. String theorists have somehow managed to convince all of these people that their field is worthy of support; I personally take the uncynical view that they have done so through obtaining interesting results.

Smolin talks a great deal about the need for physics, and academia more generally, to support plucky upstart ideas and scholars with the courage and vision to think big and go against the grain. This is a larger point than the specific argument about how to best quantize gravity, and ultimately far more persuasive; it is likely, unfortunately, to be lost amidst the conflict between string theory and its discontents. Faculty positions and grant money are scarce commodities, and universities and funding agencies are naturally risk-averse. Under the current system, a typical researcher might spend five years in graduate school, three to six as a postdoc, and another six or seven as an assistant professor before getting tenure — with an expectation that they will write several competent papers in every one of those years. Nobody should be surprised that, apart from a few singular geniuses, the people who survive this gauntlet are more likely to be those who show technical competence within a dominant paradigm, rather than those who will take risks and pursue their idiosyncratic visions. The dogged pursuit of string theory through the 1970’s by Green and Schwarz is a perfect example of the ultimate triumph of the latter approach, and Smolin is quite correct to lament the lack of support for this kind of research today.

In the real world, it’s difficult to see what to do about the problem. I would be happy to see longer-term postdocs, or simply fewer postdocs before people move on to assistant professorships. But faculty positions are extremely rare — within fundamental theory, a good-sized department might have two per decade, and it would be hard to convince a university to take a long-shot gamble on someone outside the mainstream just for the greater good of the field as a whole. And a gamble it would certainly be. Smolin stacks the deck by contrasting the “craftsmen” who toil within string theory to the “seers” who pursue alternatives, and it’s pretty obvious which is the more romantic role. Many physicists are more likely to see the distinction as one between “doers” and “dreamers,” or even (among our less politic colleagues) between “scientists” and “crackpots.”

To be clear, the scientists working on LQG and other non-stringy approaches to quantum gravity are not crackpots, but honest researchers tackling a very difficult problem. Nevertheless, for the most part they have not managed to convince the rest of the community that their research programs are worthy of substantial support. String theorists are made, not born; they are simply physicists who have decided that this is the best thing to work on right now, and if something better comes along they would likely switch to that. The current situation could easily change. Many string theorists have done interesting work in phenomenology, cosmology, mathematical physics, condensed matter, and even loop quantum gravity. If a latter-day Green and Schwarz were to produce a surprising result that convinced people that some alternative to string theory were more promising, it wouldn’t take long for the newcomer to become dominant. Alternatively, if another decade passes without substantial new progress within string theory, it’s not hard to imagine that people will lose interest and switch to other problems. I would personally bet against this possibility; string theory has proved to be a remarkably fruitful source of surprising new ideas, and there’s no reason to expect that track record to come to a halt.

Smolin is right in the abstract, that we should try to nurture a diversity of approaches to difficult questions in physics, even if his arguments on the specific example of string theory and its competitors are less compelling. But he is also right that string theorists are not always as self-critical as they could be, and can even occasionally be a mite arrogant (although I haven’t found this quality to be rare within academia). The best possible consequence of the appearance of The Trouble with Physics and similar books would be that physicists of all stripes are moved to take an honest look at the strengths and weaknesses of their own research programs, and to maintain an open mind about alternatives. (The worst possible consequence would be for large segments of the public, or the student population, or even physicists in other specialties, to misunderstand why string theorists find their field so compelling.) Sometimes a little criticism can be a healthy thing.

93 Comments

93 thoughts on “The Trouble With Physics”

  1. Sean, thanks for a very “fair and balanced” review (my apologies for using a Fox News phrase, but in your case I think that it really does in fact apply).

    You say “… The abstract argument – about academic culture and the need to nurture speculative ideas – is … important and largely correct …
    faculty positions are extremely rare – within fundamental theory, a good-sized department might have two per decade, and it would be hard to convince a university to take a long-shot gamble on someone outside the mainstream just for the greater good of the field as a whole …”.

    In an iinterview at http://www.ipm.ac.ir/IPM/news/connes-interview.pdf , Alain Connes said:

    “… From my point of view the actual system in the US really discourages people who are truly original thinkers … I believe that the most successful systems so far were these big institutes in the Soviet union, like the Landau institute, the Steklov institute, etc. Money did not play any role there, the job was just to talk about science.

    the way the young people … in the US … get their position on the market creates “feudalities” namely a few fields well implanted in key universities which reproduce themselves leaving no room for new fields. … Beginners have little choice but to find an adviser that is sociologically well implanted … so that at a later stage he or she will be able to write the relevant recommendation letters and get a position for the student … all these letters look alike in their emphatic style. The result is that there are very few subjects which are emphasized and keep producing students and of course this does not create the right conditions for new fields to emerge. …”.

    Could USA universities do what Connes suggests and copy what Connes perceives to be the useful aspects of the Soviet institutes, that is,
    create a lot of low-paying but secure research-only jobs (NOT teaching-slave-jobs), with no pressure to produce papers, thus allowing really motivated people to survive while doing whatever theoretical high-energy physics they want to do for as long as they want to do it, in an environment with good library-internet facilities and a community of like-minded others with whom they can interact, by discussion, seminars, etc., if they choose to do so ?

    Maybe the vast majority of such subsistence researchers might produce very little of note, but it would allow a Grisha Perelman type person to survive while doing work they love, and if even a very few of them make very big breakthroughs, the advances might be significant.

    Tony Smith
    http://www.valdostamuseum.org/hamsmith/

  2. I’m about to start my PhD in variants of Loop Quantum Gravity. You argue that young PhD students simply do what convinces them. To a degree that’s true, but in my case it was only through my curious reading around and some intuition that I found out about LQG. What convinced me more then anything was the difference in style you mentioned.

    However, within my University in Munich there simply would have been nobody to tell me about any other approach then String Theory. And that’s the one of the biggest faculties for Physics in Germany.

    And LQG has some remarkable results to show, however, these do not stand in the tradition of particle theory, there are no scattering amplitudes, in fact the very issue of precisely defining scattering amplitudes took over a decade. This further deepens the “cultural divide”.

    Regarding the “hype” of particles in the fabric of spacetime, these results are real, though to different degrees. Some structures of the Standard Modell do show up here and there but it’s far from clear that these behave approximately like QFT particles on a background metric in an appropriate limit. On the other hand connections between Spinfoam models and QFT have been discovered as well.

    LQG is far smaller and younger then String theory, whether or not we are heading towards a first revolution the next few years will show. However, with respect to background independence it is important to note that String theorists have come around to the point of view that it’s crucially important, that string theory in it’s background independent form implies radically new structures of space and time. Background independence means that the very concept of “short scale behaviour of gravity” makes no sense.

  3. Let me just extend on what I said regarding SpinFoams and QFT, it is becoming increasingly clear that SFs are a genuine generalization of background spacetime QFT. But these are results from right now, as opposed to the decade old Malcadena conjecture. Furthermore it’s not coming in the form of a grand conjecture but much more gradually. I think part of what Smolin is doing is simply telling people what has been achieved so they can take a look and make a judgement.

  4. The next revolution in physics will probably be when the duality between LQG and string theory is discovered. And the two warring camps can come together and realize that the only way to make progress is by helping each other.

  5. jianying

    people are looking at that, but while it makes for a nice narrative, it’s far from guaranteed. One or both theories might just as well turn out to be wrong.

  6. Gravity is a wave not a particle, Structures in space time are no more equivelent to judgment than to the Tao . Gravity is stuctured by other influences of gravity space and time reorganizes due to gravitation thus gravity and quantum loop feedbake can be equel after space time is adjusted back to itself after the a particular gravity well made in mass by gravity. It is ok to say we know electromagnetism can influence matter in unbeilievalble ways its just like yin and yang of tao. Is mass a function of a gravity well that can be moved in space and time in the universe on the linear scale also? just something to think about

  7. fh wrote:

    Background independence means that the very concept of “short scale behaviour of gravity” makes no sense.

    fh, can you expand on that? It sounds like you’re implying that background independence does away with scale entirely — I would have thought scale would still exist as a concept, just not as an absolute. (But I’m not a physicist or a mathematician, so….)

  8. Brian Greene:

    The goal of ISCAP is to bring together theoretical physicists, astrophysicists, and observational astronomers to address key problems in particle physics and cosmology that require a broad confluence of expertise and perspective.

    I think leaders in both camps are really trying to “tie up” loose ends:) And must stop thinking of “each other” as Angels and Demons:)

    To think Lee may of cut off his right arm, is a sad “state of affairs?”

    To think gravity as extended versions tested in society,”time clocks and such” one couldn’t help but wonder about such “conceptual changes” housing new perspective.

    Okay, it’s far distant from the theory of GR, but heck, moving it to the quantum region, makes it no less important?

    Good review Sean, and to the others you have mentioned.

  9. I’m just waiting for the day when some bright grad student, after making the coffee-shop round in Amsterdam, returns to prove that LQG is actually some esoteric limiting case of M-theory.

  10. I may have overstated there. It’s a trivial observation really, in a theory without a backround spacetime, that is, with diffeomorphism invariance, correlators do not depend on distances. Diffeos take W(x,y) to W(f(x),f(y)) so W(x,y) is really quite independent of x and y. In a relational picture you break diffeo invariance by thinking about matter degrees of freedom that fix locations and paths in the gravitational field.

  11. Reading the book as a person with a B.S. in physics who went on into Engineering and whose only connection to Physics departments now is to go to Alumni events and get asked for money, I think you are correct halfway in how the book struck me. The half about the sociological issues facing High Energy Physics jumped out at me also. The part about LQG I think you are seeing from the inside. I didn’t read any real big arguements for LQG in this book, other than the fact it is there along with some other theries about alternate relativity rules and reformulated algebraic models. Aside from the last none of the others seemed to me to be any more convincing than String Theory.

    I tell you this, this book would make me think twice about donating to my alma mater’s Physics department if it was heavily into String Theory or whatever it is called. What got to me was the numbers of people. It sounds like there are hundreds of people (perhaps thousands over the last 20 years) who have or are working on String Theory. Given those enormous resources, I would expect that “guage/gravity” duality would have been pinned down within 5 years, that many other simplifications would have been made. Since they have not it seems like String theory has the Mongolian Horde problem common in big projects. Less people might be more effective since it is obviously not in the last ditch effort area yet.

  12. Hi Sean, I take it all that flowed spontaneously.
    An excellent review composed of a stream of unbiased and uncynical comments on string theory ans string theorists.

    Blake, esoteric? doesn’t QG impose ‘constraints’ on M-Theory? Esoteric or exotic from Amsterdam, maybe closed (loop) strings are LQG

  13. Thanks for this review, Sean. I have a morbid curiosity about these issues, despite being a non-scientist, and your analysis was interesting. I will note my surprise at the shadow of Thomas Kuhn that appeared in the review and the comments. I was under the impression that Kuhn’s understanding of scientific truth was somewhat discredited among scientists.

  14. Steinn, gauge/gravity duality is not “formally proven,” but there is overwhelming evidence for it. You check things over and over and they keep matching. There is always the possibility that we are being fooled, but I don’t think so.

  15. Diffeos take W(x,y) to W(f(x),f(y)) so W(x,y) is really quite independent of x and y.

    One needs to be careful here. The correlation function

    W(x,y) ~ |x-y|^-2h.

    is clearly not diffeomorphism invariant. However, it is diff invariant to claim that W(x,y) asymptotically has this form when x and y approach each other. In particular, the anomalous dimension h is diff invariant. The underlying reason is that the dilatation operator D = x^u d/dx^u is independent of the metric.

    Note also that a conformal transformation in 2D is the same thing as a diffeomorphism in one complex dimension, but infinite conformal symmetry in 2D is nevertheless compatible with correlators of the form W(z,w) ~ (z-w)^-2h.

  16. The point of the argument is precisely that the correlators can not have the form W(x,y) ~ |x-y|^-2h if diffeos are a symmetry that is implemented unitarily in a straightforward naive way.

    I can’t speak on the 2D case I haven’t looked much at conformal symmetries and string theory, sorry.

  17. However, it can be possible if infinite-dimensional symmetries are implemented unitarily in a not-so-straightforward way, namely with an anomaly. A non-zero h requires a conformal anomaly, which changes the conformal algebra to the Virasoro algebra. Now guess who discovered the Virasoro-like extension of the diffeomorphism algebra in higher dimensions 🙂

  18. I appreciate the discussion, but please let’s not use this thread to hash out technical points about diffeomorphism invariance…

  19. Heh, just as described by Sean’s review, the comments have two threads going at once. I’d like to chime in and thank Tony for providing the quote from Connes in comment 5, as well as say I liked his proposal for the establishment of… what to call them… “theory hostels”? Connes’ insight is particularly compelling since he is one of few researchers to have come up with a viable TOE independent of strings and LQG.

  20. Sean,

    In the past you have criticized scientists for being ignorant of philosophy, especially for making statements about philosophy without formal training. Whenever you discuss String Theory, however, you seem to take a distinctively positivist attitude. Now from at least the 20th century philosophers I’ve read, positivism is the only philosophy universally hated by them all. One could argue that they doth protest too much since after all if positivism is a good way to go, why would we need philosophers? That being said there are plenty of good critiques of positivism.
    Basically, I’m curious how you square your views on scientists and philosophy with your views on String Theory which seem to simplify to: String Theory deserves it’s virtually monolithic position in theoretical physics because when most scienists look at it they think it’s the best bet.

Comments are closed.

Scroll to Top