Sorry for a post title that will attract the crazies. Carl Zimmer has a story in the New York Times that discusses a growing unease with the practice of science among scientists themselves.
In tomorrow’s New York Times, I’ve got a long story about a growing sense among scientists that science itself is getting dysfunctional. For them, the clearest sign of this dysfunction is the growing rate of retractions of scientific papers, either due to errors or due to misconduct. But retractions represent just the most obvious symptom of deep institutional problems with how science is done these days–how projects get funded, how scientists find jobs, and how they keep labs up and running.
However… essentially all the examples are from biologically-oriented fields. I’ll confess that Carl asked me if there is a similar feeling among physicists, and after some thought I decide that there really isn’t. There are certainly fumbles (faster-than-light neutrinos, anyone?) and scandals (Jan Hendrik Schön being the most obvious), but I don’t have any feeling that the problem is growing in a noticeable way. Biology and physics are fundamentally different, especially because of the tremendous pressure within medical sciences when it comes to any results that might turn out to be medically useful. Cosmologists certainly don’t have to worry about that.
But maybe this is a distorted view from within my personal bubble? Happy to hear informed opinion to the contrary. The relevant kind of informed opinion would actually involve a comparison of the situation today with the situation at some previous time, not just a litany of things you think are dysfunctional about the present day.
“I’ll confess that Carl asked me if there is a similar feeling among physicists, and after some thought I decide that there really isn’t.”
——-
No dissatisfaction in your field about how science is done? The direction it’s taking? Really? But wait a minute: String theory? Multiverses? Dark matter? Seems to me I just read a book called “The Trouble With Physics” by a renowned theoretical physicist…
I have always thought that a couple of the reasons for the increasing numbers of retractions in the biological sciences is the increasing pressure on scientists to publish (and in some places to publish a certain number of papers per year), and the funding situation where there is pressure for fewer and fewer research dollars.
I’m interested at the idea that the retraction phenomenon isn’t in the other sciences as well. I would assume the same outside pressures exist.
I’ll be following this discussion to hear what others think.
I’m wondering how much the rate of retractions represents *improvements* in scientific endeavor, or at least oversight. Before the Interwebz, it was likely more difficult to find these errors, bring them to attention, “crowdsource” criticisms, or track the rate of retractions. Perhaps this is shaping up the biological sciences in that sense. That’s not to say that the rest of it–how we’re funded, hired, retained or fired, published, etc.–isn’t done inside a creaky, practically pre-modern framework that desperately needs an overhaul.
Sorry… this isn’t addressing the question of non-biological sciences; just considering ideas.
There is general and growing problem with journal access that harms every field of science. Big publishers have made paper journals expensive, so you can’t find them on library shelves. The general public is entirely locked out and students have an increasing difficult time with research.
I think the main difference is that a big boom in biomed is ending, whereas physics already suffered through the ending of its postwar boom back in the 70s or so. The points about severe pressure from the increasing time and decreasing rate of success from PhD to tenure apply just as easily to physics… but long postdoc periods have been standard in physics quite a while now, so we’re used to it.
I’ve met professors who went from PhD to faculty in two years… back in the 1960s. My PhD advisor got his PhD in 1983… and went 9 years before landing a faculty position, and another 6 or so before tenure. And this is a very successful guy! I think the biosci people are more or less going through the same painful contractions that physics went through back in the 1980s, even if the details differ (fraud is a lot easier to get away with in medicine).
That’s not to say that there isn’t dysfunction in physics… it just isn’t particularly new dysfunction.
If the type of dysfunction you’re referring to is the kind which might be measured by retractions of papers, then I don’t think this exists much in physics. But there are certainly many critics (like me) who think that there are similar, correctable institutional problems leading to an over-investment in certain research avenues, especially in those sub-fields of physics which are untethered from experiment (e.g. quantum gravity, quantum foundations, cosmology). The criticisms of string theory by Woit, Smolin, and others are the most well-known examples, although I’m not sure if that qualifies as a “feeling among physicists”.
I don’t think anyone has broken out retraction rates for biological and physical sciences, so this is an open question. All the people who I talked to about the dysfunction of science were from the biomedical sciences, and their examples were all from there. Given that most of the spending on science in the US is biomedical, I suppose one could simply say that science in general is, overall, dysfunctional.
Not sure though Carl if spending and number of submitted papers correlate. Maybe they do, but in general there are quite a lot of physics papers written by fewer authors (CERN aside), therefore less cost.
Is the fact that women are getting massively pushed out of the system before reaching tenured / senior scientific positions a signature of dysfunction?
This is clearly a problem for women in scientific careers, but is also a major problem for the science itself since it indicates that scientific merit is not central to the selection process of future scientists.
About a year ago, a New York Times reporter by the name of Carl Zimmer wrote a piece (http://www.nytimes.com/2011/06/26/opinion/sunday/26ideas.htm) about the publication of incorrect results. The refrain of that article, it seemed to me, was that there was a big problem because incorrect results weren’t being retracted. Now we’re supposed to wring our hands over the fact that they are being retracted. I can’t get too worked up about it either way.
The question of whether grant- and job-chasing is making scientists too careerist is certainly a worthwhile one, but I’m not convinced that either of these two points of view (Science is in crisis because papers aren’t being retracted! Science is in crisis because papers are being retracted!) shed much light on it.
A biologist here, so I am not fully qualified to tell how things are in physics, but it has always seemed to me that in physics, especially the theoretical part of it (and even more so in math), you become successful with your ideas, and the primary way a paper can be wrong is if you got your math and logic wrong, which is quite disastrous to one’s reputation so people have a lot of incentive to check and recheck things and very little incentive to cheat. And cheating is quite difficult to begin with because of the nature of the field. In experimental physics there is probably a lot more room for that and it likely happens more often
In biology it is totally different, because it is mostly the data that makes what they are, and that data is usually directly seen and worked with only by the authors, and most often than not, only one or very few of them. On top of that, a lot more so than it is in physics, it is where you publish that builds your reputation, not what exactly you did as few people take the time to pay attention to every methodological detail, especially these days when those are increasingly relegated to supplemental sections and people are so busy writing grants and their own paper to go in depth into articles.
So both the opportunity and the incentive is there to cheat. And this is even without the other very big issues such as the cutthroat competition for funding, the multibillion dollar pharmaceutical and biomedical industry that is tied to the field, the public relevance of the results, the industrial mode of operation of many big labs, and so on.
It is good that this is being talked about more and more but I don’t see the situation improving much in the foreseeable future without changing the incentive structure
Beyond the usual complaints about the job market (which I’ll spare you as I’m long gone), I remember of plenty of conversations about how the system rewards a lot of mediocre results and bandwagonny “me too” papers. Perhaps it’s just the nature of things, but the number of interesting papers (admittedly a really subjective measure) was depressingly few and often by a very small subset of people. Hopefully things have gotten better, but I do think there was a feeling that something was off about the direction things were going (but in a very different way than, for example, Smolin was complaining about).
There were also a number of interesting conversations about the influence of the ArXiv and how it wasn’t necessarily an unalloyed good, but that’s a slightly different story.
Really? You don’t see any institutional problems with how projects in physics get funded, how scientists find jobs, and how they keep labs up and running? This has got to be one of the most ridiculous statements I’ve ever read by a physicist on the internet, and as a physicist myself I’m actually mildly offended.
I see you work at Caltech. Let me begin there.
I hear the Caltech physics department has had some problems attracting graduate students in recent years. Why do you think this might be? Perhaps it’s because when prospective physics graduate students visit Caltech, there has been more than one year in recent memory when the physics professors weren’t in town to greet them. Or perhaps it’s because the physics department discourages the current graduate students from making contact with the prospectives. Having met some Caltech physics graduate students myself, I certainly understand why this might be. I asked them what they did for fun, and what they did outside of campus, and they couldn’t really give me an answer — they said they pretty much spent their time on campus or at home. They couldn’t even come up with a suitable lie!
As far as keeping a lab up and running, I hardly know where to begin. I can’t count the number of papers I have (eventually) published where the reviewer(s) had some sort of vendetta against my research group. How many professors have turned writing grant proposals into their full-time jobs? Why do government agencies reject grant proposals because of things like improperly formatted citations? How much of scientific publishing is a race to publish the fastest or most often, rather than the most profoundly? Why do people care so much about the impact factor of journals rather than the content of their work? The justification of funding is largely responsible for many of these questions.
As far as finding jobs, you might have noticed that physics has continually had difficulty attracting females to the profession. One of the reasons for this is the uncertainty of being able to obtain stable employment. At what point in their career does a physicist have the stability to raise a family? After earning peanuts as a graduate student, and slightly more peanuts as a postdoc, most physicists don’t earn a decent salary in a stable position until their early-mid 30s or later. You could try to point to the rising number of females in physics and suggest that the problem might be getting better, but the underlying cause of the problem is the comparison between the number of available positions in comparison with the number of people being trained for those positions.
Are things getting better or worse? I honestly don’t know. The number of jobs are increasing, but not as quickly as the number of graduate students, and the percentage of foreign students who accept domestic postdocs is rising as well. All this means that there is more competition for the few available spots.
Overall, I can certainly understand why there might be a “growing sense among scientists that science itself is getting dysfunctional”. I see these problems myself and sometimes wonder why I’m doing what I’m doing.
There is a more or less relevant literature on this see:
Philip Mirowski, Science-Mart (2011) and his earlier The Effortless Economy of Science? (2004)
Are the ground for retraction the same across all disciplines? If some fields only retract due to fabrication of primary data, and others retract simply because conclusions are later proved to be incorrect, then it willbe hard to compare.
It was Lord Rutherford who said that physics was the only science and all the rest were stamp collecting.
No one is saying that modern hi-tech industry is dysfunctional because ninety nine from hundred startups are failure. Startups are also mired with frauds, incompetence and poor leadership. Nevertheless startup industry as whole is healthy and one of cornerstones of modern economy. So modern science now look less than big corporations of old and more like stratups. Some area of science could be not so healthy, but the same goes with economy and that is not the end of the world. Don’t see the problem with it as long as science as whole works.
One place to look is the retractionwatch blog: http://retractionwatch.wordpress.com/
@Serge — unfortunately the modern startup industry isn’t at all healthy right now, and it might never be anything like the 90s again. Last I heard from my friends who are VCs, is that they’re funding significantly fewer startups today. Almost none of these companies or their backers see an IPO as a possible endgame. Getting bought out by a Google, Cisco, Microsoft are the only realistic outcomes that yield a profit for the investors, which makes VC firms less likely to fund blue-sky endeavors (rather they’re funding incremental stuff that a Google, Cisco, or Microsoft type of behemoth would be interested in). A lot of VC partnerships have disbanded as investors have taken their money out. Last year I heard from a friend on Sand Hill that the number of functioning VC firms had dropped by about 50 percent since the current mini-depression started. Probably more by now. And talent is harder too find now, because Zynga drove home the point that stock options even they’ve vested are mostly worthless — unless you’re lucky enough to hold founders stock.
So what about the Bogdanov brothers?
And what do you think about Roman Jackiw’s statement “They showed some originality and some familiarity with the jargon. That’s all I ask.” ?
Yes, in a few fields of physics you have similar issues.
Climate Science because it has a lot of the data focus of biology, and cannot do any proper (controllable) scientific experiments to test theories.
I would say that biology is still kind of OK. Psychology and medicine, fields where I as a PhD in theoretical physics currently venture, is much much more broken.
It’s not just the nature of the subject they study. It is also the people who study it. They are in my experience clearly less intelligent ON AVERAGE and more emotionally driven than in a hard science.
I have some ‘survival bias’ so I won’t talk about my own experience. However, I have some friends in biology and some in physics. The physicists are in general reasonably happy, the biologists are in general quite unhappy, or even angry. They are all around 30-40 years old.
I am afraid there is something wronger in biology –
The reason there are no retractions in hep-th is that in the vast majority of cases nobody really cares if a paper is right or wrong. Most papers are probably never even read by anybody but some unlucky referee who, when reminded for the 3rd time, just wrote something to make the paper go away. Do you really seriously have to ask if that sounds like a dysfunctional process?
Besides, this might be a good place to ask:
I’m looking for stories about the dysfunctional academia for a project I’m working on. If you have something to share, something to complain about, a frustrating experience of you or a friend, please send me an email to hossi at nordita dot org, subject “academia.” I’ll respect your anonymity. It doesn’t matter which field you work in.
Is the ‘faster-than-light’ thing from Gran Sasso really a good example of a fumble, and a sign of dysfunctionality? To be honest, I think there’s a l0t there for physics to be proud of.
The team there reported a strange but potentially important result with themselves a great deal of scepticism, and asked for help to prove or disprove it. Isn’t that exactly how science ought to work – a very public and very intense peer review process.
The hype that was generated was more of a fumble, but that’s hardly their fault. Just my two cents…
Got to agree with Bee at number 23! To my mind, rather a lot of theoretical high-energy physics and cosmology consists mainly of pointless speculation. Take inflation, for instance – how many countless different models are written about every year, most of which are based on various untestable, more or less ad hoc assumptions, and all of which “predict” a scalar spectral index of around 0.96 and not much else?
Whether this is an entirely new phenomenon or not I can’t say. I wasn’t around 30 years ago, but my gut feeling is that the time lag between theoretical ideas and the experimental results that ought to drive them has grown, which leads to more pointless speculation in lieu of tackling genuinely interesting physical problems. And I think the existence of arXiv and other online citation counting websites has probably increased the temptation to rush out half-baked papers in an effort to cash in on current ‘fads’ and gain citations.