Via JenLuc Piquant’s twitter feed, here’s one time I’m not going to stick up for my colleagues in the social sciences: a misguided attempt to cast the search for empirical support as “physics envy.” It’s a New York Times Op-Ed by Kevin Clarke and David Primo, political scientists at the University of Rochester.
There is something rightly labeled “physics envy,” and it is a temptation justly to be resisted: the preference for reducing everything to simple and clean quantitative models whether or not they provide accurate representations of the phenomena under study. The great thing about physics is that we study systems that are so simple that it’s quite useful to invoke highly idealized models, from which fairly accurate quantitative predictions can be extracted. The messy real world of the social sciences doesn’t always give us that luxury. The envy becomes pernicious when we attack a social-science problem by picking a few simple assumptions, and then acting like those assumptions are reality just because the model is so pretty.
However, that’s not what Clarke and Primo are warning against. Their aim is at something altogether different: the idea that theories should be tested empirically! They write,
Many social scientists contend that science has a method, and if you want to be scientific, you should adopt it. The method requires you to devise a theoretical model, deduce a testable hypothesis from the model and then test the hypothesis against the world…
But we believe that this way of thinking is badly mistaken and detrimental to social research. For the sake of everyone who stands to gain from a better knowledge of politics, economics and society, the social sciences need to overcome their inferiority complex, reject hypothetico-deductivism and embrace the fact that they are mature disciplines with no need to emulate other sciences…
Unfortunately, the belief that every theory must have its empirical support (and vice versa) now constrains the kinds of social science projects that are undertaken, alters the trajectory of academic careers and drives graduate training. Rather than attempt to imitate the hard sciences, social scientists would be better off doing what they do best: thinking deeply about what prompts human beings to behave the way they do.
Sorry, but “thinking deeply” doesn’t cut it. People are not especially logical creatures, and we’re just not smart enough to gain true knowledge about the world by the power of reason alone. That’s why empiricism was invented in the first place, and why it’s been so spectacularly successful over the last few centuries.
Clarke and Primo seem to confuse “the need for empirical testing” with “the need for every model proposed to be backed up by data before it gets published.” If they had stuck to rejecting the latter narrow idea, they would have had a decent case. Certainly we physicists don’t require that every model be supported by data before it is published — otherwise my CV (and those of most of my friends) would be a lot shorter! But we all agree that the ultimate test of an idea is a confrontation with data, even if a theory might be too immature for that confrontation to take place just yet.
One of the main reasons psychology gets rapped as not being a “real” science is the active resistance on the part of a lot of psychologists at pushing evidence-based approaches to learning about human behaviour. The actual scientist/practitioners are fighting back but it’s an uphill battle…
http://drvitelli.typepad.com/providentia/2012/03/why-are-people-so-skeptical-about-psychology.html
Time for another Sokal hoax?
First – please don’t associate this garbage with social science. This is not how I and my colleagues see the task of studying human social behavior. You seem to be dissociating them – I just want to make sure you do!
Second – on the point of oversimplifying mathematical models in social science, I hear a lot of natural sciences level this accusation against my own discipline of economics, and more often than not they really don’t know what they’re talking about. Of course, as Einstein noted, everything should be made as simple as possible – but no simpler. I take the “but no simpler” part very seriously. But in my experience, natural scientists and even social scientists in other disciplines are a very, very poor judge of exactly what that entails.
I think of social science as a sub-field of biology, and a very important sub-field. We are studying the social behavior of the most highly evolved species we know of. The social behavior of this species is so intricate that we further sub-divide into multiple disciplines just to get a grasp of it. And I also reject the hard/soft distinction. The data we have on human social behavior is far more extensive and detailed than the data that other biologists have on the social behavior of other animals, and I’d guess our models are much better at explaining and predicting observed phenomena. Who is the hard science and who is the soft science here? In my mind – the distinction doesn’t make much sense. It’s either science or its not science.
People are not especially logical creatures, and we’re just not smart enough to gain true knowledge about the world by the power of reason alone.
However since science is logical, wouldn’t that mean to study illogical beings scientifically would be fruitless? Might as well just make some stuff up.
I was being sarcastic there. I actually agree with you that social sciences need to be studied scientifically. People are a lot like quantum particles. It’s difficult to predict what one will do, but on the average, there is a well defined expectation value. However unlike quantum mechanics there is an ethics board which makes these studies more difficult. Just think how hard it would be PETH (People for the Ethical Treatment of Hadrons) would be demanding you turn off particle accelerators.
Huh, there actually is a PETH group.
http://www.astroengine.com/2008/08/do-hadrons-feel-pain/
A model can be useful without being experimentally verifiable. Aren’t there all kinds of toy quantum field theories that help illustrate some feature of quantum field theory, without any chance of such a toy theory existing in our universe? If they are legitimate in physics, they are legitimate in the social sciences. I took that NYT op-ed to be calling for such simple, clarificatory models to be part of the social sciences repertoire. The examples in the essay – Anthony Downs’ model, or Kenneth Arrow’s theorem on voting – certainly suggest that interpretation.
I think Clarke and Primo are confused about what it means for a theoretical model to be empirically tested. They suggest two examples of models which they claim are empirically untested but valuable nonetheless. However, the first of these models IS empirically tested, while the second of them is not a theoretical model at all, but rather a theorem.
First example: Downs’ model predicts that political parties will move towards the center. They do, so his model is worth taking seriously. If it turns out that parties don’t move toward the center, his model isn’t valuable.
Second example: Arrow uses logic to demonstrate that no voting system can satisfy all of a certain set of criteria. Since this is a conclusion deduced by logic, it need not be empirically tested. But this is not an example of a model in the social sciences which doesn’t need to be tested, because it isn’t a *model* at all! Rather, it’s merely a theorem. We have those in physics, too (e.g. Noether’s theorem).
Clark and Primo also say the following, in the course of arguing that theories need not make predictions: “the test of a map lies not in arbitrarily checking random points but in whether people find it useful to get somewhere”. But use of a map for successful navigation is a form of empirical confirmation! As with the Downs example, they seem to be confused about what empirical testing means.
So, if the goal is to argue that models in the social sciences need not be empirically testable to have value, they have failed.
Now, it’s true that simplified models might have value in helping us understand how our models work, even if they are too simplified to line up with reality at all. If this is all that they were intending to argue, the best that I can say is that the article was very unclear (misleading framing of issue, misleading discussion, misleading examples, misleading title, and failure to mention that this was their point, for starters).
For completeness, I’ll mention that they also might intend to endorse two other propositions. First, that social scientists can do useful things other than create models (e.g. make theorems, like Arrow). Second, that data can be useful even without a theory. Both of these propositions seem quite uncontroversial, of course, and would not in any way be called into question by a healthy envy of physicists.
Pingback: Testing Your Theories Is Not a Matter of “Envy” | Cosmic Variance | Discover Magazine « Secularity
There’s an easy fix for this: doff the “science” conceit and call it philosophy. No one accuses a philosopher of being a charlatan if he or she spends all his or her time thinking in illuminatingly deep ways, unfettered by the constraints of empiricism. And they needn’t envy anyone, because no one requires them to be practical, just clever.
Right. Clark and Primo’s approach might produce some interesting insights and results, and they could productively develop and apply it. Good for them; more power to ’em, sincerely. Just don’t call it “science.”
“That’s why empiricism was invented in the first place, and why it’s been so spectacularly successful over the last few centuries.”
David Deutsch’s recent book — annoying, challenging, and thought-provoking — calls this “spectacular success” into question. I believe his claim is that empiricism in its strict sense (going back to the empiricist British philosophers) fails, and that the “power of reason” is at least an equal partner, and sometimes is primary. Maybe I’m not disagreeing with you (Sean).
Here is some Krugman, talking about the value of simple ideas in economics.
http://krugman.blogs.nytimes.com/2011/02/02/models-plain-and-fancy/
Quote:
You could argue that modern economics really began with David Hume’s Of the Balance of Trade, whose core is a gloriously clear thought experiment:……And Karl Smith is right: no way could Hume have published such a thing in a modern journal.
End quote.
Primo and Clarke:
Quote:
Many social scientists contend that science has a method, and if you want to be scientific, you should adopt it. The method requires you to devise a theoretical model, deduce a testable hypothesis from the model and then test the hypothesis against the world. If the hypothesis is confirmed, the theoretical model holds; if the hypothesis is not confirmed, the theoretical model does not hold. If your discipline does not operate by this method — known as hypothetico-deductivism — then in the minds of many, it’s not scientific……For the sake of everyone who stands to gain from a better knowledge of politics, economics and society, the social sciences need to overcome their inferiority complex, reject hypothetico-deductivism and embrace the fact that they are mature disciplines with no need to emulate other sciences.
….
End quote.
I see them as arguing the very same thing.
More Krugman:
http://krugman.blogs.nytimes.com/2011/02/03/more-on-simple-models/
Quote:
“Simple conceptual models can also help convince you NOT to believe in economic ideas. Real business cycle theory says that economic fluctuations are the result of technological shocks, amplified by intertemporal labor substitution. My version: think of a farmer who faces sunny and rainy days. On rainy days his labor won’t be as productive as on sunny days; this effect on his output is amplified by his rational decision to stay in bed on rainy days and work extra hard when the sun shines. I think this gets at the essence of the concept; it also makes you wonder, is this really, really what you think happens in recessions?”
End quote.
It would seem that empirical data is not sufficient to dismiss Real Business Cycle Theory, otherwise why would Krugman propose his analogy?
I also find it ironic that people who insist superstring theory is mainstream physics insist that theory must meet the test of empirical data. Forty years and counting and that has not yet happened…..
If a discipline has the word science in it is usually is not a science – computer science, social science, information science – that does not mean it is not worthwhile it simply means it isn’t science.
I read that editorial, but assumed it was an April’s Fool prank. Do you think it was not?
I read that too, thought the same thing, but then wasn’t sure if it wasn’t a joke.
The gist of Sean’s post is admirable, as it reflects the standard lofty ideals of science. It is probably impossible to be reminded too often that,
“…’thinking deeply’ doesn’t cut it. People are not especially logical creatures, and we’re just not smart enough to gain true knowledge about the world by the power of reason alone.”
It is therefore pertinent to point out a huge domain of gravitational physics where deep thinking holds sway and empirical evidence is absent. The cliche is that General Relativity has been quite thoroughly tested throughout the Solar System and that, to quote Stephen Hawking: “We already know the laws that govern the behavior of matter under all but the most extreme conditions.” By extreme is meant extreme velocities and extremely strong gravitational fields.
The domain I have alluded to is one where even Newtonian gravity has not been tested. Given a uniformly dense sphere with a hole through the center, a test object is dropped into the hole. What happens? By “thinking deeply” textbooks and professors answer: simple harmonic motion. But no observational data is ever cited—because we have none.
The media is barraged with science shows and articles about black holes and wormholes. All the while we have no empirical evidence pertaining to an extremely simple (in principle) gravitational experiment involving an ordinary hole. Aside from healthy curiosity and a humble desire to live up to the ideals of science, interested readers may like to consider other reasons why it would be a good idea to fill in the gap in our empirical knowledge of gravity:
http://astroreview.com/issue/2012/article/the-direction-of-gravity
Daniel, that’s all very well, but there has got to be a fundamental difference between the social sciences and biology. After all, when did a cat’s behaviour ever change because of a research paper about cats’ behaviour? The same ability to generate knowledge and change the way we see the world is precisely what makes human behaviour so complex. This isn’t to see we shouldn’t study it rigorously – it is to say we shouldn’t expect to get the kind of replicable results you get in biology or physics. Ever.
Hi Arun,
I agree with you that simple models might sometimes be very useful in the social sciences. Indeed, it would be silly to undertake construction of detailed quantitative models before possessing a good qualitative understanding. I think Krugman’s articles make this case well.
However, this is NOT in conflict with the scientific method. Quite the contrary. The models that Krugman talks about are of interest only because they help us understand the empirical facts.
If all that Clarke and Primo mean to argue is that simple qualitative models can help economists get on the right track, and that journals are too reluctant to publish such models, they could very well be right. But in that case, they would not be arguing against “hypothetico-deductivism” – such models are of interest because they help us understand the facts – so it would be misleading of them to claim that they are.
there is difference bw understanding how things may occur (a general theoretical framework) and how they have occurred (using that framework for analysis)
a grounded theoretical framework builds from real data, and it thus has roots in empirical “evidence”
empirical testing comes down to a testing of fit, and relevance , when it comes to testing a framework empirically
it is not about qualitative and quantitative. neither deductivity or inductivity..
empirical research can be generating a hypothesis or it can be testing one
for hypothesis generating, you need to use and compare (test) empirical data
for hypothesis testing, you need to generate empirical data
in theoretical physics, or biology, or archiology, or geology, hypothesis generating has been the regular approach
in social sciences, the trend is a little different
and then, in physics, mathematics, that’s different, too
Pingback: Social scientists argue that social scientists shouldn’t do science | The Finch and Pea
The basic issue here is what makes the ‘explanations’ provided by various disciplines of use for human survival.
This thread reveals that there’s lots of confusion on this matter among relatively sophisticated people. No wonder policy makers fail to make anywhere near optimal use of science and engineering in running governments and industries.
To try to address just this problem, I’ve drafted an essay:
“The Skeptical Scientific Mind-Set in the Spectrum of Belief: It’s about models of ‘reality’ – and the unavoidable incompleteness of evidence, for – or against – any model or fact.”
http://www.pipeline.com/~lenornst/ScienceInTheSpectrumOfBelief.pdf
In Len Ornstein’s essay, it is written:
“no scientific model can be either absolutely falsified – or absolutely proven.”
I think most science-oriented persons, including myself, regard this approach as increasing the kind of confusion that Ornstein professes to diminish.
The example that I mentioned in my previous post is a case in point. Both Newton and Einstein predict that a test object dropped into a hole through a uniformly dense sphere will oscillate back and forth in the hole. Since this prediction has never been tested, for the sake of argument we may suppose that nothing like this happens at all. Suppose instead that the test object does not even pass the center. If this is what actually happens, then it would absolutely falsify and disprove Newton’s and Einstein’s theories; they would not survive what Faraday called the “Ithuriel spear of experiment.” I don’t think any physicist would disagree with this. If the test object does not pass the center, it means we need a new theory of gravity.
This brings us back to the question as to why physicists see no need to test this particular prediction. Previously I suggested that it was due to the kind of “deep thinking” used by Clarke and Primo that Carroll is critical of. But physicists sometimes draw conclusions lacking empirical support without even troubling to think deeply. They also have a kind of “reflex reaction” for deciding whether something is true or not.
For example, if a prediction violated the energy conservation law, the model it is based on is likely to be rejected without further thought. The track record of this law tends to support this judgment. A more cautious, strictly empirical approach, however, would be to regard the new prediction as one more opportunity to test the law.
With regard to the experiment I’ve proposed, physicists are neither cautious nor empirical. They fail to see the need to carry it out because the very idea does not get past their reflexes. This characteristic of physical scientists may ultimately be even more detrimental to science than the avoidance of empirical facts via deep thinking.
Mona Chalabi, in the New York Times, is worth a read:
http://www.nytimes.com/roomfordebate/2012/04/01/how-to-teach-economics-after-the-financial-crisis/what-i-didnt-learn-in-introductory-economics
Begins:
“An economist must be a “mathematician, historian, statesman, philosopher, in some degree.” So, promisingly, begins the sixth edition of “Macroeconomics” by Greg Mankiw, a key reading for my introductory economics class at the University of Edinburgh (and still a key reading six years on). Unfortunately, the book, like the course that prescribed it, delivered on only one of those claims: to be a mathematician.
What began as eloquent and logical graphs and formulas quickly spiraled out of control and I soon found myself reading that “economics is not only a social science, it is a genuine science. Like the physical sciences” and that financial crises can be predicted by using the following formula:…”
You could equally interpret those comments as meaning “we can think about ideas in the social sciences that cannot conceivably be verified or falsified by empirical data in the foreseeable future; such topics are valid avenues of research, even though ‘thinking deeply’ is the only way to distinguish between ideas in such fields.”
Phrased like this it does not sound very different to a description of some research in string theory, or indeed issues of why the universe consists of something rather than nothing (a topic you have been championing recently). Why is one ‘science’ and the other ‘science envy’?