What Is Science?

There is an old parable — not sure if it comes from someone famous I should be citing, or whether I read it in some obscure book years ago — about a lexicographer who was tasked with defining the word “taxi.” Thing is, she lived and worked in a country where every single taxi was yellow, and every single non-taxi car was blue. Makes for an extremely simple definition, she concluded: “Taxis are yellow cars.”

Hopefully the problem is obvious. While that definition suffices to demarcate the differences between taxis and non-taxis in that particular country, it doesn’t actually capture the essence of what makes something a taxi at all. The situation was exacerbated when loyal readers of her dictionary visited another country, in which taxis were green. “Outrageous,” they said. “Everyone knows taxis aren’t green. You people are completely wrong.”

The taxis represent Science.

(It’s usually wise not to explain your parables too explicitly; it cuts down on the possibilities of interpretation, which limits the size of your following. Jesus knew better. But as Bob Dylan said in a related context, “You’re not Him.”)

Defining the concept of “science” is a notoriously tricky business. In particular, there is long-running debate over the demarcation problem, which asks where we should draw the line between science and non-science. I won’t be providing final any final answers to this question here. But I do believe that we can parcel out the difficulties into certain distinct classes, based on a simple scheme for describing how science works. Essentially, science consists of the following three-part process:

  1. Think of every possible way the world could be. Label each way an “hypothesis.”
  2. Look at how the world actually is. Call what you see “data” (or “evidence”).
  3. Where possible, choose the hypothesis that provides the best fit to the data.

The steps are not necessarily in chronological order; sometimes the data come first, sometimes it’s the hypotheses. This is basically what’s known as the hypothetico-deductive method, although I’m intentionally being more vague because I certainly don’t think this provides a final-answer definition of “science.”

The reason why it’s hard to provide a cut-and-dried definition of “science” is that every one of these three steps is highly problematic in its own way. Number 3 is probably the trickiest; any finite amount of data will generally underdetermine a choice of hypothesis, and we need to rely on imprecise criteria for deciding between theories. (Thomas Kuhn suggested five values that are invoked in making such choices: accuracy, simplicity, consistency, scope, and fruitfulness. A good list, but far short of an objective algorithm.) But even numbers 1 and 2 would require a great deal more thought before they rose to the level of perfect clarity. It’s not easy to describe how we actually formulate hypotheses, nor how we decide which data to collect. (Problems that are vividly narrated in Zen and the Art of Motorcycle Maintenance, among other places.)

But I think it’s a good basic outline. What you very often find, however, are folks who try to be a bit more specific and programmatic in their definition of science, and end up falling into the trap of our poor lexicographic enthusiasts: they mistake the definition for the thing being defined.

Along these lines, you will sometimes hear claims such as these:

  • “Science assumes naturalism, and therefore cannot speak about the supernatural.”
  • “Scientific theories must make realistically falsifiable predictions.”
  • “Science must be based on experiments that are reproducible.”

In each case, you can kind of see why one might like such a claim to be true — they would make our lives simpler in various ways. But each one of these is straightforwardly false.

I’ve talked about the supernatural issue a couple of times before. Short version: if a so-called supernatural phenomenon has strictly no effect on anything we can observe about the world, then indeed it is not subject to scientific investigation. It’s also completely irrelevant, of course, so who cares? If it does have an effect, than of course science can investigate it, within the above scheme. Why not? Science does not presume the world is natural; most scientists have concluded that the world is natural because that’s the best explanation for what we observe. If you are ever confused about what “science” has to say about something, just ask yourself what actual scientists would do. If real scientists were faced with a purportedly supernatural phenomenon, they wouldn’t just shrug their shoulders because it wasn’t part of their definition of science. They would investigate it and try to come up with the best possible explanation.

The falsifiability question is a trickier one, to which I will not do justice here. It’s a charge that is frequently leveled against string theory and the multiverse, as you probably have heard. People who like to wield the falsifiability cudgel often cite Karl Popper, who purportedly solved the demarcation problem by stating that scientific theories are ones that could in principle be falsified. (Lenny Susskind calls these folks the “Popperazzi.”) It’s the kind of simple, robust, don’t-have-to-think-too-hard philosophy that even a scientist can get behind. Of course, string theory and the multiverse aren’t at all the kinds of things Popper had in mind when he criticized “unfalsifiable” ideas. His bugaboos were Marx’s theory of history, Freudian psychoanalysis, and Adlerian psychology. The problem with these theories, he (correctly) pointed out, was that they told stories that could be made to fit literally any collection of data. Not just “data we could realistically acquire,” but absolutely anything you could imagine happening in the world. That’s completely different from the examples of string theory or the multiverse, which clearly are saying something concrete about the world (the ultraviolet completion of quantum gravity, or conditions in the universe far outside our observable region), but to which we have no experimental access (or almost none). Of course, there’s also the issue that the demarcation problem is a lot trickier than naive Popperianism makes it out to be, but that’s another discussion. The right strategy, once again, is to look at what actual scientists would do or are doing. When faced with difficult problems concerning quantum gravity or the early universe, they follow precisely the outlined program: they invent hypotheses and try to see which one is the best explanation for the data. The fact that the data are relatively crude (the existence of gravity and gauge theory, the known cosmological parameters) doesn’t prevent it from being science.

Noah Smith (an economist) wrote an interesting post related to the “reproducibility” question. It’s another bugaboo, often raised by creationists who want to take jabs at evolution. As a working cosmologist, I know perfectly well that not all good science requires reproducible experiments. We haven’t made a Big Bang in the laboratory — yet. Few of the folks who emphasize reproducibility would go so far as to claim that cosmology (and much of astrophysics) doesn’t count as “science.” Instead, they say things like “Oh, but in cosmology you’re comparing data to theories that are developed here in Earth in response to laboratory experiments, so it’s a more complicated give-and-take.” Yes it is! What they should admit is that all of science involves this more complicated and subtle kind of give-and-take between theories and experiments.

Nothing in our three-step definition of science refers to “reproducibility” (any more than it refers to “naturalism” or “falsifiability”). The key feature of science is that it is empirical — progress is made by comparing multiple plausible theories to actual data — rather than rationalist/logical — deriving truths from reason alone. But when it comes to collecting those data, the only rule is “do the best you can.” Sometimes we’re lucky enough to be able to reproduce conditions exactly (Noah’s “Level Four”), but often we are not. What matters is that there are data, and that attempting to account for them is how we choose between various hypotheses that would have otherwise been plausible or at least conceivable. This might mean that some scientific questions are harder to decide than other ones, but that sounds like the least surprising conclusion in the world.

Some will object that this conception of science is too broad, and encompasses not only economics but also fields like history. To which I can only say, sure. I’ve never really thought there was an important distinction of underlying philosophy between what scientists do and what historians do; it’s all sifting through possibilities on the basis of empirical evidence.

Which is not to say that every worthwhile intellectual endeavor is a version of science in some way. Math and logic are not science, because they don’t involve steps 2 or 3. They are all about figuring out all possible ways that things could be, whether or not things actually are that way in our real world.

On the other hand, things like aesthetics and morality aren’t science either, because they require an additional ingredient — a way to pass judgment, to say that something is beautiful/ugly or right/wrong. Science doesn’t care about that stuff; it describes how the world is, rather than prescribing how it should be. You may think that there are objectively true statements one can make within these realms (“killing babies is wrong,” “Justin Bieber sucks”). But whether or not they are objectively true (they’re not, in any useful sense), they’re not scientific statements, in the way that “the universe is expanding” is a scientific statement. If they were, we could imagine worlds in which they were not true at all (“killing babies is good,” “Justin Bieber is awesome”). Those would be absolutely conceivable worlds, just not the ones in which we happened to live. And the knowledge of which world we lived in would have to come from collecting some data, just as that’s how we learned the universe is expanding.

Sometimes the fact that science is not the only kind of respectable intellectual endeavor gets packaged as the statement that there are other “ways of knowing.” This is an unhelpful framing, since it could be true or false depending on unstated assumptions held by the speaker. Yes, mathematics is a different way of gaining true knowledge than science is, so at that minimal level there are different valid ways of knowing. But they are not merely different methods of getting at the truth, they are ways of getting at different kinds of truth. What makes science (broadly construed as empirical investigation) special is that it is the unique way of learning about the contingent truths that separate our actual world from all the other worlds we might have imagined. We’re not going to get there through meditation, revelation, or a priori philosophizing. Only by doing the hard work of developing theories and comparing them to data. The payoff is worth it.

  1. Nice article.

    I have my own opinion, of course.

    – “The demarcation problem” is a philosophical non-problem of asking what is not asked elsewhere. If one would need apriori boundaries instead of observing how the process (here of doing science) works, one wouldn’t get it off the ground. Who can say, when studying interactions for example, where physics stops and chemistry oc chemical bounds start?

    – Similarly with hypothesis testing. One needs merely to ask if it works, if it is testable. (Yes, it is a meta-question that works recursively.) That the use of hypothesis testing doesn’t predict all of science is a different problem, as long as it is a testable model.

    – “That’s completely different from the examples of string theory or the multiverse, which clearly are saying something concrete about the world (the ultraviolet completion of quantum gravity, or conditions in the universe far outside our observable region), but to which we have no experimental access (or almost none).”

    Funny how when you point out that areas of no access is not a problem for testing, persons in the comments still claims palpably erroneous “non-falsifiability”. (Counter-examples: Weinberg’s prediction of the cosmological constant 1987, the Higgs scale prediction a bit later, et cetera.) If not yet accessed areas would be a problem, I think theories like quantum field theories (shields proper charge by short-lived particle-anti-particle pairs, right?) and objects like nucleons (a lot more short-lived particle-anti-particle pair building) would be inadmissible.

    The real divide in cosmology now stands between which is the better explanation of inflationary multiverses on one side and naturalness on the other. That multiverses has made testable predictions since 1987, while naturalness has no longer any overt successes may scare some, but it seems to be a fact.

    Also, since naturalness is often taken to mean a “singleverse” theory, those variants also says things on conditions in the universe far outside our observable region.

    – “As a working cosmologist, I know perfectly well that not all good science requires reproducible experiments. We haven’t made a Big Bang in the laboratory — yet.”

    Well, any observation is an, often independent experiment. And we have many such in cosmology.

    That we have a unique objection isn’t a problem elsewhere. If I find a stone, I can subject it to many reproducible experiments, without knowing about rock formation (say, knowing that the stone is a granite). The latter helps, but is not necessary.

    Of course, like in the making of stones, a process very rarely results in just one object!

  2. I have argued elsewhere (because I believe it) that mathematics has a strong empirical component. I’ll try to be briefer here. Math started with counting (I think). It is an empirical fact that if you have five goats and sell two of them, you will have three left. A while back I spent most of a year trying (as an intellectual challenge) to find an independent proof of Fermat’s Prime Theorem (the one about primes of the form 4N+1 being the sum of two squares). A lot of my investigation was empirical, generating very long series of numbers in a spreadsheet and looking for patterns and then seeing if I could prove them. I would not have been able to construct the proof I came up with if I had merely stared at my navel and thought.

    The more important example I like to cite is that Andrew Wiles’ proof of Fermat’s Last Theorem can be traced to the Taniyama–Shimura conjecture, which was a purely empirical observation that the characteristic numbers of modular functions and elliptical curves seemed to be identical sets (if I am remembering what I read in Simon Singh’s “Fermat’s Enigma” correctly).

    I think if I were trying to define science I would focus on science as a way of solving problems, rather than who is or is not a scientist. As a design engineer and as a computer programmer I try to be scientific in my approach to finding and fixing bugs in designs and programs. It seems to work a heck a lot better than whatever the second-best method is. I guess I think of a scientist as someone who has increased the sum total of human knowledge about the universe in a non-trivial way. Which eliminates engineers and plumbers, but engineers and plumbers can and should attack their problems scientifically.

    Thanks for your interesting post.

  3. Sean,
     
    Do you consider parapsychology a science? I do not expect that you do but then, why not?
     
    Parapsychologists follow the scientific method as closely as in any other field, or even much more so since they are much more careful about doing blind analyses and other experimental techniques due to the fact that their work is picked through much more by critics than in any other field. Parapsychology experiments, specifically the Auto-Ganzfeld experiment and presentiment experiments, can and have been reproduced on demand and, put together, produce highly significant results in favor of psionic effects. In fact, the statistical significances achieved for telepathy and presentiment rival many particle physics results. Most parapsychology experiments these days use highly automated systems with quantum true RNGs and have cross-confirmed their discoveries using EEG and fMRI.
     
    A good overview of recent papers can be found here,
    http://www.deanradin.com/evidence/evidence.htm
     
    In some of these papers, the signals are very strong.
     
    Now, of course, the theoretical justification for psychic effects is murky, requiring modifications to quantum mechanics. But is this any different from the case for dark matter or dark energy? And, in general, naturalist assumptions are used. So why would someone not consider parapsychology a science?

  4. I haven’t read Popper, so I don’t know if I’m echoing his comments or not. If yes, and if you consider them sufficiently refuted, my apologies.

    I would take issue with Sean’s comments about reproducibility, particularly how he analogizes it (I’m always suspicious of analogies, so please forgive my bias). He says, “…I know perfectly well that not all good science requires reproducible experiments. We haven’t made a Big Bang in the laboratory — yet.”

    I would argue that the Big Bang is not an experiment, not and observation. It is a deduction, based on observations. No one has observed the big bang. We have observed that the universe is expanding – observations which have been reproduced many times. We deduced a big bang from that, and asked what else might be out there if the big bang happened. And we observed microwave cosmic background radiation.

    I would argue that the level of reproducibility of experimental observations allows us to assign a confidence factor to any conclusions based on those observations. This may sound like Noah Smiths “4 levels of science”, but I don’t entirely think so. A natural experiment, repeated sufficiently, with sufficiently correlating results, would have a high degree of confidence, while a lab experiment, done twice, with wildly differing results would have a low degree of confidence. Cold Fusion, anyone?

    Why else do we have sigma evaluations for things like the Higgs Boson results?

    I think reproducibility of experimental observations – not results, not conclusions -is absolutely critical to science. Otherwise we leave ourselves open to mistake, misunderstanding and fraud. Global warming denialism, anyone?

  5. If a “supernatural” something effects and/or is effected by the natural world, wouldn’t this make the “supernatural” natural? I have never seen the use of the term “supernatural”…

  6. @Colin

    I think terms like “supernatural” and “paranormal” are nonsensical. If something happens in our Universe, it is by definition natural and normal and in accordance with the laws of nature. Are we dividing phenomena into “supernatural” and “natural” categories based on the ability of current science to explain them? Then is dark matter supernatural? If not, then what is the criteria? It makes no sense so I think the terms should be disposed of.

  7. Pingback: “The Cocoon of Unfalsifiability” by Daniel Bastian [GUEST POST] | the long way home | Prodigal Paul

  8. Is the question, “what is science?” a scientific question? Can we approach “science” from “science” as defined in the steps given?

    1. Think of every possible way science could be.
    2. Label each way an “hypothesis.”
    3. Look at how science actually is. Call what you see “data” (or “evidence”).

    Where possible, choose the hypothesis that provides the best fit to the data.

    This leads to the conclusion: “science is as science does.” There is a problem in step 3 insofar as we have not answered how to determine how science “actually is” (like you say, does astrology count?) so this approach does not seem to work. What does this mean? Does this mean the question is “unscientific” after all? If it does not fit into the 3 part description of what science is and how science works, then it must not be science, correct?

    So, it appears that there is something deeper than science that defines science but is not defined by science. Is this philosophy? Does it matter? And if this deeper thing is not defined by science (empirical methods) what methods are sufficient?