Every professional football game begins with the flip of a coin, to determine who gets the ball first. In the case of the Super Bowl, the teams represent the National Football Conference (NFC) or American Football Conference (AFC). Interestingly, the last 14 coin flips have been won by the NFC.
Working out the numbers, the chances of 14 coin flips in a row being equal is 1 in 8,192. (The linked article says 1 in 16,000, which comes from 2^14; but that first coin flip has to be something, so the chances of 14 in a row are really 1 in 2^13. The anomaly would be just as strange if the AFC had won every time.) That’s a better than 3.8-sigma effect! Enough to call a press conference, if this were particle physics.
The question is … is this really a signal, or did we just get lucky? Is it a fair coin and the NFC has just been the happy recipient of a statistical fluctuation, or is there something fishy about the coin? Remember Barry Greenstein’s parable about how different people compute probabilities.
And let it be a lesson the next time you’re excited about 3-sigma anomalies.
3.8 sigma could be wrong sometimes…
No no no no no you people have it all wrong.
There may be 16 correct calls in a row, but there is only one lucky streak!
I found this gem at http://www.thewaythefutureblogs.com/2012/02/bright-sayings-of-bright-people-no-25/ . It seems appropriate here.
“Statistics are like bikinis. What they reveal is suggestive, but what they conceal is vital.”
—Aaron Levenstein
The coins are not identical. There is a lot of literature on how loaded coins from different mints are.But this is besides the point.
The 14 coins did not produce the same outcome year after year, they just resulted in 14 times one winning the outcome of the flip whatever it may have been by virtue of either guessing or of the other team guessing incorrectly. Which brings me to my point:
There is the overlooked random event: which team gets to make the call for the outcome.
The patriots won the coin toss this year, so the NFC winning streak has been broken.
Pingback: Superbowl statistics | Stats Chat
To continue on the thread of Alex @ 12,
Not only might one ask what are the odds that it will happen in the history of the Superbowl instead of the last 14 years (prior to this year), but what are the odds that such a string of coin flips would happen in a series of high-profile games. It would be just as noticed if it happened at the national-championship game for college football. It would also be noticed if a single team won (or lost) 14 coin tosses in a row. Of course we also pay attention to many other sports stats as well.
So the odds that such a coincidence will astound us are actually quite good — indeed all but certain.
I assume that the particle physics guys at the LHC are not just looking any old statistical anomaly in their millions of data points as they WILL find one. Rather the theory must say what it expects to find. When they calculate the sigma for the proposed Higgs observations, are they calculating the odds of finding anywhere in the proposed range of energies or just the odds it will be found where it was found? I hope it is the former. But even if it is not, the vast majority of statistical quirks that one one might dig out of the LHC data are not what Higgs hypothesis predicts. There is a big difference between noticing a statistical quirk and predicting one in advance.
32,766 throws, on average, to have that 14-sequence appear.
Pingback: Signal for Higgs Boson Particle Gains Strength | Journal of Technology and Economic Development | Future Technology | Green Technology | Military Technology | Business | Trading | Finance | Computer | Robots | Entertainment | Games | GPS | Software | Musi
That 3.8sigma signal is a ‘local’ figure. The analysis should take account of the ‘look elsewhere effect’ – that’s what the particle guys do. It’s exactly as childermass explains @32
See http://cms.web.cern.ch/news/should-you-get-excited-your-data-let-look-elsewhere-effect-decide
Just to spell out the point made by Rich Townsend’s link:
Three-sigma results will be observed in about 3 draws out of 1,000 from a normal distribution. So if you form a null hypothesis about a particular data-generating process (DGP) (e.g. a fair coin is being flipped), draw a sample from it, and observe a three-sigma departure from the predicted value, then if your prior probability for the alternative to the null (that it is not a fair coin toss, in this case) was not substantially less than one in 300, then you should seriously consider rejecting the null and accepting the alternative. Most of the comments here are various suggestions about what rejecting the null here would mean concretely.
But! This is not the situation described by this post. We did NOT first form a hypothesis about the DGP and then draw a sample from it. Rather, we are only talking about coin flips because we *already* observed the anomalous result. So the relevant question is, given the universe of comparably salient DGPs (in the Super Bowl, in professional sports, among stuff going on this past weekend, among stuff Sean might plausibly blog about — whatever we decide the relevant universe is) what is the probability of a three-sigma result being observed among at least one of them? And given that, for any reasonable definition of the potential universe of DGPs, there are far more than 300 of them, the answer is going to be close to 100%.
Sean gets this, obviously — the post was clearly tongue in cheek. But it seems that many commenters here don’t.
As for the application to physics, that’s way above my pay grade. But it depends whether (or to what extent) the sample with the three-sigma anomaly is drawn from a DGP identified as of interest on prior theoretical grounds, or whether it’s the result of a fishing expedition. Rich T.’s link suggests it’s often the latter; I don’t have any idea if that’s right, but it’s the right question.
(Sorry, I know we’re talking about 3.8 sigma, not 3 sigma. But I don’t think there’s any question that the universe of possibly salient DGPs — even just within the Superbowl — is considerably larger than 8,000. Or, what Childermass said.)
Pingback: Higgs signal gains strength » imcity.ir
I probably observed several sigma 5 results just walking to work today, though I wasn’t aware of it.
Pingback: Two Inform CanadaHiggs Particle is as Good as found - Two Inform Canada
Pingback: Higgs signal gains strength | science
As a street scientist, I would like to point out that this analysis would all benefit from using some Bayesian analysis. What is the pre-DGP chance of the NFC winning 14 times in a row? One would need to know the odds of one team winning a coin toss each game. Theoretically 50:50, but not necessarily, based on the unmeasurable variables that go into calling tosses and biased coins, and the performance of each team within that set of variables. Add to that the occurrence of statistical anomalies in real life, and I suspect one gets a high chance of the 14 wins in a row. One knows this analysis is likely correct, because no one has asked for a new kickoff determining methodology.
The Sigma for Higgs taken in context of the Bayesian analysis of what the pre-test expectations for the experiment are, and accounting for 2 separate experiences coinciding makes the chance of the phenomenon being real much higher than simply saying that they are likely to be wrong 1/1000 times, and that kind of error is pretty common.
By the way, same goes for the superluminal neutrinos. There are more than one set of experiments revealing the same result, which raises the Bayesian chance of it being a true finding. That does not mean that they beat the speed of light, only that they traverse known space faster than photons can. There may be other reasons besides speed that they do that (like unknown space shortcuts only they have access to). Either way, the street would say the odds of the findings being real are higher than a single result would indicate.
Uh.. Sean. I hate to admit I know more about football than another person, but since it is you: statistics aside, it is simply not true that “Every professional football game begins with the flip of a coin, to determine who gets the ball first.”
The winner of the toss gets to DECIDE whether to kick or receive.
Despite what people say about gifts, NFL coaches usually find it more blessed to receive.
@36 JW Mason: Richard Feynman skewered this problem as well:
“You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won’t believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!” — Six Easy Pieces