Swing states are dead, but did Big Data pull the trigger?

A swing state: “A state in the US in which democratic and republican candidates both have a good chance of winning”. But what if predictions are so accurate that you get all 50 states right? That’s the end of the swing state, because there’s nothing to swing anymore. The man who killed them is Nate Silver. This is what he predicted and there is no need to show the map with actual results, because that would be the same map. Nate Silver is a data cruncher and author of The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t.  He’s a former baseball statistician. He’s the one who showed that when you look at the data it’s relatively easy to predict who is going to win. Silver is also the one that proved that a lot of what was going on in the mass media was just data entertainment. The pundits are the “sultans of swing” and people love to be entertained, but it has nothing to do with real predictive analyses. We don’t have to be entertained about “will Florida go the Democrats”. If you want to know for sure you better look at the statistics. Data pulled the trigger of the swinging of states, but was it Big Data or Small Data? The best way to describe what Nate Silver did was to interpret polls predictively. If you’re a data cruncher yourself you might be interested to read the posting on his blog in which he explains his methodology. In short: he used other predictions to predict. The most important input for his predictions was curated data coming from the other polls. By weighting and adjusting the polls and not looking at the outliers of the different polls but the consensus he was able to do the trick. Some factors: Recency – More recent polls are weighted more heavily Sample size – Polls with larger samples receive more weight Pollster rating – Pollsters committed to disclosure and transparency standards receive more weight Trendline adjustment – If old polls haven’t been replaced, they are adjusted to reflect the overall trendline House effects – Some polls tilt right, some left, and this adjustment mitigates those effects Likely voter adjustment – Polls of likely voters are given a lot of credence The Big Data fundamentalists say this is not Big Data. The fact that Silver used his laptop to do the analysis was seen as the final proof. But does it matter? Maybe a decade from now we conclude that Big Data was just a word that helped us looking a data in a different way. It helped us to predict in an early stage who’s going to win. The word “swing state” disappears because “a good chance of winning” is redefined. Nothing else to conclude than that the real victim of (big) data is entertainment and excitement of what could happen. Here’s a video by the New York Times on how Nate Silver got it right