Big Data and Algorithms Rule!

The proper interpretation and linkage of data already leads to better decision-making, more sales, fewer risks and cost reduction. However . . . ‘We are still in the early, black-and-white-tv stage of Social Analytics . . .’ Big Social Today Is Still Like B&W TV This is the most respected view of Paul Barrett, Customer Management Director at Teradata. Let us examine what that literally means. Black-and-white was the tv period in which there were few channels and we required different antennas to receive them. Very often the only thing to see on television was ‘snow’. We were troubled by ‘atmospheric disturbance’, and saw only ‘snow’ (or ‘noise’) if a strong wind had turned the aerial a little. It was far from being the ideal situation, with only gray tints to represent a colorful world. And, we could only see the same program at fixed times. The Commercial Big Data Challenge It would be an exaggeration to denunciate Big Social in a comparable way, because modern Social Analytics is in far better shape. But things could be better: a single antenna please, sharper picture, more details, more channels and sources, real time, various angles, more aggregation levels, pattern recognition and, above all: the ability to predict behavior. We need to know what people really want, serve them in a timely way, bind them to us, and build up relationships with and via them. This striking improvement is the commercial Big Data challenge for organizations. A Faster and Clearer Picture If a dataset is large enough, and up to date, and above all relevant, the empirical approach often works better than a formula. We could formulate a complex model to determine how any people will go down with flu, but investigating search results produces a faster and clearer picture. Gunther Eysenbach, a professor at the University of Toronto was the first to do so, in 2006. His conclusion back then: ‘The Internet has made measurable what was previously immeasurable: the distribution of health information in a population, tracking (in real time) health information trends over time, and identifying gaps between information supply and demand.’ Most Answers Are Latent in Large Data Sets The same applies to many other things, such as the best pricing strategy for selling second-hand articles. We find that immediately in eBay data, which gives better insight into matters such as inflation and consumer confidence. In short, all kinds of answers are latent in large data sets, and we can uncover these without having to concern ourselves with models. The End of Theory As far back as 2008, the start of the first Obama term, Chris Anderson of Wired magazine spoke provocatively about a Big Data vista, in which even theory-forming and the scientific method would become superfluous; but, of course, data cannot speak for itself. At most, empiricism and theory play leapfrog and, thanks to the data explosion, the emphasis currently lies on data and algorithms rather than on traditional models. The Algorithm Is the Model This development has been ongoing for a number of years now; compare, for example, the statistical approach to that of machine learning: ‘Statisticians emphasize probabilistic models for learning, and techniques for quantifying variation in the estimated model that results from variation in the learning sample. For many machine learners, the algorithm is the model, and emphasis is placed on developing interpretable yet flexible methods of learning in challenging context (computer vision, natural language).’ Download Our “Big Social” Report Data and algorithms rule! Just read “Big Social: Predicting behavior with Big Data,” our second research report on Big Data, that is now available. It offers a multi-faceted orientation into next-generation Social Analytics and Social Media by presenting the rapid developments, the analysis of available tools, best practices and inspiring cases. [DOWNLOAD]