English | EN Big Data Review – Big Data community whitepaper: Big data as the present day synonym of Business Intelligence
Jun 18, 2012
In our research on Big Data we come across a lot of interesting reading material. In our Big Data Reviews we share some of our most valuable reads and point out some of the key concepts or statements that these sources provide.
A group of researchers from across the USA have written a collaborative whitepaper about Big Data to establish some industry consensus on the theme. The whitepaper is interesting, because it views Big Data through the eyes of the data process: from how it is collected to how it is used. It then describes the changes and challenges in the case of big data.
The big data process has similarities to the processes we had when we still called it BI, and when the terms ‘extract – transform –load’ where very much in style. Yet, at the same time, much has changed and many new themes have become urgent: scale, timeliness, heterogeneity, but also privacy and human collaboration. In that respect, an important element seems to be missing in the diagram above: to take action and measure the outcomes of these actions – the proof of true data driven management: not only use data to make decision, but also verify if the decisions and actions have the predicted effect.
Part of the discussion turns to the data itself: can you trust the data at its most elementary level? Is what people have entered correct, or is their deception or fraud? The more steps there are between data and the interpretation, the trend or pattern that forms the basis of a decision, the harder it becomes to track the quality of the data, and therefore the reliability of your insight. The example given is that perhaps unemployed people are less likely to report their employment status, so how would you make decisions based on anything related to ‘employment’ without first doing further (thorough) analyses into the elementary data. This becomes especially relevant if we increasingly try to tap into other data sources such as social media, profiles and online behavior: how much of this is true?
One of the conclusions of the paper is straightforward: we are entering a new era that is yet uncharted. More research is needed to learn how to deal with these new uncertainties and opportunities. In that sense, it’s BI, but not as we know it.
About the author
