The 4 research reports on Big Data have been quite a success. Now, you can read the whole story. VINT combined their latest report updates in one volume: No More Secrets with Big Data Analytics. The book provides an overview of the new big data reality, how to unlock your own Big Data potential and how to leverage social data and privacy aspects. No more Secrets provides the basis for updating or refining your understanding of Big Data Analytics and for exploring new ground. The first part sheds light on the Big Data phenomenon in general. Part 2 presents ample suggestions for determining your specific Big Data potential. These you can readily apply to gain insight in what exactly makes your customers tick using social data, the topic of part 3. The triad of privacy, technology and the law concludes the book. The desire of charting Your Big Data Potential arises from the purposeful data focus on the combination of business, organization and technology. The book helps businesses with answering ten questions to create a big data strategy: Question 1 Why Big Data intelligence? Question 2 What new insights can I expect? Question 3 How will these insights help me? Question 4 What skills do I need? Question 5 How do Big Data pioneers organize data management and IT processes? Question 6 How can I merge my structured and unstructured data? Question 7 Which new technologies should I be watching? Question 8 What is looming on the horizon? Question 9 What does this mean in organizational terms? Question 10 How does this affect everyday life? Transformative Big Data initiatives begin with “magic moments”: by choosing a domain in which your organization wishes to excel, while taking into account the risks and side effects. Performance Big Data initiatives are directed to existing projects with the aim of improving the performance. With No More Secrets with Big Data Analytics VINT aims to create clarity by putting experience and vision in perspective: independent and supported by examples.
Download the book by clicking the cover
What differs between the big data analysis and ordinary statistics?
Supervised learning- regression?
Unsupervised learning- singular value decomposition?
Transduction-training and test set?
Reinforcement-optimizarion of model?
Sparse-variable selection?
In the unstructured big data, is anything more predictive than a linear model in all variables?
Hi Sven, Statistics is extremely relevant to Big Data Analytics. To answer you AND remain intelligible for our main audience, I refer to this here article: Big Data and the Role of Statistics > http://community.amstat.org/Blogs/BlogViewer/?BlogKey=737fd276-0225-4c87-b7cb-0cfc7cd9e124
Online, you’ll find courses from Stanford among others > http://www.kdnuggets.com/2013/03/stanford-data-mining-statistics-courses-online.html
Regards, Jaap
Oh yes, Sven, I also remember these remarks from the Dutch Statistics Bureau CBS >
Re: Big Data and statistics
• Preparing Big data for statistics is time consuming
• Exploration phase takes a lot of time
• Try to reduce amount of data without losing information (‘making big data
small’, noise reduction)
• Risk: ‘garbage in’ ‘garbage statistics out’
• Traditional approach does not suffice
• Big data sources are definitely not ‘large’ sample surveys or admin data
• Often a selective but a large part of the ‘population’ is included
• Events are registered, not units!
• Careful with using ‘traditional’ statistical analysis (everything is significant!)
• More need for:
• Visualisation methods (to rapidly gain insight)
• Methods & models specific for large dataset (fast and ‘robust’)
• Learn from ‘computational statistics’ & (try to) use dedicated hardware
• Beware of privacy issues!
Source: http://omegate.astro.rug.nl/~target_conference/presentations/Splinter5B/Piet_Daas.pdf