The end of Big Data? Based on his discussions with CIOs, reports Derrick Harris at GigaOm, Opera Solutions’ CEO Arnab Gupta “thinks the analytics market will crest around the end of next year as CIOs face enormous data spikes.” Is this what he means by “turning Big Data into Small Data?” Apparently saying “crest” is a very convincing way to get $84 million, but does he really believe that the Big Data flood is going to start tapering off next year?
Participants in the O’Reilly Strata conference on Big Data think not, as Marshall Kirkpatrick at ReadWriteWeb reports under the convincing title “Bankers go bonkers over Big Data’s future”: “Huge opaque markets are about to become transparent because of new regulations and that means a whole lot of new data available for analysis. Scalable processing of that data will require outsourcing, giving birth to new industries. Millions of people will need to be trained to deal with all this.”
Indeed, outsourcing could be good way to raise a glass or two of Big Data, for those who are not going bonkers and are in general slow to catch up with new developments (most enterprises?). But “outsourcing” is outmoded, so Brad Howarth at CIO Magazine employs the more fashionable “data-as-a-service” (DaaS), discussing companies such as Infochimps and Factual ho do all the dirty data work for you. What really got my attention, however, was the quote from Dennis Claridge, cofounder of Melbourne-based doubleIQ, summing up nicely the state of data today: ”[Enterprises] don’t make as much use of their information as they could, and it gets lost over time. Data’s never valued correctly in an organisation.”
In contrast, says DJ Patil on O’Reilly Radar, “I’ve found that the strongest data-driven organizations all live by the motto ‘if you can’t measure it, you can’t fix it’ (a motto I learned from one of the best operations people I’ve worked with). This mindset gives you a fantastic ability to deliver value to your company by:
• Instrumenting and collecting as much data as you can. Whether you’re doing business intelligence or building products, if you don’t collect the data, you can’t use it.
• Measuring in a proactive and timely way. Are your products, and strategies succeeding? If you don’t measure the results, how do you know?
• Getting many people to look at data. Any problems that may be present will become obvious more quickly — ‘with enough eyes all bugs are shallow.’
• Fostering increased curiosity about why the data has changed or is not changing. In a data-driven organization, everyone is thinking about the data.”
As the chart above shows, there are more and more data-driven organizations hiring more and more data scientists and using analytics to find value in the data.
