The end of Big Data? Based on his discussions with CIOs, reports Derrick Harris at GigaOm, Opera Solutions’ CEO Arnab Gupta “thinks the analytics market will crest around the end of next year as CIOs face enormous data spikes.” Is this what he means by “turning Big Data into Small Data?” Apparently saying “crest” is a very convincing way to get $84 million, but does he really believe that the Big Data flood is going to start tapering off next year?
Big Data Bytes of the Week: The End of Big Data?
Posted in Data Growth, Data Scientists, Predictions
Leave a comment
The Landscape of the Internet of Things
Source: Entrepreneur and Media Lab researcher David Rose talks ‘enchanted objects’
The book on Amazon: Enchanted Objects: Design, Human Desire, and the Internet of Things
Posted in Internet of Things
Leave a comment
Scenarios for the Future of the IT Industry
In November 1998, I sent to my then-colleagues at EMC an email with the subject line “The Demise of Dell.” I wrote:
“My fail-proof crystal ball just talked to me again: By the end of 2000, Dell’s market cap (today at $80B) will be cut in half.
Dell’s only strength, as we all know, is in low-cost distribution. Distribution (of everything) is going to undergo a radical change in the near future because of the Internet. There will be new players in the PC market that will figure out how to sell PCs over the Internet at half the cost of Dell’s distribution infrastructure. On top of that, the corporate PC market will grind to a halt and we may even see a slight drop in PC revenues in the year 2000. On the consumer side, appliances is where the action will be—led by new players. “
After I sent my email, Dell’s stock went on to almost double to a peak of just over $56 in March 2000. It closed yesterday at $14.09, about half of where it was in late 1998.
Posted in Predictions
Leave a comment
The Digitization of IT
In many companies today, the “consumerization of IT” is turning into the “Digitization of IT.” The spreading of consumer technologies and services into the workplace is being expanded into a larger set of IT practices, borrowed from Silicon Valley innovators and adapted to the needs of enterprises in a variety of industries.
The old IT was analog IT: A single-purpose function designed to automate specific business activities, provide support and governance, and “keep the trains running on time.” The new IT is digital: Multi-purpose, extremely flexible, weaved into every aspect of the business, and gushing with unexplored and previously unknown opportunities.
The digitization of IT means that the IT organization is both stable and innovative, fault tolerant and fast learning, reliable and experimental. It solves the paradox of “safe is risky, stable is dangerous.” It promotes a culture of constant change which ensures resilience, and experimentation which safeguards continuity. Yes, you can have the best of both worlds.
Posted in Misc
Leave a comment
Big Data: Who, Why, and How (Infographic)
“Early adopters of Big Data analytics have gained a significant lead over the rest of the corporate world. Examining more than 400 large companies, we found that those with the most advanced analytics capabilities are outperforming competitors by wide margins.”
Source: Bain & Company
Posted in Big Data Analytics, Stats
Leave a comment
Big Data Analytics and Data Science at Netflix (Video)
Chris Pouliot, the Director of Analytics and Algorithms at Netflix: “…my team does not only personalizations for movies, but we also deal with content demand prediction. Helping our buyer down in Beverly Hills figure out how much do we pay for a piece of content. The personalization recommendations for helping users find good movies and TV shows. Marketing analytics, how do we optimize our marketing spin. Streaming platform, how do we optimize the user experience once I press play. There’s a wide range of data, so theres a lot of diversity. We have a lot of scale, a lot of challenging problems. The question then is, how do we attract great data scientists that can just see this as a playground, a sandbox of really exciting things. Challenging problems, challenging data, great tools, and then just the ability to have fun and create great products.”
[youtube http://www.youtube.com/watch?v=pJd3PKm9XUk]
The Data Science Interview: Yun Xiong, Fudan University
The Goal of Data Science is to Study the Phenomena and Laws of Datanature
Yun Xiong is an Associate Professor of Computer Science and the Associate Director of the Center for Data Science and Dataology at Fudan University, Shanghai, China. She received her Ph.D. in Computer and Software Theory from Fudan University in 2008. Her research interests include dataology and data science, data mining, big data analysis, developing effective and efficient data analysis techniques for various applications including finance, economics, insurance, bioinformatics, and sociology. The following is an edited version of our recent email exchange.
How has data science developed in China? Continue reading
Posted in Data Scientists
Leave a comment
A Digital Nation (Infographic)
[Source: U.S. Census Bureau]
Big Data Observations: The Science of Asking Questions
“I am a firm believer that without speculation there is no good and original observation”—Charles Darwin
“It is the theory that determines what we can observe”—Albert Einstein
“I suspect, however, like as it is happening in many academic fields, the NSA is sorely tempted by all the data at its fingertips and is adjusting its methods to the data rather than to its research questions. That’s called looking for your keys under the light”—Zeynep Tufekci
“Large open-access data sets offer unprecedented opportunities for scientific discovery—the current global collapse of bee and frog populations are classic examples. However, we must resist the temptation to do science backwards by posing questions after, rather than before, data analysis. A scant understanding of the context in which data sets were collected can lead to poorly framed questions and results, and to conclusions that are plain wrong. Scientists intending to make use of large composite data sets need to work closely with those responsible for gathering the data. Standard scientific principles and practice then demand that they first frame the important questions, then design and execute the data analyses needed to answer them”—David B. Lindenmayer and Gene E. Likens
“The wonderful thing about being a data scientist is that I get all of the credibility of genuine science, with none of the irritating peer review or reproducibility worries… I thought I was publishing an entertaining view of some data I’d extracted, but it was treated like a scientific study… I’ve enjoyed publishing a lot of data-driven stories since then, but I’ve never ceased to be disturbed at how the inclusion of numbers and the mention of large data sets numbs criticism”—Pete Warden
Posted in Big Data Analytics, Data Science, Quotes
Leave a comment
The Big Data Debate: Correlation vs. Causation
In the first quarter of 2013, the stock of big data has experienced sudden declines followed by sporadic bouts of enthusiasm. The volatility—a new big data “V”—continues this month and Ted Cuzzillo summed up the recent negative sentiment in “Big data, big hype, big danger” on SmartDataCollective:
“A remarkable thing happened in Big Data last week. One of Big Data’s best friends poked fun at one of its cornerstones: the Three V’s. The well-networked and alert observer Shawn Rogers, vice president of research at Enterprise Management Associates, tweeted his eight V’s: ‘…Vast, Volumes of Vigorously, Verified, Vexingly Variable Verbose yet Valuable Visualized high Velocity Data.’ He was quick to explain to me that this is no comment on Gartner analyst Doug Laney’s three-V definition. Shawn’s just tired of people getting stuck on V’s.”
Indeed, all the people who “got stuck” on Laney’s “definition,” conveniently forgot that he first used the “three-Vs” to describe data management challenges in 2001. Yes, 2001. If big data is a “revolution,” how come its widely-used “definition” is based on a dozen year-old analyst note?