Data is eating the world, continuing to impact every aspect of our lives. Here’s the data on the state-of-data in June 2021, from current attitudes towards self-driving cars to the dramatic reduction in the price of lithium-ion batteries supporting the creation and consumption of data everywhere, including work from home, to the data lost in cybersecurity attacks.
IDC predicts that 175 trillion gigabytes of new data will be created worldwide in 2025
[July 23, 2109]
Data is eating the world. All businesses, non-profits, and governments around the world are now in full digital transformation mode, figuring out what data can do to the quality of their decisions and the effectiveness of their actions. In the process, they tap into IT resources and landscape that have changed dramatically over the last decade, offering unprecedented choice, flexibility, and speed, facilitating the management of data eating work.
Launched in 2012, DraftKings is a prime example of a new breed of data-driven, perpetually-learning companies. One of the few players in the market for fantasy sports, it has faced “unique challenges that haven’t been solved by other businesses yet,” says Greg Karamitis, Senior Vice President of Fantasy Sports. To solve these challenges, “we have to lean on our analytical expertise and our ability to absorb and utilize vast amounts of data to drive our business decisions.”
Founded by serial entrepreneur Ash Ashutosh 10 years ago, Actifio is a prime example of the new breed of IT vendors transforming the IT landscape from a processor-centric to data-centric paradigm, from a primary emphasis on the speed of computing to a new focus on the speed of accessing data. “It used to be that only the backup people cared about data,” says Ashutosh. “Then it was the CIO, and later, the Chief Data Officer or CDO. Now, every CEO is a data-driven CEO. If you are not data-driven, most likely you are not the CEO for long.”
Data “as a strategic asset” was the vision driving Ashutosh and Actifio in 2009, bringing to the enterprise the same attitude towards data that has made the fortunes of consumer-oriented, digital native companies such as Amazon. “We wanted to facilitate getting to the data as fast as possible, to make it available to anybody, anywhere,” recalls Ashutosh. Since only backup people cared about enterprise data 10 years ago, they were the customers Actifio initially targeted.
The value proposition for these customers centered on reduced cost, as Actifio helped them maintain only one copy of any piece of data, available for multiple uses, instead of maintaining numerous copies, each one for a specific application or data management activity. Actifio achieved this magic trick (e.g., reducing 50 terabytes of data to only 2 terabytes) by capitalizing on another trend shaking the tech world 10 years ago, virtualization. Replacing analog data with digital data gave rise to data-driven companies and replacing the physical with the logical—virtualization—created a new IT landscape.
It took a while for enterprises to adapt to the new IT realities. But around 2015 the cloud flood gates opened because of the business pressures to do everything faster and faster, especially the development of new (online) applications, and the fact that more and more enterprise roles and activities required at the very least some creation, management, manipulation, analysis, and consumption of data. These changes manifested themselves in Actifio’s business. In its most recent quarter (ended April 30, 2019), “60% of our customers used Actifio to accelerate application development, up from close to 0% in 2016,” reports Ashutosh, “and over 30% of our workloads today are in cloud platforms.”
The new attitude towards data as a strategic asset and the widespread availability of cloud computing have opened up new uses for Actifio’s offerings such as compliance with data regulations and near real-time security checks on the data. But possibly the most important recent development is the increased use by data scientists for machine learning and artificial intelligence-related tasks.
A very significant chunk of data scientists’ time (and their most popular complaint) is the time they spend on data preparation. And a significant chunk of the time spent on data preparation is simply waiting for the data. In the past, they had to wait between eight to forty days for the IT department to deliver the data to them. Now, says Ashutosh, “they have an automated, on-demand process,” providing them data from the relevant pool of applications, in the format they require. Bringing up the new term “MLOps” (as in machine learning operations), Ashutosh defines it as “allowing people to make decisions faster by not having data as a bottleneck.” The end result? “The more you give people access to data in self-service way, the more they find new and smarter ways of using it.”
As 40% of Actifio’s sales come from $1 million-plus deals, these new uses of data are not “tiny departmental stuff,” says Ashutosh. “Large enterprises are beginning to use data as a strategic asset, as a service, on premises or in multi-cloud environments.”
Large enterprises today learn from, compete with, and often invest in or acquire the likes of DraftKings, a startup that runs on data. Growing the business for DraftKings means making their contests bigger and bigger. “But if we make our contests too big and we don’t get enough users to fill 90% of the seats, we start to lose money really fast,” says DraftKings’ Karamitis. Balancing user satisfaction and engagement with the company’s business performance requires accurate demand predictions for each of the thousands of contests which DraftKings runs daily in a constantly changing sports environment.
“We need to absorb tons of data points to figure this out on a daily basis,” explains Karamitis. But these data points are not only based on DraftKings accumulated experience with past contests. Befitting a business living online, another important data source is social networks—“our users are giving us enormous amount of data in terms of what they are twitting to us, what they engage with and what they don’t, allowing us to understand better which ways we want to shift as a company and which way we want to build a product,” says Karamitis.
There is yet another source of the data that is driving decisions at DrafKings, possibly the most important one: The data DraftKings creates by constantly experimenting. “We create data points by running structured tests,” says Karamitis. They cannot run A/B tests, he explains, because running smaller size contests will not produce the same effect of larger size contests and because their users tend to communicate a lot among themselves and compare notes about their experiences. “We are willing to take the risk testing our underlying beliefs around user behavior,” says Karamitis, by changing the top prize or changing the marketing treatment, for example.
In the past, domain expertise was the key to a company’s success. Today, it is data expertise, and the skill of applying it instantly to new opportunities as they arise. “Our analytical expertise allows us to learn really fast, learn in the traditional meaning of learning from experience and figuring out what matters to our users and also learn in the more modern sense of building machine learning algorithms around the key principles our users care about,” says Karamitis. Learning from data has become a core competency that can be applied to new markets, a competency that should serve DraftKings well as it pursues the new business opportunity of legalized sports betting.
DraftKings—and the new breed of data-driven companies—are data science labs, creating data and acquiring new insights with continuous experimentation. Like good scientists, they test their hypotheses through carefully structured experiments, challenging their own assumptions about customers and markets. Karamitis recalls the start of the 2017 NFL season when DraftKings offered a new contest site: ”We had a very specific expectation as to who it’s going to appeal to and how big it will be. We were totally wrong, 100% wrong.” But because the new product was developed through experimentation, the data led to what is now a “super valuable product, so different from what we initially offered.”
Ashutosh predicts that over the next few years we will see a bifurcation of the economy into two segments, one focused on producing physical assets and the other comprised of data-driven companies. Like DraftKings, these companies, whether startups or established enterprises, will view data as a strategic asset and its analysis as a core competency and a key competitive differentiator.
And like DraftKings, these companies will increasingly resemble scientific labs, continuously learning through experimentation and creating new data points. Data growth will drive business growth as data continues eating the world.
Originally published on Forbes.com
“Let us cultivate the mathematical sciences with ardor, without wanting to extend them beyond their domain; and let us not imagine that one can attack history with formulas, nor give sanction to morality through theories of algebra or the integral calculus”–Augustin-Louis Cauchy, 1821, quoted by Matthew Jones, Columbia University
“…the common language of business is not going to be Chinese or Spanish. It’s going to be math”–Michael Rhodin, IBM
“The future is going to be owned by people who are comfortable in the quant world but have deep business knowledge”–Christine Poon, Max M. Fisher College of Business, Ohio State
“[One false promise that some proponents of Big Data hold out is that somehow vast oceans of digital data can be sifted for nuggets of pure enterprise gold.] It is not going to happen magically. The software only finds correlations, not causations. In order to find causal relationships you have to do work. If you take any sufficiently large data sets, you are going to find correlations. You need a human in the loop to work out which are important”–Stephen Sorkin, Splunk
Continue reading“With big data, you have only two concerns, but they are, naturally, big ones: where the data will come from and what your company will do with it. Solve these and you have big data licked… IT projects have to be fully buzzword-compliant or they’ll fail. For a big data project, this means Hadoop. If you don’t want to invest staff time and energy learning this technology, do what my client did: Build a virtual server, install MySQL on it, and assign the name “Hadoop” to the server. When your BDSC (big data steering committee) asks if you’ve installed Hadoop, you can answer in the affirmative with a clear conscience”—Bob Lewis Continue reading
“I am a firm believer that without speculation there is no good and original observation”—Charles Darwin
“It is the theory that determines what we can observe”—Albert Einstein
“I suspect, however, like as it is happening in many academic fields, the NSA is sorely tempted by all the data at its fingertips and is adjusting its methods to the data rather than to its research questions. That’s called looking for your keys under the light”—Zeynep Tufekci
“Large open-access data sets offer unprecedented opportunities for scientific discovery—the current global collapse of bee and frog populations are classic examples. However, we must resist the temptation to do science backwards by posing questions after, rather than before, data analysis. A scant understanding of the context in which data sets were collected can lead to poorly framed questions and results, and to conclusions that are plain wrong. Scientists intending to make use of large composite data sets need to work closely with those responsible for gathering the data. Standard scientific principles and practice then demand that they first frame the important questions, then design and execute the data analyses needed to answer them”—David B. Lindenmayer and Gene E. Likens
“The wonderful thing about being a data scientist is that I get all of the credibility of genuine science, with none of the irritating peer review or reproducibility worries… I thought I was publishing an entertaining view of some data I’d extracted, but it was treated like a scientific study… I’ve enjoyed publishing a lot of data-driven stories since then, but I’ve never ceased to be disturbed at how the inclusion of numbers and the mention of large data sets numbs criticism”—Pete Warden
“Big data is what happened when the cost of storing information became less than the cost of making the decision to throw it away”—George Dyson (quoted by Tim O’Reilly)
“If the engineers have their way, every idea, memory, and feeling—the recorded consciousness of a single lifetime—will be stored in the cloud… ‘Information overload’ once referred to the difficulty of absorbing intelligently the data produced by others. Now we face the peril of choking on our own…By remembering everything, we may become haunted by our pasts and immobilized by digital distractions—or we may gain new powers to prevent the bad and promote the good”—G. Pascal Zachary
“[I]n a world where massive datasets can be analysed to identify patterns not easily identified using simpler analogue methods, what happens to genius of the Einstein variety?
Genius is about big ideas, not big data. Analysing the attributes and characteristics of anything is guaranteed to find some patterns. It is inherently a theoretical exercise, one that requires minimal thought once you’ve figured out what you want to measure. If you’re not sure, just measure everything you can get your hands on. Since the number of observations — the size of the sample — is by definition huge, the laws of statistics kick in quickly to ensure that significant relationships will be identified. And who could argue with the data?
Unfortunately, analysing data to identify patterns requires you to have the data. That means that big data is, by necessity, backward-looking; you can only analyze what has happened in the past, not what you can imagine happening in the future. In fact, there is no room for imagination, for serendipitous connections to be made, for learning new things that go beyond the data. Big data gives you the answer to whatever problem you might have (as long as you can collect enough relevant information to plug into your handy supercomputer). In that world, there is nothing to learn; the right answer is given…
What if Albert Einstein lived today and not 100 years ago? What would big data say about the general theory of relativity, about quantum theory? There was no empirical support for his ideas at the time — that’s why we call them breakthroughs.
Today, Einstein might be looked at as a curiosity, an ‘interesting’ man whose ideas were so out of the mainstream that a blogger would barely pay attention. Come back when you’ve got some data to support your point”—Sidney Finkelstein
“Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it…”—Dan Ariely
“I’m a data janitor. That’s the sexiest job of the 21st century. It’s very flattering, but it’s also a little baffling”–Josh Wills, a senior director of data science at Cloudera
“Given enough data, everything is statistically significant”–Douglas Merrill
CB Insights: Media buzz around AI, robotics, and automation increased significantly towards the end of 2016.
The forward march of automation technologies — which include hardware (e.g. robots, digital kiosks), software (e.g. AI), and customer self-service (e.g. mobile ordering) — continues to reshape the world economy. Automation has already begun to reshape every company’s workforce, including yours. Leaders across all roles, companies, and verticals are taking note; right now, my report The Future of Jobs, 2027: Working Side-by-Side with Robots is one of the five best-read among all reports at Forrester. We forecast a world in which automation cannibalizes 17% of US jobs by 2027, partly offset by the growth of 10% new jobs from the automation economy. Most importantly, we see human-machine teaming as a key workforce trend in the future, as more and more human employees find themselves working side-by-side with robotic colleagues.
Steve Crowe, Editor of The Robot Report, catches up with 10-year-old robotics prodigy Michael Wimmer at the Robotics Summit & Expo 2019.
Developed at the Artificial Intelligence Center of the Stanford Research Institute (SRI) from 1966 to 1972, SHAKEY was the world’s first mobile intelligent robot. According to the 2017 IEEE Milestone citation, it “could perceive its surroundings, infer implicit facts from explicit ones, create plans, recover from errors in plan execution, and communicate using ordinary English. SHAKEY’s software architecture, computer vision, and methods for navigation and planning proved seminal in robotics and in the design of web servers, automobiles, factories, video games, and Mars rovers.”
Read more here
