Source: CB Insights Trends
McKinsey Updates Estimates of Big Data Potential Value
Source: McKinsey, 2011
[In 2011], we estimated the potential for big data and analytics to create value in five specific domains. Revisiting them today shows uneven progress and a great deal of that value still on the table (exhibit). The greatest advances have occurred in location-based services and in US retail, both areas with competitors that are digital natives. In contrast, manufacturing, the EU public sector, and healthcare have captured less than 30 percent of the potential value we highlighted five years ago. And new opportunities have arisen since 2011, further widening the gap between the leaders and laggards.
Source: McKinsey, 2016
10 Most Successful Big Data Technologies
As the big data analytics market rapidly expands to include mainstream customers, which technologies are most in demand and promise the most growth potential? The answers can be found in TechRadar: Big Data, Q1 2016, a new Forrester Research report evaluating the maturity and trajectory of 22 technologies across the entire data life cycle. The winners all contribute to real-time, predictive, and integrated insights, what big data customers want now.
Here is my take on the 10 hottest big data technologies based on Forrester’s analysis:
- Predictive analytics: software and/or hardware solutions that allow firms to discover, evaluate, optimize, and deploy predictive models by analyzing big data sources to improve business performance or mitigate risk.
- NoSQL databases: key-value, document, and graph databases.
- Search and knowledge discovery: tools and technologies to support self-service extraction of information and new insights from large repositories of unstructured and structured data that resides in multiple sources such as file systems, databases, streams, APIs, and other platforms and applications.
- Stream analytics: software that can filter, aggregate, enrich, and analyze a high throughput of data from multiple disparate live data sources and in any data format.
- In-memory data fabric: provides low-latency access and processing of large quantities of data by distributing data across the dynamic random access memory (DRAM), Flash, or SSD of a distributed computer system.
- Distributed file stores: a computer network where data is stored on more than one node, often in a replicated fashion, for redundancy and performance.
- Data virtualization: a technology that delivers information from various data sources, including big data sources such as Hadoop and distributed data stores in real-time and near-real time.
- Data integration: tools for data orchestration across solutions such as Amazon Elastic MapReduce (EMR), Apache Hive, Apache Pig, Apache Spark, MapReduce, Couchbase, Hadoop, and MongoDB.
- Data preparation: software that eases the burden of sourcing, shaping, cleansing, and sharing diverse and messy data sets to accelerate data’s usefulness for analytics.
- Data quality: products that conduct data cleansing and enrichment on large, high-velocity data sets, using parallel operations on distributed data stores and databases.
Forrester’s TechRadar methodology evaluates the potential success of each technology and all 10 above are projected to have “significant success.” In addition, each technology is placed in a specific maturity phase—from creation to decline—based on the level of development of its technology ecosystem. The first 8 technologies above are considered to be in the Growth stage and the last 2 in the Survival stage.
Forrester also estimates the time it will take the technology to get to the next stage and predictive analytics is the only one with a “>10 years” designation, expected to “deliver high business value in late Growth through Equilibrium phase for a long time.” Technologies #2 to #8 above are all expected to reach the next phase in 3 to 5 years and the last 2 technologies are expected to move from the Survival to the Growth phase in 1-3 years.
Finally, Forrester provides for each technology an assessment of its business value-add, adjusted for uncertainty. This is based not only on potential impact but also on feedback and evidence from implementations and market reputation. Says Forrester: “If the technology and its ecosystem are at an early stage of development, we have to assume that its potential for damage and disruption is higher than that of a better-known technology.” The first 2 technologies in the list above are rated as “high” business value-add, the next 2 as “medium,” and all the rest “low,” no doubt because of their emerging status and lack of maturity.
Why did I add to the list of hottest technologies two that are still in the Survival phase—data preparation and data quality? In the same report, Forrester also provides the following data from its Q4 2015 survey of 63 big data vendors:
What is the level of customer interest in each of the following capabilities? (% answering “very high”)
Data preparation and discovery 52%
Data integration 48%
Advanced analytics 46%
Customer analytics 46%
Data security 38%
In-memory computing 37%
While Forrester predicts that a few standalone vendors of data preparation will survive, it believes this is “an essential capability for achieving democratization of data,” or rather, its analysis, letting data scientists spend more time on modeling and discovering insights and allowing more business users to have fun with data mining. Data Quality includes data security from the table above, in addition to other features ensuring decisions are based on reliable and accurate data. Forrester “expects that data quality will have significant success in the coming years as firms formalize a data certification process. Data certification efforts seek to guarantee that data meets expected standards for quality; security; and regulatory compliance supporting business decision-making, business performance, and business processes.”
“Big Data” as a topic of conversation has reached mainstream audiences probably far more than any other technology buzzword before it. That did not help the discussion of this amorphous term, defined for the masses as “the planet’s nervous system” (see my rant here) or as “Hadoop” for technical audiences. Forrester’s report helps clarify the term, defining big data as the ecosystem of 22 technologies, each with its specific benefits for enterprises and, through them, consumers.
Big data, specifically one its attributes, big volume, has recently gave rise to a new general topic of discussion, Artificial Intelligence. The availability of very large data sets is one of the reasons Deep Learning, a sub-set of AI, has been in the limelight, from identifying Internet cats to beating a Go champion. In its turn, AI may lead to the emergence of new tools for collecting and analyzing data.
Says Forrester: “In addition to more data and more computing power, we now have expanded analytic techniques like deep learning and semantic services for context that make artificial intelligence an ideal tool to solve a wider array of business problems. As a result, Forrester is seeing a number of new companies offering tools and services that attempt to support applications and processes with machines that mimic some aspects of human intelligence.”
Prediction is difficult, especially about the future, but it’s a (relatively) safe bet that the race to mimic elements of human intelligence, led by Google, Facebook, Baidu, Amazon, IBM, and Microsoft, all with very deep pockets, will change what we mean by “big data” in the very near future.
Originally posted on Forbes.com
The Data Index, March 2021
Data is eating the world and there are numerous indicators of its ubiquitous presence in our lives.
2018 Predictions for AI, Big Data, and Analytics
A recent Forrester Research report, Predictions 2018: The Honeymoon For AI Is Over, predicts that in 2018 enterprises will finally move beyond the hype to recognize that AI requires hard work—planning, deploying, and governing it correctly.
But Forrester also promises improvements: Better human and machine collaboration due to improved interfaces; enhancing business intelligence and analytics solutions by moving resources to the cloud; new AI capabilities facilitating the redesign of analytics and data management roles and activities and driving the emergence of the insights-as-a-service market.
Posted in Artificial Intelligence, Big Data, Predictions
Leave a comment
Graduate Programs in Data Science and Big Data Analytics
Graduate Programs
in Data Science and Big Data Analytics
Last updated: July 9, 2019
See below for online programs and courses and other options and resources
Aegis School of Business, Data Science & Telecommunication (India)
Post Graduate Program in Business Analytics and Big Data
American Sentinel University
Master of Health Care Informatics
Arizona State University
Barcelona Graduate School of Economics (Spain)
Barcelona Technology School (Spain)
Ben-Gurion University of the Negev (Beer-Sheva, Israel)
MSc in Information Systems Engineering with focus on Data Mining and Business Intelligence
Bentley University
Berlin School of Economics and Law (Germany)
Master of Science in Business Intelligence and Process Management
Brown University
Carnegie Mellon University
Master of Computational Data Science
Catholic University of America
The Master of Science in Business Analysis
Central Connecticut State University
Master of Science in Data Mining
Graduate Certificate in Data Mining
Central Michigan University
Graduate Certificate in Data Mining
City University London (UK)
Clarkson University
Columbia University
Master of Science in Data Science
Master of Science in Information and Knowledge Strategy
Certification of Professional Achievement in Data Sciences
Cornell University
Dartmouth College
PhD in Quantitative Biomedical Sciences
Data ScienceTech Institute (Paris and Nice, France)
DePaul University
Drexel University
Master of Science in Business Analytics
Donau-Universität Krems (Austria)
Dublin City University (Ireland)
MSc in Computing (Data Analytics Major)
Dublin Institute of Technology (Ireland)
MSc in Computing (Data Analytics)
Ecole Central Paris (France)
Eindhoven University of Technology (Netherlands)
Fordham University
George Mason University
PhD in Computational Sciences and Informatics
Georgetown University
Master of Science in Analytics with a concentration in Data Science
George Washington University
Harrisburg University
Harvard University
Master of Science (SM) in Computational Science and Engineering (CSE)
Heriot-Watt University (Edinburgh, UK)
IE University (Madrid, Spain)
Master in Business Analytics and Big Data
Illinois Institute of Technology
Master of Science in Marketing Analytics and Communication
Imperial College (London, UK)
MSc in Data Science and Management
Indiana University Bloomington
Indiana University-Purdue University Indianapolis School of Informatics
Iowa State University
Jacksonville University
Master of Science in Applied Business Analytics
Jheronimus Academy of Data Science (Netherlands)
Data Science Master’s Programs
John Hopkins University
Master of Science in Bioinformatics
Kennesaw State University
Master of Science in Applied Statistics
Lipscomb University
Master of Science in Informatics and Analytics
Loras College
Loyola University Maryland
Master of Science in Data Science
Louisiana State University
Master of Science in Analytics
Maastricht University (Maastricht, the Netherlands)
MSc in International Business/Business Intelligence
Macquarie University (Sydney, Australia)
Merrimack College
Master of Science in Data Science
Michigan State University
MIT
Master of Business Analytics (M.B.An.)
New College of Florida
New Jersey Institute of Technology
Graduate Certificate in Data Mining
New York University
MS in Applied Urban Science and Informatics
North Carolina State University
Northeastern University
Graduate Certificate in Data Science
Northwestern University
Master of Science in Analytics
Nova Northeastern University
Graduate Certificate In Business Intelligence / Analytics
Oakland University
Master of Science in IT Management – Business Analytics
Pace University
MS in Customer Intelligence and Analytics
Purdue University
Rensselaer Polytechnic Institute
Master of Science in Business Analytics
Robert Gordon University (Aberdeen, UK)
Rutgers University
MBA in Analytics and Information Management
Saint Mary’s College
Master of Science in Data Science
Saint Peter’s University
Master of Science in Data Science
Stanford University
Matser of Science in Biomedical Informatics
Mining Massive Data Sets Graduate Certificate
Stevens Institute’s Wesley J. Howe School of Technology Management
Master of Science in Business Intelligence and Analytics
Graduate Certificate in Business Intelligence and Analytics
Swansea University (Wales, UK)
Syracuse University School of Information Studies
Graduate Certificate of Advanced Studies in Data Science
Texas A&M University
Master of Science in Analytics
Università di Pisa
University College Dublin (Ireland)
University College London (UK)
University of Auckland (Auckland, New Zealand)
Master of Professional Studies–Data Science
University of California, San Diego
Master of Advanced Study in Data Science and Engineering
University of Chicago
Master of Science in Analytics
M.S. in Computer Science/Data Analytics
University of Cincinnati
University of Connecticut
MS in Business Analytics and Project Management
University of Dundee (Dundee, UK)
University of Edinburgh (UK)
University of Essex (Colchester, UK)
MSc Big Data and Text Analytics
University of Glasgow (UK)
University of Illinois at Urbana-Champaign
Master of Science in Statistics/Analytics Concentration
University of Iowa
Graduate Certificate in Business Analytics
University of Magdeburg (Magdeburg, Germany)
MSc in Data and Knowledge Engineering
University of Maryland
Master of Science in Business for Marketing Analytics
University of Maryland University College
Master of Science in Data Analytics
University of Massachusetts, Dartmouth
University of Michigan-Dearborn
University of Rochester Simon School of Business
University of San Francisco
University of Southern California Marshall School of Business
Master of Science in Business Analytics
University of Stirling (UK)
University of Tennessee
Master’s in Business Analytics
Graduate Certificate in Business Analytics
University of Texas at Austin
Master of Science in Business Analytics
University of Virginia
Master of Science in Data Science
University of Warwick (UK)
University of Washington
Virginia Commonwealth University
Master of Science in Business with a decision sciences and business analytics concentration
Worcester Polytechnic Institute
York University
Fellowship/Training Programs
Online University Programs
Aegis School of Business, Data Science and Telecommunication
Post Graduate Program in Business Analytics and Big Data
American Sentinel University
Master of Geospatial Information Systems Program
Arizona State University
Master of Advanced Study in Health Informatics
Bay Path University
Brandeis University
Master of Science in Strategic Analytics
Carnegie Mellon University
Master of Science in Business Analytics
City University of New York
Elmhurst College
Harrisburg University
Indiana University
Lewis University
Master of Science in Data Science
Maryville University
Master of Science in Business Data Analytics
Northwestern University
Master of Science in Predictive Analytics
Nova Northeastern University
Graduate Certificate In Business Intelligence / Analytics
Saint Joseph’s University
Master of Science in Business Intelligence & Analytics
Saint Mary’s College
Southern Methodist University
Master of Science in Data Science
Thomas Edison State College
University of North Carolina
MBA (Data Analytics and Decision Making Concentration)
University of California, Berkeley
Master of Information and Data Science
University of California, Riverside
Master of Science in Engineering (Data Science specialization)
University of Wisconsin
Online Courses (Free)
Coursera
Data Analysis and Statistical Inference
Process Mining: Data science in Action
Data Society Online Courses
Intro to Data Science, Machine Learning, etc.
edX
Data Science and Machine Learning Essentials
Wiretaps to Big Data: Privacy and Surveillance in the Age of Interconnection
Harvard University
Udacity
Introduction to Artificial Intelligence
Online Courses (for a fee)
365DataScience
Booz Allen Hamilton
Coursera
Executive Data Science Specialization
MIT
Tackling the Challenges of Big Data
Udacity
Introduction to Hadoop and MapReduce
University of California, Irvine
Note: For similar lists, including undergraduate programs, see here and here and here and here
Resources
2019 Salaries Of Data Scientists
A Very Short History of Data Science
A Very Short History of Big Data
A Very Short History Of Artificial Intelligence (AI)
Posted in Big Data, Data Science
Leave a comment
Graduate Programs in Big Data Analytics/Data Science
Posted in Big Data, Data Science
Leave a comment
3 Recent Books on Data Mining, Data Science and Big Data Analytics
Now that most of the hype around big data has died down, overtaken by the buzz over the Internet of Things, we are sometimes treated to serious discussions of the state-of-the-art (or science, for that matter) in data analysis. If you are planning a career as a data scientist or you are a business executive trying to understand what the data scientists are telling you, three recent books provide excellent and accessible overviews:
Data Mining For Dummies by Meta S. Brown
Data Science For Dummies by Lillian Pierson
Bill Franks is the Chief Analytics Officer for Teradata, and his specialty is translating complex analytics into terms that business users can understand. The Analystics Revolution follows Franks’ Taming the Big Data Tidal Wave, which was listed on Tom Peters’ 2014 list of “Must Read” books.
“With all the hype around big data, it is easy to assume that nothing of interest was happening in the past if you don’t know better from experience” says Franks. The over-excitement about big data caused many organizations to re-create solutions that already exist and build new groups dedicated to big data analysis, separate from their traditional analytics functions. As a correction, Franks advocates “a new, integrated, and evolved analytics paradigm,” combining traditional analytics on traditional data with big data analytics on big data.
The focus of this new approach–and the book–is Operational Analytics. It takes us from the descriptive and predictive analytics of traditional and big data analytics to prescriptive analytics. It pays close attention to the numerous decisions and actions, mostly tactical, taking place every day in your business. Most important, it places great emphasis on the process of analytics, on embedding it everywhere, and on automating the required response to events and changing conditions.
“Of course,” says Franks, “it takes human intervention to decide that an operational analytics process is needed and to build the process.” But once the process is designed and turned on, the process accesses data, performs analysis, makes decisions, and then actually causes actions to occur. And humans are crucial to the success of this new brand of automated analytics, not only at the design phase, but also in the on-going monitoring and tweaking of the process.
An example of operational analytics is the development of an improved maintenance schedule using sensor data. There will be no value in the Internet of Things without an automated process for data analysis and action based on that analysis. “As traditional manufacturers suddenly find themselves embedding sensors, collecting data, and producing analytics for their customers, industry lines blur. Not only are new competencies needed, but the reason customers choose a product may have less to do with traditional selection criteria than with the data and analytics offered with the product,” says Franks.
The practical advice Franks provides in the book ranges from how to set up an analytics organization to developing and maintaining a corporate culture dedicated to discovery (finding new insights in the data and quickly acting on them) to implementing operational analytics. The Analytics Revolution is an excellent guide to the new business world of blurred industry lines and innovative data products.
If you are ready to move on from understanding the why of analytics today and how to think about it in a broad business and organizational context to a more specific understanding of the how of analyzing data, Data Mining for Dummies by Meta Brown should be your first step. The book was written for “average business people,” showing them that you don’t need to be a data scientist and “you don’t need to be an expert in statistics, a scientist, or a computer programmer to be a data miner.”
Brown is a consultant, speaker and writer with hands-on experience in business analytics. She’s the creator of the Storytelling for Data Analysts and Storytelling for Tech workshops. In Data Mining for Dummies, Brown tells the story of what data miners do.
It starts with a description of a day in the life of a data miner and goes on to discuss in clear, easy-to-understand prose all the key data mining concepts, how to plan and organize for data mining, getting data from internal, public and commercial sources, how to prepare data for exploration and predictive modeling, building predictive models, and selecting software and dealing with vendors. Data Mining for Dummies is an excellent step-by-step guide to understanding data mining and how to become a data miner.
If you are ready to move on from understanding data mining and being a data miner to more advanced tools and applications for data analysis, Data Science for Dummies by Lillian Pierson should be your first step. The book was written for readers with some technical and math skills and experience, but it aims to provide a general introduction to one and all: “Although data science may be a new topic for many, it’s a skill that any individual who wants to stay relevant in her career field and industry needs to know.”
Pierson is a data scientist and environmental engineer and the founder of Data-Mania, a start-up that focuses mainly on web analytics, data-driven growth services, data journalism, and data science training services. “Data scientists,” she explains, “use coding, quantitative methods (mathematical, statistical, and machine learning), and highly specialized [domain] expertise in their study area to derive solutions to complex business and scientific problems.
Data Science for Dummies is an excellent practical introduction to the fundamentals of data science. It provides a guided tour of the data science landscape today, from data engineering and processing tools such as Hadoop and MapReduce to supervised and unsupervised machine learning, statistics and mathematical modeling, using open-source applications such as Python and the R statistical programming language, finding resources for publicly-available data, and data visualization techniques for showcasing the results of your analysis. Stressing the importance of domain expertise for data scientists, Pierson provides detailed examples of applying data science in specific domains such as journalism, environmental intelligence, and e-commerce.
“A lot of times,” says Pierson, “data scientists get caught up analyzing the bark of the trees that they simply forget to look for their way out of the forest.” The three books reviewed here provide a handy map to the maze of data analysis and a safe conduct pass for business executives, IT staff, and students, ensuring that they successfully get in and out of the data forest. Remember, as ones and zeros eat the world, data is the new product and operational analytics, data mining, and data science is the new process of innovation.
Posted in Big Data, Data Science
Leave a comment