Most recent update: June 2, 2012
International Conference on Advancements in Information Technology 2012
June 2-3, Hong Kong
Data Analysis Conference: Tools of the Trade
June 4-5, Atlantic City, New Jersey
TDWI Solution Summit: Big Data Analytics for Real-Time Business Advantage
June 4-6, San Diego
June 4-7, University of Maryland, College Park, MD
Continue reading
The third annual Burtch Works Study: Salaries of Data Scientists April 2016 is out, documenting the continuation of a very favorable market for those with the sexiest job of the 21st century. However, the salaries of data scientists appear to be leveling off: Every job category except one (entry-level individual contributors) experienced a marginal single-digit shift in median base salary over the past year. This compared to the overall increase in compensation of 14% in last year’s report.
The Burtch Works Study is based on compensation and demographic data for 374 data scientists collected in interviews conducted by Burtch’s recruiting staff during the 12 months ending March 2016. It focuses on data scientists as distinguished from other analytics professionals, defining them as follows:
Data scientists apply sophisticated quantitative and computer science skills to both structure and analyze massive unstructured datasets or continuously streaming data, with the intent to derive insights and prescribe action. The depth and breadth of their coding skills distinguishes them from other predictive analytics professionals and allows them to exploit data regardless of its source, size, or format. Through the use of one or more general-purpose coding languages and data infrastructures, data scientists can tackle problems made very difficult by the size and disorganization of the data.
Here are the highlights of the new report.
Individual contributors: Median base salaries range from $97,000 at level 1 to $152,000 at level 3 plus bonuses ranging from $10,000 to $21,000 (over 73% of all individual contributors are eligible for bonuses).
Managers: Median base salaries range from $140,000 at level 1 to $240,000 at level 3 plus bonuses ranging from $15,000 to $80,000 (over 80% of managers are eligible for bonuses).
Salary changes from last year’s study: Base salaries for individual contributors have increased 7% at level 1 and 1% at level 3, while salaries remained steady at level 2. For managers, salaries remained steady at level 1 while those at level 2 increased 3%. At level 3, the median base salary decreased by 4% ($10,000).
Data scientists continue to get top compensation for analytics professionals: Data scientists earn base salaries up to 39% higher than other predictive analytics professionals depending on job category.
A shift in the educational background of data scientists: 59% of level 1 individual contributors’ highest degree is a Master’s, a significant increase from last year’s 48%.
An increase in the number of U.S. citizens in the data science talent pool: Among level 1 individual contributors, only 43% of this year’s professionals are foreign-born vs. 53% last year.
It appears that the increase in the number of graduate-level programs in data science has started to make its mark and is contributing to an increase in the supply of entry-level data scientists with a Master’s degree. Other trends Burtch Works has observed in its recent conversations with data scientists are increased desire to work for “more mission-driven organizations attempting to make an impact on society” rather than large companies such as Facebook or Google and “the increasing pressure on many startups to show their value,” otherwise known as the coming burst of the Unicorn Bubble.
If we do see a contraction in startup activity and attractiveness over the next year, it may well be that larger and more stable companies, even in traditional industries, will become more desirable for budding—and even experienced—data scientists, regardless of their desire to “change the world.” The job opportunities—and the high compensation—will certainly be there as the practice of data science spreads into all corners of the economy. As Burtch Works predicts: “The use of data science will become more ubiquitous, the talent supply will improve, and there will be even more use cases for these techniques.”
Originally published on Forbes.com
Source: @Annu297
Vincent Müller and Nick Bostrom of FHI conducted a poll of four groups of AI experts in 2012-13. Combined, the median date by which they gave a 10% chance of human-level AI was 2022, and the median date by which they gave a 50% chance of human-level AI was 2040.
According to Bostrom, the participants were asked when they expect “human-level machine intelligence” to be developed, defined as “one that can carry out most human professions at least as well as a typical human”. The results were as follows. The groups surveyed are described below.
| Response rate | 10% | 50% | 90% | |
| PT-AI | 43% | 2023 | 2048 | 2080 |
| AGI | 65% | 2022 | 2040 | 2065 |
| EETN | 10% | 2020 | 2050 | 2093 |
| TOP100 | 29% | 2022 | 2040 | 2075 |
| Combined | 31% | 2022 | 2040 | 2075 |
Figure 1: Median dates for different confidence levels for human-level AI, given by different groups of surveyed experts (from Bostrom, 2014).
Surveyed groups:
PT-AI: Participants at the 2011 Philosophy and Theory of AI conference. By the list of speakers, this appears to have contained a fairly even mixture of philosophers, computer scientists and others (e.g. cognitive scientists). According to the paper, they tend to be interested in theory, to not do technical AI work, and to be skeptical of AI progress being easy.
AGI: Participants at the 2012 AGI-12 and AGI Impacts conferences. These people mostly do technical work.
EETN: Members of the Greek Association for Artificial Intelligence, which only accepts published AI researchers.
TOP100: The 100 top authors in artificial intelligence, by citation, in all years, according to Microsoft Academic Search in May 2013. These people mostly do technical AI work, and tend to be relatively old and based in the US.
Source: AI Impacts
To get a more accurate assessment of the opinion of leading researchers in the field, I turned to the Fellows of the American Association for Artificial Intelligence, a group of researchers who are recognized as having made significant, sustained contributions to the field.
In early March 2016, AAAI sent out an anonymous survey on my behalf, posing the following question to 193 fellows:
“In his book, Nick Bostrom has defined Superintelligence as ‘an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.’ When do you think we will achieve Superintelligence?”
…In essence, according to 92.5 percent of the respondents, superintelligence is beyond the foreseeable horizon.
See also Oren Etzioni on Building Intelligent Machines
From Oren Etzioni’s presentation at the O’Reilly AI conference, September 2016:
“Let us cultivate the mathematical sciences with ardor, without wanting to extend them beyond their domain; and let us not imagine that one can attack history with formulas, nor give sanction to morality through theories of algebra or the integral calculus”–Augustin-Louis Cauchy, 1821, quoted by Matthew Jones, Columbia University
“…the common language of business is not going to be Chinese or Spanish. It’s going to be math”–Michael Rhodin, IBM
“The future is going to be owned by people who are comfortable in the quant world but have deep business knowledge”–Christine Poon, Max M. Fisher College of Business, Ohio State
“[One false promise that some proponents of Big Data hold out is that somehow vast oceans of digital data can be sifted for nuggets of pure enterprise gold.] It is not going to happen magically. The software only finds correlations, not causations. In order to find causal relationships you have to do work. If you take any sufficiently large data sets, you are going to find correlations. You need a human in the loop to work out which are important”–Stephen Sorkin, Splunk
Continue reading
Gartner: The emerging technologies on the Gartner Inc. Hype Cycle for Emerging Technologies, 2017 reveal three distinct megatrends that will enable businesses to survive and thrive in the digital economy over the next five to 10 years.
Artificial intelligence (AI) everywhere, transparently immersive experiences and digital platforms are the trends that will provide unrivaled intelligence, create profoundly new experiences and offer platforms that allow organizations to connect with new business ecosystems.
See also
Gartner Hype Cycle for Emerging Technologies 2016: Deep Learning Still Missing
Most Hyped Technologies: Self-Driving Cars, Self-Service Analytics, IoT; No More Big Data Buzz
A new survey of 316 executives from large global companies, conducted by Forbes Insights and sponsored by Teradata in partnership with McKinsey, provides a fresh look at the state of big data analytics implementations. Here are the highlights.
The hype gone, big data is alive and doing well
About 90% of organizations report medium to high levels of investment in big data analytics, and about a third call their investments “very significant.” Most important, about two-thirds of respondents report that big data and analytics initiatives have had a significant, measurable impact on revenues.
59% of the executives surveyed consider big data and analytics either a top five issue or the single most important way to achieve a competitive advantage. This attitude is slightly more prevalent in financial services and much more prevalent in Asia-Pacific, where 41% of executives (compared to the survey average of 21%) consider big data and analytics the single most important way for companies to gain a competitive advantage.
The right organizational culture is key to big data success
No matter how many times you say “data-driven,” decisions are still not based on data. Sounds familiar? 51% of executives said that adapting and refining a data-driven strategy is the single biggest cultural barrier and 47% reported putting big data learning into action as an operational challenge. 43% cited fostering a culture that rewards use of data and valuing creativity and experimentation with data as key challenges.
Companies that don’t get the data-driven culture right tend to fall behind their peers. 47% of executives surveyed do not think that their companies’ big data and analytics capabilities are above par or best of breed. And the survey found that the more the respondents know about big data and analytics, the less likely they are to judge the organization as above average or best of breed. For example, among data scientists, only 8% call their organizations best of breed and 10% think they are above average.
Big data is top of mind when the CEO loves data
If you take big data analytics seriously, you get results. 51% of organizations where big data is viewed as the single most important way to gain competitive advantage are led by CEOs who personally focus on big data initiatives. In organizations where big data is viewed as a top-five issue that gets significant time and attention from top leadership, the sponsor is typically one level below top leadership. Finally, companies that have established data and analytics positions at the CxO level are more likely to have above average data analytics capabilities.
Going from the right attitude to the right action is a long big data journey
Even if you have top leadership sponsorship and the right culture, getting data to drive action and strategy is a challenge. 48% of executives surveyed regard making fact-based business decisions based on data as a key strategic challenge, and 43% cite developing a corporate strategy as a significant hurdle. Other obstacles to realizing the benefits of big data analytics are focusing resources to get the most insights from data (43%) and viewing data as a valuable asset (41%).
There’s gold in them thar brontobyte data mountains
The survey found that big data is driving opportunities for innovation in three key areas: creating new business models (54%); discovering new product offers (52%); and monetizing data to external companies (40%). To pursue these opportunities, companies that are gaining the most traction are looking beyond transactional data—exploring a wide variety of many data types.
The most-cited was location data (used to identify an electronic device’s physical location), collected by over half of the respondents, followed by text data (unstructured data like email messages, slides, Word documents, and instant messages). Social media is tracked and its unstructured data collected by 43% of companies surveyed and about a third finds golden nuggets in images, weblogs, videos, sensor data and speech files.
Big data miners still very much wanted
Realizing the business and innovation opportunities hidden in the mountains of data requires the right set of skills and experiences. 46% of the executives surveyed, however, reported that hiring the talent that can recognize innovations in data is a challenge.
Originally published on Forbes.com
Two new surveys tell us a lot about both the supply and demand sides of the hot market for data scientists, “the sexiest job of the 21st Century.”
On the demand side—the challenges of recruiting, training, and integrating data scientists—we have the MIT Sloan Management Review and SAS fifth annual survey of 2,719 business executives, managers and analytics professionals worldwide. On the supply side—the talent available and what salaries it commands—we have the second annual Burtch Works Study, surveying 371 data scientists in the U.S. (see also the video presentation at the end of this post).
The median salary of a junior level data scientist is $91,000, but those managing a team of ten or more data scientists earn base salaries of well over $250,000, according to Burtch Works. Supply is still tight and top managers enjoyed over the last year an eight percent increase in base salary and median bonuses over $56,000. When changing jobs, data scientists see a 16 percent increase in their median base salary.
Who are these data scientists that are so much in demand? The vast majority have at least a master’s degree and probably a Ph.D., and one in three are foreign-born. But with a younger generation of data scientists, freshly minted from more than 100 graduate programs worldwide, the median years of experience dropped from 9 in 2014 to 6 in 2015.
As data science is increasingly adopted by all companies in all industries, the proportion of data scientists employed by startups—the firms that have dominated the application of big data analytics— declined from 29 percent in 2014 to 14 percent in 2015.
It is the mainstreaming of data science and the specific challenges of acquiring and benefiting from this still-scarce talent pool that is the focus of the MIT Sloan Management Review survey. Four in ten (43%) companies report their lack of appropriate analytical skills as a key challenge but only one in five organizations has changed its approach to attracting and retaining analytics talent.
As a result of the scarcity of data scientists, 63 percent of the companies surveyed are providing formal or on-the-job training in-house. “One big plus of developing analytics skills among current employees,” says the report, “is that they already know the business.” These companies are also doing more to train existing managers to become more analytical (49%) and train their new data scientists to better understand their business (34%). Still, half of the survey respondents cited turning analytical insights into business actions as one of their top analytics challenges.
To better manage these challenges, the study recommends giving preference to people with analytical skills when hiring and promoting, developing analytical skills through formal in-house training, and integrating new talent with more traditional data workers.
“Infusing new analytics talent without proper support and guidance can alienate traditional data workers and undermine everyone’s contributions,” says the report. Yet only 27% of companies report that they successfully integrate new analytics talent with more traditional data workers. So even after managing to find (and pay for) the data science talent, there is no guarantee for the desired results, either because of the lack of understanding of the business by the new recruits, resistance from current employees engaged in data preparation and analysis, or failure to translate new insights into meaningful action.
Many companies have responded to these challenges by creating new roles and responsibilities and devising new organizational structures. The report points out that the range of analytics skills, roles and titles within organizations has broadened in recent years. What’s more, new executive roles, such as chief data officers, chief analytics officers and chief medical information officers, have emerged to ensure that analytical insights can be applied to strategic business issues.
Whether the work is centralized or decentralized, data science and analytics should be perceived and managed by companies as a professional function with its own clear career path and well-defined roles. Tom Davenport asked in a recent essay: “When was the last time you saw a job posting for a ‘light quant’ or an ‘analytical translator’? But almost every organization would be more successful with analytics and big data if it employed some of these folks.”
Davenport defines a “light quant” as someone who knows something about analytical and data management methods, and a lot about specific business problems, and can connect the two. An “analytical translator” is someone who is extremely skilled at communicating the results of quantitative analyses.
Data science is a team sport that requires the right blending of people with different skills, expertise, and experiences. Data science itself is an emerging discipline, drawing people with diverse educational backgrounds and work experiences. Typical of the requirements for a graduate degree is what we find in a recent announcement from the University of Wisconsin’s first system-wide online master’s degree in data science: “The Master of Science in Data Science program is intended for students with a bachelor’s degree in math, statistics, analytics, computer science, or marketing; or three to five years of professional experience as a business intelligence analyst, data analyst, financial analyst, information technology analyst, database administrator, computer programmer, statistician, or other related position.”
As with any team sport, there are stars that are paid more than the average player. According to Glassdoor (HT: Illinois Institute of Technology Master of Data Science program), the average salary for data scientists is a bit more than what Burtch Works reported, at over $118,000 per year. (By the way, Glassdoor reports the average salary for statistician is $75,000 and $92,000 for a senior statistician).
It’s possible that the Glassdoor numbers include more of what Burtch Works calls “elite data scientists.” Do we know who is in the elite of top data science players? The closest we get to identify the MVP of data science is the Kaggle ranking of the data scientists participating in its competitions. Currently, Owen Zhang is number one. Zhang says on his profile that “the answer is 42” and his bio section tells us that he is “trying to find the right question to ask.” He lists his skills as “Excessive Effort, Luck, and Other People’s Code.”
Zhang is currently the Chief Product Officer at DataRobot, a startup helping other data scientists build better predictive models in the cloud. He is also yet another example of how experience and skills still matter today more than formal data science education. His educational background? Master of Applied Science in Electrical Engineering from the University of Toronto.
This Burtch Works webinar provides highlights from the 40+ pages of compensation and demographic data in the report, which is available for free download here: http://goo.gl/RQX1xd
[youtube https://www.youtube.com/watch?v=aEkpVr8Q6oI?rel=0]
A new study of LinkedIn profiles by RJMetrics has found that the number of data scientists has doubled over the last 4 years . This reflects the increasing demand for sophisticated data analysis skills, combining computer programming with statistics, and the growth in the popularity of the term “data science” both in job openings and the words people use to describe their work on LinkedIn. At least 52% of all current 11,400 data scientists on LinkedIn have added that title to their profiles within the past 4 years.
In the chart above, the cumulative number of data scientists in any given year corresponds to the number of present-day data scientists who started their first job that year. We can safely assume that those who started their first jobs between 1995 and 2009 were not called then “data scientists,” but the data shows the cumulative growth in the number of professionals who have this title today.
Here are the other highlights of the study:
The high-tech industry (LinkedIn classification: Information Technology and Services industry, Internet and Computer Software industries) employs 44.9% of the professionals identified on LinkedIn as data scientists, followed by education (8.3%, probably employed mostly by universities), Banking and Financial Services (7.2%), and Marketing and Advertising (5.2%).
The top ten companies employing data scientists are Microsoft, Facebook, IBM, GlaxoSmithKline, Booz Allen Hamilton, Nielsen, GE, Apple, LinkedIn, and Teradata. Note that Google is not at the top ten, possibly because the data science Googlers on LinkedIn adhere to the title Google bestows on them: quantitative analyst.
Both Microsoft and Facebook, according to RJMetrics’ analysis, appear to be on a hiring spree, accelerating their data scientist recruiting during the 2014 calendar year by at least 151% and 39%, respectively, when compared to 2013. But given the scarcity of experienced data scientists, it’s a revolving door, with Microsoft also losing the largest number of data scientists over that period.
So how do you become one of these unicorn data scientists, commanding annual salaries of $200,000 plus? The study provides fresh data on the skills and background of data scientists.
RJMetrics analyzed 254,000 skill records of the data scientists on LinkedIn and ranked each skill by the number of people listing it on their profile. In addition to the catch-all categories of “data analysis,” “data mining,” and “analytics,” the top skills are R, Python, machine learning, statistics, SQL, MATLAB, Java, statistical modeling, and C++. Hadoop (20.9%) is at the bottom of the top 20, as a specific skill, behind SAS (22.78%).
An analysis of skills by job levels revealed that chief data scientists appear to be less technical on average: Only 27% and 26% listed Python and R, respectively, compared to 52% and 53% of junior data scientists, along with 38% and 43% of senior practitioners. Those at higher level jobs may not need to emphasize their technical skills or may not need them in positions where management experience and knowledge of a business domain are valued more than technical proficiency.
Over 79% of data scientists listing their education have earned a graduate degree, with 38% of all data scientists who had an education record earning a PhD, and close to 42% listing a Master’s degree as the highest degree attained.
Computer Science is the dominant field of study among data scientists, followed by business administration/management, statistics, mathematics, and physics. Only 4.6% of data scientists list “machine learning/data science” as their graduate degree, a number that will probably increase in coming years due to the proliferation of new Master in Data Science programs, supplanting the older Master in Analytics programs.
Note that RJMetrics included in their sample only data scientists associated with specific companies, assuming that those listing “data scientist” in their profile without an association with an actual company may only have aspirations about a career in data science, but not actual experience. They analyzed 60,200 records of professional experiences, 27,700 records of education, and 254,600 records of skills, and information about 6,200 unique companies that employed self-identified data scientists as of June 1, 2015.
For other recent studies of the skills and salaries of data scientists see here and here.
Data are to this century what oil was to the last one: a driver of growth and change. Flows of data have created new infrastructure, new businesses, new monopolies, new politics and—crucially—new economics. Digital information is unlike any previous resource; it is extracted, refined, valued, bought and sold in different ways. It changes the rules for markets and it demands new approaches from regulators. Many a battle will be fought over who should own, and benefit from, data…
The problem [with personal data] is the opposite to that with corporate data: people give personal data away too readily in return for “free” services. The terms of trade have become the norm almost by accident, says Glen Weyl, an economist at Microsoft Research. After the dotcom bubble burst in the early 2000s, firms badly needed a way to make money. Gathering data for targeted advertising was the quickest fix. Only recently have they realised that data could be turned into any number of AI services.
Whether this makes the trade of data for free services an unfair exchange largely depends on the source of the value of the these services: the data or the algorithms that crunch them? Data, argues Hal Varian, Google’s chief economist, exhibit “decreasing returns to scale”, meaning that each additional piece of data is somewhat less valuable and at some point collecting more does not add anything. What matters more, he says, is the quality of the algorithms that crunch the data and the talent a firm has hired to develop them. Google’s success “is about recipes, not ingredients.”
That may have been true in the early days of online search but seems wrong in the brave new world of AI. Algorithms are increasingly self-teaching—the more and the fresher data they are fed, the better. And marginal returns from data may actually go up as applications multiply, says Mr Weyl.
See also:
Data is Eating the World: 163 Trillion Gigabytes Will Be Created in 2025
Data Is Eating the World: Enterprise Edition
